Model card
A plain description of the AI components in marketscout, what they are for, and how well they have been measured to work. Written 26 October 2026; the table in section 5 is generated from the latest evaluation run.
1. System summary
| Name | marketscout |
| Purpose | Weekly cited research briefs on crypto, equities, gold and silver |
| Output | A structured brief (JSON, schema version 1.0) and a readable version, each claim linked to its sources |
| Language model use | Normally three calls per brief: draft, verify, summarise |
| Autonomy | Runs on a schedule. It reads public data and publishes briefs. It has no tools beyond reading its configured sources and takes no action outside producing the brief |
2. Intended and out-of-scope use
Intended: a starting point for a person who wants a sourced summary of recent price movement and news for a handful of assets, and a demonstration of an auditable AI research pipeline.
Not intended for: investment, trading, tax or legal decisions; real-time pricing; anything where a wrong figure causes harm without a person checking it; personal or retail price quotes; regulated advice of any kind.
3. Models
The pipeline calls any provider with an OpenAI-compatible interface, tried in the order set in config/config.yaml. At the time of writing the first choice is openai/gpt-oss-120b on Groq, falling back to openai/gpt-oss-20b, then other hosted and local models. Temperature is 0.2. The model that actually answered is recorded in every brief and run ledger. A brief written by more than one model says so in its run notes.
No model is trained or fine-tuned by this project.
4. Data
Prices come from CoinGecko, Yahoo Finance chart data, and COMEX gold and silver futures; currency rates from the Frankfurter service (European Central Bank rates); filings from SEC EDGAR; news headlines from news search and publisher feeds. No personal data about readers is processed. Source terms have not yet been formally reviewed (see the risk register, R4).
5. Measured performance
Measured on hand-written test cases (synthetic headlines from invented outlets) and on the published briefs. Cases are in eval/cases.json; full results are in eval/results/REPORT.md.
Latest run: 2026-10-27, test cases version 1.1.
| Check | Result |
|---|---|
| Citation rules | 5 of 5 cases pass |
| Number check | 14 of 14 cases pass; wrong numbers caught 5/5; false alarms 0/9 |
| Filler and fragment guard | 14 of 14 cases pass |
| Telegram news-line pick | 7 of 7 cases pass |
| Verifier (groq/openai/gpt-oss-120b) (measured 2026-10-26) | strict accuracy 86%; flawed claims wrongly accepted 0% of 14; good claims wrongly rejected 0% of 8 |
| Published briefs (2026-10-26) | 52 claims in 6 briefs; citations valid 100%; numbers found in sources 100% |
How to read it: the first two rows check the code against labelled cases and must stay at 100%. The verifier row measures a model and varies by model and by run; on repeated runs the same model made three misses each time but not always the same three, so treat a single run as accurate to within about one or two cases. The number that matters most is flawed claims wrongly accepted, because that is a flawed claim reaching a published brief. The test set is small and was written by the developer, so these figures show the checks work on known failure types, not that they catch every error.
6. Known limitations
- Headline-only news. Claims about news are limited to what a headline says. Context in the full article is not available to the model or the verifier.
- Correlated checking. The drafting and verification calls usually use the same model, so a mistake the model tends to make may not be caught. Using a different model for verification is planned.
- Over-trimming. The verifier sometimes repairs a claim in a way that is accurate but loses the outlet's name. These are visible in the claim trail.
- Number check is not a gate. A figure that the cited evidence does not contain is flagged in the trail but is not automatically removed.
- Futures, not spot. Gold and silver prices are futures prices, labelled as such.
- Small and narrow scope. Six subjects, English-language sources, mostly US sources.
7. Human oversight and monitoring
- Every run saves a ledger with flags for token use over budget, fallback to another model, long rate-limit waits and missing token counts.
- Every claim has a trail showing the decision and the reason, including the claims that were removed.
- The evaluation can be re-run at any time with
marketscout eval(free, offline) andmarketscout eval --live(scores the verifier). - There is currently no human approval step before a brief is published. A review queue for flagged claims is planned.
8. Change policy
The brief schema changes only by a deliberate, versioned decision. The model chain and checks are set in configuration and code that are versioned in git, and each run ledger records the configuration used.