How to Score Stock News Sentiment with an LLM

Turn a headline into a number

News sentiment analysis compresses an article into one figure on a fixed scale, almost always -1.0 for the most bearish read up to +1.0 for the most bullish, with 0.0 as neutral. A number sorts, thresholds and alerts. Nobody reads three hundred headlines before the open; they filter down to the few prints that are actually a surprise, then read those properly.

The job splits in two. The halves are not the same size:

• Score one article. A good prompt does this well. You can have it working in a minute.
• Trust the score across thousands of articles, every hour, for months. This is the part that quietly turns into an engineering project.

The prompt that works

Most first attempts read like "is this news good or bad for Tesla?" That produces mush, because "good news" and "good for the share price" are different things: a recall is bad news that the market shrugs off, a small guidance bump is dry reading that gaps the stock up. Score the price reaction, on a fixed scale, using only the text in front of the model:

Score how the news below is likely to affect the share price of {TICKER}
over the next few trading days.

Return a single number from -1.0 to 1.0:
  -1.0  extremely bearish
   0.0  neutral, mixed or irrelevant
   1.0  extremely bullish

Rules:
- Judge price impact, not whether the news is "good" or "bad" in general.
- Use ONLY the text provided. Do not rely on prior knowledge of {TICKER}.
- If the news is not about {TICKER}, or is too vague to move the price, return 0.0.
- Respond with the number only. No words, no explanation.

News:
{ARTICLE_TEXT}

Four choices are doing the work. The fixed scale with written anchors tells the model what the endpoints mean, so -0.5 stops being a vibe. Judging price impact keeps it from scoring the morality of the story. Pinning it to the provided text is what stops the model reciting what it already believes about the ticker. And number-only output means you can parse it without a regex fighting a paragraph of hedging.

When you feed the score into code, ask for JSON instead so a rationale rides along for auditing without polluting the number:

Respond as JSON only: {"ticker": "TSLA", "score": -0.35, "rationale": "one short sentence"}

Set temperature to 0. It cuts run-to-run variance without removing it, since near-ties can still flip on batching and floating-point order. It does nothing at all about the provider changing the model under you.

One limit is worth naming while the prompt is still fresh. Price impact is surprise against expectations, so a scorer that is banned from prior knowledge cannot know whether 435,000 was already the whisper number. The prompt reads the sentence, not the setup. Feed it the consensus figure and the recent move if you want it to judge the thing it was asked to judge.

From headline to score: TSLA

Take an example headline and run it through the JSON version of the prompt:

A capable model returns something like:

{
  "ticker": "TSLA",
  "score": -0.35,
  "rationale": "A clear delivery miss is bearish, softened by reaffirmed full-year guidance."
}

That is a reasonable read. The miss is the headline, the reaffirmed guidance is the cushion. A moderate negative captures both.

Now put the same sentence through the engine behind the Adanos news sentiment data, a finance-tuned hybrid of VADER and Twitter-RoBERTa. It returns 0.011, labelled neutral. Checking that costs nothing: paste the headline into the free Finance Sentiment Analyzer, which runs the same engine on anything you give it.

Same text, two scores, no agreement. The prompt was told to judge price impact, so it reads a 20,000-unit shortfall against consensus as something that should hurt the stock. The engine scores the polarity of the words, where "miss" pulls down while "reaffirms full-year guidance" pulls back up, so the sentence nets out near zero. Most desks would weight the shortfall harder than the reaffirm, which is close to boilerplate. That gap is the standing cost of scoring financial text with a general-purpose lexicon plus a transformer trained on tweets: "miss", "beat", "guidance" and "dilution" carry weight in this domain that neither model learned.

Neither number is the truth. The practical question is which one you can run on every article, every hour, for a year without the scale shifting under you. That is where the prompt starts to struggle.

Six ways a single LLM score misleads you

Everything below shows up once you go from one headline to a feed:

• False precision. The model will happily return -0.35, then -0.3 on a rerun, then -0.4 after a tiny wording change. The two-decimal look promises a precision the model does not have. Treat the output as three or four buckets rather than a continuous score.
• No calibration across runs. A -0.4 today and a -0.4 next week are not guaranteed to be the same severity. The scale drifts as prompts, models and context windows change, so a time series of raw scores can move for reasons that have nothing to do with the news.
• Priors over text. Ask about a household name and the model leans on what it absorbed in training, not the article in front of it. That is why the "use only the text provided" line matters. Test it with a made-up ticker to see whether the score follows the words or the reputation.
• Cost, volume and latency. One call costs nothing worth counting. Five hundred tickers, every source, refreshed hourly, is tens of thousands of calls a day, plus the retries, the rate limits and the bill. Breaking news makes the round trip its own problem: a scorer that answers in seconds is a different product from one that answers in minutes.
• Lookahead leakage. Backtest the prompt on 2023 headlines and the model already knows how the quarter turned out. It is not predicting, it is remembering. Every LLM score computed over historical news is contaminated by knowledge the market did not have at the time, which flatters the backtest and tells you nothing about tomorrow. A model with no memory of the outcome cannot cheat this way.
• Aggregation is the actual job. A single opinion piece is noise, though an 8-K is not. The signal you want is the read across every source on a ticker, deduplicated so the same wire story syndicated eight times does not count eight times, with some view on which sources deserve the weight. That is a pipeline, not a prompt.

The prompt survives all six as a component rather than as the answer. The moment you want a trustworthy number per ticker instead of per article, you are running and maintaining the ingestion, scoring, dedup and aggregation around it.

Why the News Sentiment API does not use an LLM

The Adanos Stock News Sentiment API does not run a prompt like the one above on any article. It scores with the VADER and Twitter-RoBERTa hybrid from the example. The reasons map onto the list you just read. The ensemble has no memory of what happened next, so a backtest over old news measures prediction rather than recall. It is cheap enough to score every article from every source hourly. Within an engine version it is deterministic, so when the number moves, the news moved.

That determinism has a boundary worth naming. The response carries an engine_version, currently 5.4. That field exists because scoring does change when the engine changes. Same text on the same version returns the same score; a version bump can move it. Determinism is reproducibility, not accuracy: a score that is wrong reproduces exactly as well as one that is right.

The trade cuts both ways. The example above is the evidence. VADER is a general-purpose lexicon and Twitter-RoBERTa learned sentiment from tweets, so neither carries a native sense of what "miss", "beat" or "guidance" do to a share price. That is the standing objection to this class of model. It is why finance-specific alternatives such as FinBERT exist. The 0.011 on the delivery miss is that limitation expressed as a single number.

You can point the engine at a passage yourself. The analyze endpoint scores arbitrary text and needs a Professional plan, though the free Analyzer above runs the same thing without a key:

{
  "sentiment_score": 0.011,
  "sentiment_label": "neutral",
  "components": {
    "engine_version": "5.4",
    "vader_compound": -0.0772,
    "roberta_score": 0.0728,
    "emoji_score": 0.0,
    "phrase_adjustment": 0.0
  }
}

For a ticker rather than a passage, the per-ticker endpoint returns those same engine scores rolled up across every source:

and the ticker comes back already scored, counted and broken down by source:

{
  "ticker": "TSLA",
  "company_name": "Tesla Inc",
  "sentiment_score": 0.30,       // -1 bearish .. +1 bullish, aggregated
  "bullish_pct": 60,
  "bearish_pct": 20,
  "mentions": 5,
  "source_count": 3,
  "buzz_score": 41.4,
  "trend": "stable",
  "top_sources": [
    { "source": "market-watch",    "sentiment_score": 0.60, "mentions": 1 },
    { "source": "the-motley-fool", "sentiment_score": 0.63, "mentions": 1 },
    { "source": "tipranks",        "sentiment_score": 0.09, "mentions": 3 }
  ]
}

The value is in the fields a single prompt cannot produce. The sentiment_score is one read across every source rather than one article. The top_sources array exposes the disagreement that any single number hides. Read the shape of this one: TipRanks supplies three of the five mentions at the lowest score, which is why the aggregate sits at 0.30 instead of near the 0.6 the other two report. An outlet that publishes more pulls harder. Worth knowing before you treat 0.30 as a verdict.

Because the scoring runs identically on every article, the series is at least consistent with itself, which drifting prompt runs never manage. Consistency is not accuracy. It only means that when the number moves, the news moved rather than the scorer.

The Finance Sentiment CLI wraps the same endpoints for terminal use. The Stock News Sentiment API page carries the full field list plus the sector and trending endpoints.

FAQ

Related