Why the latest System One models could be great for PR

It's twenty to eight on a Tuesday morning, and a story about a supply-chain breach at a big retailer has just landed. Somewhere inside Agentcy, a model is asked whether it's a newsjack for one of our cybersecurity clients. A few seconds later it answers: yes, 0.87.

That answer decides whether a consultant sees the story over their first coffee or misses it entirely. Now picture the same thing happening for every article and coverage alert we pull in, for every client, all day long.

Hardly any of those calls involve writing. Is this coverage about our client, or about a company with a similar name? Which part of the business does it belong to? They're decisions, and today we hand them to chat models that were built to hold a conversation.

Those models do a good job. But that 0.87 has always bothered us, because nobody can tell you what it actually means. This month, a new kind of model gave us a way to find out.

The problem with "0.87"

When we ask a language model whether an article is a newsjack for a client, it returns JSON: approved or not, plus scores for confidence, authority, value-add and relevance. Our code then applies a bar. At 0.75 or above, it goes to the team.

But 0.87 from a chat model is an opinion in the shape of a number. No probability stands behind it, and it drifts between runs and model versions. It also runs high, because models trained on human feedback (RLHF) learn that sounding sure gets rewarded.

So our thresholds can't be reasoned about. We set each one by looking at examples until it felt right. We can't say "at 0.8 we're wrong one time in five", because the score never promised to mean that.

What Typesafe has built

On 15 September, Typesafe AI came out of stealth with a new class of model it calls System One, after Kahneman's fast, intuitive mode of thinking. Its first model, Jev, doesn't generate text at all. You send it material and a set of questions, and it returns typed answers with probabilities:

• Choice: pick one option from a list, with the probability of every option.
• Score: rate the material against a rubric, with the probabilities across levels.
• Noul: is this statement true? One number from 0 to 1.

The training recipe is new too. Typesafe calls it RLCD, reinforcement learning for calibrated decisions. RLHF rewards answers people prefer; RLCD rewards probabilities that match outcomes. When a calibrated model says 70%, it's right about seven times in ten.

Typesafe puts it best with the description:

"If a model can do a task 95% of the time but doesn't say when it's in the 5%, it can't automate that task."

Typesafe quotes 70–500 ms end to end, against seconds for a chat model, and $0.042 per million input tokens with output free. Answers come back as typed values against a schema defined in advance, so there's never a malformed response to catch. And a model that doesn't write sentences can't invent one.

Why PR is a near-perfect fit

PR monitoring is volume. Agentcy reads articles from hundreds of news feeds, plus coverage alerts, forums, LinkedIn, podcasts and journalist enquiries, around the clock, for every client. Almost every item needs a few small, bounded questions answered before a person should see it.

System One models are built for that shape:

• The answers come from a short list. Which sector, which client, is it spam.
• The volume is high. Today, cost forces us to filter first and send only a shortlist to the model that judges. At Typesafe's prices, asking every item every question costs less than the shortlist does now.
• Speed matters. Newsjacking is a race. A decision in under a second means an alert while the story is still moving.
• We already have the labels. Every coverage correction, spam flag and newsjack a consultant approves or bins is ground truth. We can measure calibration on our own data.
• The stakes vary. Some calls are safe to automate; others need a consultant. Calibrated odds tell you which is which.

What we're testing

We're testing Jev on the three jobs below. We're comparing its accuracy and cost with today's models, and plotting its stated confidence against what actually happened.

1. Article categorisation

Every article that enters Agentcy gets a structured card covering industry, subtopics, entities, geography, audience and angle. Those fields drive the newsfeed and story clustering. Today a small language model writes the card for every article.

Most of that card is classification in disguise. Industry is a Choice from our taxonomy. "Is this about the UK?" is a yes-or-no. We're testing whether Jev answers those fields more consistently at a fraction of the cost, and whether its confidence flags the cards that deserve a second look. The summary stays with a language model, because that part is writing.

2. Newsjack confidence rating

This is where calibration matters most. Each article gets a handful of small questions, asked in parallel:

• Is this a newsjack opportunity for this client? (yes/no, with a probability)
• Which of the client's themes does it touch? (Choice)
• How time-sensitive is it, 1 to 5? (Score)

Then the probability decides where it goes. High confidence goes straight to the team with the suggested angle. The middle band goes to a review queue, where the odds tell a consultant how hard to look. Low confidence is dropped, and for the first time we can defend that cut-off with a number.

Newsjack angles and draft comment still come from our reasoning models. Jev decides which stories are worth that effort.

3. Coverage client matching and categorisation

Every coverage alert passes through around ten separate model calls before a client sees it: is this genuinely about the client, how good is the mention, is it spam or a scraper copy, is it wire copy, how trusted is the source, and which account does it belong to (parent brand or subsidiary).

Every one of those is a Choice, a Score or a yes-or-no. With a decision model they collapse into one call that returns every answer in under half a second. The spam line changes meaning too. Today we mark something as spam when the model says it's 0.7 sure, which is only the model's opinion. With calibrated output, 0.7 means spam seven times in ten.

A person already checks every result here, which makes it the easiest place to measure calibration before anyone relies on it.

Fast, slow and find

Decision models sit alongside language models. The stack becomes three kinds of model doing three kinds of job:

• Fast: decision models for the high-volume calls, like relevance and urgency. Cheap enough to ask about everything, with odds you can trust.
• Slow: reasoning models for the angle and the campaign plan. Slower and dearer, and worth it where judgement matters.
• Find: embeddings for search. They build the shortlist the other two work on.

Agentcy already has the last two. The first is the missing piece, and it's the one that lets us automate the low-stakes calls with confidence and point our consultants' time at the ones that matter.

What comes next

We're excited about what System One models could do for Agentcy, and the early results we're seeing are really promising.

For our consultants, the prize is time. If a model can sort the routine calls in a fraction of a second and say honestly when it's unsure, people can spend their mornings on the stories that need a human eye. For our clients, it means faster alerts and cut-offs we can explain with a number.

It also changes which questions are worth asking. Once a decision costs almost nothing, we can stop sampling and ask it of every article and every mention. We already have a long list of places across the platform where that could help.

We'll share what we find.