Decision Model vs. Conventional LLM - Building a natural language bridge to a formal travel planning system

Why we tried this

TripSnek plans multi-city trips around Europe. The core of it is an optimizer: a genetic algorithm that searches thousands of candidate itineraries (which cities, in what order, how many nights each, train or car or plane between them) and evolves them toward the best fit for what you asked for.

What you ask for comes in two kinds:

People don't think in constraint forms, though. They write "two weeks in October, landing in London and flying home from Rome, three nights in Paris, and we'd rather take trains." So we've been playing with a side project: a plain-language front door that reads a description like that and fills in the planner for you. It's an experiment, not a core part of TripSnek, and this page is about one branch of it: what happens if the model reading the post is Jev instead of an LLM?

The task: slot filling

If you want the textbook name, this is slot filling: read free text and fill a fixed set of typed fields. It's the same shape as a booking bot pulling "destination" and "date" out of "I need a flight to Boston on Friday." Our form just has more slots, and some of them repeat, like the list of stops.

Our test set is real posts from r/Europetravel, each hand-annotated with what it actually asks for. Here's a short one:

Most posts are longer and messier, with typos, hedging ("maybe 2–2.5 weeks"), cities they've already seen, and places that aren't places ("Oktoberfest"). We score one thing: how much of what was asked for gets captured, feature by feature, weighted by how clearly the post states each one. Invented constraints cost points as well as missed ones. Turning that into a plannable trip is a separate step, and it's the same code whichever model did the reading.

Two very different tools for the job

Jev is a decision model. You send it a piece of text plus a list of typed questions, and it answers every question in one pass with a calibrated probability: a Choice over options you define, a Score on a rubric, or a Noul, the probability that a claim is true. What it can't do is write. It can't say "Lisbon"; it can only pick Lisbon from a list you gave it. Here's how that plays out on this problem:

Same job, same words

To keep the comparison fair, both pipelines are given the task in literally the same words. Every decision criterion lives in one file, and the same strings are interpolated into Claude's system prompt and attached to Jev's questions as the descriptions of their options. Change one and both change. Here's the definition of a ruled-out place, and how each model receives it:

forbidden: "Somewhere they implicitly or explicitly rule out: they say they
  do not want it, indicate that one or more travelers have already seen or
  visited, or are explicitly skipping it."

## What each place is to the trip

Every place plays exactly one of
these roles:
- **start** -- The trip begins here…
- **forbidden** -- Somewhere they
  implicitly or explicitly rule out…

choice({
  task: "What is this place to the trip?",
  mention: "Vienna",
  context: "…we've already done Vienna…"
}, { start, end, stop,
     forbidden, none })

Claude also gets the list of places, dates and lengths our scanner found in the post, as hints it's free to ignore, and both outputs go through the same validation and scoring code. A few rules exist only in Claude's prompt, for mistakes Jev can't make by construction, like writing a date in the wrong format.

What comes back

Here's the Prague post again, and what each model returned for it:

{
  "startLocation": { "name": "Prague" },
  "endLocation": { "name": "Milan" },
  "stopLocations": [
    { "name": "Florence" },
    { "name": "Venice" },
    { "name": "Pisa",
      "priority": "optional" },
    { "name": "Santorini",
      "priority": "optional" }
  ],
  "totalNights": 14,
  "notes": "Order Prague-Florence-
    Venice-Milan taken as start/end
    per listing; Pisa and Santorini
    mentioned only as ideas…"
}

Prague     role     stop 0.94
Milan      role     stop 1.00
Santorini  priority optional 0.95
tripStart           unstated 0.99
tripEnd             unstated 1.00
totalNights         "14 days" 1.00
pace                unstated 0.49
                    BlitzTour 0.42
allowFlights        0.51
interest_Ocean      0.50
… 36 more

Jev's numbers become the same kind of record only after our own code applies a threshold to each one, breaks ties, and fills in fallbacks. Here that gives six stops, two of them optional, 14 nights, and no start or end city. Nothing else clears its bar.

Sonnet's output reads like a person wrote it, notes included. Jev's is a pile of numbers, and every rule for turning them into a trip is ours to write. On this post the difference happened to favour Jev: Sonnet decided the first and last cities in the list must be the start and end, which the post never says, while Jev put 0.99 on "not stated". Across the corpus, that kind of thing goes both ways.

The easy parts and the hard parts

Some of the form mapped onto Jev's question types with no effort at all. The rest needed tricks, because the answer isn't one of a list you can write down in advance.

The hard rows are where an LLM's robustness is easy to take for granted. It doesn't care how a traveler phrases something, it can do arithmetic, and it can connect a sentence at the top of a post to one at the bottom. With Jev, every one of those has to be something your code anticipated.

That's the general shape of Jev's hardest failure: if the scanner doesn't surface something, it's invisible to everything downstream, and no question design gets it back. A phrasing the regexes don't know, a number that only exists after some arithmetic, a date written as "the week after Easter": an LLM shrugs these off, and for Jev each one is either another rule in the scanner or a guaranteed miss.

What made the difference

We started with the most obvious Jev pipeline and added one idea at a time, keeping everything else fixed. Three ideas did nearly all the work.

Look up every city name we know in the post, ask Jev what role each one plays (start, end, stop, ruled out, or not part of the trip), and take its top answer. Every other field becomes a fixed-option question: trip length picked from "3 days … 7 weeks", pace from three labels, each interest a yes/no.

That gets , which is already more than Claude Haiku 4.5 manages (), though Haiku ran without thinking because it doesn't accept the adaptive-thinking setting the other Claude models used. The failures show what's missing. Dates vanish, because no question could return one. Places outside our gazetteer are never asked about. And trip length is picked from a menu even when the post says "14 days", or says nothing at all.

If the model can only choose, the options should be exactly what the traveler wrote. A deterministic scanner finds every candidate span: places we know, capitalized names we don't, dates in any format we could think of, and duration phrases like "2-week" or "15 to 18 days". Jev's job shrinks from extraction to tagging. As a bonus, a made-up value becomes impossible: every place in the output is a substring of the post.

Asking about each mention separately is right for stops, because a trip can have any number of them. It's wrong for things there's exactly one of. Asked "what is this place to the trip?", Jev reliably says a trip's last city is a stop, which is true, and almost never says it's the end. So the one-per-trip facts get asked once, about the whole trip: where does it start, where does it end, does it come back to where it began?

Two smaller rules of the same kind. Don't ask what the text can't answer: "how many nights here?" is only asked where a number appears near the mention, because otherwise Jev will happily read one off the neighbouring city. Ask positive claims: "are flights OK?" as one yes/no read silence as "no", and 17 of 30 trips came back ground-only. Splitting it into "expects to fly between stops" and "rules out flying" let silence mean silence.

Jev gives you probabilities, so you get to choose where each line is drawn, and "take the top answer" leaves that choice unmade. We set each threshold by what a wrong answer costs. The output is a map the traveler adjusts: an extra city pinned on it costs one click to remove, but a city silently ruled out of the search is invisible. So a stop needs only a clear plurality (0.35 against a uniform 0.20), ruling a place out needs 0.6, and an interest, which quietly reweights the whole trip, needs 0.7. Where a threshold could be measured instead of argued, we measured it.

We also tried a fourth idea, re-asking the answers that sit close to a threshold. It didn't help.¹

Where it landed

With those three ideas, Jev captures of what was asked for, against for Claude Sonnet and for Claude Opus, which barely improves on Sonnet. Jev does it for of Sonnet's cost ( of Opus's), in about per post against . That's the first pair of charts at the top.

It's also steadier. Across three runs over the same posts, Jev's score moved by . The Claude models moved by (Opus), (Sonnet) and (Haiku).

And one failure caught our eye because it can't happen to Jev. In of Sonnet extractions, the output wasn't valid JSON, even though the request asked for schema-constrained output. That post scored zero for the run. A retry would usually fix it, but it's a kind of failure a decision model doesn't have: Jev never writes anything, so there's nothing to parse.

What it cost to build

Jev is cheaper to run. It was not cheaper to build. Both pipelines sit on the same lines of shared code: place grounding, validation and trip assembly. On top of that, Claude needs a prompt, an output schema and one API call. Jev needs a scanner to find the candidates, code to turn each one into questions, and the thresholds and assembly that turn answers per post back into a trip. The second chart at the top puts numbers on it.

The Jev side is the size of Claude's, but most of the difference is in kind rather than amount. About a third of the Claude side is the prompt, which is English. The Jev side has decision points against , and hand-set thresholds, each of which we had to pick by reading traces and then defend with a measurement.

Things to keep in mind

• Read "comparable", not "better". Both pipelines were tuned against this corpus. The Claude prompt went through several rounds of revision, and the annotations were widened to accept reasonable alternative readings of both models' output. But the Jev pipeline got more of that attention, and some of its lead probably reflects that. The 15 posts collected in 2026 were never used for tuning; on those alone, Jev scores and Sonnet .
• It took a lot more work: the code (see What it cost to build), and each trick above was something we found by reading traces.
• You own recall. Anything the scanner can't find, Jev can't find either.
• Some things are simply out of reach. It can't do arithmetic, can't tell whether 4/22 is April or May, and can't write a note about what it did.

When this might be worth trying

We don't know yet how far this generalizes, but on this problem the pattern seemed to fit because:

• The output is a structured record, not prose.
• Most fields are yes/no or one of a fixed set.
• The open values (names, dates, amounts) can mostly be found in the input by a scanner, a gazetteer, or a parser.
• Volume or latency makes an LLM call per request expensive or slow.
• We had a labelled set to measure against, which you'll need, because you'll be tuning a lot of small decisions.