The itinerary builder got a brief for a roughly ninety-day trip across four countries. It broke four times, in four different places, and every one of them was the same mistake.
This is the tool I’ve written about a couple of times already: an advisor fills in a questionnaire, a model plans the trip in stages, and the result comes out as a multi-tab spreadsheet. It had been tested properly. On trips of a week or two, which is what people mostly book.
Break one: the skeleton ran out of room
The first stage of a build produces the skeleton — trip metadata plus a stub for every day of the trip. One day, one object. Every later stage hangs off it.
The ninety-day brief failed at that stage with this:
skeleton: model did not return meta + days
Which reads like the model got confused and returned the wrong shape. It hadn’t. It had been cut off.
Each stage has a cap on how many tokens the model is allowed to produce, and the skeleton’s was 10,000. A fortnight fits in that comfortably. Ninety days of stubs, at the verbosity the prompt was asking for, needed something like two to three times that. The model got part of the way through the days, hit the ceiling, and stopped mid-string. Half a JSON object isn’t a JSON object, so the parse failed, and the error reported the parse rather than the reason.
The response told us, too. The API returns a stop_reason on every call, and on this one it said max_tokens rather than end_turn. The code just never looked.
Three fixes:
- Raise the skeleton cap to 24,000. There’s plenty of headroom on the model’s side, and the client timeout is ten minutes.
- Make the skeleton lighter per day. One image search per day instead of up to three, one-sentence highlights, short transport notes, and an explicit instruction to stay terse on long trips. The detailed prose belongs to later stages anyway.
- Read
stop_reasonwhen a parse fails. If it’smax_tokens, tell the advisor what actually happened and what to do about it.
$body = json_decode( (string) ( $res['body'] ?? '' ), true );
$stop = is_array( $body ) ? (string) ( $body['stop_reason'] ?? '' ) : '';
$msg = ( $stop === 'max_tokens' )
? 'This itinerary is too long to plan in one pass. Split it into two shorter date ranges and build each separately.'
: 'skeleton: model did not return meta + days';
After that, the same brief came back complete: 91 days, about 19,000 tokens, end_turn.
Break two: finished, and still broken
The next run failed at the same stage. Same error. But this time stop_reason was end_turn and the response was around 16,000 tokens, well under the new cap.
Somewhere in sixteen thousand tokens of JSON there was a malformed character. The more interesting part: I replayed the identical input and it parsed perfectly.
So it’s not deterministic, and it’s not a length limit either. Roughly, the longer a single object the model has to emit in one go, the more chances it has to slip once, and one slip voids the lot. You can’t parse ninety-nine percent of a JSON document.
This got a three-part fix, cheapest first.
A retry. If the single-shot skeleton comes back unparseable, try once more — a regenerate usually clears it. But skip the retry if the response was truncated or was large and still malformed, because doing the same thing again is unlikely to go better.
A firmer instruction. The shared stage prompt now spells out what valid JSON means in practice: escape inner quotes, no literal newlines inside strings, no trailing commas. I’m not claiming that fixed anything on its own. It’s cheap.
And a fallback that doesn’t depend on the model being perfect across a huge object. First an outline call that returns only the headers for each day — date, location, trip phase, nothing else. Ninety of those came back in about 7,500 tokens. Then the day details get filled in parallel, in chunks of 25 days, and merged back onto the outline:
$outline = skeleton_outline_call( $brief ); // headers only
$chunks = array_chunk( $outline['days'], 25 );
$details = run_in_parallel( $chunks, function ( $days ) use ( $brief ) {
return skeleton_detail_call( $brief, $days ); // prose for these days
} );
$skeleton = merge_day_details( $outline, $details );
Every payload stays small enough that neither truncation nor a single stray character can sink the whole build. And it doesn’t care how long the trip is. That’s the property the original design was missing.
Break three: the same thing, one stage later
With the skeleton sorted, the build got further and fell over at hotels.
Same class of failure. The four section stages (hotels, restaurants, transport, extras) each carry a per-day map that grows with the length of the trip. I measured them on the ninety-day brief: hotels needed about 19,700 output tokens, restaurants about 17,500, extras about 13,400. Their caps were 10,000, 8,000 and 8,000.
Raise the caps: 24k, 22k, 20k, and 12k for transport while I was there.
And the non-deterministic slip showed up here too — hotels failed once at the old cap and parsed on the very next run. The sections already run in parallel, so the fix was a single retry pass that regenerates only the sections that failed, rather than throwing away the three that worked:
$retry = array();
foreach ( $section_keys as $key ) {
$parsed = parse_stage_result( $results[ $key ] );
if ( is_array( $parsed ) ) {
$sections[ $key ] = $parsed;
} else {
log_stage_retry( $key, stop_reason_of( $results[ $key ] ) );
$retry[ $key ] = $requests[ $key ];
}
}
if ( $retry ) {
$results = run_parallel_requests( $retry ); // one more go, failures only
// parse again; anything still broken fails the build
}
Verified on the same brief: all four sections completed, hotels parsed on the retry.
Break four: stuck at 28 of 29
Now the build got all the way to the end and then sat there.
Before the spreadsheet is written, the browser checks each hotel so the workbook can link to a real listing rather than a guessed URL. One request per hotel, five at a time. A two-week trip might have five or six hotels. Ninety days had twenty-nine, and the progress line read “Verifying hotels (28/29)” for as long as anyone was willing to watch it.
One of those requests had hung. The fetch() had no timeout, and the whole batch waits for every worker to finish, so a single slow call held up everything behind it indefinitely.
const PER_HOTEL_MS = 35000;
const PASS_BUDGET_MS = 180000;
const deadline = Date.now() + PASS_BUDGET_MS;
const worker = async () => {
while (idx < jobs.length && Date.now() < deadline) {
const job = jobs[idx++];
const ctl = new AbortController();
const timer = setTimeout(() => ctl.abort(), PER_HOTEL_MS);
try {
const res = await fetch(verifyUrl, { method: 'POST', body: JSON.stringify(job), signal: ctl.signal });
const data = await res.json().catch(() => null);
if (data && !data.error) results.set(job.key, data);
} catch (_) {
// timed out or failed: leave it unresolved
} finally {
clearTimeout(timer);
}
}
};
A per-hotel timeout, and a budget for the whole pass. Anything that doesn’t resolve in time falls back to a search link, which the workbook already handled. Verification was always best-effort. It just hadn’t been bounded.
The same mistake, four times
None of these were really about the model. They were about limits that had been sized, quite sensibly, for the trips people had tested with. A token cap that’s generous for fourteen days. A single JSON object that’s reliable at fourteen days. A verification pass that’s quick with six hotels.
Everything that scales with the input needs either a limit that scales with it, or a design that doesn’t care. Raising the caps was the first kind. Chunking the skeleton was the second, and it’s the one I trust.
The other thing I’d pass on is smaller. If you’re parsing model output and the parse fails, look at why the model stopped before you report anything. “Unparseable” and “ran out of room” need completely different fixes, and one field in the response tells you which one you’ve got.
Every failure now logs its stop reason. Next time somebody asks for six months, I’d like to find out from a log line rather than a spinner.