Four calls instead of one, running side by side instead of end to end, and the wall clock went up. Not by a little.
That took a while to admit out loud, so let me get the whole shape of it down before I start defending myself.
Where this comes from
A few weeks back I wrote about a long-running generation and the timeout that kept killing it. The fix was to stop measuring elapsed time and start measuring silence — no total cap, a low-speed watchdog instead. That worked, and I still think it’s right.
But it was a fix at the wrong altitude, and I knew it while I was typing it.
The plugin is an advisor tool inside a travel site. An agent answers a dozen guided questions, the model builds a full itinerary — days, hotels, restaurants, transport, budget, a checklist — and the server turns that into a spreadsheet they can send a client. The generation takes minutes. Legitimately.
So I sat down and wrote out every hack in the file that existed only because of that. It’s an unflattering list when you see it together:
| What | What it’s compensating for |
|---|---|
@set_time_limit(900) | PHP killing a healthy build |
CURLOPT_TIMEOUT => 0 plus a low-speed watchdog | a fixed timeout cutting a live stream |
| blocking fallback raised 180 → 600 | three timeouts in a row, then a 502 |
no-gzip, X-Accel-Buffering, no-transform | hosts that gzip text/event-stream |
| a whole self-test endpoint and admin notice | detecting the above, per host |
| a forced client-side re-prompt | build turns that came back as the wrong phase |
Every one of those is a sensible response to a real symptom. Together they’re a diagnosis: one HTTP request is one unit of work, and that unit had quietly grown from “a conversational reply” into “several minutes of generation with live web searches in the middle”. Everything that hurt followed from holding a request-response cycle open past what the web stack around it was built to hold.
The plugin ships to client sites on hosting I don’t control. A build behind Cloudflare meets the edge timeout at about a hundred seconds and there is no apache_setenv() that saves you from that.
Getting it off the request
The obvious move. POST returns a job id immediately, a background worker does the build and writes its progress to a table, the browser polls.
A custom table rather than a post type — the rows are working state with a TTL and none of it needs WP_Query. Status, stage, a slot for each section’s output, an attempt counter, an expiry. A daily cron sweeps the dead ones.
Three ways to kick the worker off, tried in order, detected at runtime:
- Action Scheduler, if the site has it. It’s on anything with WooCommerce and a fair few others. Own runner, own retry semantics, an admin UI you get for free.
- A non-blocking loopback POST to
admin-ajax.php. The originating request returns instantly; the spawned one does the work. wp_schedule_single_event()as a last resort, which fires on the next page view and therefore not at all on a quiet site.
I picked Action Scheduler as the preferred path without much agonising. It’s the mature option. It has retries. Somebody else maintains it. If you’d asked me to justify the ranking I’d have said reliability, and I’d have thought that was the end of the question.
That shipped, and it was better. No multi-minute PHP request in front of anyone, builds survived a closed tab, the diagnostics subsystem went from load-bearing to nice-to-have.
Then I split the build up
The worker was still making one enormous call. Twenty-odd thousand tokens of output in a single envelope, against a 32k cap — a single point of failure with no partial credit, and a wall clock equal to the sum of everything the model had to write.
So: a stage machine. Each invocation of the worker runs exactly one stage, writes its output, and re-enqueues itself for the next one.
private const STAGES = ['skeleton', 'sections', 'assemble']; /** Progress % stamped when each stage STARTS (assemble bumps to 100 on done). */ private const STAGE_PROGRESS = ['skeleton' => 15, 'sections' => 45, 'assemble' => 90]; /** The parallel passes inside the `sections` stage. */ private const SECTION_KEYS = ['hotels', 'restaurants', 'transport', 'extras'];
Skeleton is one cheap call — the shape of the trip, the day stubs everything else aligns to. Then four bounded calls in parallel, each with its own prompt, its own token cap, its own retry. Then a deterministic merge.
The parallel part is curl_multi, not the WordPress HTTP API, which has no concurrent primitive. That’s fine — the streaming path was already raw cURL, so there was nothing new to vendor, which matters because this machine doesn’t have Composer on it.
for ($round = 1; $round <= self::MAX_ATTEMPTS && $pending; $round++) {
$mh = curl_multi_init();
// add a handle per pending section, drive the multi stack to completion,
// then collect. Only the retriable failures go into the next round, so a
// single flaky section never re-runs the ones that already succeeded.
}
The property I actually wanted is the one that has nothing to do with speed: no single PHP process spans more than one stage. A host with a hard max_execution_time you can’t raise stops being the same class of problem, because nothing you run is anywhere near as long. The sections stage is still the fat one — a couple of minutes, five in the worst case — so this isn’t a licence to stop caring about execution caps. It’s the difference between one number you have to argue with and four.
Speed was supposed to be the bonus. Sum of the sections becomes the slowest section.
The measurement
It was slower than the monolith. On the quiet test site, meaningfully slower.
I spent an embarrassing amount of time on the wrong suspect. Checked the section prompts for accidental bloat. Checked whether curl_multi was actually running concurrently or whether I’d built an expensive sequential loop by mistake. Logged per-call latency. The calls were fine. The calls were faster, exactly as advertised.
The time wasn’t in the work. It was in the gaps between the work.
Action Scheduler, on a site with no traffic, runs its queue off WP-cron — and WP-cron on a site with no traffic runs when somebody loads a page. Nobody was loading pages. It was a dev box with one person on it, and that person was sitting watching a progress banner rather than browsing.
So each hop waited for the runner. Around a minute, give or take. Three dispatches — one to start, two between stages — and I’d bought roughly three minutes of queueing to save maybe ninety seconds of generation.
The monolith had exactly one of those gaps. That’s the whole story.
Loopback first
The fix is a reordering, which is the least satisfying kind of fix and often the right one.
public static function dispatch(int $id): void {
// 1. Immediate loopback - the fast path.
wp_remote_post(admin_url('admin-ajax.php'), [
'blocking' => false,
'timeout' => 0.01,
'sslverify' => false, // loopback to self
'body' => [
'action' => self::AJAX_ACTION,
'job' => $id,
'token' => self::token($id),
],
]);
// 2. Backstop in case the loopback never ran.
if (function_exists('as_enqueue_async_action')) {
as_enqueue_async_action(self::AS_HOOK, [$id], 'jobs');
} elseif (!wp_next_scheduled(self::CRON_SINGLE, [$id])) {
wp_schedule_single_event(time() + 60, self::CRON_SINGLE, [$id]);
}
}
Fire the loopback immediately, every time, and arm the reliable-but-slow path behind it. Where loopback works — most places — the stage starts within a second. Where it’s firewalled or the request gets dropped, the backstop picks it up about a minute later and nobody but me notices.
Both of them firing is now the normal case rather than an edge case, and that’s only survivable because of one line already sitting there:
public static function claim(int $id): bool {
global $wpdb;
$n = $wpdb->query($wpdb->prepare(
"UPDATE $table SET status = 'running', attempts = attempts + 1, updated_at = %s
WHERE id = %d AND status = 'queued'",
self::now(), $id
));
return $n === 1;
}
Claiming a job is a conditional update, and whether you got it is rows_affected === 1. Whichever dispatcher reaches the row second finds it isn’t queued any more and quietly does nothing. I wrote that guard for double-dispatch as the thing that might go wrong — two runners colliding, a sweeper racing a retry. It turned out to be what let me make double-dispatch the design.
The HMAC on the loopback body is doing the other half of the work — admin-ajax.php is public, so the handler needs to know the request came from us and not from someone who read this blog post.
One more thing fell out of it. The stall sweeper — the cron that finds jobs stuck in running and re-dispatches them — had a threshold of fifteen minutes, set when a stalled job was indistinguishable from a monolith that was slow but alive. With short stages and a heartbeat at every hop, stalled now means stalled, so that came down to eight. Not zero, because a re-dispatched sections stage means four more model calls you’re paying for, and I’d rather wait than pay twice.
What I’d take from it
When you move work out of the request lifecycle you do not get the time back. You trade a problem you can see — a request that hangs, a 502, a timeout with a stack trace — for one you can’t, which is latency spread thin across handoffs that don’t log anything on their own.
Two things I’d do differently, and one I’d keep.
Rank your dispatchers on the axis that matters for this workload. I ranked on reliability because that’s the obvious virtue of a mature queue library, and reliability genuinely is Action Scheduler’s virtue. It just wasn’t the constraint. Something waiting at the other end changes the question from “will this run” to “when”, and I never asked the second one.
Count the hops, then multiply. Three dispatches at a minute each is three minutes, and three minutes was more than the entire saving I’d gone in there to get. That arithmetic takes ten seconds and I did it afterwards.
The thing I’d keep is the atomic claim. It cost one WHERE clause and it’s the reason the fix was a reordering rather than a rewrite — I could add a second dispatcher without thinking hard about what happens when both land, because the answer was already written down. Cheap idempotency early buys you options later, usually ones you didn’t have in mind at the time.
And the stage machine stands on its own merits regardless. Even if it had come out exactly even on wall clock, the stages are short enough to stop fighting execution caps, and a section that comes back 429 or 503 retries on its own without re-running the three that already landed. It’s not partial credit — a section that won’t parse still fails the whole job, deliberately, because assembling around a hole is worse than failing loudly. What you get is a cheap isolated retry instead of an expensive total one. I just would have liked to find that out from a graph rather than from a week of being confidently wrong.