Argues that the core guarantees workflow engines sell — exactly-once step execution, retries, and crash recovery — reduce to an append-only event log plus idempotent step IDs, both of which SQLite in WAL mode provides natively. Demonstrates this with a ~400-line reference implementation that handles memoization, retries, timeouts, and recovery without a separate runtime, worker pool, or gRPC layer.
Frames the workflow-engine space as having consolidated around a complexity assumption that doesn't hold for most teams. Most internal job runners don't need a distributed coordinator — they need durable steps, which a single SQLite file delivers with dramatically less operational surface area.
Roughly half the thread consisted of engineers with Temporal experience confirming that the SQLite approach captures Temporal's essential mechanism: an event log of completed steps that gets replayed on recovery. Their take is that Temporal's value-add is the distributed coordination and language SDKs, not the durability primitive itself, which is genuinely this simple.
Pushed back that SQLite-as-workflow-engine hits a wall when you need to fan out across 50 workers or run workflows spanning days that require durable timers. At that point you're rebuilding a distributed coordinator on top of SQLite — and you should just adopt the tool that already solved it.
Conceded the boundary honestly: the SQLite approach is bounded by single-machine scale, and once you cross that threshold you are genuinely buying a distributed coordinator. His framing was that teams should make that purchase consciously rather than by default.
A post titled *SQLite is all you need for durable workflows* hit 470 points on Hacker News, pulled from obeli.sk's engineering blog. The thesis is simple and uncomfortable for anyone who just finished a Temporal migration: the durability guarantees that workflow engines sell — exactly-once step execution, automatic retries, replay after crash — reduce to an append-only event log plus idempotent step IDs. SQLite, with WAL mode enabled, gives you both for free in a single file you can `scp` to a backup server.
The author's reference implementation is roughly 400 lines of code and handles workflow execution, step memoization, retries, timeouts, and crash recovery — the full Temporal feature surface that ships in your average internal job runner. The pattern: each workflow run gets a row, each step within it gets an idempotency key, completed steps cache their output, the orchestrator is a plain function you re-invoke after a crash and it skips anything already in the log. No separate worker pool. No gRPC. No "workflow versioning" headache. The whole runtime is `import sqlite3`.
The HN thread — 280+ comments at submission time — split predictably. Half were ex-Temporal engineers nodding along ("this is what Temporal does, minus the YAML"). The other half pushed back on scale: what about fan-out across 50 workers? What about long-running workflows that span days and need durable timers? The author's response was honest: at single-machine scale, this works; past that, you're buying a distributed coordinator and you should know that's what you're buying.
The workflow-engine space has been quietly consolidating around a complexity assumption that doesn't hold for most teams. Temporal, Airflow, Prefect, Inngest, Trigger.dev, Restate — all of them solve the same core problem (durably execute a multi-step process across failures) by introducing a separate runtime, a separate datastore, a separate worker fleet, and a separate mental model. For a team running fewer than ~10,000 workflows per second on a single box, that infrastructure is a tax paid in operational overhead for capacity you will never use.
The interesting part isn't "SQLite is fast" — that's been litigated since Litestream shipped. It's the recognition that durable execution is a database problem, not a runtime problem. Temporal's actual innovation was the event-sourcing pattern (deterministic replay over an immutable log). Everything else — the SDKs, the server, the visibility UI — is packaging. If you have transactional writes and you're disciplined about idempotency keys, the packaging is optional. The Restate team has been making a softer version of this argument for two years; obeli.sk just stated it without the marketing varnish.
The pushback worth taking seriously is around schema evolution and long-lived workflows. Temporal's versioning machinery exists because a workflow that started under code v1 and resumes after a deploy of v2 will deterministically diverge from its replay log unless you explicitly handle the version skew. The blog post hand-waves this — "just don't change step signatures" — which is fine for jobs that complete in minutes and unrealistic for anything that runs for days. If your workflows outlive your deploy cadence, you need versioning primitives the 400-line approach doesn't give you, and bolting them on gets you 80% of the way back to Temporal.
The comments also flagged the observability gap. Temporal's UI — being able to click into a stuck workflow, see which step failed, replay from a specific point — is the feature most people actually pay for, separate from the durability story. SQLite-as-engine means you're writing that dashboard yourself, or living with `SELECT * FROM workflow_steps WHERE status = 'failed'`. For internal tools that's fine. For a payments pipeline where the on-call needs to debug a stuck transfer at 2am, the missing UI becomes the operational cost.
The practitioner read: audit what your workflow engine is actually doing. If you're running Temporal or Airflow and your workflows are short-lived (< 1 hour), single-region, with < 1,000 concurrent runs, and your team writes the same `try/except/retry/log` pattern over and over in workflow code — you're paying for a distributed system to do what a transaction would. The migration cost from Temporal to a SQLite-backed event log is real but bounded: you're trading one well-defined abstraction for another, not rebuilding the world.
Where you still want the heavier engine: cross-region durability, workflows that span deploys, fan-out across heterogeneous worker pools, regulatory audit trails that need a dedicated history service, or any team where the cost of writing your own 400-line orchestrator exceeds the cost of running a Temporal cluster. That's not a small set. But it's also not every team, and the default of "reach for Temporal" has been wrong for the long tail of internal job runners that just need durable retries.
If you're starting fresh, the calculus shifted today. Start with SQLite + an event log + idempotency keys. If you hit a scaling wall — and you'll see it coming, because SQLite's single-writer ceiling is measurable, not mysterious — port to Postgres with the same schema, then to a managed engine if you outgrow that. The upgrade path is linear. The downgrade path from Temporal to "a function and a table" is a rewrite.
The workflow-engine market is about to feel what the queue market felt when Postgres caught up to RabbitMQ for 90% of use cases: a slow leak of greenfield projects choosing the boring option, while the heavyweights consolidate around the genuinely hard problems — cross-region, multi-tenant, regulatory. Expect Temporal to lean harder into the enterprise story (audit, compliance, multi-region replication) and let the long tail go. The 400-line proof-of-concept won't kill the category, but it correctly named what most of the category was selling: a database transaction wrapped in a marketing deck.
The cycle of expertise :- what is X, I just do Y- wow I can see so many limits of Y, now I do X- I use X for literally everything- now that I properly understand the limits of Y but also the heavy constraints of X ... maybe Y is enough- I use Y for literally everythingrinse & repeat. The thing i
I don't understand this obsession with SQLite for real, production apps. SQLite is an embedded database, completely unsuitable for managing concurrency. This is what database _servers_ are for, e.g., Postgres, MySQL, etc. Their entire job is to allow you to modify data from multiple processes,
I've replaced all of these with Go + SQLite:1. Intercom 2. Zendesk 3. Email marketing 4. Kanban 5. Todo 6. Our billing stack 7. Our issue tracker 8. Our forum 9. Uptime monitor 10. PagerDuty (clone)I have dozens of products I sell, so I thought: why not build everything ourselves?All of these r
SQLite is surprisingly performant for single node applications even when comparing to Postgres. Postgres consumes a lot more memory and requires IO to hop through IPC whereas you can keep everything in process in SQLite with a shared connection pool.I've been testing different storage engines f
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.
I started setting up my workflows using Temporal. It deploys as relatively light weight local app. For an isolated local installation it uses SQLite. It makes the process of dealing with API retries and organizing workflows and tasks really simple. I recommend giving it a try. It is, philosophically