Twelve Years Of Distributed SQLite

TL;DR

Buying for a business?Offer from Amazon

Get business pricing on monitors, keyboards and dev gear

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

A retrospective by rqlite creator Philip O’Toole traces distributed SQLite from rqlite’s 2014 open-source release through a growing mix of replication and recovery projects. It describes three distinct approaches—consensus clusters, backup streaming and conflict-free merging—and argues they address different application needs.

Philip O’Toole, the creator of rqlite, has published a retrospective marking twelve years of distributed SQLite, tracing the field from rqlite’s 2014 open-source release to projects that replicate, back up or merge SQLite data. The account matters because it sets out how a once-narrow engineering approach developed into a broader category of tools and hosted database services.

O’Toole says the original rqlite idea was to place a Raft consensus log in front of SQLite and replicate writes across a cluster, so losing one node would not mean losing the data. He released the source on GitHub in September 2014. He describes rqlite as, to his knowledge, the first open-source system to combine Raft and SQLite, while acknowledging other related work existed inside companies and in projects emerging around the same time.

Rqlite’s defining choice was to make it a strong-consistency, or CP, system: writes go through a leader, a majority of nodes must acknowledge them, and a minority partition refuses writes. O’Toole says the project replaced its initial consensus library with HashiCorp’s Raft implementation in early 2016. That design differs from later projects aimed at recovering a single SQLite database, supporting multiple independent writers, or providing replication as a managed service.

The retrospective covers several milestones: Canonical’s dqlite, developed for products including LXD and MicroK8s; Ben Johnson’s Litestream, which streams SQLite’s write-ahead log to object storage for recovery; and later projects including LiteFS, cr-sqlite, libSQL and SQLite Cloud. It also describes Cloudflare D1 as a managed SQLite offering. These products do not all use the same replication method or promise the same consistency behavior.

Three Ways to Protect SQLite Data

The history helps clarify that “distributed SQLite” is not one architecture. A consensus cluster, a streamed backup and a system that merges concurrent changes solve different problems. The practical choice depends on whether an application needs a live replica during a machine failure, inexpensive recovery after a failure, or the ability for multiple writers to work independently.

That distinction matters for developers evaluating database infrastructure. A backup stream can be a simpler fit when downtime for restoration is acceptable; it is not equivalent to a cluster that continues serving writes after a node fails. Conversely, a strongly consistent cluster adds coordination and can refuse writes when a majority is unavailable. Merge-based approaches allow different trade-offs, including eventual consistency, but are designed for a different class of application.

The commercial direction has also changed. O’Toole describes a shift from software users operate themselves toward hosted and platform-integrated services. Managed offerings may lower deployment work, but the retrospective does not compare their reliability, cost or performance. Readers should treat the article as a history and architectural account, not as a comparative benchmark.

From Open-Source Tool to Category

O’Toole says the initial need arose at a company where a clustered system required a small relational store and there was no clear answer about which node should hold it. He continued developing rqlite after the company chose another approach, aiming to make replicated SQLite a general-purpose tool. Its model was a server that users could run alongside their application.

The account also qualifies the claim of being first. Expensify had used a Paxos-style protocol to replicate SQLite internally and released Bedrock in 2016. ActorDB, an Erlang system that gave each actor a SQLite database replicated with Raft, appeared around rqlite’s early period. O’Toole says these examples show the underlying idea was not exclusive to rqlite; his claim is that rqlite made it available as a standalone, general-purpose open-source system.

Later developments broadened the design space. Canonical built dqlite as an embeddable library, rather than a server. Litestream focused on sending database changes to storage for disaster recovery, while cr-sqlite used conflict-free replicated data types to support changes that can be merged. The retrospective says Turso, built around the libSQL fork of SQLite, was acquired by Supabase in 2026. These milestones show the category expanded in both technical design and business model.

Claims the Retrospective Cannot Settle

The account is written by rqlite’s creator and reflects his view of the technology’s history. His description of rqlite as the first open-source Raft-and-SQLite system is explicitly qualified as far as he can tell; the supplied material does not provide an independent survey of projects or archival evidence establishing a definitive first.

The article does not offer common benchmarks comparing the projects’ throughput, failover times, availability, costs or operational burden. Nor does it provide detailed technical assessments of each managed service’s replication behavior. Its descriptions establish that approaches differ, but do not show which performs best for a particular workload. The material also does not state the terms or financial details of Turso’s acquisition by Supabase.

Choosing a Model for the Workload

The retrospective does not announce a new rqlite release or set out a future product timetable. Its immediate contribution is a historical account and a framework for understanding the choices now available. Developers considering these tools will need to check each project’s current documentation for supported versions, consistency guarantees, failure behavior and deployment requirements.

For readers, the key next step is to define the failure the database must handle: continued service during a node outage, restoration after data loss, or writes from multiple locations. Those requirements determine which approach is relevant. The retrospective leaves open how the category will develop, including whether managed services will make their underlying replication guarantees more transparent and how the different models will perform as applications scale.

Key Questions

What is distributed SQLite?

It refers to approaches that extend SQLite with replication, recovery or multi-writer coordination. The projects covered in O’Toole’s retrospective use different methods, so the term does not describe one shared architecture.

How does rqlite replicate SQLite?

Rqlite uses Raft consensus. Writes pass through a leader and require acknowledgment from a majority of the cluster; a minority partition does not accept writes.

How is Litestream different from rqlite?

Litestream streams SQLite’s write-ahead log to storage to support recovery of a single-node database. Rqlite is designed to replicate writes across a cluster, addressing the case where an application cannot wait to restore after a machine failure.

Does the retrospective prove rqlite was the first project of its kind?

No. O’Toole says that, as far as he can tell, rqlite was the first open-source system combining Raft and SQLite. He also discusses earlier internal work at Expensify and the contemporaneous ActorDB project, and the supplied account is not an independent historical audit.

Which distributed SQLite approach should an application use?

That depends on the required behavior. A consensus cluster, streamed backup and merge-based system address different needs; teams should compare their consistency guarantees, recovery expectations and operational requirements against the application’s workload.

Source: hn