Skip to content

A durable job queue in your own Postgres, for OCaml 5 and Eio.

Fewer moving parts: jobs live in the database you already run, commit or roll back with your own changes, and are backed up with your data.

Terminal window
opam repo add hypha https://github.com/hyphatech/opam-repository.git
opam install spool
(* An order's receipt: its kind, the order taken with its receipt owed in
one transaction, and a runner that sends it. quick_start.ml runs it. *)
open Result.Syntax
module Pg = Rowtype_postgres
let send_receipt order =
Printf.printf "Receipt for order %d sent.\n%!" order;
Ok ()
(* {"order": 42} *)
let order =
Wiretype.Object.map Fun.id
|> Wiretype.Object.mem "order" Wiretype.int ~enc:Fun.id
|> Wiretype.Object.finish
let receipt = Spool.Kind.make ~queue:"mail" "receipt.send" order
let insert_order =
Rowtype.find ~params:Rowtype.text ~row:Rowtype.int
"insert into orders (email) values ($1) returning id"
(* The order and its receipt, in one transaction: a rollback leaves neither. *)
let take_order conn ~now email =
Pg.Transaction.within conn @@ fun conn ->
let* order = Pg.run conn insert_order email in
Spool.enqueue conn ~now ~unique:(string_of_int order) receipt order
let runner ~sw env conninfo =
Spool.Runner.create ~sw ~net:(Eio.Stdenv.net env)
~mono_clock:(Eio.Stdenv.mono_clock env)
~now:Ptime_clock.now
~queues:[ ("mail", 2) ]
[ Spool.handle receipt (fun _job order -> send_receipt order) ]
conninfo

In your transaction

A job is a row written with the work that made it owed, so a rollback leaves no job.

Several processes, one queue

Claims that never wait on each other, leases a heartbeat renews, and a lapsed one rescued.

Retries with backoff

A handler retries, retries after what an upstream said, or discards.

Asked twice, one job

A unique key folds a second request into the job already waiting.

Scheduled and prioritised

At an instant, after a span, until an expiry, at a priority from 0 to 9.

Tests that never sleep

Runner.drain runs every job due at the test’s own clock.