pg_net on Supabase under load: timeouts, batches and delivery, measured
How pg_net, the extension behind Database Webhooks, behaves on a hosted Supabase project when the receiver is slow, the database is busy or the endpoint fails some of the time. It is for anyone sending HTTP from a trigger or a scheduled job who needs to know what a timed_out row means, how long a queued call can wait, and how many messages arrive. The architecture around it, a trigger posting to a Worker, is in Cloudflare Workers + Supabase; the egress risk of pg_net is in Locking down Supabase.
All measured rows come from the lab’s pgnet-under-load experiment, run 2026-10-26 on throwaway projects on a Pro-plan organisation in ap-southeast-1, each created through POST /v1/projects and deleted by NL99: NL01 and NL02 on one Micro; NL04 run 1 on a first Large (where an NL03 attempt aborted in setup with 22003: integer out of range), NL03 on a second, NL04 run 2 on a third, NL05 on a fourth. NL03 (00:30:17 to 00:55:24 UTC) ran alongside both NL04 runs (00:28:01 to 00:56:26 UTC between them), against the same receiver. Postgres 17.11 and pg_net 0.20.4 were recorded on the Micro project and on NL03’s Large; the NL04 and NL05 projects ran the same code and did not record them (NL04 recorded pgmq 1.5.1). The Large projects’ metrics endpoint listed 2 CPUs. The receiver was a Cloudflare Worker on workers.dev that writes every arrival to a Durable Object ledger before it answers, so a call the sender gave up on still leaves a record. Receiver times are the Worker’s clock at request entry minus the enqueue time the database wrote into the body, so they include the offset between the two hosts’ clocks (not separated). Each module ran once except NL04, which ran twice. The figures are quoted from the RUNLOG and the artifacts listed in Modules. Rows marked documented come from the pages cited at the claim and were not run.
TL;DR:
timeout_millisecondsdefaults to 5000 in the hostednet.http_postsignature, and calls with the argument omitted timed out at 5000 ms. The Supabasepg_netguide showsdefault 2000.pg_net.max_timeout_ms, which the upstream README documents, does not exist at 0.20.4 (42704: unrecognized configuration parameter); 600001 and 0 were accepted at enqueue, and 0 meant no timeout (a 90 s call answered 200).- The worker takes up to 200 queued requests and starts the next batch only when the whole batch is done. One 4.5 s call delayed the 200 calls behind it to 6010 to 6065 ms after enqueue; one call timing out at 30 s delayed them to 31031 to 31083 ms. None of the delayed calls timed out at 5000 ms: the timeout clock starts at send, not at enqueue.
- A burst of 200 calls sent after the
pg_networker had been quiet (the project’s first outbound calls, or 120 s idle) had 89 and 139 of 200 time out at the 5000 ms timeout although every one had reached the receiver, within 825 ms of enqueue. The two bursts sent straight after burst 1 had none. Atimed_outrow is “maybe delivered”. - Saturating both CPUs of a Large project with 4 spinning backends caused no timeout: 0 of 3009 calls across a Worker, a trivial Edge Function and an Edge Function querying the same database. The one timeout in that run came, with the database idle, from the function that queries the same database. Cold starts happened on most invocations but did not cause timeouts.
- The
error_msgof a timeout splits the time into DNS, TCP/TLS and request phases, and the split does not say where the time went: eight timeouts of calls that had reached the receiver put the whole time under “DNS time” (about 5001 ms for the six at 5000, about 30002 ms for the two at 30000). - Of 1,000 messages to a receiver that refused 20 % with 503 and held 10 % for 8 s, raw
pg_netlost 194 to 274 per arm: every message the receiver refused, plus 35 calls of one cold first burst that never arrived. Apg_cronretry job and apgmqqueue with apg_cronconsumer lost 0 in all eight arms and processed 11 to 14 % of messages twice or more. CPU saturation changed no delivery count beyond the receiver’s random fault mix.
Topology
Section titled “Topology”Text version: a trigger, function or scheduled job calls net.http_post, which writes a row to net.http_request_queue; the pg_net background worker takes up to pg_net.batch_size (200) rows, sends them, and takes the next batch only when every call in the current one has finished or timed out; each call’s timeout counts from when the worker sends it; the outcome lands in net._http_response, whose created column is the moment the worker took the batch.
Which do I pick
Section titled “Which do I pick”| What the receiver must get | Mechanism | What it cost here (NL04, 1,000 messages, two runs) |
|---|---|---|
| Notifications where a lost one is acceptable | raw net.http_post | lost 194 to 274 per arm: every message the receiver refused (194 to 239), plus in one arm 35 calls that never arrived; none resent |
| Every message, receiver idempotent | pg_net plus a pg_cron job that re-enqueues on 5xx or timeout | 0 lost; 11.8 to 14.4 % processed twice or more (derived from 118 to 144 of 1,000); all confirmed in 73 to 103 s |
| Every message, retries spaced by a visibility timeout | pgmq queue plus a pg_cron consumer, ack on 2xx | 0 lost; 11.1 to 14 % processed twice or more (derived from 111 to 140 of 1,000); about 186 to 187 s to confirm all, or 1 to 2 left at a 180 s deadline |
Both at-least-once paths duplicate for the same reason: a call that times out at the sender may already have been processed, so the resend is a second delivery. A receiver of either has to be idempotent.
Settings and the timeout (NL01)
Section titled “Settings and the timeout (NL01)”NL01 read the hosted settings on a Micro project and then sent 28 calls to a receiver that answers after 4, 8, 30 or 90 s, crossed with timeout_milliseconds 5000, 30000, 120000, 600000, 600001, 0 and omitted.
| Read | Value |
|---|---|
pg_net% settings | pg_net.batch_size=200, pg_net.ttl=6 hours, pg_net.database_name=postgres, pg_net.username=postgres, nothing else |
show pg_net.max_timeout_ms | 42704: unrecognized configuration parameter "pg_net.max_timeout_ms" |
net.http_post signature, timeout_milliseconds default | 5000 |
| Calls | Outcome |
|---|---|
| timeout above the receiver’s sleep | 200, including 30 s and 90 s calls at 120000 and above |
| timeout at or below the sleep, other than 0 | timed_out |
| timeout omitted | as 5000 |
| timeout 0 | no timeout: 200 after a 90 s sleep |
| timeout 600001 | accepted at enqueue, completed a 90 s call |
None of the 28 was refused at enqueue and all 28 arrived at the receiver. The Supabase guide shows timeout_milliseconds int default 2000 for http_get, http_post and http_delete,1 which the hosted signature and the omitted-timeout calls contradict. The upstream README documents pg_net.max_timeout_ms with a default of 600000 and says a request outside 1..pg_net.max_timeout_ms is not sent;2 at 0.20.4 the setting does not exist, and 0 and 600001 were sent. Nothing in the run lasted 600 s, so whether any cap applies above 600000 was not measured.
The guide also says the extension “is configured to reliably execute up to 200 requests per second”;1 no module measured a rate.
What net._http_response.created holds
Section titled “What net._http_response.created holds”created is when the worker took the request’s batch, not when the response arrived. In NL02c call 1 timed out after 30 s and its row’s created was 48 ms after enqueue. In NL01 the 28 calls were enqueued one statement at a time. The first call’s row was created 7 ms after its enqueue; for the other 27, created minus enqueue falls from 4500 to 1607 ms in enqueue order, consistent with one pickup about 4.5 s after the first enqueue, after that first call (a 4 s sleep) finished, and with the enqueues spread over 2893 ms (per-call enqueue times are not published). Read that way, the worker took call 1 alone, then the other 27 as one batch. Response latency cannot be computed from created.
The error_msg phase split
Section titled “The error_msg phase split”A timeout’s error_msg has this shape:
Timeout of 5000 ms reached. Total time: 5001.621000 ms (DNS time: 5001.621000 ms, TCP/SSL handshake time: 0.000000 ms, HTTP Request/Response time: 0.000000 ms)| Where | Calls | Split |
|---|---|---|
| NL01 | 8 timeouts | the whole time under “DNS time” (6 at about 5001 ms, 2 at about 30002 ms), although every one of those requests had reached the receiver, so name resolution, TCP and TLS had completed |
| NL02c | the 30 s timeout | DNS time: 0.294000 ms, TCP/SSL handshake time: -0.294000 ms, HTTP Request/Response time: 30000.796000 ms |
| NL03 | 1 timeout | all 5000.566 ms under “DNS time”; whether that call reached the receiver is not recorded |
| NL05 | one sampled timeout per cold burst | DNS time: 35.321000 ms, TCP/SSL handshake time: 112.676000 ms, HTTP Request/Response time: 4888.822000 ms (burst 1) and 15.926000 / 235.276000 / 4769.802000 ms (burst 4), for requests that had equally arrived |
The split is inconsistent across runs and includes a negative handshake time, so it does not locate a timeout. Locate one from the receiver’s arrival log or from timing. Why the figures come out this way was not investigated.
Batches and head-of-line blocking (NL02)
Section titled “Batches and head-of-line blocking (NL02)”NL02 enqueued 400 calls in one transaction on the Micro project. Call 1 went to a fast receiver (control), a 4.5 s receiver, or a 60 s receiver with timeout_milliseconds 30000; calls 2 to 400 went to a fast receiver with the default 5000 ms timeout. Arrival delay is the receiver’s arrival minus the enqueue time.
| Part | Call 1 | Calls 2-200 arrival p50 / max | Calls 201-400 arrival min / p50 / max | Gap after the last of 2-200 | timed_out 2-200 / 201-400 |
|---|---|---|---|---|---|
| NL02a control | fast, HTTP 200 | 134 / 374 ms | 2132 / 2151 / 2187 ms | 1775 ms | 0 / 0 |
| NL02b slow | 4.5 s receiver, HTTP 200 | 67 / 126 ms | 6010 / 6031 / 6065 ms | 5904 ms | 0 / 0 |
| NL02c hang | 60 s receiver, timeout 30000, timed_out | 65 / 96 ms | 31031 / 31048 / 31083 ms | 30957 ms | 0 / 0 |
- Calls 201 to 400 waited for the whole first batch, so one slow call held the next 200 until it finished: they arrived 6010 to 6065 ms after enqueue behind a 4.5 s receiver and 31031 to 31083 ms behind a 30 s timeout, against 2132 to 2187 ms in the control.
- None of them timed out at 5000 ms after waiting about 6 s and about 31 s in the queue (0 of 200 in both parts): the timeout clock starts when the call is sent, not when it is enqueued.
- Even with no slow call (NL02a) the second batch started 1775 ms after the last arrival of the first, 2132 ms after enqueue at the earliest. A wait between batches in the worker loop would produce it; the run does not separate that.
NL04 showed the same pacing at the receiver: with 10 % of each 200-call batch held past the 5 s timeout, each batch of the raw arm arrived 6 s after the previous one (0, 6, 12, 18 and 24 s; ledger read by hand, not in the published artifact).
Under CPU saturation: who times out (NL03)
Section titled “Under CPU saturation: who times out (NL03)”NL03 ran on a Large project (2 CPUs). Per arm, a pg_cron job every 10 s enqueued 17 calls to each of three targets with timeout_milliseconds 5000, for 600 s: the Worker, a trivial Edge Function (ef_echo), and an Edge Function that runs one query (nl_work(), a count over 2M rows, 181.361 ms alone by EXPLAIN ANALYZE) through the Data API (ef_query). The saturated arm added 4 session-pooler backends spinning a PL/pgSQL loop, all 4 seen active. CPU busy from the metrics endpoint: 0.162 over the idle arm (presumably the ef_query load; not separated), 0.954 over the saturated arm, 1 in nine of its ten 60 s samples.
| Arm | Target | Calls | 2xx | timed_out | Enqueue to arrival p50 / p95 / max | Query inside the function p50 / p95 / max |
|---|---|---|---|---|---|---|
| idle | Worker | 1020 | 1020 | 0 | 8 / 20 / 363 ms | - |
| idle | ef_echo | 1020 | 1020 | 0 | 104 / 146 / 2579 ms | - |
| idle | ef_query | 1020 | 1019 | 1 | 104 / 151 / 1387 ms | 1326 / 1397 / 3870 ms |
| saturated | Worker | 1003 | 1003 | 0 | 19 / 31 / 146 ms | - |
| saturated | ef_echo | 1003 | 1003 | 0 | 109 / 136 / 2551 ms | - |
| saturated | ef_query | 1003 | 1003 | 0 | 110 / 139 / 2623 ms | 1692 / 1790 / 4482 ms |
- The worker was not starved: 0 of 3009 calls timed out in the saturated arm. Pickup (
createdminus enqueue) went from p50 4 ms, max 14 ms (Worker target, idle) to p50 14 ms, max 42 ms, and the Worker’s arrival p50 from 8 to 19 ms. Thepg_cronjob ran 60 times idle (average 14 ms, max 16 ms) and 59 times saturated (average 41 ms, max 53 ms). - The one timeout, with the database idle, came from the target that queries the same database. The query inside the function took 1326 ms at p50 and 3870 ms at most, against 181 ms alone by
EXPLAIN ANALYZE; saturation moved that to 1692 ms and 4482 ms, still under 5 s. 17 calls arriving together every 10 s on 2 CPUs probably accounts for most of the gap (inferred: no arm isolated concurrency, and the function’s figure also includes the Data API round trip). The function answered the other 1019 calls of that arm. - Cold starts happened but did not cause timeouts:
function_logscounted a Boot for most invocations (865 and 858 of 1020 idle, 833 and 839 of 1003 saturated), yet arrival stayed at p95 136 to 151 ms and at most 2623 ms, well under 5 s. Boots were counted per function over each arm’s window plus 30 s; whether some belong to a neighbouring window was not checked. - The worker’s own
pg_stat_activityrow, sampled every 1 s, read idle onExtension/Extensionin 402 samples (idle arm) and 397 (saturated), and active in 214 and 223 (derived: active onExtension/Extension119 and 104, active with no wait event 95 and 118, and oneLWLock/ProcArraywhen saturated). The largest queue seen was 51 in both arms.
This rules out worker-side starvation at 4 spinning backends on 2 CPUs and nothing heavier: more backends, I/O or memory pressure, lock waits, other sizes, Micro’s shared CPU and a target slower than 5 s on a cold boot were not run.
A cold burst times out delivered calls (NL05)
Section titled “A cold burst times out delivered calls (NL05)”NL04’s first run left an excess of sender-side timeouts in the idle raw arm (240 against 81 held requests, and 35 calls that never arrived), and its second run repeated the excess with every call arriving (277 timeouts against 97 held). Both were the project’s first outbound burst. NL05 tested that on a fresh Large: four bursts of 200 calls to the receiver’s /echo with the default 5000 ms timeout, each in one transaction after the queue drained, each to a fresh ledger.
| Burst | HTTP 200 | timed_out (of them arrived at the receiver) | Never arrived | Arrival p50 / p95 / max |
|---|---|---|---|---|
| 1, the project’s first outbound calls | 111 | 89 (89) | 0 | 484 / 781 / 816 ms |
| 2, at once after 1 | 200 | 0 | 0 | 55 / 85 / 92 ms |
| 3, at once after 2 | 200 | 0 | 0 | 44 / 69 / 187 ms |
| 4, after 120 s idle | 61 | 139 (139) | 0 | 493 / 785 / 825 ms |
- A burst of 200 after the
pg_networker had sent nothing for a while had 89 and 139 calls time out at 5 s; bursts sent straight after a previous one had none. A quiet period alone triggered it: burst 4, on a project that had already sent 600 calls, had 139 time out. - Every timed-out call had reached the receiver within 825 ms of enqueue, and
/echoanswers at once after its ledger write. The slower cold arrivals (p50 484 and 493 ms against 55 and 44 ms warm) fit 200 new TLS connections with none reused (reasoned, not measured). - From the lab Mac in Singapore the same receiver answered 200 concurrent
POSTs to/echoon a fresh ledger in under 1 s, after about 150 s idle (time_totalp50 / p95 / max 0.079 / 0.557 / 0.898 s, then 0.522 / 0.890 / 0.986 s; curl from the lab Mac, not in the published artifact). The receiver does not reproduce the stall from that vantage, which points at the database host’s side, the path between it and Cloudflare, or a receiver stall specific to the colo serving the database; those three were not separated, and neither was whether a smaller first batch or a longer first timeout avoids it. n = 1 run, two cold bursts.
Delivery of 1,000 messages (NL04)
Section titled “Delivery of 1,000 messages (NL04)”The receiver’s /flaky route answered 20 % of requests 503, held 10 % for 8 s then answered 200, and answered the rest 200 at once; every call used timeout_milliseconds 5000. Three mechanisms, each with the database idle and then saturated (CPU busy 0.999, 4 backends): raw net.http_post per message (deadline 90 s), pg_net plus a pg_cron job that re-enqueues on 5xx or timeout, and a pgmq 1.5.1 queue with a pg_cron consumer, visibility timeout 30 s, ack on 2xx (deadlines 180 s). Delivered means the receiver recorded a 200 or a held request for the message at least once; a held request whose sender timed out still counts as processed.
Run 1:
| Arm | Mechanism | Delivered | Lost | Processed twice or more (extra deliveries) | Receiver arrivals (503 / held) | Sender: calls, confirmed, timed_out | All confirmed |
|---|---|---|---|---|---|---|---|
| idle | raw | 726 | 274 | 0 (0) | 965 (239 / 81) | 1000, 555, 240 | no (90 s) |
| idle | retry | 1000 | 0 | 144 (168) | 1468 (300 / 168) | 1468, 1000, 168 | after 103 s |
| idle | pgmq | 1000 | 0 | 111 (125) | 1414 (289 / 126) | 1414, 999 (1 left in queue), 126 | no (180 s) |
| saturated | raw | 806 | 194 | 0 (0) | 1000 (194 / 104) | 1000, 702, 104 | no (90 s) |
| saturated | retry | 1000 | 0 | 133 (152) | 1444 (292 / 152) | 1444, 1000, 152 | after 73 s |
| saturated | pgmq | 1000 | 0 | 122 (142) | 1450 (308 / 144) | 1450, 998 (2 left in queue), 144 | no (180 s) |
Run 2, a third fresh Large:
| Arm | Mechanism | Delivered | Lost | Processed twice or more (extra deliveries) | Receiver arrivals (503 / held) | Sender: calls, confirmed, timed_out | All confirmed |
|---|---|---|---|---|---|---|---|
| idle | raw | 797 | 203 | 0 (0) | 1000 (203 / 97) | 1000, 566, 277 | no (90 s) |
| idle | retry | 1000 | 0 | 118 (137) | 1425 (288 / 137) | 1425, 1000, 137 | after 93 s |
| idle | pgmq | 1000 | 0 | 125 (139) | 1384 (245 / 139) | 1384, 1000 (0 left in queue), 139 | after 187 s |
| saturated | raw | 792 | 208 | 0 (0) | 1000 (208 / 113) | 1000, 679, 113 | no (90 s) |
| saturated | retry | 1000 | 0 | 127 (144) | 1405 (261 / 144) | 1405, 1000, 144 | after 73 s |
| saturated | pgmq | 1000 | 0 | 140 (158) | 1452 (294 / 158) | 1452, 1000 (0 left in queue), 158 | after 186 s |
- Raw
pg_netlost every message the receiver refused: lost equals the receiver’s 503 count in the run 2 arms, and in run 1’s saturated arm. Run 1’s idle raw arm also had 35 calls that never arrived, all in the first batch of 200 (ledger read by hand, not in the published artifact); NL05’s cold bursts had none that never arrived, so the cold-burst effect above does not explain these 35. Measured from the sender, raw also counted the held-then-answered requests as failures: it confirmed 555, 702, 566 and 679 where the receiver processed 726, 806, 797 and 792. - The retry job and
pgmqdelivered all 1,000 in every arm, at a duplicate cost: each held request that timed out at 5 s had been processed and was sent again. Extra deliveries equalled held requests in every retry row, and in run 2’spgmqrows; run 1’spgmqrows had 125 of 126 and 142 of 144; the remaining 1 and 2 were still in the queue at the deadline, already delivered and waiting out their 30 s visibility timeout. - CPU saturation changed no delivery count beyond the receiver’s own random mix. The retry arm confirmed sooner saturated (73 s, both runs) than idle (103 s and 93 s); the random fault draw is as likely a cause as anything, not separated at n = 2.
- The idle raw arm was each project’s first burst, and its sender-side timeouts fit the cold-burst effect (240 for 81 held, 277 for 97; NL05 ran on another project, so this is inferred); its receiver-side delivered and lost counts do not depend on it. The saturated raw arms, which followed other traffic, matched exactly (104 and 113 timeouts against 104 and 113 held).
Docs against runtime
Section titled “Docs against runtime”| Claim | Documented | Measured | Filed upstream |
|---|---|---|---|
timeout_milliseconds default | int default 2000 in the Supabase guide1 | 5000 in the hosted signature; omitted-timeout calls timed out at 5000 ms (NL01) | not filed; the lab RUNLOG records no report |
| A timeout ceiling | pg_net.max_timeout_ms, default 600000, requests outside 1..max not sent (upstream README at HEAD)2 | the setting does not exist at 0.20.4; 0 and 600001 were sent, 0 as no timeout (NL01) | not filed; no release README through v0.20.5 documents it, only HEAD (ddd4485), so this is version drift |
| Throughput | “configured to reliably execute up to 200 requests per second”1 | not measured | - |
Reading the numbers
Section titled “Reading the numbers”| Claim | Measured or documented | How it was checked |
|---|---|---|
Default 5000 ms, no max_timeout_ms, 0 = no timeout | measured, one run (NL01), Micro | show, the function signature, 28 calls against 4 to 90 s sleeps |
| Batches of 200 block on their slowest call; timeout counts from send | measured, one run per part (NL02), Micro | receiver arrival minus enqueue for 400 calls per part |
created is batch pickup | measured (NL01, NL02c) | created against enqueue and the known outcome |
error_msg phase split does not locate a timeout | measured across NL01, NL02c, NL03, NL05 | the split against receiver arrivals |
| CPU saturation caused no timeout | measured, one run (NL03), Large, 4 spinning backends | 600 s per arm, three targets, metrics endpoint CPU, worker pg_stat_activity |
| Cold burst times out delivered calls | measured, one run, two cold and two warm bursts (NL05), Large | ledger arrivals against timed_out; receiver control from the lab Mac |
| Delivery counts per mechanism | measured, two runs (NL04), Large | receiver ledger per message id against the sender’s rows |
| 200 requests per second | documented, not run | the Supabase guide |
Every figure comes from one region, a Pro organisation, Micro (NL01, NL02) or Large (NL03 to NL05), one receiver on Cloudflare, and a database clock against Cloudflare’s for every arrival delay. n = 1 per module except NL04.
What to do about it
Section titled “What to do about it”| Practice | Evidence | Module |
|---|---|---|
Pass timeout_milliseconds explicitly on every call. | Omitted means 5000 ms on 0.20.4; the Supabase guide shows 2000. | NL01 |
| Do not rely on a server-side timeout ceiling; set a finite timeout yourself. | pg_net.max_timeout_ms is absent at 0.20.4; 0 was accepted and meant no timeout. | NL01 |
| Keep slow endpoints off the queue that carries fast ones. | One 4.5 s call delayed the next 200 calls to 6010 to 6065 ms; one 30 s timeout to 31031 to 31083 ms. | NL02 |
| Size timeouts for the call, not for its time in the queue. | Calls queued 6 s and 31 s behind a slow batch did not time out at 5000 ms: the clock starts at send. | NL02 |
Treat timed_out as “maybe delivered” and make receivers idempotent. | Cold bursts: 89 and 139 of 200 timed out, every one already at the receiver within 825 ms. | NL05 |
| Expect timeouts on the first burst after a quiet period. | 200 calls after 120 s idle had 139 time out; bursts 2 and 3, sent straight after burst 1, had 0. Smaller batches or longer first timeouts were not tested. | NL05 |
Locate a timeout from the receiver’s log, not from error_msg. | The whole time (about 5001 ms, or 30002 ms at a 30000 timeout) reported as “DNS time” for calls that had arrived; a negative handshake time in NL02c. | NL01, NL02c, NL05 |
Do not compute response latency from net._http_response.created. | A call that timed out after 30 s had created 48 ms after enqueue. | NL01, NL02c |
Add a retry job or pgmq when every message must arrive. | Raw lost 194 to 274 of 1,000 per arm; retry and pgmq lost 0 in all 8 arms. | NL04 |
| Budget for 11 to 14 % of messages arriving twice or more on an at-least-once path. | 111 to 144 of 1,000 messages processed twice or more; extra deliveries equalled held-then-timed-out requests: 137 to 168 per retry arm. | NL04 |
Look past CPU for pg_net timeouts: a receiver waiting on the same database first. | 0 of 3009 timeouts with both CPUs saturated; the one timeout was the function querying the same database. | NL03 |
Decision guide
Section titled “Decision guide”Text version:
- If a lost message is acceptable, use raw
net.http_postwith an explicittimeout_milliseconds. - If every message must arrive, add a
pg_cronjob that re-enqueues on 5xx or timeout, or usepgmqwith apg_cronconsumer when retries should wait out a visibility timeout. Either way, make the receiver idempotent: 11 to 14 % of messages arrived twice or more. - If
timed_outrows appear, check three causes in the receiver’s own log before blaming load: a first burst after a quiet period (calls arrived, then timed out), a slow call holding its whole batch, or a receiver whose work queries the same database.
Not measured
Section titled “Not measured”- Load from
pg_netitself: throughput, the documented 200 requests per second, and apg_net.batch_sizeother than 200. - Heavier database pressure than 4 spinning backends on 2 CPUs: I/O, memory, lock waits; Micro’s shared CPU under saturation; other compute sizes for NL03 to NL05.
- A
timeout_millisecondscap above 600000 (nothing ran 600 s), andpg_netversions other than 0.20.4. - The cause of the cold-burst stall (database host side, the path to Cloudflare, or the receiver’s colo), and whether a smaller first batch or a longer first timeout avoids it.
- The source of the about 1.8 s gap between batches with no slow call.
- Database Webhooks as configured from the dashboard (the run called
net.http_postdirectly),http_getandhttp_delete, and receivers other than one Worker and two Edge Functions. - Repeats of NL01 to NL03 and NL05.
Reproducing
Section titled “Reproducing”The experiment provisions its own projects and deploys its own receiver; it needs a Pro-plan organisation slug in PVLAB_ORG_PRO, a personal access token in SUPABASE_ACCESS_TOKEN, and a Cloudflare token for the Worker.
make receiver-up # deploys the workers.dev receiver, prints its URLmake micro RECEIVER=<url> # NL01, NL02, NL99 on a Micro; about 3 minutes on 2026-10-26make large RECEIVER=<url> # NL03, NL04, NL99 on a Large; NL03 alone took about 25 minutes and NL04 about 14 on 2026-10-26make large LARGE=NL05,NL99 RECEIVER=<url>make clean # list leftover pvlab-nl-* projects; DELETE=1 deletesmake receiver-downmake publish-evidence RUN=evidence/<ts>/run-<stamp>.jsonPVLAB_NL_ARM_S (default 600) and PVLAB_NL_MECH_S (default 180) set the NL03 arm length and the NL04 retry and pgmq deadline (raw is fixed at 90 s).
Modules
Section titled “Modules”| Module | Experiment | Test | Artifact |
|---|---|---|---|
| NL01 | pgnet-under-load | nl01-timeout-ceiling.ts | run-2026-10-26T00-24-41-120Z |
| NL02 | pgnet-under-load | nl02-head-of-line.ts | run-2026-10-26T00-24-41-120Z |
| NL03 | pgnet-under-load | nl03-attribution.ts | run-2026-10-26T00-30-17-690Z |
| NL04 | pgnet-under-load | nl04-delivery.ts | run 1 run-2026-10-26T00-28-01-744Z, run 2 run-2026-10-26T00-42-43-410Z |
| NL05 | pgnet-under-load | nl05-first-burst.ts | run-2026-10-26T00-57-29-512Z |
Related docs
Section titled “Related docs”- Cloudflare Workers + Supabase: the trigger-to-Worker path these calls ran on, and the authentication and idempotency rules for its receiver.
- Locking down Supabase:
pg_netas an egress path from SQL, and why the project owner cannot revoke it. - Edge Function limits: the cold start and idle timeout of the Edge Functions used as targets in NL03.
- Postgres entity graphs: a queue in Postgres with
pgmqbesidepg_cronandpg_net, the pattern NL04’s queue arm used.
References
Section titled “References”-
Supabase, “pg_net: Async Networking,” Supabase Docs. https://supabase.com/docs/guides/database/extensions/pg_net ↩ ↩2 ↩3 ↩4
-
Supabase,
README.md, supabase/pg_net at ddd4485. https://github.com/supabase/pg_net/blob/ddd44859303c/README.md ↩ ↩2