It REALLY depends on the server you're using...
For public-inbox[1], I ended up writing a custom PSGI server
(since I already needed a C10K NNTP server) to handle
backpressure when generating gigantic responses using a
"pull"-based model rather than a traditional "push" one[2]
I wrote more about push vs. pull models on the rack-devel (Ruby)
list here, since rack uses the "push" model:
https://public-inbox.org/rack-devel/20161115-slow-cli...@80x24.org/
The end result is public-inbox-httpd and -nntpd can stream
arbitrarily large responses while never having more than a
single email loaded in memory for a given client connection.
But yeah, it is very specialized and requires a fair bit of
knowledge about response sizes, pipelining backpressure, and
careful coding to manage the state machines.
To that end, having lightweight concurrency (e.g. Coro for Perl
or "auto-fibers" for Ruby) built into the language to expose a
synchronous flow to the scripter would be much easier. And I'm
hoping I can get "auto-fibers" into Ruby for 2.7.
[1] git clone
https://public-inbox.org/ public-inbox
see:
lib/PublicInbox/HTTP.pm
lib/PublicInbox/HTTPD/Async.pm
git grep -F pi-httpd.async
[2] the "push" vs. "pull" terminology is common from audio APIs