Fwd: Dispatching a State Synchronization Working Group
Michael Toomim
-------- Forwarded Message --------
| Subject: | A State Synchronization Working Group |
|---|---|
| Date: | Wed, 25 Oct 2023 15:04:42 -0700 |
| From: | Michael Toomim <too...@gmail.com> |
| To: | disp...@ietf.org <disp...@ietf.org> |
I would like to dispatch the idea of forming a general State Synchronization Working Group, to coordinate duplicate efforts in defining various state synchronization protocols across existing IETF working groups.
Many networking protocols implement some form of state
synchronization. At the application layer, SCIM, SIP, COAP,
ALTO, NETCONF, JMAP, CalDAV, and IPP all define state
synchronization protocols over HTTP. Many non-IETF protocols
also implement sync-over-HTTP for specific uses, such as WebSub
(to synchronize blogs), ActivityPub (to synchronize social
feeds), and Matrix (to synchronize chat). Outside of HTTP, we
do state synchronization in DNS, IMAP, LDAP, NFS, RSYNC, and
Git. Dropping to the lower-level OSI layers, we have protocols
to synchronize IP routing tables, network reachability, and
header compression dictionaries like ROHC, OSPFv2, IS-IS, and
BGP. State synchronization also comes up in distributed systems
and databases, with algorithms such as OT and CRDT, and each
system defines its own distinct protocol.
We design these as separate protocols because the *general* version of the State Synchronization Problem has seemed too challenging to tackle. However, the last decade of research has brought breakthroughs in general state synchronization algorithms that allow (a) multiple editors, (b) editing arbitrary data types, (c) over a distributed network, (d) of arbitrary network topology, (e) experiencing arbitrary network partitions and delays, (f) with arbitrary merge resolution semantics—all with reasonable (and improving) performance. These general algorithms bring up the possibility of designing general *protocols* for state sync.
If IETF working groups could rely on general standards for state sync, they could eliminate redundant consensus and specification work, as well as implementation work, because implementers could re-use common libraries and algorithms instead of writing their own algorithms. Furthermore, general libraries could afford investment in advanced sync features such as peer-to-peer networking, offline modes, delta-compression, merge resolution, consistency guarantees, and fast-replay; which applications might not implement on their own, but now could benefit from for free. This would improve networking robustness across the board. Finally, interoperability would improve. Network sysadmins, for example, could inspect and debug the state and history of their routers, emails, web applications, email, and file systems with intercompatible tools over a general protocol.
This new working group would integrate research with practice by working in symbiosis with existing IETF Working Groups. Our new group would gather experts and researchers in state synchronization and interface with individual Working Groups. It would learn the needs of individual working groups, and produce general knowledge, protocols, and libraries in return.
We have had initial success with this model in the informal
Braid.org group, which was roughly modeled on an IETF Working
Group, and has produced the Braid-HTTP internet draft that is
coming up for discussion in HTTPbis. The Braid group has met
every two weeks for 2.5 years on zoom, with average attendance
of 4 or 5 people, and has produced a number of research results
in generalizing performant state synchronization:
- Diamond-Types:
World's fastest P2P text editing CRDT
- List-CRDTs: First text editing CRDT
with general pluggable merge semantics (video presentation)
- Antimatter: World's first history-pruning P2P text editing
CRDT (draft) (implementation)
- Sync9:
First true replace text CRDT
- Portals: Generalized patch
semantics for moves, replaces, splices
- Time Machines:
Generalization of OT & CRDT theory
- Braid-HTTP:
Synchronization for HTTP
We could imagine formalizing the Braid group into this State
Synchronization group. It would look more broadly at
synchronization than any individual application, and would
produce specs that could be offered to other groups for
standardization, similar to how the QUIC group seeded HTTP/3.
I am very curious what the IETF thinks about this idea for a
working group, and would be very grateful to hear thoughts and
opinions from everyone. Thank you very much!
Michael
[1] Relevant Braid-HTTP draft: https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http
Michael Toomim
Separately, our Braid-HTTP protocol is on the agenda for the HTTP Working Group, along with Rahul Gupta's PREP proposal:
We will have about 10 minutes to present Braid-HTTP, and make a case for the HTTP Working Group to adopt state synchronization as an extension to standardize into HTTP, plus time for questions & discussion.
Michael
Duane Johnson
--
You received this message because you are subscribed to the Google Groups "Braid" group.
To unsubscribe from this group and stop receiving emails from it, send an email to braid-http+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/braid-http/1689ad11-e5cc-a025-4dd0-c9fb60304a07%40gmail.com.
Michael Toomim
Ah, yes Duane! I'm glad you're pointing this out, because I feel
it too!
This is one step— but it's a step into a new world of people who care. The IETF is filled with protocol-designers, and many (or most?) of those protocols do some forms of state synchronization. All the work we've done will now be flowing amongst a great number of new minds!
I feel the excitement of establishing a beachhead of connection with this new community. We're starting to engage in dialogue. We can get these ideas into HTTP, and other protocols too!
One small step for man can sometimes be part of a giant leap forward for mankind. :)
Michael