Create streams vs subjects

345 views
Skip to first unread message

Peter Spradling

unread,
Dec 14, 2022, 6:05:37 PM12/14/22
to nats
I am trying to understand Jetstream a little better, and it seems like there are two different levels to group messages.  The stream level and the subject level.

If, for example, I want to create a chat app with different groups, each with their own group id.  Would folks recommend a single stream with different subjects (the group id), or different streams per group id.  

These chats would be ephemeral, so I would like to be able to purge them.  Looking at the api, I couldn't find ways to delete specific subjects, so this makes me think I would make each chat a different stream, so I could delete the stream when the chat was finished.

That said, I don't know if it is bad practice to create millions of short lived streams, each with just a single subject.  In this example it would look something like:
Account: Chat
Stream.subject_examples: <group_id_1>.messages, <group_id_2>.messages, ... <group_id_1000000>.messages, etc etc

Alberto Ricart

unread,
Dec 16, 2022, 8:11:47 AM12/16/22
to nats
A stream can host any number of subjects. I would start with a single stream and different subjects and then make your consumers filter on the subject for the chat group. You can assign a maximum amount of time to live for a message so the cleaning could happen automatically for you.
Depending on the type of stream (replicated or not) you also impose additional work on the server to replicate etc. Definitely, a single stream with many different subjects is better.

Take a look at https://docs.nats.io/nats-concepts/jetstream/streams

The API for the server lets you purge messages on a specific subject on a stream - this should be accessible through the nats cli and from clients that implement management of streams, for example, the javascript clients support this. See https://github.com/nats-io/nats.deno/blob/d72048b415ad0fc5127bb36da1cbab71f01cf7e4/nats-base-client/types.ts#L1386
Reply all
Reply to author
Forward
0 new messages