WebSocket Integration
50 views
Skip to first unread message
Landreville
May 13, 2020, 6:51:49 PM5/13/20
to pylons-discuss
Hi All,
I've started experimenting with integrating WebSockets into Pyramid applications and came up with this framework/library for implementing them: https://gitlab.com/landreville/rbow
I haven't used it in a production application yet. Right now I'm looking for some feedback if this seems useful and if there are any ideas for improving this.
In general it runs an ASGI application that wraps the Pyramid app and uses Venusian to register websocket "views". These are classes that manage the websocket functionality. They also provide options for working with websockets asynchronously, synchronously, or with a Pyramid request (kinda).
Lastly it also provides "channels" that can send messages to all websockets subscribed to a channel and can call a function to send data periodically (such as grabbing stats and sending them to the clients every 5 seconds).
Thanks for any feedback,
Landreville
Michael Merickel
May 13, 2020, 7:31:23 PM5/13/20
to pylons-...@googlegroups.com
This looks awesome!
Things that aren’t clear to me are how the config and potentially other services / settings are shared as well as authentication and data store connections. And what the pitfalls may be there.
It looks like it’s using asgiref so I assume you are supposed to use its mechanisms for jumping between sync and async apis?
Also what can you say about Pyramid’s apis being used at all in the async context?
--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/4db62ffd-09cb-4ccc-a277-73c5104a47e9%40googlegroups.com.
Steve Piercy
May 13, 2020, 7:46:36 PM5/13/20
to pylons-...@googlegroups.com
Would you please tweet something and mention @pylonsproject, so
I can retweet it to garner more feedback?
--steve
On 5/13/20 at 6:31 PM, mmer...@gmail.com (Michael Merickel) pronounced:
------------------------
Steve Piercy, Eugene, OR
I can retweet it to garner more feedback?
--steve
On 5/13/20 at 6:31 PM, mmer...@gmail.com (Michael Merickel) pronounced:
Steve Piercy, Eugene, OR
Landreville
May 30, 2020, 1:32:42 PM5/30/20
to pylons-discuss
On Wednesday, May 13, 2020 at 7:31:23 PM UTC-4, Michael Merickel wrote:
This looks awesome!Things that aren’t clear to me are how the config and potentially other services / settings are shared as well as authentication and data store connections. And what the pitfalls may be there.It looks like it’s using asgiref so I assume you are supposed to use its mechanisms for jumping between sync and async apis?Also what can you say about Pyramid’s apis being used at all in the async context?- Michael
Correct, asgiref's async_to_sync and sync_to_async decorators are used anytime it's necessary. I also extended asgirefs WsgiToAsgi class which handles making calls to the WSGI application instance. It makes the calls to the WSGI (Pyramid) instance using sync_to_async.
I think my ultimate goal here is to have a high-level interface where on receipt of a websocket message or on a given time interval a function will run. That function will be treated as if it were a view-callable and will be given a Pyramid request instance (based on the initial websocket upgrade request). The return content should also use the Pyramid renderers, except what would normally be an HTTP response body will instead be sent as a websocket message. Ideally this would also include transaction support.
I'm a couple steps towards that goal. There is a "AsyncWebsocketConsumer" that can respond asynchronously to websocket messages. This is the lowest level interaction with the websocket in the library.
Then there is SyncWebsocketConsumer which uses normal synchronous methods to interact with the websocket.
The last one is the RequestConsumer which isn't complete. I feel like I'm currently re-writing a lot of stuff Pyramid does automatically and I have to figure out a better way to treat the "receive" method as a view-callable. For example I'm building a request context manually and using the registry to find request extensions to apply to it. Perhaps I could register a "fake" view in the Pyramid instance and call it using invoke_subrequest. Then send the response body on the websocket.
The SyncWebsocketConsumer and RequestConsumer use async_to_sync to override the send method of the websocket (which is normally async because they come from the ASGI implementation). And sync_to_async to convert the receive method to asynchronous so it can be called by the ASGI implementation.
This means that the receive method is run in a threadpool by sync_to_async. The receive method calls send which is run using async_to_sync. Async_to_sync runs the code by scheduling a task on the main event loop (its gets a hold of this through a threadlocal).
There's one place where this goes even further and its sync_to_async -> sync_to_async -> async_to_sync. Which I think means it's going to use multiple threads from the threadpool to execute that.
As far as using Pyramid's APIs in an async context. I wouldn't trust ever using them asynchronously. Any time a Pyramid API is used it's going to be wrapped in sync_to_async to ensure it's running in the threadpool.
I think my ultimate goal here is to have a high-level interface where on receipt of a websocket message or on a given time interval a function will run. That function will be treated as if it were a view-callable and will be given a Pyramid request instance (based on the initial websocket upgrade request). The return content should also use the Pyramid renderers, except what would normally be an HTTP response body will instead be sent as a websocket message. Ideally this would also include transaction support.
I'm a couple steps towards that goal. There is a "AsyncWebsocketConsumer" that can respond asynchronously to websocket messages. This is the lowest level interaction with the websocket in the library.
Then there is SyncWebsocketConsumer which uses normal synchronous methods to interact with the websocket.
The last one is the RequestConsumer which isn't complete. I feel like I'm currently re-writing a lot of stuff Pyramid does automatically and I have to figure out a better way to treat the "receive" method as a view-callable. For example I'm building a request context manually and using the registry to find request extensions to apply to it. Perhaps I could register a "fake" view in the Pyramid instance and call it using invoke_subrequest. Then send the response body on the websocket.
The SyncWebsocketConsumer and RequestConsumer use async_to_sync to override the send method of the websocket (which is normally async because they come from the ASGI implementation). And sync_to_async to convert the receive method to asynchronous so it can be called by the ASGI implementation.
This means that the receive method is run in a threadpool by sync_to_async. The receive method calls send which is run using async_to_sync. Async_to_sync runs the code by scheduling a task on the main event loop (its gets a hold of this through a threadlocal).
There's one place where this goes even further and its sync_to_async -> sync_to_async -> async_to_sync. Which I think means it's going to use multiple threads from the threadpool to execute that.
As far as using Pyramid's APIs in an async context. I wouldn't trust ever using them asynchronously. Any time a Pyramid API is used it's going to be wrapped in sync_to_async to ensure it's running in the threadpool.
-- Landreville
Reply all
Reply to author
Forward
0 new messages