Extensions for Shen in Host Languages

[Reuben]

I’ve been thinking about libraries that are useful to call from Shen but difficult to implement efficiently in portable Shen alone. Cryptography is an obvious example: it can be written entirely in Shen, but most host languages already provide mature, heavily optimized implementations. I’m experimenting with an extension model where the public API is written in Shen, while each port may optionally provide a host-language implementation of the same contract. Shen programs call the same functions regardless of port. If a native backend is available, the extension uses it; otherwise it can fall back to a pure Shen implementation where that makes sense. I’ve prototyped this here: https://github.com/pyrex41/shen-extensions The SHA-256 extension currently uses: - Go’s

crypto/sha256

on shen-go - OpenSSL through FFI on shen-lua - the Rust

sha2

crate on shen-rust - an optimized Common Lisp implementation on shen-cl - a pure Shen implementation as a portable fallback and correctness reference The pure implementation is not intentionally naive—it has already undergone a basic optimization pass—but I'm sure still has room for improvement. The README links each port implementation and includes a reproducible benchmark, with the results explicitly presented as a comparison of the current implementations rather than a limit on pure Shen performance. There are definite tradeoffs: coordinating contracts across ports, handling feature discovery, deciding which extensions deserve pure fallbacks, and keeping behavior consistent. I’m not convinced this is necessarily the best abstraction, but it seems promising. I’d be interested in other approaches or prior art. How would you structure optional, host-accelerated libraries across arbitrary Shen ports?

[Raoul Duke]

sync vs. async will be an issue i suspect 

[Reuben]

Yeah, I think the portable interface probably has to remain synchronous, with any async machinery staying inside the host implementation’s black box.

The repo above includes a basic ZMQ extension that uses polling to provide synchronous access to event-driven behavior. It works as a lowest-common-denominator approach, though it’s somewhat awkward.

I did find a brief mention that Shen Professional supported concurrency, but I’m not sure how that mapped across its various ports: https://shenlanguage.org/OSM/History.html. Does anyone know how that was handled?

[Bruno Deferrari]

It is experimental and the API is not final, but this may help:

module definitions: https://github.com/tizoc/shen-batteries/blob/master/docs/shen-module-v1.md
feature-gated expansion: https://github.com/tizoc/shen-batteries/blob/master/docs/features.adoc

--
You received this message because you are subscribed to the Google Groups "Shen" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qilang+un...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/qilang/6627ea24-97ee-4c5f-8182-5df425e08a85n%40googlegroups.com.

--

BD