JSON-LD critical

[Nicholas Car]

Dear RDFLib Developers,

JSON-LD support within RDFLib is now at a critical stage: we have essentially no activity on the other Python JSON-LD package, pyLD, and ever-increasing demands on RDFLib to support all the various JSON-LD modes and related JSON functions like framing. Also, some members of the JSON-LD community that might have helped here are no longer in the domain, including one prominent one who recently passed away.

Are any of you able to contribute to the push for an RDFLib v8 that we are currently undertaking that might include an enhance handing of JSON-LD?

Thanks, Nick

[Miel Vander Sande]

Hi Nicolas,

Top what extent can pyLD be migrated into RDFLib or are we talking from-scratch implementation here?

In any case, I'll look at related code base and try to estimate the level of know-how and effort needed

Best,

Miel

Op donderdag 9 oktober 2025 schreef Nicholas Car <ni...@kurrawong.ai>:

--
http://github.com/RDFLib
---
You received this message because you are subscribed to the Google Groups "rdflib-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rdflib-dev+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/rdflib-dev/TLSx2qBWUsjMom8z1mWM8AmK2-qbj4dVxIB7SSwQeovehN07OzIiq6DRzyTC0KwRTUFaAALI0-20vYp9bOoE7zUKppseXs631_dNnShigT8%3D%40kurrawong.ai.

--

Miel Vander Sande Data Architect meemoo vzw | Ham 175, 9000 Gent | https://www.meemoo.be/ t +32 9 298 05 01 | m +32 492 83 21 29

[Nicholas Car]

Hi Miel,

We’ve looked at merging pyLD in previously but haven’t worked out a simple path for that yet. Edmond is probably most experienced in that.

I do think though that the scope of pyLD isn’t too huge, so the conversions, context handing and osso on that it does could be either copied into RDFLib or re-implemented, informed by pyLD code, pretty easily.

I suspect we have more challenges in ensuring that all the JSON-LD 1.1 elements are handles to/from RDF conversions and parsing/serialising in native RDFLib JSON-LD handling than just the pyLD conversions.

I’ve just tried to improve JSON-LD issue tagging for this list:

https://github.com/RDFLib/rdflib/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22format%3A%20JSON-LD%22

Further early work could be to check that all the JSON-LD problems we know about are actually listed in tagged issues.

Cheers, Nick

To unsubscribe from this group and stop receiving emails from it, send an email to rdflib-dev+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/rdflib-dev/CAHeRLWuTBWVvBkcd-AVW_cHznXEqNxGYxO0TY5eAgyG-TZaTpA%40mail.gmail.com.

[Niklas Lindström]

Dear all,

I'm also still active with TRLD [1], which from the start strives to be true to the letter with the JSON-LD 1.1 API spec algorithms; i.e. expansion, compaction, flattening, to and from RDF. (For reasons (e.g. separating querying and serialization), it lacks full framing support; but it has a simple means of framing blank nodes.)

(Like PyLD it's independent of RDFLib. It also comes with a TriG parser/serializer and some extras; including transpilation of its implementation into Java and JS. I'm also working on aligning it with compliant RDF 1.2 support (as part of the upcoming work on JSON-LD 1.2 and 1.3).)

Best regards,

Niklas

[1]: <https://github.com/niklasl/trld>

--

http://github.com/RDFLib
---
You received this message because you are subscribed to the Google Groups "rdflib-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rdflib-dev+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/rdflib-dev/TLSx2qBWUsjMom8z1mWM8AmK2-qbj4dVxIB7SSwQeovehN07OzIiq6DRzyTC0KwRTUFaAALI0-20vYp9bOoE7zUKppseXs631_dNnShigT8%3D%40kurrawong.ai.

[Miel Vander Sande]

Hi Niklas,

What's the focus of the TRLD project? Just the algorithms or does it extend to a JSON-LD parser too? Of is TRLD a library that can be used by a JSON-LD library?

@Nicholas Car should the parser/serialiser hook into https://github.com/RDFLib/rdflib/blob/main/rdflib/parser.py and https://github.com/RDFLib/rdflib/blob/main/rdflib/serializer.py? 

Best,

Miel

Op do 9 okt 2025 om 12:16 schreef Niklas Lindström <linds...@gmail.com>:

To view this discussion visit https://groups.google.com/d/msgid/rdflib-dev/CADjV5jf8t0SOks_sQS36RNHSOdMm_%3DbBL54oLuUq0XU2CbP0sg%40mail.gmail.com.

[Niklas Lindström]

On Tue, Oct 21, 2025 at 9:57 AM Miel Vander Sande
<miel.van...@meemoo.be> wrote:
>
> Hi Niklas,

Hi Miel,

Sorry for the late reply (some health issues got in the way)!

> What's the focus of the TRLD project? Just the algorithms or does it extend to a JSON-LD parser too? Of is TRLD a library that can be used by a JSON-LD library?

The focus is to implement the JSON-LD algorithms verbatim, mainly
expansion, compaction and flattening, and to be able to transpile
these into other languages (currently Java and JS). Some algorithm
issues have cropped up by doing that.

It has since been extended to do some extras, including automatic
framing of "object-only" linked blank nodes, and indexing by @id
and/or @type, TriG and N-quads parsing and serializing, and a
vocabulary mapper (based on RDFS and OWL). In some way, it functions
as a low-level RDF library, with the internal JSON-LD data model as an
"abstract syntax tree".

Library-wise, I have not stabilized the processor API, and it is still
at a semantic 0.x version, so I do not generally advocate it as
API-stable. I do use it daily, mainly through its `trld` CLI [1], but
also in some other tools, importing most things from trld.api [2] (I
think peeking into those can give a generally good glimpse of how to
use this in other tooling). The Java-transpiled code is also used in
the library system I work with on a daily basis, mainly for speedy
output of RDF (Turtle/Trig) for indexing and conneg, but in the recent
years also for mapping and compaction.

I have not deeply compared the PyLD implementation "step by step", so
I cannot fully say how TRLD differs, but I think I've heard that PyLD
isn't always 1:1 with the spec steps (which doesn't necessarily matter
if the same output is achieved). That said, TRLD also optimizes some
things in the context processing (to avoid reprocessing); something
that *might* be valuable to factor out in the specs as well (unless a
note about optimization is more appropriate).

I think PyLD maintenance being picked up by the RDFLib community is
very promising, and I'll think about what the best long-term strategy
is for avoiding duplicate work or fragmentation. Having multiple
implementations is hopefully healthy, but it also risks someone
"betting" on eventually unmaintained code. There are also differing
ideals, such as minimal components cooperating using stable API:s, vs.
efficiency through deeper integration. I will think more about this
too.

(The best I've hitherto been able to do is to publish this as 0BSD to
make it clear that copying chunks of any parts of the code, e.g. into
PyLD, is fully OK, without any licensing issues AFAICS, nor
attribution required (though still appreciated). This implementation
was made by looking at the spec, not other implementations. The TriG
parser and serializer are mainly made "the hacker way" by throwing
data at them until tests pass. I'm sure others, and maybe some LLMs,
have or will eventually do that much better.)

All the best,
Niklas

[1]: <https://github.com/niklasl/trld/blob/main/trld/cli.py>
[2]: <https://github.com/niklasl/trld/blob/main/trld/api.py>

PS. One of my more ambitious goals with the transpiler was initially
to generate the spec algorithms in pseudo-code (perhaps INFRA...). It
"should" be simpler than the other real languages, but I have not
dedicated much time on that yet.

> To view this discussion visit https://groups.google.com/d/msgid/rdflib-dev/CAHeRLWuij2WACxAeXm4H9JLj40abSTX7eqPZxj%2B%3D-C6C9q-gkg%40mail.gmail.com.

[Miel Vander Sande]

Hi Niklas,

Thanks for coming back to me. Happy to hear you healthy again!

I've got some answers inline

Op di 4 nov 2025 om 13:24 schreef Niklas Lindström <linds...@gmail.com>:

On Tue, Oct 21, 2025 at 9:57 AM Miel Vander Sande
<miel.van...@meemoo.be> wrote:
>
> Hi Niklas,

Hi Miel,

Sorry for the late reply (some health issues got in the way)!

> What's the focus of the TRLD project? Just the algorithms or does it extend to a JSON-LD parser too? Of is TRLD a library that can be used by a JSON-LD library?

The focus is to implement the JSON-LD algorithms verbatim, mainly
expansion, compaction and flattening, and to be able to transpile
these into other languages (currently Java and JS). Some algorithm
issues have cropped up by doing that.

It has since been extended to do some extras, including automatic
framing of "object-only" linked blank nodes, and indexing by @id
and/or @type, TriG and N-quads parsing and serializing, and a
vocabulary mapper (based on RDFS and OWL). In some way, it functions
as a low-level RDF library, with the internal JSON-LD data model as an
"abstract syntax tree".

I think those are maybe not a priority, they are definitely great assets for developers, so it's good to at least try to incorporate the ideas.

 

Library-wise, I have not stabilized the processor API, and it is still
at a semantic 0.x version, so I do not generally advocate it as
API-stable. I do use it daily, mainly through its `trld` CLI [1], but
also in some other tools, importing most things from trld.api [2] (I
think peeking into those can give a generally good glimpse of how to
use this in other tooling). The Java-transpiled code is also used in
the library system I work with on a daily basis, mainly for speedy
output of RDF (Turtle/Trig) for indexing and conneg, but in the recent
years also for mapping and compaction.

I have not deeply compared the PyLD implementation "step by step", so
I cannot fully say how TRLD differs, but I think I've heard that PyLD
isn't always 1:1 with the spec steps (which doesn't necessarily matter
if the same output is achieved). That said, TRLD also optimizes some
things in the context processing (to avoid reprocessing); something
that *might* be valuable to factor out in the specs as well (unless a
note about optimization is more appropriate).

Yeah I think these observations are correct. TRLD seems more technologically sound, but PyLD has a lower entry-barrier (code structure, documentation, comments), at least for me. Since my time and that of the few contributors is limited, I need to be pragmatic about this and start from PyLD. 

 

I think PyLD maintenance being picked up by the RDFLib community is
very promising, and I'll think about what the best long-term strategy
is for avoiding duplicate work or fragmentation. Having multiple
implementations is hopefully healthy, but it also risks someone
"betting" on eventually unmaintained code. There are also differing
ideals, such as minimal components cooperating using stable API:s, vs.
efficiency through deeper integration. I will think more about this
too.

That would be great and we'll look at parts of PyLD that can possibly be replaced with ideas and implementations from TRLD. There's (among others) the question of what internal RDF representation rdflib, PyLD and TRLD use and to what extent that makes them compatible.

 

(The best I've hitherto been able to do is to publish this as 0BSD to
make it clear that copying chunks of any parts of the code, e.g. into
PyLD, is fully OK, without any licensing issues AFAICS, nor
attribution required (though still appreciated). This implementation
was made by looking at the spec, not other implementations. The TriG
parser and serializer are mainly made "the hacker way" by throwing
data at them until tests pass. I'm sure others, and maybe some LLMs,
have or will eventually do that much better.)

Thanks that is very much appreciate it. As far as I am concerned, attribution is a given for both you and digitalbazaar. But I would have to check the current practice in rdflib about that.

Cheers,

Miel

 

To view this discussion visit https://groups.google.com/d/msgid/rdflib-dev/CADjV5jdd2rByX%2BMDkxAdy%2B-YZx1WrDA1jUmNE3iJR2R1T1ez0A%40mail.gmail.com.

[Nicholas Car]

Hi Niklas, Miel and all,

Great to be seeing all this discussion. 

Clearly there are a bunch of intersections of rdflib, pyLD & trld that I haven't got my head around yet, so I'll likely have to play with them all a bit more to know where integrations can be made.

Just getting the use - on parse and serialize - of JSON-LD contexts in RDFLib is a much requested feature, so we might be able to achieve that even without full absorption of either pyLD or trld into RDFLib.

Cheers, Nick