RDFLib roadmap & governance
36 views
Skip to first unread message
Stefano Cossu
Apr 26, 2018, 10:55:26 PM4/26/18
to rdfli...@googlegroups.com
I am not sure if there is a "non-dev" group for RDFLib but I might as
well start asking here.
I find RDFLib to be an amazing project and the only RDF libray for
Python (if we don't count the Redland Python bindings). Lately I have
been wondering how the roadmap is laid out (I understand 5.0.0 is the
next release and a major overhaul) and if there is some governance
around the project. Is it completely volunteer-based or is there an
institutional commitment? Is there a group that works on the roadmap and
high-level goals? Is there any maintenance for 4.x planned once 5.x is out?
This would be helpful for me to understand how the project is doing and
what to expect from it. As RDFLib is a fundamental library for handling
an increasingly popular data format with one of the most popular
programming languages, I would love to see more PRs merged, more issues
closed, more questions answered, more test coverage, more exhaustive
documentation, more regular commits; in short, a mature project. I may
not have the bandwidth to commit to the code directly but I may be able
to volunteer some of my spare time for strategic planning.
Thanks for your great efforts so far and for any information you may
want to share.
Stefano
--
Stefano Cossu
Director of Application Services, Collections
The Art Institute of Chicago
116 S. Michigan Ave.
Chicago, IL 60603
312-499-4026
well start asking here.
I find RDFLib to be an amazing project and the only RDF libray for
Python (if we don't count the Redland Python bindings). Lately I have
been wondering how the roadmap is laid out (I understand 5.0.0 is the
next release and a major overhaul) and if there is some governance
around the project. Is it completely volunteer-based or is there an
institutional commitment? Is there a group that works on the roadmap and
high-level goals? Is there any maintenance for 4.x planned once 5.x is out?
This would be helpful for me to understand how the project is doing and
what to expect from it. As RDFLib is a fundamental library for handling
an increasingly popular data format with one of the most popular
programming languages, I would love to see more PRs merged, more issues
closed, more questions answered, more test coverage, more exhaustive
documentation, more regular commits; in short, a mature project. I may
not have the bandwidth to commit to the code directly but I may be able
to volunteer some of my spare time for strategic planning.
Thanks for your great efforts so far and for any information you may
want to share.
Stefano
--
Stefano Cossu
Director of Application Services, Collections
The Art Institute of Chicago
116 S. Michigan Ave.
Chicago, IL 60603
312-499-4026
Gunnar Aastrand Grimnes
Apr 27, 2018, 6:07:55 AM4/27/18
to rdfli...@googlegroups.com
Hi Stefano,
RDFLib has had a series of volunteer developers since 2002 (!)
The last few years it's mainly been myself and Jörn Hees who make
releases, but we've both had increasingly little time. (Jörn finishing
his PhD, me working on things not related to RDF).
Roadmap is a big word, and I don't think I would say we have anything
like it :), but if I had some time I would try get the following done:
* a real interface for triple sinks and support for streaming parsers:
https://github.com/RDFLib/rdflib/pull/411
* move a bunch of things out of rdflib-core - like microdata/rdfa
parsers (already weirdly copied in their own repos)
* remove SPARQLWrapper dependency: https://github.com/RDFLib/rdflib/pull/744
It's not very likely to happen very soon from my side though.
Jörn should chime in himself, but I think we'd both be happy to have
someone else come in and take more responsibility - if you want to
help code-review PRs, discuss future changes, make releases etc. I
would only be glad to see RDFLib be more alive!
- Gunnar
> --
> http://github.com/RDFLib
> --- You received this message because you are subscribed to the Google
> Groups "rdflib-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to rdflib-dev+...@googlegroups.com.
> To post to this group, send email to rdfli...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/rdflib-dev/c3fa068d-ea99-df1d-b5da-27f527dbcb06%40artic.edu.
> For more options, visit https://groups.google.com/d/optout.
--
http://gromgull.net
RDFLib has had a series of volunteer developers since 2002 (!)
The last few years it's mainly been myself and Jörn Hees who make
releases, but we've both had increasingly little time. (Jörn finishing
his PhD, me working on things not related to RDF).
Roadmap is a big word, and I don't think I would say we have anything
like it :), but if I had some time I would try get the following done:
* a real interface for triple sinks and support for streaming parsers:
https://github.com/RDFLib/rdflib/pull/411
* move a bunch of things out of rdflib-core - like microdata/rdfa
parsers (already weirdly copied in their own repos)
* remove SPARQLWrapper dependency: https://github.com/RDFLib/rdflib/pull/744
It's not very likely to happen very soon from my side though.
Jörn should chime in himself, but I think we'd both be happy to have
someone else come in and take more responsibility - if you want to
help code-review PRs, discuss future changes, make releases etc. I
would only be glad to see RDFLib be more alive!
- Gunnar
> http://github.com/RDFLib
> --- You received this message because you are subscribed to the Google
> Groups "rdflib-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to rdflib-dev+...@googlegroups.com.
> To post to this group, send email to rdfli...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/rdflib-dev/c3fa068d-ea99-df1d-b5da-27f527dbcb06%40artic.edu.
> For more options, visit https://groups.google.com/d/optout.
--
http://gromgull.net
Jörn Hees
Apr 27, 2018, 9:12:52 AM4/27/18
to rdfli...@googlegroups.com
Hi,
full +1 wrt. next steps and any help is obviously welcome.
I'll hopefully find some time to work on integration of HDT as storage backend soon, which might be a good chance to also finally integrate a sink.
But before doing that i'll probably focus on releasing 5.0.0, which i had pushed away to free some time.
Best,
Jörn
> To view this discussion on the web visit https://groups.google.com/d/msgid/rdflib-dev/CAGm1OD%3DMxNnF9by4WEPOfctAVe48SV6nU7O9m%2B1E8tcqDzhqLQ%40mail.gmail.com.
full +1 wrt. next steps and any help is obviously welcome.
I'll hopefully find some time to work on integration of HDT as storage backend soon, which might be a good chance to also finally integrate a sink.
But before doing that i'll probably focus on releasing 5.0.0, which i had pushed away to free some time.
Best,
Jörn
scossu
Apr 27, 2018, 4:47:58 PM4/27/18
to rdflib-dev
Gunnar, Jörn,
The goals you mention seem very important for the project.
All the code writing & reviewing that I can afford I am spending on a side project, but I would be interested in increasing sustainability for RDFLib, e.g. identifying other projects that run mission-critical applications and depend on RDFLib and trying to establish a collaboration, as well as finding ways to make the code more accessible to contribution.
I would be available for a phone call or teleconference to kick ideas around, if you are interested.
Best,
Stefano
p.s. on a side note—Jörn, speaking of storage back ends, I implemented a back end for LMDB which I am quite happy with so far: https://github.com/scossu/lakesuperior/blob/master/lakesuperior/store/ldp_rs/lmdb_store.py It is quite embedded in specific features of my project but I thought it would be interesting to share (and probably not too hard to make it an RDFLib module).
The goals you mention seem very important for the project.
All the code writing & reviewing that I can afford I am spending on a side project, but I would be interested in increasing sustainability for RDFLib, e.g. identifying other projects that run mission-critical applications and depend on RDFLib and trying to establish a collaboration, as well as finding ways to make the code more accessible to contribution.
I would be available for a phone call or teleconference to kick ideas around, if you are interested.
Best,
Stefano
p.s. on a side note—Jörn, speaking of storage back ends, I implemented a back end for LMDB which I am quite happy with so far: https://github.com/scossu/lakesuperior/blob/master/lakesuperior/store/ldp_rs/lmdb_store.py It is quite embedded in specific features of my project but I thought it would be interesting to share (and probably not too hard to make it an RDFLib module).
Stefano Cossu
May 3, 2018, 5:55:10 PM5/3/18
to rdfli...@googlegroups.com
If there is interest in this discussion, I came up with a list of
high-level thoughts about RDFLib.
I want to preface that what I described as issues are not meant to put
anyone in a corner. I am aware that this is a volunteer-based project
and no one is expected to be a full-time maintainer of it.
As for the points, on the technical side:
1. Performance could use some improvement. There are great libraries for
Python that use more of the underlying C libraries. Especially for
computationally intensive operations such as serializing and
deserializing, as well as SPARQL, rewriting some functions as C
extensions or using CFFI or Cython could make RDFLib competitive with
several triple stores. (I am also aware that this is not a trivial task...)
And the more management-related:
2. RDFLib has great functionality, unfortunately a lot is hidden in
code. I find myself looking through the code more often than at the
documentation, which is very good in certain areas and completely absent
in others. If I have a #1 qualm about RDFLib, that is documentation
coverage. This also makes it hard to attract new contributors.
3. There is a large number of PRs that are passing tests, most of them
also apparently agreed upon and ready for merging. A quicker turnaround
in the community contribution process would definitely encourage more
contributions.
4. Similarly, the number of issues that receive no response (let alone
being resolved) is an indicator of the health of the project. That is at
least one thing that I look at when I evaluate open source software.
With RDFLib, the overall quality of the code and the lack of
alternatives were stronger factors for adoption, but one can't always
rely on those.
5. Knowledge retention. I recall an email (or github issue?) exchange
where I inqured about a design decison and learned that nobody actively
contributing to the code may have an answer. I think that reaching out
to previous key contributors and ask for some documentation, or at least
an informal explanation, of certain code would go a long way.
These are my thoughts, put very plainly. Again, this is not meant to be
a string of complaints, but rather a way to point out areas to work on.
I recognize that there is a great deal of hard work in RDFLib and I
think it would be much more convenient to improve it than starting
something else from scratch. Users may need to be made aware of this.
Let me know what the preferred way to continue this conversation would be.
Best,
Stefano
> >> email to rdflib-dev+...@googlegroups.com <javascript:>.
> <https://groups.google.com/d/optout>.
> <https://groups.google.com/d/msgid/rdflib-dev/CAGm1OD%3DMxNnF9by4WEPOfctAVe48SV6nU7O9m%2B1E8tcqDzhqLQ%40mail.gmail.com>.
> <https://groups.google.com/d/optout>.
>
> --
> http://github.com/RDFLib
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "rdflib-dev" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/rdflib-dev/9XAyxUbawlQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> rdflib-dev+...@googlegroups.com
> <mailto:rdflib-dev+...@googlegroups.com>.
> <https://groups.google.com/d/msgid/rdflib-dev/57f57963-60a6-4406-9fad-02a5729fb0dc%40googlegroups.com?utm_medium=email&utm_source=footer>.
high-level thoughts about RDFLib.
I want to preface that what I described as issues are not meant to put
anyone in a corner. I am aware that this is a volunteer-based project
and no one is expected to be a full-time maintainer of it.
As for the points, on the technical side:
1. Performance could use some improvement. There are great libraries for
Python that use more of the underlying C libraries. Especially for
computationally intensive operations such as serializing and
deserializing, as well as SPARQL, rewriting some functions as C
extensions or using CFFI or Cython could make RDFLib competitive with
several triple stores. (I am also aware that this is not a trivial task...)
And the more management-related:
2. RDFLib has great functionality, unfortunately a lot is hidden in
code. I find myself looking through the code more often than at the
documentation, which is very good in certain areas and completely absent
in others. If I have a #1 qualm about RDFLib, that is documentation
coverage. This also makes it hard to attract new contributors.
3. There is a large number of PRs that are passing tests, most of them
also apparently agreed upon and ready for merging. A quicker turnaround
in the community contribution process would definitely encourage more
contributions.
4. Similarly, the number of issues that receive no response (let alone
being resolved) is an indicator of the health of the project. That is at
least one thing that I look at when I evaluate open source software.
With RDFLib, the overall quality of the code and the lack of
alternatives were stronger factors for adoption, but one can't always
rely on those.
5. Knowledge retention. I recall an email (or github issue?) exchange
where I inqured about a design decison and learned that nobody actively
contributing to the code may have an answer. I think that reaching out
to previous key contributors and ask for some documentation, or at least
an informal explanation, of certain code would go a long way.
These are my thoughts, put very plainly. Again, this is not meant to be
a string of complaints, but rather a way to point out areas to work on.
I recognize that there is a great deal of hard work in RDFLib and I
think it would be much more convenient to improve it than starting
something else from scratch. Users may need to be made aware of this.
Let me know what the preferred way to continue this conversation would be.
Best,
Stefano
> >> To post to this group, send email to rdfli...@googlegroups.com
> <javascript:>.
> >> To view this discussion on the web visit
> >>
> https://groups.google.com/d/msgid/rdflib-dev/c3fa068d-ea99-df1d-b5da-27f527dbcb06%40artic.edu
> <https://groups.google.com/d/msgid/rdflib-dev/c3fa068d-ea99-df1d-b5da-27f527dbcb06%40artic.edu>.
> >>
> https://groups.google.com/d/msgid/rdflib-dev/c3fa068d-ea99-df1d-b5da-27f527dbcb06%40artic.edu
> <https://groups.google.com/d/optout>.
> >
> >
> >
> > --
> > http://gromgull.net
> >
> > --
> > http://github.com/RDFLib
> > ---
> > You received this message because you are subscribed to the
> Google Groups "rdflib-dev" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send an email to rdflib-dev+...@googlegroups.com <javascript:>.
> >
> >
> > --
> > http://gromgull.net
> >
> > --
> > http://github.com/RDFLib
> > ---
> > You received this message because you are subscribed to the
> Google Groups "rdflib-dev" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > To post to this group, send email to rdfli...@googlegroups.com
> <javascript:>.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/rdflib-dev/CAGm1OD%3DMxNnF9by4WEPOfctAVe48SV6nU7O9m%2B1E8tcqDzhqLQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/rdflib-dev/CAGm1OD%3DMxNnF9by4WEPOfctAVe48SV6nU7O9m%2B1E8tcqDzhqLQ%40mail.gmail.com>.
> <https://groups.google.com/d/optout>.
>
> --
> http://github.com/RDFLib
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "rdflib-dev" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/rdflib-dev/9XAyxUbawlQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> rdflib-dev+...@googlegroups.com
> <mailto:rdflib-dev+...@googlegroups.com>.
> To post to this group, send email to rdfli...@googlegroups.com
> <mailto:rdfli...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/rdflib-dev/57f57963-60a6-4406-9fad-02a5729fb0dc%40googlegroups.com
> <https://groups.google.com/d/msgid/rdflib-dev/57f57963-60a6-4406-9fad-02a5729fb0dc%40googlegroups.com?utm_medium=email&utm_source=footer>.
Reply all
Reply to author
Forward
0 new messages