Python based SPARQl endpoint

790 views
Skip to first unread message

richarddi...@gmail.com

unread,
May 30, 2021, 6:59:01 AM5/30/21
to rdflib-dev
As far as I know no Python based sparql-endpoint (like Fuseki, https://jena.apache.org/documentation/fuseki2/) implementations exist.

Do (previous) initiatives exist with this goal?

Wes Turner

unread,
May 30, 2021, 10:38:38 AM5/30/21
to rdfli...@googlegroups.com
Rdflib has support for SPARQL queries on local graphs and remote SPARQL endpoints, but doesn't specify a framework to use to send results back over HTTP per e.g.  SPARQL 1.1 UPDATE or SPARQL 1.1 Graph Store HTTP Protocol.

https://rdfextras.readthedocs.io/en/latest/working_with.html#working-with-sparql

SPARQL 1.1 Update

SPARQL 1.1 Graph Store HTTP Protocol

FWIU, LDP is an alternate way to share subgraphs over HTTP which specifies how to do pagination, where SPARQL does not and something should enforce a default and a maximum LIMIT clause.

What does this query do when run by many users concurrently?

SELECT ?s,  ?p, ?o LIMIT 1000000000000;

That's okay though, because like any database can be trivially DOS'd if the DB interface is exposed to end users, right?

So, there is not an opportunity to eliminate the web application server tier because SPARQL endpoints cannot be exposed to end users without creating a Denial-of-Service vulnerability.

Which framework should a reference e.g. LDP or SOLID (LDP+[...]) implementation be written for?

- Django, on top of DRF
- Flask, because I want to reinvent auth/authz for every implementation
- Flask, because it's more async
- ?



On Sun, May 30, 2021, 06:59 richarddi...@gmail.com <richarddi...@gmail.com> wrote:
As far as I know no Python based sparql-endpoint (like Fuseki, https://jena.apache.org/documentation/fuseki2/) implementations exist.

Do (previous) initiatives exist with this goal?

--
http://github.com/RDFLib
---
You received this message because you are subscribed to the Google Groups "rdflib-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rdflib-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rdflib-dev/067098ee-f08e-424c-9cdf-d9e234f635c3n%40googlegroups.com.

richarddi...@gmail.com

unread,
May 30, 2021, 3:18:06 PM5/30/21
to rdflib-dev
Thank you for the confirmation and the additions.
Thinking about these DOS attacks will be an issue for all implementations imo; so I will not start taking this a major criterion.

Local graph queries and the RDF store can be used behind this sparql endpoint. Queries supply optionally a graph name in the query or as parameter.

I have already developed RDF model driven apps using web2py. I would like start using py4web (successor of web2py) for the sparql endpoint because of the easy use of py4web. Py4web is rather new but I think it will be sufficient mature within 1 year. Now we have a momentum for RDF based solutions. This type of solutions require always software development. A productive Python ecosystem will lower the effort required for initial RDF model driven applications because specific query might return specific output like charts, graphs, json or formatted tables using specific vocabulaires.

My experience is also with TQ TB Composer and TB EDG. My wep2py apps used EDG as a sparql endpoint for both ontology and data store. TB EDG use GraphQL for form generation; I have been using pydal for instant form generation and data validation based on SHACL.

I am also in contact with the py4web community for their interest in this initiative.

Wes Turner

unread,
May 31, 2021, 4:55:40 AM5/31/21
to rdfli...@googlegroups.com
## Objectives
- Develop schema/models and map those onto objects/tables/rows/nodes/edges/RDF/RDF* in order to store instances
- Generate HTML + JS forms with clientside validation in order to CRUD instances and assure data quality (JSONschema?)
- Generate HTML + JS forms with serverside validation in order to CRUD instances and assure data quality (JSONschema?)
- Generate HTTP API for use with:
  - JS client app (browser, desktop, mobile)
  - Mobile app (mobile)
  - Clients that authenticate before making requests
- Pass through SPARQL queries made to the app to the backend SPARQL endpoint in order to validate/modify the query (e.g. the LIMIT clause)
- Expose all data through SPARQL endpoint
  - Prevent resource exhaustion ("DoS") due to reasonable use by more than one user
  - Support HTTP Content-Range requests and/or Paging/Pagination
  - Support [pass through?] authentication and possibly also authorization
  - Determine whether ther ultimate SPARQL server supports rate limiting and/or anything like an X-Forwarded-For header

*****

- TIL that pydal is the web2py SQL ORM and forms generation and validation library:

- Django ModelForms (and class-based views) are unfortunately somewhat tightly coupled to the Django ORM:

  "Unfortunate" because there may not be a good way to make a graph query interface quack exactly like a SQL ORM interface; so more abstraction *would be* necessary to maximally reuse the forms validation, CRUD scaffolding, and REST API features -- that we use batteries-included frameworks for -- with anything other than a SQL database. So, instead of trying to do SPARQL on top of SQL -- which is trivial with e.g. Virtuoso -- SQL applications can also just publish cached, URI-named JSON-LD documents and/or provide an e.g. W3C LDP HTTP API that other apps can easily copy data from into an on-disk or in-memory e.g. triplestore database as necessary.

- DRF (Django REST Framework) does lots of APIs very well.
  - I've mentioned why I think it's worth it to do LDP/SPARQL on DRF for all on Twitter before in the past;
    because the Django community is huge so high impact and 
    Linked Data in Django could be as easy as models.py / admin.py / api.py.
  - IIRC, DRF Validators and (De-)Serializers are not coupled to the Django ORM:
  - It's probably pretty easy to frame DRF API responses as JSON-LD; with an optional explicit @context that maps the framed JSON document structure to RDF; though then what are the named graph URIs for each [paged, possibly windowed] HTTP API endpoint responses?

- Generate HTML+JS forms with validation from one or more {RDFS, SHACL, JSONschema, OWL, Fresnel} schema

  - From https://github.com/schemaorg/schemaorg/issues/2790#issuecomment-839263042 re: schema validation [of template output in CI test runs] :
    > Perhaps there's some combination of e.g. https://github.com/google/schema-dts and ( https://github.com/vazco/uniforms , https://github.com/rjsf-team/react-jsonschema-form ) or https://github.com/mulesoft-labs/json-ld-schema that would be usable offline in frequent, automated builds?

    - Is there a good way to get {pySHACL,} data validation messages to display in an HTML+JS form?
    - Can SHACL validation also be done in JS in a browser; for live form validation without server load?
    - What are the standards for returning SHACL validation results in RDF over HTTP?

- LDP and/or / instead_of SPARQL:

    TIL about "A LDP server in Python3 using asyncio, aiohttp and api_hour" named glutton:
    - name: glutton


      > Two Flask web-apps for quickly setting up a SPARQL 1.0 Endpoint or a LOD app for RDFLib

      > 
      > This module contains a small Python module which is intended to be added (imported) into a Python Flask installation to add a small library of Renderer classes which can be used to handle requests and return responses in a manner consistent with Linked Data principles of operation.
      > The intention is to make it easy to "Linked Data-enable" web APIs.

These for Django would be cool; it'd be great to be able to use the batteries included in the framework with Linked Data data sources instead of SQL to Object models.

To view this discussion on the web visit https://groups.google.com/d/msgid/rdflib-dev/75b8b124-da49-4b34-96e5-6ef361eaf047n%40googlegroups.com.

richarddi...@gmail.com

unread,
May 31, 2021, 3:53:37 PM5/31/21
to rdflib-dev
Hi Wes,

I am impressed by your invertory of relevant available libraries.
IMO it's a pitty you have learned pydal just today, it is great. Pydal is maintained as a seperate library; it has been part of web2py. Pydal is also part of the py4web stack. Pydal and sqlforms (https://py4web.com/_documentation/static/en/chapter-12.html) can be a productive combination for RDF model driven UI's.

The objectives are complete, we might arrange them by priority.

We could start with a 'simple' sparql endpoint only reading graphs, serving a general usecase. With rich output options would be already more useful than Fuseki.

Additions to possible useful libraries:
- Shaclgen https://pypi.org/project/shaclgen/ for generating shapes of relational models
- https://github.com/Babibubebon/vue-shacl-form is a a JS library for Vue forms based on shacl.
- https://github.com/rob-metalinkage/django-rdf-io , Utilities to link Django to RDF stores and inferencers
- also a minimal s-put (SPARQL over HTML, written in Ruby) exists in Python.

Richard

Nicholas Car

unread,
Jun 22, 2021, 7:21:59 PM6/22/21
to rdfli...@googlegroups.com
Just following up on this:

* pyLDAPI
    * initially a LDP implementation in Python Flask
    * updated to support ConnegP (https://w3c.github.io/dx-connegp/connegp/)
    * recently updated to work with Pythons newer FastAPI - see https://github.com/rdflib/pyLDAPI/tree/feature/fastapi
    * a variant of it also supports the OGC's OGC API

We (my company, Australian government) use this API a lot, so it's under active and constant development and use.

We tend to use it on top of a triplestore like Fuseki or GraphDB but also use it on top of non-RDF sources like Oracle, Postgres, WFS etc. for RESTful HTML & RDF too.

Example endpoints:

* https://asgsld.net/2016/ - Postgres back-end
* http://asgs.surroundaustralia.com/collections - same data as above, Fuseki back-end
* https://gnafld.net/?_view=reg - large Postgres back-end

We are working on a next-gen API that auto-configures a lot more: just give it data and it will work out how to make the API.

Cheers,

Nick

To view this discussion on the web visit https://groups.google.com/d/msgid/rdflib-dev/389452c5-b7f7-4dd9-b432-6dda7bf173e0n%40googlegroups.com.

richarddi...@gmail.com

unread,
Nov 12, 2021, 11:20:08 AM11/12/21
to rdflib-dev
I found today https://github.com/vemonet/rdflib-endpoint looks promissing.
Reply all
Reply to author
Forward
0 new messages