vega(-lite) with SPARQL endpoint

Greg Trzeciak

unread,

Apr 8, 2019, 7:58:30 AM4/8/19

to vega-js

I want to use vega and/or vega-lite with SPARQL enpoint, an example of the query I would like to provide as a parameter to vega(lite) would be:

the results are then given in "application/sparql-results+json" format: https://www.w3.org/TR/sparql11-results-json/

At the moment I am planning to

1. get the results

2. convert the json,

3. save as file

4. provide url to the converted json file

But is there a better more dynamic way of achieving that in both vega and vega-lite?

Thanks

G.

Roy I

unread,

Apr 8, 2019, 9:19:05 AM4/8/19

to vega-js

Instead of saving as file (steps 3 and 4), the final data can be:

1. Defined within the Vega spec as "values":

https://vega.github.io/vega/docs/data/

2. Inserted into a running Vega visualization using Vega View API:

https://vega.github.io/vega/docs/api/view/#view_insert

Greg Trzeciak

unread,

Apr 8, 2019, 9:51:58 AM4/8/19

to vega-js

Thanks Roy

1. I want to keep Vega spec and data separately, also this wouldn't be more dynamic than the file url.

2. This makes sense, the query itself will be editable so using view API sounds like a good idea.

BTW is there a way of using native vega methods to do the conversion of the results on the fly ("application/sparql-results+json" -> json). I see there is

https://vega.github.io/vega/docs/transforms/

but is it capable of such transformation for both vega and vega-lite?

Roy I

unread,

Apr 8, 2019, 12:09:09 PM4/8/19

to vega-js

To use data transforms within Vega spec, the data must first be in a structure that Vega can parse (array, JSON, csv, tsv, dsv, or topojson). In other words, Vega cannot read arbitrary files. See:

https://vega.github.io/vega/docs/data/

Vega has javascript data utility library "datalib":

https://github.com/vega/datalib

https://github.com/vega/datalib/wiki/API-Reference

Vega is built on top of D3 (d3.js) and D3 API has many functions for data manipulation:

https://github.com/d3/d3/blob/master/API.md

For your requirements to convert the structure of the data object, take as look at "lodash":

http://shzhangji.com/blog/2017/03/13/why-use-lodash-when-es6-is-available/

https://lodash.com/docs/4.17.11

On Monday, April 8, 2019 at 7:58:30 AM UTC-4, Greg Trzeciak wrote:

Yuri Astrakhan

unread,

Apr 8, 2019, 12:43:23 PM4/8/19

to Greg Trzeciak, vega-js

Greg, Wikipedia supports it too using this code:

Converts sparql results to flat values:

https://github.com/nyurik/wd-type-parser

Lib to extend "url" parameter to support non-http urls and call various handlers:

https://github.com/nyurik/mw-graph-shared

Example:

https://www.wikidata.org/wiki/Template:Graph:Pie_chart

--
You received this message because you are subscribed to the Google Groups "vega-js" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vega-js+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Greg Trzeciak

unread,

Apr 8, 2019, 1:21:57 PM4/8/19

to vega-js

Thank you Yuri!

This is to a T what I wanted to achieve - especially the extended url parameter! Now I am glad I asked this question - these are great extensions you have written.

And thank you Roy for your informative links.

G.

To unsubscribe from this group and stop receiving emails from it, send an email to veg...@googlegroups.com.

Yuri Astrakhan

unread,

Apr 8, 2019, 1:27:29 PM4/8/19

to Greg Trzeciak, vega-js

Sure thing Greg, but I would recommend using a more recent approach I did in Elastic's Kibana -- treat `url` as an Object rather than a string, and have proper subfields in it. This way you will not need to url-encode SPARQL queries, which will make everything far more readable. Vega passes the "url" value as is to the loader, without analyzing, so this will work just fine.

To unsubscribe from this group and stop receiving emails from it, send an email to vega-js+u...@googlegroups.com.

Greg Trzeciak

unread,

Apr 8, 2019, 1:39:19 PM4/8/19

to vega-js

That's really interesting - I didn't think you could do that with Vega.

For future reference this is the code in question: https://github.com/nyurik/kibana-vega-vis

Cheers

Greg

Dominik Moritz

unread,

Apr 8, 2019, 1:56:11 PM4/8/19

to Greg Trzeciak, vega-js

If you're interested in custom data loaders and transforms, check out https://github.com/vega/vega-loader-arrow and https://github.com/vega/vega-transform-omnisci-core.

--

Greg Trzeciak

unread,

Apr 8, 2019, 2:09:16 PM4/8/19

to vega-js

OK, I am now oficially spoiled - thank you guys!

G.

To unsubscribe from this group and stop receiving emails from it, send an email to veg...@googlegroups.com.

Xavi Gimenez

unread,

Oct 11, 2019, 7:40:45 AM10/11/19

to vega-js

Thanks Yuri and Dominik for the pointers!

BTY, I've started to develop a custom Vega data transform to specifically solve this (fetching and parsing data from SPARQL endpoints).

It's still a WIP, so any feedback is appreciated. The transform just expects the URL of the endpoint and a sparl query.

if you want to check it out, a demo online here, and :

repo - https://github.com/XavierGimenez/vega-transform-sparql

demo repo - https://github.com/XavierGimenez/vega-transform-sparql-demo

Best,

Xavi

Reply all

Reply to author

Forward