Fetching pathway SVG via CORS

[Eric Weitz]

Hi WikiPathways,

Is there a way to get raw SVG for a pathway diagram using CORS HTTP requests?  

I'm aware of two similar options, but neither quite suits.

getColoredPathway: perfect, but offline

First is the WikiPathways API endpoint for getColoredPathway, which seems broken.  Trying to plug in values in the Swagger example returns a message that the service was offline.  It's also deprecated in related software.  This `getColoredPathway` functionality seems like precisely what I need, as it's CORS-enabled and returns a pathway diagram graphic.

PathwayWidget: iframe makes reuse hard

Second is the PathwayWidget API, which doesn't fit my use case given its iframe approach.  Basically, I'd like to fetch the pathway diagram SVG and use the neat pvjs library more directly.  Using iframes raises security concerns with some downstream embedding applications, and substantially increases complexity in modern JavaScript components that otherwise use CORS.

Could you restore the getColoredPathway API endpoint?  Or is there another way to fetch pathway SVG via CORS, i.e. directly via web browser JavaScript on another domain?

Thanks,

Eric

P.S.: My use case for this is to show pathway diagrams in gene tooltips for a related genes ideogram.  E.g. if a user searches for GNAQ and sees OXTR as an interacting gene, they could hover over OXTR, and see not only the name of the pathway as they can now, but also the biochemical drawing of how the two genes interact.

The diagram is uniquely rich and valuable.  Seeing it in context would supercharge this gene discovery UI, and enable many other use cases!

[Egon Willighagen]

I do not know what all other options are right now, but I think this is one of them:

https://www.wikipathways.org//wpi/wpi.php?action=downloadFile&type=svg&pwTitle=Pathway:WP4904

Egon

--
You received this message because you are subscribed to the Google Groups "wikipathways-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wikipathways-discuss/833b06ba-ded3-446a-8175-f88d5d0a98d2n%40googlegroups.com.

--

----

BiGCaT received a NWO Open Science grant to support our research into interoperability of biological data and knowledge: https://www.nature.com/articles/d41586-021-03418-1 and https://www.nwo.nl/en/researchprogrammes/open-science/open-science-fund/open-science-fund-2021-awarded-grants

-----

E.L. Willighagen
Department of Bioinformatics - BiGCaT
Maastricht University (http://www.bigcat.unimaas.nl/)

Twitter/Mastodon: @egonwillighagen / @egonw
Homepage: http://egonw.github.io/
Blog: http://chem-bla-ics.blogspot.com/
PubList: https://www.zotero.org/egonw
ORCID: 0000-0001-7542-0286
ImpactStory: https://impactstory.org/u/egonwillighagen

[Eric Weitz]

Thanks, Egon, for pointing me in a promising direction.  For this use case, I would need that option with two adjustments:

• Enable CORS, e.g. by adding the HTTP header Access-Control-Allow-Headers: * to the response.
• No download to file system, e.g. by removing the HTTP header Content-Disposition: attachment from the response.

Source code for wpi.php suggests no such option exists in that service.  

But your input has given me a key thread to follow!  I will update here if my exploration yields any fruit.

[Mark Fortner]

It would be useful if there was a webcomponent wrapper for the SVG that would allow people to apply colour to nodes in the network. For example, you might want to set the colour of the target that you’re currently working on to red, and set the targets used by potential collaborators to green. Or you might want to add popups to the nodes with more detailed information about the target or the interaction.

A w3c-compliant webcomponent would overcome most of the limitations of iframes and would make the pathways more easily embeddable in modern web applications without tying the component directly to a framework like react, or vue.

Best,

Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/wikipathways-discuss/bcd16333-ee40-4a8f-9d4c-c26f28ec8af7n%40googlegroups.com.

[Anders Riutta]

Eric,

Is there a specific domain name we could add to an "allow list" for CORS, or would it need to be any IP address?

-Anders

To view this discussion on the web visit https://groups.google.com/d/msgid/wikipathways-discuss/CAMPqvY-3akkt-Oa_BXrPMq2JUbZbtoZKmoMvC3nTr-j1VNy_tQ%40mail.gmail.com.

--

Anders Riutta

anders...@gladstone.ucsf.edu | Bioinformatics Core | Institute of Data Science and Biotechnology

Gladstone Institutes | Science Overcoming Disease

[Eric Weitz]

Hi Anders, it would need to be any domain.

If restoring the getColoredPathway WikiPathways API endpoint is infeasible, then I could see adding a new (and simpler) REST endpoint like "getPathwayGraphic" that would only respond with the raw SVG for now.  That would ensure all proper HTTP headers for CORS are in place by default, ease discovery of the functionality, etc.

Adding CORS headers to wpi.php -- and changing its content disposition per my previous message -- seems like it would work too.

For convenience, here are example steps to reproduce the desired WikiPathways functionality I describe:

1. Go to https://eweitz.github.io/ideogram/related-genes?q=GNAQ -- or any domain that doesn't have wikipathways.org in it
2. Open DevTools, go to Console panel
3. Enter and execute: fetch("https://webservice.wikipathways.org/getPathwayGraphic?pwId=WP289") -- or, if adjusting wpi.php instead of using REST API, perhaps URL would be e.g. https://www.wikipathways.org/wpi/wpi.php?action=getGraphic&type=svg&pwTitle=Pathway:WP289.
   
4. Go to Network panel in DevTools
5. Verify the request from step 3 has returned only raw SVG (without including style and script tags like PathwayWidget, and without auto-downloading the SVG to the file system)

I hope that helps!

[Alex Pico]

Hi Eric,

It was a couple years ago that we took down the getColoredPathway method. I think it was due to heavy bandwidth we could not afford on AWS, IIRC. We invested dev time into making a more robust embedded solution, which supports better color and referencing options (via labels or Xref IDs) and was a more popular option among our users (easy for devs to embed into their own websites).  

I’m sure you read through the docs already (https://www.wikipathways.org/index.php/PathwayWidget), but maybe it’s worth pointing out that that you can access the SVG from the resolvable URLs and skip the iframe business if you want, e.g., here’s an advanced case with a few nodes colored:

https://www.wikipathways.org/wpi/PathwayWidget.php?id=WP528&xref[]=10400,Entrez%20Gene&xref[]=62-49-7,CAS&xref[]=1119,Entrez%20Gene&colors=green,red,blue  (the SVG is contained within, e.g., view source)

So, we are not likely to bring back the older, inferior getColoredPathways method. It was a pain to maintain and support, and the widget is just a better solution overall.

However, all our code is open source and we actively promote its reuse. So, if you want to perform GPML-to-SVG conversions yourself or even setup your own SVG service, you’d have our support.  Here’s one place to start:

https://github.com/wikipathways/gpml2unifiedsvg

Cheers,

 - Alex

To view this discussion on the web visit https://groups.google.com/d/msgid/wikipathways-discuss/c81b6008-eb5c-470a-b06d-d3b2d372900en%40googlegroups.com.

[Eric Weitz]

I appreciate that context, Alex.  PathwayWidget has some seriously useful features.

For broad reuse in modern JavaScript-centric web components, PathwayWidget would need to support CORS.  If you do those example steps from my previous message, and in step 3 use the URL you provided, you'll see an error like this:

Access to fetch at 'https://www.wikipathways.org/wpi/PathwayWidget.php?id=WP528&xref%5B%5D=10400,Entrez%20Gene&xref%5B%5D=62-49-7,CAS&xref%5B%5D=1119,Entrez%20Gene&colors=green,red,blue' from origin 'https://eweitz.github.io' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

Resolving that requires CORS for PathwayWidget.  This means adding HTTP headers like "Access-Control-Allow-Origin: *" to the response, and I think also supporting OPTIONS requests.  That comes out-of-the-box with the WikiPathways REST API.  But if getColoredPathway were taken down due to heavy bandwidth / egress cost as you recall, then I suspect the adjusted PathwayWidget would have the same problem.

Thank you for endorsing a separate SVG service -- I'll try that out.  It looks like https://wikipathways-data.wmcloud.org/current/svg/ has all the raw SVG data.  I think I'll copy that, but with each WikiPathway's SVG file individually URL-addressable by WPID, run through an SVG compressor, and hosted on a free service (e.g. GitHub Pages, or a JS package CDN) that's CORS-enabled by default.

I'll follow up here if I get something interesting set up.

[Anders Riutta]

Hi Eric,

This URL supports CORS, but I'm not sure it's the SVG format you're looking for:

https://webservice.wikipathways.org/getPathwayAs?fileType=svg&pwId=WP279

The SVG is Base64 encoded inside an XML file in that response. If that doesn't work, let me know, and I'll see if I can come up with a better solution.

-Anders

To view this discussion on the web visit https://groups.google.com/d/msgid/wikipathways-discuss/c81b6008-eb5c-470a-b06d-d3b2d372900en%40googlegroups.com.

[Eric Weitz]

Anders, nice, does pvjs work with that?  If so, I think that'd suffice for my use case!

Also, after Alex's nod, I set up a separate SVG service.  An experimental version works.  See e.g.:

https://eweitz.github.io/cachome/wikipathways/WP289.svg

The Cachome WikiPathways SVG is simpler, faster and maybe more reliable than the official WikiPathways SVG, but it's also staler, feature-poorer, and more experimental.  Below is a more specific comparison of Cachome vs. official API.

Pros:

• Serves only raw SVG.  No Base64 decoding needed, so byte-for-byte it's faster and simpler to consume.
• Smaller, thus faster.  Response size is ~5x smaller.  WikiPathways SVG is compressed via Scour, then custom lossless and lossy algorithms.
• Perhaps better uptime.  Served via GitHub Pages.  Compare availability statistics for Cachome and official equivalent this time next month.

Cons:

• Content is less up-to-date.  It's a cache, so its data will likely be less current than that in WikiPathways.
• Not as feature rich.  It's simply static files served over HTTP.  It could be enriched via client-side software, but it's not an API in itself.
• Experimental.  I set this up last weekend.  It's incomplete and might not pan out.

Maybe Cachome's lossless SVG compression is interesting to WikiPathways folks, maybe not :).   In any case, I think I've got enough to unblock showing pathway diagrams in gene tooltips for a related genes ideogram!

[Anders Riutta]

Hi Eric,

We're working on an experimental new API endpoint. I'll send you a private message with more details.

-Anders

To view this discussion on the web visit https://groups.google.com/d/msgid/wikipathways-discuss/0935a0a5-1839-4a8d-9bfd-d33bc0bf0165n%40googlegroups.com.