Trouble using local .json data

[snipa org]

Hi,
We are running the SNiPA webserver which is a tool for annotating and browsing genetic variants.

We recently decided to move our visualization (see attached img) to LocusZoom.
I successfully loaded the library and included an example plot using the "umich" api data.
However, I struggle to use our own association data instead of the 'https://portaldev.sph.umich.edu/api/v1/' api.
On pageload we supply our old plot with data for the specified region. I transformed that data to look like the response from the api. Now I'm trying to supply that json to LocusZoom.
Specifically, when replacing
.add('assoc', ['AssociationLZ', { url: apiBase + 'statistic/single/', source: 45, id_field: 'variant' }])
with
.add('assoc', [' AssociationLZ ', { url:  'tmpdata/' + randid + '/plot_snps_annotations.json', source: null, id_field: 'variant' }])
The supplied plot_snps_annotations.json is located on our server and built like the API response json.
But when used with 'AssociationLZ ' the path adds "results/?filter=analysis in null and chrom..."
When used with 'StaticSource' a console error of "must provide data as required option 'config.data' " is thrown.
In the end I would also like to replace the LD api with our own LD data which is a mix of 1kG and TopLD data.

Thanks in advance for any pointers!
Nick

PS: The wiki page for a "simple GWAS template" is a 404 redirect

[Andy Boughton]

Thanks for the question.

I'd offer several notes: 

• First, AssociationLZ is intended to talk to one very specific web server, which does indeed have its own ideas about how data is arranged (source code). It's not meant to be a broad drop in for "any sort of assoc data". (the demo offline mode does use a json file, and it sort of hacks around the question by treating the extra path segments as part of a query string- pretty crude trick!)
• StaticSource does indeed require a config option called "data" (much like you'd pass `url` to the `AssociationLZ` adapter). See docs
• Personally, if you don't want to write a full api server, but still want dynamic plots, then I recommend a third approach: our tabix tracks demo (see: source code, docs, generic line parsers) can read data directly from a tabixed file. The problem with reading a ginormous json file all at once is that it doesn't scale to the whole gwas, whereas querying a tabix-indexed file lets you see any region dynamically. Loading only the data you need is way more efficient. Since so many researchers already use tabix, it also tends to be way less work for you to get data into the format you need!
• The demo includes detailed instructions on how to prepare the dataset to work with this approach

Our my.locuszoom.org site hosts many user-uploaded GWAS for public sharing, and we use a variant of the tabix strategy to serve user datasets. This makes features like the "go to region" search box possible (demo). (Our api reads tabix files on the backend, not the frontend, because we wanted to enforce access controls... but conceptually, it's the same basic idea.) We generate region data for any arbitrary region as needed, rather than preparing pre-set json blobs in advance.

-Andy Boughton
abo...@umich.edu

Applications Programmer/Analyst, Lead
Center for Statistical Genetics
University of Michigan

<snipa_browser.jpg>
Thanks in advance for any pointers!
Nick

PS: The wiki page for a "simple GWAS template" is a 404 redirect

--
You received this message because you are subscribed to the Google Groups "locuszoom" group.
To unsubscribe from this group and stop receiving emails from it, send an email to locuszoom+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/locuszoom/62036633-4664-4c05-b74c-afcc61334431n%40googlegroups.com.
<snipa_browser.jpg>

[snipa org]

Hi Andy,
First of all, thank you for the fast reply!
I am now implementing your suggestion about using tabix files.
After preparing a tabix file in the backend and following the tabix track demo, I am now having a hard time getting it recognized in my LocusZoom plot.
Since the demo uses an URL call to url_data: amazonaws.com I tried to get it to recognize my local tabix.gz file by providing a /path/to/tabix.gz, but failed.
You mentioned that on locuszoom.org you read tabix files on the backend, is that basically what i need to do? And is there a piece of code you can point me to how this is done?
Thank you again for you help!

[Andy Boughton]

You're on the right track!

It looks like you're trying to read an HTML file on your hard drive, and asking that HTML file to read another file, also on your local hard drive? The "direct approach" won't work here, due to a major security feature of your web browser that prevents JS code from reading the local filesystem directly. (imagine if an ad could run code to steal your private  files: very bad!)

There is a way to make this work for local development, which involves running a local server from the folder containing your files (and using relative paths, like `webroot/tabix/myfile.gz`). See this article for helpful tips about an example server that most bioinformatics people probably already have installed:

https://realpython.com/python-http-server/

Once you deploy to production, this isn't a problem- you're always running a web server anyway**. But it does cause confusion when testing locally!

** If you are running your web server on one host, but your static files on a different provider like Amazon S3 or Google Cloud Storage, note that you may need to configure some additional request headers before Tabix will work properly (due to CORS). If the web page and the tabix files are on the same host (like https://snipa.org/), then you likely won't encounter this problem. Core message: browsers have lots of gnarly security rules, and when in doubt, check the JS devtools console for error messages!

 

-Andy Boughton
abo...@umich.edu

Applications Programmer/Analyst, Lead
Center for Statistical Genetics
University of Michigan

To view this discussion on the web visit https://groups.google.com/d/msgid/locuszoom/8161de75-4eba-4314-b2e0-9021369f4be4n%40googlegroups.com.

[snipa org]

The tabix file is read successfully and displayed in the plot!
Sorry to bother you again, but I'm having difficulties providing extra information through the tooltip and I am not that familiar with javascript unfortunatly.
Our current tooltip (see below) adds extra annotation for a given snp/position. I'm trying to pass this annotation from the tabix.gz file to LZ makeGWASParser, but failed.
I saw in some examples on how to customize the tooltip within layouts via tooltip: { html: "..." }, but they take the "expected" fields like pvalue.
The tabix.gz could provide the annotation in ready-to-use HTML format or in separate columns.
Basically, what I am asking is how to get extra (unexpected) columns from the tabix file into the tooltip.



I was also wondering if it is possible to filter between certain values, for example in allele frequency (0.001< show this <0.1)
I found this snippet I wanted to adapt:
config.toolbar.widgets.push( { type: 'filter_field', position: 'right', layer_name: 'assoc', field: 'assoc:allele_freq_col', field_display_html: 'MAF', operator: '>=', data_type: 'number', }

Thank you again for your support!
Nick

[Andrew Boughton]

Short version: for extra columns, you will need a custom parser function that returns all fields

If you look at the source code for the parser, you will notice it ONLY handles the provided columns. This is because the parser was designed to match our "share harmonized gwas" website (my.locuszoom.org), and we were very strict about only handling fields we knew we could harmonize across datasets. 

Then, you will be able to modify the tooltip template to use a field with the same name as what your parser returns.

It's been a few years since I used the filter field widget, but it should be possible to extend the behavior to filter within a range. Iirc the allowed operators are extensible via the LocusZoom.MatchFunctions registry.

https://statgen.github.io/locuszoom/docs/api/module-LocusZoom_MatchFunctions.html

-Andy Boughton

abo...@umich.edu

On Jun 21, 2024, at 8:45 AM, snipa org <serbu...@gmail.com> wrote:

The tabix file is read successfully and displayed in the plot!

To view this discussion on the web visit https://groups.google.com/d/msgid/locuszoom/de0b6ce2-db3e-429e-bc68-daf36ac37927n%40googlegroups.com.