Density plots , combined chromosomes

Bina, Minou

unread,

Dec 12, 2016, 11:29:41 AM12/12/16

to gen...@soe.ucsc.edu

Hello,

I have a set of data that includes density plots to display values along each chromosomes

with bigwig, I was unable to have data for all the chromosomes uploaded at the same time.

My dataset is not large; would you please advice how I could display them without having to load each chromosome separately

I do not have a server;

Minou Bina

Purdue University

Brian Lee

unread,

Dec 16, 2016, 7:20:32 PM12/16/16

to Bina, Minou, gen...@soe.ucsc.edu

Dear Minou,

Thank you for using the UCSC Genome Browser and your question about loading your data.

wig vs. bigWig

If you are using a bigWig file, there will be a server step for the browser to access the file, I have a feeling you may have not generated a bigWig. Binary indexed bigWig files allow the browser to only grab small sections of the file, specified by the regions of the genome a user is viewing, and by necessity must be viewed via a server.

From a previous question, it sounded as though you have a wig file for each chromosome. You can put all that wig data together into one file bigger wig file (not the same a same a bigWig), below is such a combination of wig data with just one data point per chromosome and a track line at the very top to allow adding this as a custom track:

track type=wiggle_0 name=exampleWig
variableStep chrom=chr1 span=1
368244 2
variableStep chrom=chr2 span=1
368250 2
variableStep chrom=chr3 span=1
368256 2
variableStep chrom=chr4 span=1
368262 2
variableStep chrom=chr5 span=1
368268 2

If your dataset is truly not that large, it should work to upload such a text file. However, we would rather have people use bigWigs to view their data when it is quite sizeable. The benefit is that instead of uploading a large file and only viewing a few sections, a bigWig allows the browser to only pass the small sections of data that are currently being viewed. The trick is that this requires the server to accept byte-ranges so just small junks are sent.

To create a bigWig from such an above set of input data named wigInput, where the track line is first removed, and one uses a tool like wigToBigWig available here: http://hgdownload.soe.ucsc.edu/admin/exe/

wigToBigWig wigInput http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/hg19.chrom.sizes out.bw

The resulting bigWig, out.bw, could then be put on a custom track line such as the following:

track type=wiggle_0 name=exampleWig bigDataUrl=http://location/file/out.bw

Or even directly in a link to the browser such as:

http://genome.ucsc.edu/cgi-bin/hgTracks?db=hg19&hgct_customText=http://location/file/out.bw

Hosting data locally on a Virtual Machine called a Genome Browser in a Box (GBiB)

However, the problem would remain on where to host the data in your case, yet there is a solution to avoid the server problem by installing your own version of the Genome Browser on your machine. This Genome Browser in a Box (GBiB) approach may seem a high hurdle, but the benefit is it allows you to directly pass local files to a Virtual Machine and so avoid needing a server. Here is a video blog to learn about what is involved with a GBiB, http://blog.openhelix.eu/?p=2019 and an introduction to starting GBiB: http://genome.ucsc.edu/goldenpath/help/gbib.html#GetStarted

Downloading a GBiB

If you feel up to going down the GBiB path the first steps to using a GBiB is to install the free VirtualBox program that will allow loading the Genome Browser image on your laptop. Here is the link to VirutalBox software, https://www.virtualbox.org/, where on the left-hand side you can click Downloads, and then select “OS X hosts” or “Windows hosts” to pick your platform. Once the download has finished, install the file so it will be ready to launch for the next step.

The next step is to acquire our GBiB from our store, where we require you verify you aren’t going to use the code for commercial uses, and request an email account. To access the store you can go to the top “Downloads” menu in the blue bar and then select "Genome Browser Store”: https://genome-store.ucsc.edu/

At the store, you will want to go click the Register/Login link and register an email address that will require an email verification step. Next go to the first "Genome Browser in a Box (GBiB)” section, and click "Add to cart” and then click the “Cart(1)” Link at the top menu bar. This will take you to a page where you can “Proceed to checkout” and at the checkout, be sure to click the “No” selection under "Are you intending to use this software for commercial use?” and then agree with the terms and conditions.

Now you can click the “Download” button and it will reveal a link you can click. Please be prepared for a long download when you click the link, the file is over 6 gigabytes in size.

Once downloaded, the next steps will depend on your operating system. If you are using an Apple computer double-click on the gbib.zip to extract the file, and then find and double-click the browserbox.vbox file. This will launch the virtual box and load up the machine. On Windows you need further tools to extract and open the tool.

Starting a GBiB and Adding Local Files

In Virtual Box you will click to Start the machine, see http://genome.ucsc.edu/goldenpath/help/gbib.html#UsingGbib, and it may take a moment to update the first time it runs as well. Once it is running a local virtual machine version of GBiB will be available through your machine at: http://127.0.0.1:1234/

With the GBiB running you can start locally serving large files, as you will no longer be using our public site. You can still paste in text for new custom tracks as you have been doing, but you can also tell the GBiB where your bigWigs or other large binary indexed files are located by informing Virtual Box where a shared folder is located: http://genome.ucsc.edu/goldenPath/help/gbib.html#YourTracks

For instance, if you had a file on your desktop like GSM1186795.bw, after using Virtual Box with the GBiB turned off to add your Desktop as a source for shared files you could start up your GBiB again and go to the MyData menu and find the GBiB Shared Data Folder. By clicking through that menu location and opening folders you would find a file that you can copy the URL location to by doing a right-click: http://127.0.0.1:1234/folders/sf_Desktop/GSM1186795.bw

That URL, specific for the shared file in your GBiB could then be used in GBiB with all our tools, and it will be interpreted as though it was coming from a server.

For example, this URL represents the GBiB running and loading the above example track locally from the shared folder on the Desktop:http://127.0.0.1:1234/cgi-bin/hgTracks?db=hg19&position=chrM%3A1-16571&hgct_customText=http://127.0.0.1:1234/folders/sf_Desktop/GSM1186795.bw

However, in the long run, it might be much easier to ask your IT department about the avaiilablity of hosting such files from your institution's server.

Thank you again for your inquiry and using the UCSC Genome Browser. If you have any further questions, please reply to gen...@soe.ucsc.edu. All messages sent to that address are archived on a publicly-accessible forum. If your question includes sensitive data, you may send it instead to genom...@soe.ucsc.edu.

All the best,

Brian Lee
UCSC Genomics Institute

--

Brian Lee

unread,

Dec 16, 2016, 7:38:57 PM12/16/16

to Bina, Minou, gen...@soe.ucsc.edu

Dear Minou,

Here is a working link to the Open Helix blog post about installing a Genome Browser in a Box and the related video:

http://blog.openhelix.eu/?p=20193
https://www.youtube.com/watch?v=1DzZZgB1gvQ

Also, here is a correction about "type=" portion in the track line example for the resulting bigWig, out.bw: