Question about using UCSC genome browser

[SeungJ...@mdc-berlin.de]

Dear UCSC Genome browser,

hi, I am trying to use UCSC Genome browser and I have questions for using it.

My work has been done making BedToBigBed file and I am trying to look at it.

When I went to Custom track and upload my .bb file it comes with this message:

Error File 'ccs1.bb' - It appears that you are directly uploading binary data of type bigBed. Custom tracks of this type require the files to be accessible by public http/https/ftp, and file URLs must be passed as the bigDataUrl setting on a "track" line. See bigBed custom track documentation for more information and examples.

Can you explain it to me how can I access data by http or ftp?
Although I read explanation, I cannot really understand. Some discussion mentioned I have to
upload data file to Dropbox and use it. But I am not sure this is the right way.
Then, is there any way to access data through directly from server?

Also, when I upload my BED file, it works well and directly move to Genome browser.
Can you tell me what is difference between them and also, is it possible to analyze data with BED instead of BB file?

I am stuck in this step and I really want to utilize UCSC for my analyzing data.
Thanks in advance and I look forward to your reply.

Best regards,

Seung Kim



 

[Brian Lee]

Dear Seung Kim,

Thank you for using the UCSC Genome Browser and your question about using bigBed data in the Browser.

Unlike the BED files that you have been successfully uploading to the Browser, bigBeds instead must reside at an internet accessible location, referring to the location as a "bigDataUrl" that is accessed by ftp/http.

How bigBeds work is that only small segments of the files are passed over the internet, so even if your bigBed is several gigabytes in size, only the small section you are currently visualizing in the Browser is sent over the internet.

To do these internet transfers of only small segments of the files, the location that is hosting the bigBed must allow byte range requests. Most free services like DropBox do not enable this kind of access, as it could also result in people hosting pirated video on their free servers.

A free science based file sharing service does exist called CyVerse, sort of like the DropBox of scientific data. By uploading your bigBeds to their file server, you can create "Send to Browser" or "View in Browser" links that allow the file to have byte-range requests enabled.

Please see this previously answered question about using CyVerse:
https://groups.google.com/a/soe.ucsc.edu/d/msg/genome/_Ws2jxNfJV4/0c5aGb_PAQAJ

Once you have a file at CyVerse (or any location with byte-range access, your institution may provide a location as well), you can paste the URL to your track on the Custom Track page.

For example, here is a bigBed at our local server that could be pasted in: http://genome.ucsc.edu/goldenPath/help/examples/bigBedExample.bb

And here is the same file hosted at CyVerse: http://de.cyverse.org/anon-files/iplant/home/brianlee/bigBedExample.bb

Please note you can add additional parameters to your custom track lines, such as defining a name beyond the file name, and giving a description and other attributes, see this link for more information: http://genome.ucsc.edu/goldenPath/help/customTrack.html#TRACK

track type=bigBed name="CV bigBed" description="A bigBed file at Cyverse" color=0,255,255 visibility=full bigDataUrl="http://de.cyverse.org/anon-files/iplant/home/brianlee/bigBedExample.bb"

Thank you again for your inquiry and using the UCSC Genome Browser. If you have any further questions, please reply to gen...@soe.ucsc.edu. All messages sent to that address are archived on a publicly-accessible forum. If your question includes sensitive data, you may send it instead to genom...@soe.ucsc.edu.

All the best,

Brian Lee
UCSC Genomics Institute

Edit Watch

Also available in: Atom PDF

Powered by Redmine © 2006-2016 Jean-Philippe Lang

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "UCSC Genome Browser Public Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to genome+un...@soe.ucsc.edu.
> To post to this group, send email to gen...@soe.ucsc.edu.
> Visit this group at https://groups.google.com/a/soe.ucsc.edu/group/genome/.
> To view this discussion on the web visit https://groups.google.com/a/soe.ucsc.edu/d/msgid/genome/B2AF25D96D54DE4E89697F8383F4136B31267D08%40DAGONE.mdc-berlin.net.
> For more options, visit https://groups.google.com/a/soe.ucsc.edu/d/optout.

[SeungJ...@mdc-berlin.de]

Thanks, now it works well!! Thanks for the advice

Seung Kim

---

보낸 사람: Brian Lee [bria...@soe.ucsc.edu]
보낸 날짜: 2017년 8월 1일 화요일 오후 8:36
받는 사람: Kim, Seung Joon
참조: gen...@soe.ucsc.edu
제목: Re: [genome] Question about using UCSC genome browser

[SeungJ...@mdc-berlin.de]

Hi,

can I ask you about using psl2bed and BedToBigBed as well?
I did alignment using BLAT and the output is psl format.
I want to change to Bed format to see the result on UCSC browser so first I used psl2bed and made a Bed file and I follow the like website to make bb file
http://blog.naver.com/jinp7/221064735045
However, using BedToBigBed the below message comes out

bedToBigBed: Relink `/gnu/store/88wvqp60hbrdvbp0xsqad5c6njjfshcw-libpng-1.6.28/lib/libpng16.so.16' with `/gnu/store/ybpgv1v7606xw7mafda66w10hiynpiw2-glibc-2.25/lib/libpthread.so.0' for IFUNC symbol `longjmp'
pass1 - making usageList (452 chroms): 14763 millis
Error line 1 of sorted_ccs1a.bed: thickStart after thickEnd

Can you explain about the error message?
Also, can you advise me any better way to do from psl -> bb file?

Thank you in advance and I look forward to your reply.

Best wishes,

Seung Kim

---

보낸 사람: Kim, Seung Joon
보낸 날짜: 2017년 8월 2일 수요일 오후 2:20
받는 사람: Brian Lee
참조: gen...@soe.ucsc.edu
제목: 회신: [genome] Question about using UCSC genome browser

[SeungJ...@mdc-berlin.de]

Dear Ann

Hi,I send you because Brian is on vacation.

Can I ask you about using psl2bed and BedToBigBed?

[Hiram Clawson]

Good Morning Seung Kim:

Can you verify if you are using the binaries from:

http://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64/

or are you working in a gBiB virtual machine image ?

You can also use your PSL file directly without the transformation
to bed format:
https://genome.ucsc.edu/goldenpath/help/bigPsl.html

--Hiram

> Thank you again for your inquiry and using the UCSC Genome Browser. If you have any further questions, please reply to gen...@soe.ucsc.edu<mailto:gen...@soe.ucsc.edu>. All messages sent to that address are archived on a publicly-accessible forum. If your question includes sensitive data, you may send it instead to genom...@soe.ucsc.edu<mailto:genom...@soe.ucsc.edu>.

>
> All the best,
>
> Brian Lee
> UCSC Genomics Institute
>
> Edit Watch
>
> Also available in: Atom PDF
>
> Powered by Redmine © 2006-2016 Jean-Philippe Lang
>

> On Tue, Aug 1, 2017 at 5:08 AM, SeungJ...@mdc-berlin.de<mailto:SeungJ...@mdc-berlin.de> <SeungJ...@mdc-berlin.de<mailto:SeungJ...@mdc-berlin.de>> wrote:
>>
>> Dear UCSC Genome browser,
>>
>> hi, I am trying to use UCSC Genome browser and I have questions for using it.
>>
>> My work has been done making BedToBigBed file and I am trying to look at it.
>>
>> When I went to Custom track and upload my .bb file it comes with this message:
>>

>> Error File 'ccs1.bb<http://ccs1.bb>' - It appears that you are directly uploading binary data of type bigBed. Custom tracks of this type require the files to be accessible by public http/https/ftp, and file URLs must be passed as the bigDataUrl setting on a "track" line. See bigBed custom track documentation for more information and examples.

>>
>> Can you explain it to me how can I access data by http or ftp?
>> Although I read explanation, I cannot really understand. Some discussion mentioned I have to
>> upload data file to Dropbox and use it. But I am not sure this is the right way.
>> Then, is there any way to access data through directly from server?
>>
>> Also, when I upload my BED file, it works well and directly move to Genome browser.
>> Can you tell me what is difference between them and also, is it possible to analyze data with BED instead of BB file?
>>
>> I am stuck in this step and I really want to utilize UCSC for my analyzing data.
>> Thanks in advance and I look forward to your reply.
>>
>> Best regards,
>>
>> Seung Kim
>>
>>
>>
>>
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups "UCSC Genome Browser Public Support" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to genome+un...@soe.ucsc.edu<mailto:genome%2Bunsu...@soe.ucsc.edu>.
>> To post to this group, send email to gen...@soe.ucsc.edu<mailto:gen...@soe.ucsc.edu>.

[Cath Tyner]

Hello Seung Kim,

Thank you for contacting the UCSC Genome Browser support team. Below are some options for visualizing your psl results in the browser.

Web-based BLAT: Directly add as custom track

1. We recently released an update to the browser which provides the ability to save web-based BLAT search results as a custom track there is no need to use any utilities. This update is only for the web-based BLAT tool, not command-line BLAT utility.

Command-line BLAT utility

2. It is recommended to use option 3, below, but I wanted to note that the uploading of a psl file is supported as a custom track in the browser. You can simply load your url to your file into custom tracks. For example, at the Custom Tracks tool in the "Paste URLs or data" box, you can paste in "http://hgwdev.cse.ucsc.edu/~cath/temp/smallExample.psl" to see psl results in the browser, as seen in this session.

3. psl to bigPsl

The best way to visualize psl results in the UCSC Genome Browser as acustom track is to first compress your psl file into a binary, indexed bigPsl file for faster performance. While it is possible to convert a psl to a bed file, it's not recommended because you will loose information in the conversion process.

You can follow these detailed instructions to convert psl to bigPsl and then load the bigPsl as a custom track in the browser:

Instructions for converting from psl to bigPsl and uploading a bigPsl custom track

You'll be using several utilities from the utilities directory (pslToBigPsl, bedToBigBed, and fetchChromSizes) to convert your psl file.

A summary of steps, which are detailed in the instructions, are:

1. Download utilities (pslToBigPsl, bedToBigBed, and fetchChromSizes)
2. Use pslToBigPsl to convet your psl file to a sorted bigPsl.txt file.
3. Use 'fetchChromSizes' to get the chrom.sizes files for whichever assembly you need (e.g., get the file hg38.chrom.sizes). 

Note: Using the utility "fetchChromSizes" is not necessary if you instead wish to 1) simply download the needed file from the downloads directory (e.g., here is the hg38.chrom.sizes file for hg38), OR 2) you can simply provide the url to the file in your command. 

4. Follow this link to the needed bigPsl.as file and save the file to the directory you are working in.

You should now have 3 files, 1)bigPsl.as 2)bigPsl.txt 3) hg38.chromsizes. The bedToBigBed utility needs all three of these files to create the output bigPsl.bb file.

5. Use the bedToBigBed utility to create the final bigPsl.bb file, which is the compressed binary version of your psl file.
6. Move the final bigPsl.bb file to a byte-range supported web-accessible server (such as Cyverse).
7. Paste in the url to your bigPsl.bb file in Custom Tracks and load the file into the browser.

Please carefully read the bigPsl instructions page, where you will find detailed examples that you can copy.

If you have problems or questions, please feel free to reply to this forum!

Thank you for contacting the UCSC Genome Browser support team. 

​Please send new and follow-up questions to one of our UCSC Genome Browser mailing lists below:

  * Post to the Public Help Forum: E

mail 

gen...@soe.ucsc.edu

​ or search the Public Archives

​  * Post to the Mirror Help Forum: Email

 genome...@soe.ucsc.edu 

or search the Mirror Archives​

​  * Confidential/private help: Email

 genom...@soe.ucsc.edu

UCSC Genome Browser Announcements List (email alerts for new data & software):

  * Subscribe: Email genome-announce+subscribe...@soe.ucsc.edu 
  * Unsubscribe: Email genome-announce+unsubscri...@soe.ucsc.edu

Join us on Social Media! Facebook, Twitter, Wordpress Blog, YouTube

​Enjoy,​

Cath
. . .

Cath Tyner

UCSC Genome Browser, Software QA & User Support

UC Santa Cruz Genomics Institute

UCSC Genome Browser

To view this discussion on the web visit https://groups.google.com/a/soe.ucsc.edu/d/msgid/genome/B2AF25D96D54DE4E89697F8383F4136B31267E4A%40DAGONE.mdc-berlin.net.

[SeungJ...@mdc-berlin.de]

Thank you for the advice.
I will try using bigPsl and see how it works.
Thank you very much!!!

Best regards,

Seung Kim

---

보낸 사람: Cath Tyner [ca...@ucsc.edu]
보낸 날짜: 2017년 8월 3일 목요일 오후 7:34

받는 사람: Kim, Seung Joon
참조: gen...@soe.ucsc.edu

제목: Re: 전달: [genome] Question about using UCSC genome browser