Re: GBiB is too slow......

[Maximilian Haeussler]

Hi Dahun,

Thank you for your email, I'm sure we can fix this. This is with online-mode, correct? You cannot use offline-mode for some reason, I imagine? I'm curious: if you use offline mode , is the browser fast again? The command is "gbibOffline", documented here: https://genome.ucsc.edu/goldenpath/help/gbib.html#Commands You can switch it back as before with "gbibOnline".

To identify why the GBIB is so slow, can you run these two commands on your GBIB and send us the output:

gbibUcscTablesLog - this will use the Apache log to find the MySQL tables that were loaded from the UCSC public Mysql server since the last reboot. This should give us a good idea why GBIB is so slow

gbibUcscGbdbLog - this will use the Apache log to find the binary files under /data/gbdb that were loaded from the UCSC download server (hgdownload.soe.ucsc.edu). This is less likely to be a problem but could be one.

Also, you can always go to /cgi-bin/hgHubConnect on your GBIB and click the button "Show timings" under "Show load times". Then, the genome browser will output the time it takes to load every track under the track image, the timings will be milliseconds. You can also send us these timings.

thanks!

Max

From: "엄다훈" <dah...@gmail.com>
To: UCSC Genome Browser Discussion List <gen...@soe.ucsc.edu>
Cc: 
Bcc: 
Date: Thu, 10 Mar 2022 18:54:18 +0900
Subject: GBiB is too slow......

Hello, 

In advance, thank you for such a nice tool. It is really helpful in my study. 

Thesedays, genome browser is too slow when using GBiB. 

Also, after 7~10 seconds later, error message came out. 

Memory setting is almost 150GB and I downloaded default tracks in mirror track. But, I don't know why browser is too slow......

Is there any suggestion to deal with this? T.T I think it is slower than original UCSC genome browser now.

Thank you for reading

Best,

Dahun

[엄다훈]

Hi Max,

Thank you for kind explanation!

First, I set the 'Offline' mode already.... but it is too slow.... 

Next, I tried two commands but there is no text. 

And I tried to check the times but I could not enter the /cgi-bin/hgHubConnect. Can you explain in detailed? 

Thank!

Dahun

2022년 3월 10일 (목) 오후 7:04, Maximilian Haeussler <mhae...@ucsc.edu>님이 작성:

[Maximilian Haeussler]

Hi Dahun,  

OK, if it's already in offline mode, it should not be slow. Especially when you're using the default tracks.

I don’t know the address of your mirror, so I just told you the relative address. You can find hgHubConnect in the menu under My Data - Track Hubs. There click "hub development" and then click the button "Show timings" under "Show load times". From then on, the genome browser will output the time it takes to load every track under the track image, the timings will be milliseconds. You can send us these timings.

gbibUcscTablesLog and gbibUcscGbdbLog will only work if your GBIB has been accessed a few times since the last boot, but I imagine that that's the case...

best

Max

[엄다훈]

Thank you Max!

It is faster than before!! Our lab member said that loading of human genome is especially slow. So, I have downloaded mirror tracks(default tracks). After that, it is faster than before!

If I upload many tracks(more than 20 tracks), it must be slow, right? I think I should not upload too many tracks..haha..

Anyway, thank you so much!!

Best,

Dahun

2022년 3월 10일 (목) 오후 10:01, Maximilian Haeussler <mhae...@ucsc.edu>님이 작성:

[Maximilian Haeussler]

Hi Dahun,

>It is faster than before!! Our lab member said that loading of human genome is especially slow. So, I have downloaded mirror tracks(default tracks). After that, it is faster 

> than before!

Great! can you tell us what you did to make it faster? You downloaded (again, so at least for the second time) the "default tracks" ? And you ran "gbibOffline" again (just to make sure that it's really offline) ?

If I upload many tracks(more than 20 tracks), it must be slow, right? I think I should not upload too many tracks..haha..

Anyway, thank you so much!!

It depends on how you upload your tracks: 1) do you paste URLs into the custom track box, 2) do you load track hubs via "My Data > My Hubs" or 3) do you paste the data itself as text into the custom track box? If (1) or (2), how far is the data server away from your GBIB ? If (3) how big is the data?

Usually custom tracks of either type should not slow the browser down, but if you use certain formats in (3) and they're big or if you use data stored on slow servers for (1) and (2), it may get very slow, yes. 

best

Max 

[엄다훈]

Hi max,

Great! can you tell us what you did to make it faster? You downloaded (again, so at least for the second time) the "default tracks" ? And you ran "gbibOffline" again (just to make sure that it's really offline) ?

=> In human genome, 'default track' in mirror track is 'partially downloaded'. So, I tried to download default tracks again. After that, it is faster than before. Yes, I have set the 'gbibOffline' mode now. 

It depends on how you upload your tracks: 1) do you paste URLs into the custom track box, 2) do you load track hubs via "My Data > My Hubs" or 3) do you paste the data itself as text into the custom track box? If (1) or (2), how far is the data server away from your GBIB ? If (3) how big is the data?

=> Actually, I use '1) do you paste URLs into the custom track box'. I don't know exactly the way '2)  do you load track hubs via "My Data > My Hubs" '. In this case, is it slower than the way 2)?? If it is true, I will try to set that 

Also, it is really close from the server(maybe...2~3m?). 

In last year, I asked you about storage of GBiB. I heard that you will develop GBiB for larger capacity(It is limited for 4TB now). Is there updated?? I'm really looking forward to new version!!!

Anyway, thank you for your help! Have a nice weekend!

Best,

Dahun

2022년 3월 11일 (금) 오후 8:09, Maximilian Haeussler <mhae...@ucsc.edu>님이 작성:

[Daniel Schmelter]

Hello Dahun,

Thank you for your interest in the Browser and your patience with this delayed reply.

It sounds like you have already made improvements to your GBIB speed by downloading the data and setting "gbibOffline". Those should help enormously. As Max mentioned, uploading and visualizing large datasets takes time, especially when the server is distant. It sounds like your server is very close and so that shouldn't be an issue. Changing your viewing settings to show less data is one way to increase loading speed.

As far as the different methods to upload custom data, I will say that Custom Tracks and Track Hubs should be similar in speed. Data that is compressed into binary tracks like bigBed and bigWig will be visualized faster.

As far as increasing the storage capacity to more than 4Tb, this is not a feature we have developed yet, but we have noted your request and will add to our queue among our many other engineering priorities.

For reference, here is our GBiB page on increasing speed. There may be options still available to you, such as increasing RAM.

https://genome.ucsc.edu/goldenpath/help/gbib.html#GbibMirroring

I hope this was helpful. If you have any more questions, please reply-all to our public support email at gen...@soe.ucsc.edu. For private communication, please reply-all to genom...@soe.ucsc.edu.

All the best,

Daniel Schmelter

UCSC Genome Browser; UCSC Genomics Institute
Twitter | Facebook | YouTube

--

---
You received this message because you are subscribed to the Google Groups "UCSC Genome Browser Mirror-Specific Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to genome-mirro...@soe.ucsc.edu.
To view this discussion on the web visit https://groups.google.com/a/soe.ucsc.edu/d/msgid/genome-mirror/CABbK9tohcKBoRy8jqKC8m%2ByaYVL6zN808FmuGNkydyS8KSHUGw%40mail.gmail.com.