Mirroring specific Versions and MySQL questions

21 views
Skip to first unread message

Martin

unread,
Oct 23, 2017, 10:40:33 AM10/23/17
to genome...@soe.ucsc.edu
Hello,

I've looking around how to get specific versions or even seeing somewhere which version of UGB is currently running.

Is there some kind of http endpoing (preferrably machine readable) that will allow to read this version?

Also wrt GBIC I am struggling to provide a more cloud-ish installation. Currently the mirroring of MySQL assumes that one has actual access to the MySQL server.

Because the GBIC script[1] does an rsync of the binary data files I don't see how it would be possible to load a migration script from version X to X+1 in "plain SQL". How would one go to have this kind of setup so that I could leverage say AWS RDS instead of having to maintain this.

Are there plain dumps from MySQL available somewhere or will I have to automate that myself?

Something that also caught my eye was that all the data is stil using MyISAM which, IIRC, has been the default in the past but was switched away from for quite some time because it won't give you relational integrity. Is there any roadmap to switching to a better storage engine?

Thanks,
Martin


Christopher Lee

unread,
Oct 23, 2017, 12:48:58 PM10/23/17
to mar...@marcher.name, genome-mirror

Hi Martin,

Thank you for your questions about the GBiC installer. Firstly, please note that a license is required to obtain the GBiC software for commercial use:
https://genome-store.ucsc.edu/

Can you explain further what you mean by "version of UCSC GB is currently running"? Do you mean the version of the software behind the browser, the version of the data in the tables, or the MySQL version we are running? We update the CGI's on a roughly 3 week schedule, and we update MySQL tables whenever we release a new dataset, which could be every day/week/month/etc, and you can determine the MySQL version we are running via the following command:

mysql --user=genome --host=genome-mysql.soe.ucsc.edu \
-A -e 'show variables;' hg38 | grep -w version
version 5.6.26-log

Can you also explain further what you mean by 'load a migration script from version X to X+1 in "plain SQL"'? What are you migrating in this case?

You can find SQL dumps for an individual database, for example hg38, here:
http://hgdownload.cse.ucsc.edu/goldenPath/hg38/database/

One of our engineers adds that "We use MyISAM because it is simple and trivial to rsync tables directly from one running instance of the MySQL server to a second running instance of a MySQL server without any interruption on either side."

Please let us know if you have any further questions!

Thank you again for your inquiry and using the UCSC Genome Browser. If
you have any further questions, please reply to gen...@soe.ucsc.edu.
All messages sent to that address are archived on a
publicly-accessible forum. If your question includes sensitive data,
you may send it instead to genom...@soe.ucsc.edu.

Christopher Lee
UCSC Genomics Institute


--

---
You received this message because you are subscribed to the Google Groups "UCSC Genome Browser Mirror-Specific Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to genome-mirror+unsubscribe@soe.ucsc.edu.
To post to this group, send email to genome...@soe.ucsc.edu.
Visit this group at https://groups.google.com/a/soe.ucsc.edu/group/genome-mirror/.
To view this discussion on the web visit https://groups.google.com/a/soe.ucsc.edu/d/msgid/genome-mirror/CAK1mKES5rMRwSp_EKA0UNe02XkdYA-Wv63zLdBH0S9-xb1rpow%40mail.gmail.com.
For more options, visit https://groups.google.com/a/soe.ucsc.edu/d/optout.

Martin

unread,
Oct 24, 2017, 11:38:27 AM10/24/17
to Christopher Lee, genome-mirror
Hello,

The license has been purchased way back in 2016 by my client

By version I mean that the composition of the 3 components you mentioned (web, tools, data) should be versioned in some way so one can refer to it when asking questions about this. It would be enough if there was some kind of information that said:

* Web Frontend Version: X
* Tools (CGIs) Version: Y
* Database Version: Z

I was trying to provide information so you know which version I'm referring to.

By migration script I mean a rake task or flywaydb or any other of the tools that do schema migrations. Even plain SQL files having the correct statements. An engineer will be familiar those terms.

The problem I am facing:

1. I am running MySQL on RDS, that means that no shell access to the server is possible
2. I want to update from the currently present MySQL data version on the MySQL RDS to the version you are providing while keeping the custom tracks and other local data present

How would I do that?

Thanks,
Martin

Brian Lee

unread,
Oct 31, 2017, 3:21:02 PM10/31/17
to mar...@marcher.name, Christopher Lee, genome-mirror
Dear Martin,

Thank you for using the UCSC Genome Browser and your administrative mirror questions. If you are using the browser code for commercial purposes, could you please share more details about your commercial license holder, after looking at our records it isn't clear who is your client that purchased a GBiC installation in 2016.

You may want to search through our archives of previously answered mirror installation questions to learn more about our system: https://groups.google.com/a/soe.ucsc.edu/forum/#!forum/genome-mirror

Also please be aware of our mirror documentation: 

Thank you again for your inquiry and using the UCSC Genome Browser. If you have any further questions, please reply to genome...@soe.ucsc.edu. All messages sent to that address are archived on a publicly-accessible forum. If your question includes sensitive data, you may send it instead to genom...@soe.ucsc.edu.

All the best,

Brian Lee
UC Santa Cruz Genomics Institute

Reply all
Reply to author
Forward
0 new messages