New hubClone.c program to copy architecture of a remote hub locally

[Brian Lee]

For track hub developers who might want to copy a remote track hub architecture, there is a new tool built within the kent tool system that can make a local copy called hubClone: https://genome-source.gi.ucsc.edu/gitlist/kent.git/raw/master/src/hg/utils/hubClone/hubClone.c

This tool was originally created to help temporarily debug problems reported by users who are developing hubs. For example, if there was a remote assembly hub and one wanted to make a local copy of the trackDb architecture, one could provide hubClone the URL to the hub.txt and replicate a working copy.  Of note, however, almost all of the files for the hub data would still be remote, and pointed to with URLs. For instance, here is a plant assembly hub, https://genome-test.gi.ucsc.edu/~hiram/hubs/Plants/hub.txt, which in turn points to a genomes.txt, https://genome-test.gi.ucsc.edu/~hiram/hubs/Plants/genomes.txt, and then several trackDb.txt files and related assembly hub support files.  Here is the usage of hubClone to copy this hub:

hubClone http://genome-test.gi.ucsc.edu/~hiram/hubs/Plants/hub.txt 

The result is a copy of the hub architecture in a local directory, but wherever there was a local reference to a file, such as in the above genomes.txt for this stanza for a Arabidopsis assembly hub, the remote locations of the file data are substituted in, so the copy of the hub will continue to work in the UCSC Genome Browser when the new hub.txt URL is pasted in:

Original assembly hub stanza in the genomes.txt:

genome araTha1

trackDb araTha1/trackDb.txt

groups groups.txt

description Feb. 2011 Thale cress

twoBitPath araTha1/araTha1.2bit

organism Arabidopsis thaliana

defaultPos chr1:1000000-2000000

orderKey 4700

scientificName Arabidopsis thaliana

htmlPath araTha1/description.html

New hubClone generated copy of the genomes.txt:

genome araTha1

description Feb. 2011 Thale cress

defaultPos chr1:1000000-2000000

scientificName Arabidopsis thaliana

htmlPath http://genome-test.gi.ucsc.edu/~hiram/hubs/Plants/araTha1/description.html

organism Arabidopsis thaliana

orderKey 4700

groups http://genome-test.gi.ucsc.edu/~hiram/hubs/Plants/groups.txt

twoBitPath http://genome-test.gi.ucsc.edu/~hiram/hubs/Plants/araTha1/araTha1.2bit

trackDb araTha1/trackDb.txt

The one exception for the remote files is that local version of the trackDb.txt is created. This tool was primarily created to help UCSC create a temporary copy of a hub from a user emailing for help so that we could locally debug trackDb.txt errors with small edits to the user's existing trackDb.txt.  The result is an ability to report to the user a few key changes that might fix the problem they were experiencing.  The hubClone tool also can be a way to copy a working hub and then slowly substitute in local files and local data as a starting point to reformat an existing hub architecture for a new hub with new data.