cleanup of lost tables in customTrash database
Mullan, James Patrick
we are operating a UCSC mirror site here at UCC which includes the facility to add a custom track. I am looking for some help with cleaning up lost tables in the customTrash database.
The customTrash database is located on a different server to where the UCSC browser is located. The clean-gb-ctdb script is configured in a cron job to cleanup the customTrash database.
However we have noticed that the customTrash database seems to be growing and there are 163 lost tables mentioned in the log file for the clean-gb-ctdb script. See below.
cat 2016-04-27.txt
# specified age = 168.000000 hours = 604800 seconds
# current time: 2016-04-27 06:33:20 1461735200
# drop time: 2016-04-20 06:33:20 1461130400
# drop requested: TRUE
# historyToo: FALSE
# extFile: FALSE
# extDel: FALSE
# tableStatus: FALSE
# topDir: /usr/local/apache/trash
# database: customTrash
# drop t1_gwips_4704_6a48a0
# Dropped 1 tables, no size info, 163 lost tables
# pid=27004: VmPeak: 157204 kB
As time goes on the number of lost tables is increasing. I would like to be able to cleanup lost tables older than a week and recover some disk space. Any help with this would be much appreciated.
Regards
James
James Mullan
Brian Lee
Dear James Mullan,
Thank you for running a mirror of the UCSC Genome Browser and your question about a growing number of lost tables regarding your clean-gb-ctdb script.
From the information you provided, it looks as though you are not following our source documentation, but rather a helpful external post here:http://www.hybridstat.gr/index.php/installation-of-ucsc-genome-browser-in-a-local-server/
From that external site, there is a step "E. Setting up the custom track database" that includes a description of the script you name, clean-gb-ctdb, to be used with a cron job to periodically clean the custom tracks database.
While it may be difficult to help in regards to this externally created script, we can direct you to our internal source code, and specifically a directory src/product/scripts where there are some tools that could be of use: http://genome-source.cse.ucsc.edu/gitweb/?p=kent.git;a=tree;f=src/product/scripts
In that directory you will find a README file: http://genome-source.cse.ucsc.edu/gitweb/?p=kent.git;a=blob_plain;f=src/product/scripts/README.trashCleaning;hb=HEAD
The trash cleaning README will direct you to cleanLostTables.sh, which uses the perl script 'lostTables.pl' to identify tables not recorded in the 'metaInfo' table and that have aged enough to confirm they are not in any custom track. Both of these scripts are in the src/product/scripts directory.
Thank you again for your inquiry and using the UCSC Genome Browser. If you have any further questions, please reply to gen...@soe.ucsc.edu. All messages sent to that address are archived on a publicly-accessible forum. If your question includes sensitive data, you may send it insteadtogeno...@soe.ucsc.edu.
All the best,
Brian Lee
UCSC Genomics Institute
--