Brower migration to new server

[McCully, Dwayne (NIH/NIAMS) [C]]

Hello Genome-Mirror support,

 

I need to migrate from an existing installation of UCSC Genome Browser to a new server and keep my existing

Information in tact (users, custom tracks, on site gnomes, setup, etc.).  

 

Do you have instructions for such a task?

 

I’m have been successfully using the browser GBIC browserSetup.sh script for many years and wish to continue.

 

Dwayne

 

Dwayne McCully

NIAMS, NIH

Contractor

[McCully, Dwayne (NIH/NIAMS) [C]]

[Maximilian Haeussler]

Hi Dwayne,

You may want to "clean" the machine first, with the "browserSetup.sh clean" command, but on the other hand, if your users have not saved their custom tracks into user accounts/session yet, then you may not want to remove old temporary files under "customTrash", so it's probably safer now not to run "clean" on the old server.

The user accounts and custom assemblies are stored in the hgcentral database (as you know). The custom tracks are stored in the customTrash database and also as files under trash/ct.

So the following should work to copy over sessions and custom tracks and local assemblies, I have tested this only partially, with a single user account and single custom track today, migrating a Rocky8 mirror to one running on Ubuntu 23:

mysqldump customTrash | gzip -c > customTrash.sql.gz

mysqldump hgcentral | gzip -c > hgcentral.sql.gz

tar cvfz ct.tar.gz /usr/local/apache/trash/ct/*

copy over the three files to the new machine, then run on this new machine:

zcat customTrash.sql.gz | mysql customTrash

zcat hgcentral.sql.gz | mysql central

tar xvfz ct.tar.gz -C /

Make sure that the directory and the files in /usr/local/apache/trash/ct/* are owned by the apache user on the new machine, e.g. on Ubuntu:

chown -R www-data:www-data /usr/local/apache/trash/ct

When you go to the new machine, you should be able to login with the same usernames and passwords, and the sessions should be the same.

Note that if your new server has a new hostname AND your users saved data without creating username/passwords, then these users will get a new session identifier (in a cookie and also the hgsid in the URL), as the old cookie cannot be shared across servers, new hgsid values will be created and since their data is not associated to a username, but only a hgsid session identifier, their custom track will not be visible. You can still find these old hgsids in the sessionDb table and point them to hgsid=<id>_<sessionKey> created from that table, but I don't think we want to get into that...

To copy over your custom assemblies, you have to copy over /gbdb/<assemblyId> and also the <assemblyId> mysql databases, you can use rsync for that.

Let us know if this works, we should probably document it.

best

Max

--

---
You received this message because you are subscribed to the Google Groups "UCSC Genome Browser Mirror-Specific Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to genome-mirro...@soe.ucsc.edu.
To view this discussion on the web visit https://groups.google.com/a/soe.ucsc.edu/d/msgid/genome-mirror/DM8PR09MB6104E920D570959EF3CD8BC0E7452%40DM8PR09MB6104.namprd09.prod.outlook.com.

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Max,

 

Before starting the below tasks.

 

1. Should I run the “browserSetup.sh install” command on the new server or copy everything over?
2. If a copy, what RPM’s will I need (Apache, Mariadb, etc)?
1. It used to also include ghostscript rsync ImageMagick R-core MySQL-python curl
3. Should I use the existing hg.conf file on the new server? 

 

 

 

BTW:  We are using RHEL 9 as the life span of this server and operating system will need to last at least 5-6 years.

            I have seen genome-mirror e-mail about mirror sites using Rocky 9.  

 

zcat hgcentral.sql.gz | mysql central   ß  did you mean hgcentral?

 

 

Dwayne

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.

 

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Max,

 

For item 1. I should have asked to run the browserSetup.sh install and then copy or just copy?

 

Dwayne

[Maximilian Haeussler]

1. Should I run the “browserSetup.sh install” command on the new server or copy everything over?

Yes, because this will take care of installing all the requirements and it will create the configuration for your new OS. E.g. mysql locations may be different on the new OS or socket file names.

 

1. If a copy, what RPM’s will I need (Apache, Mariadb, etc)?

I'd let the browserSetup.sh script do that.

 

> Should I use the existing hg.conf file on the new server?

The "install" command will create one. I think you can also use your old one... unless the mysql socket file location has changed. Probably the new one is easier, unless you made changes to it.

BTW:  We are using RHEL 9 as the life span of this server and operating system will need to last at least 5-6 years.

            I have seen genome-mirror e-mail about mirror sites using Rocky 9.  

Yes, we're running rocky9 here and I changed the installation script last week a little for rocky9, but it worked.

  

zcat hgcentral.sql.gz | mysql central   ß  did you mean hgcentral?

Yes, sorry! Was a typo. It's "mysql hgcentral". 

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Max,

 

Thanks for clearing up a few questions.

 

“Yes, we're running rocky9 here and I changed the installation script last week a little for rocky9, but it worked.”

 

Which URL should I use for the wget command?

 

Dwayne

 

 

From: Maximilian Haeussler <mhae...@ucsc.edu>
Sent: Thursday, February 8, 2024 10:09 AM
To: McCully, Dwayne (NIH/NIAMS) [C] <dmcc...@mail.nih.gov>
Cc: genome...@soe.ucsc.edu

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.

 

[Maximilian Haeussler]

Ohh, is the URL not clear? I see... we're pointing to genome-store.

The question for the URL is excellent, thanks, yes, use this URL:

https://raw.githubusercontent.com/ucscGenomeBrowser/kent/master/src/product/installer/browserSetup.sh

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Max,

 

Ran into an issue with the browerSetup.sh script. 

 

[root ~]# subscription-manager repos --enable codeready-builder-for-rhel-9-$(arch)-rpms

Repository 'codeready-builder-for-rhel-9-x86_64-rpms' is enabled for this system.

[root ~]# dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm

Updating Subscription Management repositories.

Red Hat CodeReady Linux Builder for RHEL 9 x86_64 (RPMs)                                                                 13 MB/s | 6.6 MB     00:00

epel-release-latest-9.noarch.rpm                                                                                        148 kB/s |  19 kB     00:00

Dependencies resolved.

========================================================================================================================================================

Package                                Architecture                     Version                           Repository                              Size

========================================================================================================================================================

Installing:

epel-release                           noarch                           9-7.el9                           @commandline                            19 k

 

Transaction Summary

========================================================================================================================================================

Install  1 Package

 

Total size: 19 k

Installed size: 26 k

Is this ok [y/N]: y

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction

  Preparing        :                                                                                                                                1/1

  Installing       : epel-release-9-7.el9.noarch                                                                                                    1/1

 Running scriptlet: epel-release-9-7.el9.noarch                                                                                                    1/1

Many EPEL packages require the CodeReady Builder (CRB) repository.

It is recommended that you run /usr/bin/crb enable to enable the CRB repository.

 

  Verifying        : epel-release-9-7.el9.noarch                                                                                                    1/1

Installed products updated.

 

Installed:

  epel-release-9-7.el9.noarch

 

Complete!

 

[root@niamsapp02 ~]# /usr/bin/crb enable

Enabling CRB repo

Repository 'codeready-builder-for-rhel-9-x86_64-rpms' is enabled for this system.

CRB repo is enabled and named: codeready-builder-for-rhel-9-x86_64-rpms

 

 

Updating Subscription Management repositories.

Error: No matching repo to modify: powertools.

Exit error 1 occurred on line 1

The UCSC Genome Browser installation script exited with an error.

Please contact us at genome...@soe.ucsc.edu and send us an output log

of the command prefixed with "bash -x", e.g.

bash -x browserSetup.sh install 2>&1 > install.log

 

Any idea how to fix?

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Max,

 

I found the problem for running on RHEL 9. 

 

[root  ~]# grep -i powertool browserSetup.sh

    # try to activate the powertools repo. Exists on CentOS and Rocky but not Redhat

    yum config-manager --set-enabled powertools

[root ~]#

 

Dwayne

 

From: McCully, Dwayne (NIH/NIAMS) [C]
Sent: Thursday, February 8, 2024 11:10 AM
To: Maximilian Haeussler <mhae...@ucsc.edu>
Cc: genome...@soe.ucsc.edu

[Maximilian Haeussler]

Yes sorry, I know this repo doesn’t exist on Rhel but I couldn’t test it, we don’t have RHEL licenses at the University. I’ll try to add an if-then-else around it. For now, can you remove this line and restart ? 

[McCully, Dwayne (NIH/NIAMS) [C]]

Hello Genome-Mirror support,

 

Running the “bash browserSetup.sh update” gets stuck trying to download “gnomeAD/v3.1.1/genome.bb.

I’ve tried several times running the update but always get stuck at this point.   Any help would be appreciated in getting

past this point in the update.

 

 

 

Dwayne

 

Dwayne McCully

NIAMS, NIH

Contractor

301-471-4324

[Christopher Lee]

Hi Dwayne,

That file is just enormous, as listed here:

http://hgdownload.soe.ucsc.edu/gbdb/hg38/gnomAD/v3.1.1/

Are you sure it is actually stuck, or just downloading? Also it probably shouldn't be downloading in the first place I suppose, maybe we can fix that. Unless you want the track locally?

Thanks,

Christopher Lee

UCSC Genomics Institute

--

---
You received this message because you are subscribed to the Google Groups "UCSC Genome Browser Mirror-Specific Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to genome-mirro...@soe.ucsc.edu.

To view this discussion on the web visit https://groups.google.com/a/soe.ucsc.edu/d/msgid/genome-mirror/BLAPR09MB6100F0DAC386310619AF380EE7052%40BLAPR09MB6100.namprd09.prod.outlook.com.

[Christopher Lee]

Hi Dwayne,

I think this means the permissions of the file are wrong on your end right? They seem to be ok on our end as of this morning:

$ rsync rsync://hgdownload.soe.ucsc.edu/mysql/hg18/bigFiles.*
-rw-rw----         11,496 2024/04/08 06:22:22 bigFiles.MYD
-rw-rw----         10,240 2024/04/08 06:22:22 bigFiles.MYI
-rw-rw----          1,463 2024/04/08 06:22:22 bigFiles.frm

On Fri, Apr 12, 2024 at 5:00 AM McCully, Dwayne (NIH/NIAMS) [C] <dmcc...@mail.nih.gov> wrote:

Hi Christopher,

 

You were correct about the size of the file.   I have run into a new problem that I believe is on your side of the browserSetup.sh script.

Seems I’m unable to download some of the hg18 mysql/mariadb files on your site.

 

Dwayne

 

sent 61 bytes  received 96,390 bytes  38,580.40 bytes/sec

total size is 2,244,561,167,080  speedup is 23,271,517.84

| syncing gbdb: sacCer3

receiving incremental file list

 

sent 36 bytes  received 1,660 bytes  1,130.67 bytes/sec

total size is 195,369,701  speedup is 115,194.40

| syncing full mysql database: hg18

receiving incremental file list

rsync: [sender] send_files failed to open "/hg18/bigFiles.MYD" (in mysql): Permission denied (13)

rsync: [sender] send_files failed to open "/hg18/bigFiles.MYI" (in mysql): Permission denied (13)

rsync: [sender] send_files failed to open "/hg18/bigFiles.frm" (in mysql): Permission denied (13)

rsync: [sender] send_files failed to open "/hg18/tableList.MYD" (in mysql): Permission denied (13)

rsync: [sender] send_files failed to open "/hg18/tableList.MYI" (in mysql): Permission denied (13)

rsync: [sender] send_files failed to open "/hg18/tableList.frm" (in mysql): Permission denied (13)

 

sent 15,818 bytes  received 809,224 bytes  235,726.29 bytes/sec

total size is 814,919,500,095  speedup is 987,730.93

rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1823) [generator=3.2.3]

Exit error 23 occurred on line 1

The UCSC Genome Browser installation script exited with an error.

Please contact us at genome...@soe.ucsc.edu and send us an output log

of the command prefixed with "bash -x", e.g.

bash -x browserSetup.sh install 2>&1 > install.log

 

 

 

 

From: 'Christopher Lee' via UCSC Genome Browser Mirror-Specific Support <genome...@soe.ucsc.edu>
Sent: Thursday, April 11, 2024 6:23 PM
To: McCully, Dwayne (NIH/NIAMS) [C] <dmcc...@mail.nih.gov>

To view this discussion on the web visit https://groups.google.com/a/soe.ucsc.edu/d/msgid/genome-mirror/CAC7r7-JD1fo8frg%2Bcs%2BCr%2BDM%3Dm%3DHMEGisSYE%2B3_RRS-%2BYJ-hhQ%40mail.gmail.com.

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Dwayne,

To view this discussion on the web visit https://groups.google.com/a/soe.ucsc.edu/d/msgid/genome-mirror/CAC7r7-JD1fo8frg%2Bcs%2BCr%2BDM%3Dm%3DHMEGisSYE%2B3_RRS-%2BYJ-hhQ%40mail.gmail.com.

[Christopher Lee]

When did you try the download that failed? There was a power outage at UCSC yesterday that you may have just happened to notice?

On Fri, Apr 12, 2024 at 11:15 AM McCully, Dwayne (NIH/NIAMS) [C] <dmcc...@mail.nih.gov> wrote:

Hi Christopher,

 

This might be a problem with the browserSetup.sh script as I can download the files to the mysql

directory hg18 using your rsync command without issue.

 

Can you try with the browserSetup.sh script?

 

Dwayne

[Christopher Lee]

No this is a permissions problem on our end, you are totally right.
I'll get this fixed up and let you know when it's fixed.

On Fri, Apr 12, 2024 at 11:29 AM McCully, Dwayne (NIH/NIAMS) [C]
<dmcc...@mail.nih.gov> wrote:
>
> Hi Christopher,
>
>
>
> Do I need to run a newer version of the browserSetup.sh script?

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Christopher,

 

This might be a problem with the browserSetup.sh script as I can download the files to the mysql

directory hg18 using your rsync command without issue.

 

Can you try with the browserSetup.sh script?

 

Dwayne

 

From: Christopher Lee <chm...@ucsc.edu>
Sent: Friday, April 12, 2024 11:53 AM

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Christopher,

 

I have received other files from running the browserSetup.sh update command.  Directory and file permissions look correct.  

 

 

-rw-rw-r--   1 mysql mysql       2032 Apr  2 11:03 geneReviews.frm

-rw-rw-r--   1 mysql mysql      46472 Apr  2 11:03 geneReviews.MYD

-rw-rw-r--   1 mysql mysql      46080 Apr  2 11:03 geneReviews.MYI

-rw-rw-r--   1 mysql mysql       1533 Apr  2 11:03 geneReviewsDetail.frm

-rw-rw-r--   1 mysql mysql     149292 Apr  2 11:03 geneReviewsDetail.MYD

-rw-rw-r--   1 mysql mysql       1024 Apr  2 11:03 geneReviewsDetail.MYI

-rw-rw-r--   1 mysql mysql       1506 Apr  6 05:03 tableDescriptions.frm

-rw-rw-r--   1 mysql mysql     163840 Apr  6 05:03 tableDescriptions.MYI

-rw-rw-r--   1 mysql mysql    2647592 Apr  6 05:03 tableDescriptions.MYD

drwxr-xr-x.  2 mysql mysql    1073152 Apr  8 09:22 .

drwxr-xr-x. 15 mysql mysql       4096 Apr 12 14:18 ..

 

Dwayne

 

 

From: Christopher Lee <chm...@ucsc.edu>
Sent: Friday, April 12, 2024 11:53 AM

[McCully, Dwayne (NIH/NIAMS) [C]]

I’ve just tried it again and got the same results.    Its only happening for the hg18 database.

 

Fri Apr 12 02:22:35 PM EDT 2024

[root@niamsapp02 ~]# bash browserSetup.sh update

 

receiving incremental file list

./

encode/

greatData/

hgCgiData/

hgCgiData/Human/

hgCgiData/Mouse/

hgCgiData/Rat/

hgCgiData/Zebrafish/

hgGeneData/

hgGeneData/C_elegans/

hgGeneData/C_elegans/ce2/

hgGeneData/C_elegans/ce4/

hgGeneData/C_elegans/ce6/

hgGeneData/D_melanogaster/

hgGeneData/D_melanogaster/dm1/

hgGeneData/D_melanogaster/dm2/

hgGeneData/D_melanogaster/dm3/

hgGeneData/Human/

hgGeneData/Human/hg15/

hgGeneData/Human/hg16/

hgGeneData/Human/hg17/

hgGeneData/Human/hg18/

hgGeneData/Human/hg19/

hgGeneData/Human/hg38/

hgGeneData/Mouse/

hgGeneData/Mouse/mm10/

hgGeneData/Mouse/mm3/

hgGeneData/Mouse/mm39/

hgGeneData/Mouse/mm4/

hgGeneData/Mouse/mm5/

hgGeneData/Mouse/mm6/

hgGeneData/Mouse/mm7/

hgGeneData/Mouse/mm8/

hgGeneData/Mouse/mm9/

hgGeneData/Rat/

hgGeneData/Rat/rn2/

hgGeneData/Rat/rn3/

hgGeneData/Rat/rn4/

hgGeneData/S_cerevisiae/

hgGeneData/S_cerevisiae/sacCer1/

hgGeneData/S_cerevisiae/sacCer2/

hgGeneData/S_cerevisiae/sacCer3/

hgGeneData/Zebrafish/

hgGeneData/Zebrafish/danRer3/

hgNearData/

hgNearData/C_elegans/

hgNearData/C_elegans/ce2/

hgNearData/C_elegans/ce4/

hgNearData/C_elegans/ce6/

hgNearData/D_melanogaster/

hgNearData/D_melanogaster/dm1/

hgNearData/D_melanogaster/dm2/

hgNearData/D_melanogaster/dm3/

hgNearData/Human/

hgNearData/Human/hg17/

hgNearData/Human/hg18/

hgNearData/Human/hg19/

hgNearData/Human/hg38/

hgNearData/Mouse/

hgNearData/Mouse/mm5/

hgNearData/Mouse/mm6/

hgNearData/Mouse/mm7/

hgNearData/Mouse/mm8/

hgNearData/Mouse/mm9/

hgNearData/Rat/

hgNearData/Rat/rn3/

hgNearData/Rat/rn4/

hgNearData/S_cerevisiae/

hgNearData/Zebrafish/

hgNearData/Zebrafish/danRer2/

hgNearData/Zebrafish/danRer3/

hgPhyloPlaceData/

hgPhyloPlaceData/GCF_000855545.1/

hgPhyloPlaceData/GCF_002815475.1/

hgPhyloPlaceData/GCF_014621545.1/

hgPhyloPlaceData/RGCC_A/

hgPhyloPlaceData/RGCC_B/

hgPhyloPlaceData/wuhCor1/

hgcData/

hgt/

hgtIdeo/

hgtSide/

loader/

pyLib/

pyLib/MySQLdb/

pyLib/__pycache__/

pyLib/pymysql/

pyLib/pymysql/__pycache__/

pyLib/pymysql/constants/

pyLib/pymysql/constants/__pycache__/

visiGeneData/

 

sent 2,379 bytes  received 24,064 bytes  17,628.67 bytes/sec

total size is 5,574,952,712  speedup is 210,829.06

| Updating Apache htdocs

receiving incremental file list

./

FAQ/

Neandertal/

RNA-img/

admin/

cauldron/

covid/

ebolaPortal/

encodeDCC/

evoFold/

evoFoldV2/

geneExtra/

goldenPath/

goldenPath/FAQ/

goldenPath/admin/

goldenPath/certificates/

goldenPath/customTracks/

goldenPath/customTracks/help/

goldenPath/customTracks/hg17/

goldenPath/customTracks/hg18/

goldenPath/customTracks/mm9/

goldenPath/help/

goldenPath/help/example/

goldenPath/help/examples/

goldenPath/help/examples/barChart/

goldenPath/help/examples/barChart/singleCell/

goldenPath/help/examples/decorator/

goldenPath/help/examples/hubDirectory/

goldenPath/help/examples/hubDirectory/hg19/

goldenPath/help/examples/hubExamples/

goldenPath/help/examples/hubExamples/hubAssembly/

goldenPath/help/examples/hubExamples/hubAssembly/plantAraTha1/

goldenPath/help/examples/hubExamples/hubAssembly/plantAraTha1/araTha1/

goldenPath/help/examples/hubExamples/hubAssembly/plantAraTha1/araTha1/bbi/

goldenPath/help/examples/hubExamples/hubBigGenePred/

goldenPath/help/examples/hubExamples/hubBigGenePred/hg38/

goldenPath/help/examples/hubExamples/hubCheckSupportLevels/

goldenPath/help/examples/hubExamples/hubCheckSupportLevels/requiredLevels/

goldenPath/help/examples/hubExamples/hubCheckSupportLevels/requiredLevels/data/

goldenPath/help/examples/hubExamples/hubCheckUnsupportedSettings/

goldenPath/help/examples/hubExamples/hubCheckUnsupportedSettings/hg19/

goldenPath/help/examples/hubExamples/hubGenArkExample/

goldenPath/help/examples/hubExamples/hubGenArkExample/genArkTrackHub/

goldenPath/help/examples/hubExamples/hubGroupings/

goldenPath/help/examples/hubExamples/hubGroupings/hg19/

goldenPath/help/examples/hubExamples/hubIndexedBigBedSearchable/

goldenPath/help/examples/hubExamples/hubIndexedBigBedSearchable/hg18/

goldenPath/help/examples/hubExamples/hubIndexedBigBedSearchable/hg19/

goldenPath/help/examples/hubExamples/hubMetadata/

goldenPath/help/examples/hubExamples/hubPlants/

goldenPath/help/examples/hubExamples/hubPlants/cshl2013/

goldenPath/help/examples/hubExamples/hubPlants/cshl2013/araTha1/

goldenPath/help/examples/hubExamples/hubPlants/cshl2013/braRap1/

goldenPath/help/examples/hubExamples/hubPlants/cshl2013/ricCom1/

goldenPath/help/examples/hubExamples/hubRemap/

goldenPath/help/examples/hubExamples/hubRemap/remap/

goldenPath/help/examples/hubExamples/hubRemap/remap/araTha1/

goldenPath/help/examples/hubExamples/hubSearchable/

goldenPath/help/examples/hubExamples/hubUseOneFile/

goldenPath/help/examples/hubQuickStartFilter/

goldenPath/help/examples/interact/

goldenPath/help/examples/sessions/

goldenPath/help/examples/trackDbIndexBb/

goldenPath/help/trackDb/

goldenPath/licenses/

goldenPath/lost+found/

goldenPath/mapPlots/

goldenPath/statistics/

goldenPath/trackDescriptions/

images/

images/covid19/

images/encode/

images/gtex/

images/logos/

images/mammalPsg/

images/newsArchImages/

images/phylo/

images/singleCell/

images/training/

inc/

js/

license/

phylo/

pubs/

staticStyle/

style/

style/images/

training/

training/ashg2009/

training/ashg2014/

training/ashg2015/

training/education/

training/vids/

urw-fonts/

 

sent 7,082 bytes  received 72,720 bytes  53,201.33 bytes/sec

total size is 256,131,667  speedup is 3,209.59

| Creating or updating the BLAT servers table

2024-04-12 14:22:47 URL:http://hgdownload.soe.ucsc.edu/admin/hgcentral.sql [825401/825401] -> "-" [1]

updating GBDB: hg19 hg38 mm10 mm39 mm9 sacCer3

| syncing gbdb: hg19

receiving incremental file list

 

sent 285 bytes  received 2,520,021 bytes  201,624.48 bytes/sec

total size is 8,637,270,638,505  speedup is 3,427,072.20

| syncing gbdb: hg38

receiving incremental file list

 

sent 233 bytes  received 979,081 bytes  217,625.33 bytes/sec

total size is 6,169,337,374,824  speedup is 6,299,651.98

| syncing gbdb: mm10

receiving incremental file list

 

sent 80 bytes  received 594,733 bytes  237,925.20 bytes/sec

total size is 2,448,744,144,049  speedup is 4,116,830.24

| syncing gbdb: mm39

receiving incremental file list

 

sent 46 bytes  received 6,000 bytes  4,030.67 bytes/sec

total size is 851,635,939,109  speedup is 140,859,401.11

| syncing gbdb: mm9

receiving incremental file list

 

sent 61 bytes  received 96,390 bytes  38,580.40 bytes/sec

total size is 2,244,561,167,080  speedup is 23,271,517.84

| syncing gbdb: sacCer3

receiving incremental file list

 

sent 36 bytes  received 1,660 bytes  1,130.67 bytes/sec

total size is 195,369,701  speedup is 115,194.40

| syncing full mysql database: hg18

receiving incremental file list

rsync: [sender] send_files failed to open "/hg18/bigFiles.MYD" (in mysql): Permission denied (13)

rsync: [sender] send_files failed to open "/hg18/bigFiles.MYI" (in mysql): Permission denied (13)

rsync: [sender] send_files failed to open "/hg18/bigFiles.frm" (in mysql): Permission denied (13)

rsync: [sender] send_files failed to open "/hg18/tableList.MYD" (in mysql): Permission denied (13)

rsync: [sender] send_files failed to open "/hg18/tableList.MYI" (in mysql): Permission denied (13)

rsync: [sender] send_files failed to open "/hg18/tableList.frm" (in mysql): Permission denied (13)

 

sent 15,818 bytes  received 809,224 bytes  330,016.80 bytes/sec

total size is 814,919,500,095  speedup is 987,730.93

rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1823) [generator=3.2.3]

Exit error 23 occurred on line 1

The UCSC Genome Browser installation script exited with an error.

Please contact us at genome...@soe.ucsc.edu and send us an output log

of the command prefixed with "bash -x", e.g.

bash -x browserSetup.sh install 2>&1 > install.log

 

 

 

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Christopher,

 

Do I need to run a newer version of the browserSetup.sh script?

[Christopher Lee]

Yes this appears to have been fixed, for at least hg18/hg19/hg38 bigFiles.

On Tue, Apr 16, 2024 at 5:15 AM McCully, Dwayne (NIH/NIAMS) [C] <dmcc...@mail.nih.gov> wrote:

Hi Chris,

 

Has the permissions of the files on your side been corrected?

 

Dwayne

To view this discussion on the web visit https://groups.google.com/a/soe.ucsc.edu/d/msgid/genome-mirror/BLAPR09MB6100122DCB44EC98F39D67E5E7042%40BLAPR09MB6100.namprd09.prod.outlook.com.

[Christopher Lee]

/gbdb files just need to be readable, so they can be whatever you want. I would probably not make them apache:apache just in case one of your users needs to delete a file or something without root.

On Tue, Apr 16, 2024 at 9:23 AM McCully, Dwayne (NIH/NIAMS) [C] <dmcc...@mail.nih.gov> wrote:

Hi Christopher,

 

Great and thanks for the following up.  

 

BTW:  What should be the UID/GID for /gbdb files?   Maybe apache:apache?

I see the /var/lib/mysql are mysql:mysql. 

 

Dwayne

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Chris,

 

Has the permissions of the files on your side been corrected?

 

Dwayne

 

From: 'McCully, Dwayne (NIH/NIAMS) [C]' via UCSC Genome Browser Mirror-Specific Support <genome...@soe.ucsc.edu>
Sent: Friday, April 12, 2024 8:00 AM
To: Christopher Lee <chm...@ucsc.edu>

To view this discussion on the web visit https://groups.google.com/a/soe.ucsc.edu/d/msgid/genome-mirror/BLAPR09MB6100122DCB44EC98F39D67E5E7042%40BLAPR09MB6100.namprd09.prod.outlook.com.

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Christopher,

 

Great and thanks for the following up.  

 

BTW:  What should be the UID/GID for /gbdb files?   Maybe apache:apache?

I see the /var/lib/mysql are mysql:mysql. 

 

Dwayne

 

 

From: Christopher Lee <chm...@ucsc.edu>
Sent: Tuesday, April 16, 2024 11:57 AM

[McCully, Dwayne (NIH/NIAMS) [C]]

Hi Max,

 

I’m finally working on the migration and performed all the steps below.   However, the users are unable to login to the new UCSC Genome Browser.    They can login to the original machine and browser but not the new one.

What could be the problem?

 

Dwayne

 

From: Maximilian Haeussler <mhae...@ucsc.edu>
Sent: Thursday, February 8, 2024 8:49 AM
To: McCully, Dwayne (NIH/NIAMS) [C] <dmcc...@mail.nih.gov>
Cc: genome...@soe.ucsc.edu

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.

 

[Maximilian Haeussler]

Hi Dwayne,

If the passwords are not working, it could be related to the "salt" in the hg.conf file, for the password encryption. It could also be related to the cookie domain also set in hg.conf.

Did you copy over all/most of the hg.conf settings?

best

Max