Blat installation question

[Ronald Cutler]

Hello!

I have been using AlingGraph to aling contigs I have to a reference genome. This program uses blat, however only supports up to version 34. I downloaded the blatSrc34.zip source code file and followed the README for installation on a Mac OSX 10.10.5. I get this error when executing the last step by running the make file using 'make' in the blastSRC directory:

ip-128-239-123-174:blatSrc confocal$ make

cd lib && /Applications/Xcode.app/Contents/Developer/usr/bin/make

gcc -O  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DMACHTYPE_ -DJK_WARN -Wall -Werror -I../inc -I../../inc -I../../../inc -I../../../../inc -I../../../../../inc  -o aliType.o -c aliType.c

aliType.c:5:19: error: unused variable 'rcsid' [-Werror,-Wunused-const-variable]

static char const rcsid[] = "$Id: aliType.c,v 1.4 2003/05/06 07:33:41 kate Exp $";

                  ^

1 error generated.

make[1]: *** [aliType.o] Error 1

make: *** [all] Error 2

Any help with this?

Thanks,

Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

Class of 2018

Double Major: Biology & Computer Science 

[Maximilian Haeussler]

Hi Ronald,

This is not an error, but only a warning. Newer versions of BLAT do not contain this line anymore.

On OSX, you can install blat v36 with this simple command:

> brew install blat

But I see that you mentioned that you need this particular version of BLAT.

I also see that you also posted a copy of this question to seqanswers:

http://seqanswers.com/forums/showthread.php?t=70764

Similar to the first comment over there, I am wondering if you really need v34, BLAT tried to stay backwards compatible.

You may be fine with "brew install blat"

If you really want v34, to ignore warnings like this, you should be able to remove the -Wall option from the makefile or alternatively set the variable HG_WARN to an empty value, as it contains -Wall by default, which you can see in inc/common.mk. So it should compile if instead of running just "make" you run it like this:

make HGWARN=""

Thanks for your feedback and if you have any other questions about the genome browser tools, do not hesitate to contact us.

cheers

Max

--

[Ronald Cutler]

Hi Max,

Thanks for the quick reply. I responded to a post on seqanswers that will provide further reference to my problem. I successfully installed blat through brew before running AlignGraph but align graph does not support blat v35 and up. Would be nice if there were archives of previous versions in brew. 

I used the command, HG_WARN="", and it seemed to be compiling, but got this error:

make[1]: *** No rule to make target `../lib//jkweb.a', needed by `blat'.  Stop.

make: *** [all] Error 2

I have attached a log file for your reference.

Once we find a solution for this I'll update the post so other people using AlignGraph can gain insight on this. 

http://seqanswers.com/forums/showthread.php?p=197644#post197644

Best,

Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

Class of 2018

Double Major: Biology & Computer Science 

[Maximilian Haeussler]

Hi Ronald,

Did you do a "make clean" ?

jkweb is created by "cd lib; make". 

Have you set the variable "MACHTYPE" as explained in the file README ?

[Ronald Cutler]

Here is the steps I took to install blat v34:

From the README in the blat installation folder these are the steps I followed:

1. Check that the environment variable MACHTYPE exist on your system.

 $echo $MACHTYPE
x86_64-apple-darwin14

2. Make the directory ~/bin/$MACHTYPE where (non-web) executables will go.

$mkdir /usr/local/bin/$MACHTYPE

3. Add the directory made above to the path. In /.bash_profile

PATH=$PATH:/usr/local/bin/$MACHTYPE

4. Make the directory $MACHTYPE in the lib directory

$mkdir /usr/local/bin/blastsrc/lib/$MACHTYPE

5. create jkweb in lib directory

lib$ sudo make HG_WARN=""

6. In the blastSrc directory type 'make'

blastSrc$ sudo make HG_WARN=''"

This gives me the error:

make[1]: *** No rule to make target `../lib//jkweb.a', needed by `blat'.  Stop.

make: *** [all] Error 2

When I check the lib directory to see if `/jkweb.a' is there I get a confirmation:

lib confocal$ make

make: `/jkweb.a' is up to date.

I'm not sure when I should do a 'make clean' because I went through these steps right after unzipping the file 

Best,

Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

Class of 2018

Double Major: Biology & Computer Science 

[Hiram Clawson]

Good Afternoon Ronald:

You are losing all your shell environment when you use 'sudo' to run the
build commands. You do not need to run the build under 'sudo', the results
of the build will end up in the directory $HOME/bin/$MACHTYPE under your
ownership. If you want that directory copied to /usr/local/bin/ then you
can use 'sudo cp $HOME/bin/$MACHTYPE/* /usr/local/bin/'

Also, please note the full MACHTYPE instructions:

> 2. Check that the environment variable MACHTYPE
> exists on your system. It should exist on Unix/Linux.
> (And making this on non-Unix systems is beyond
> the scope of this README). The default MACHTYPE is often a
> long string: "i386-redhat-linux-gnu"
> which will not function correctly in this build environment.
> It needs to be something simple such as one of:
> i386 i686 sparc alpha x86_64 ppc etc ...
> with no other alpha characters such as: -
> To determine what your system reports itself as, try the
> uname options: 'uname -m' or 'uname -p' or 'uname -a'
> on your command line. If necessary set this environment variable.
> Do this under the bash shell as so:
> MACHTYPE=something
> export MACHTYPE
> or under tcsh as so:
> setenv MACHTYPE something
> and place this setting in your home directory .bashrc or .tcshrc
> environment files so it will be set properly the next time you
> login. Remember to "export" it as show here for the bash shell.

> <https://www.linkedin.com/pub/ronald-cutler/a5/383/409>

>>> <https://www.linkedin.com/pub/ronald-cutler/a5/383/409>

>>>>> *aliType.c:5:19: **error: **unused variable 'rcsid'
>>>>> [-Werror,-Wunused-const-variable]*

>>>>>
>>>>> static char const rcsid[] = "$Id: aliType.c,v 1.4 2003/05/06 07:33:41
>>>>> kate Exp $";
>>>>>

>>>>> * ^*

>>>>>
>>>>> 1 error generated.
>>>>>
>>>>> make[1]: *** [aliType.o] Error 1
>>>>>
>>>>> make: *** [all] Error 2
>>>>>
>>>>> Any help with this?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

>>>>> <https://www.linkedin.com/pub/ronald-cutler/a5/383/409>

[Ronald Cutler]

Here is also a log file from step 4

Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

Class of 2018

Double Major: Biology & Computer Science 

[Ronald Cutler]

Okay, so if the variable MACHTYPE = x86_64-apple-darwin14 on my system, I should set the MACHTYPE variable to the report of 'uname -m' which is X86_64? And then export it?

Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

Class of 2018

Double Major: Biology & Computer Science 

[Hiram Clawson]

Correct: x86_64

On 8/8/16 2:07 PM, Ronald Cutler wrote:
> Okay, so if the variable MACHTYPE = x86_64-apple-darwin14 on my system, I
> should set the MACHTYPE variable to the report of 'uname -m' which is
> X86_64? And then export it?
>
> Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

[Hiram Clawson]

There can be different names for these files. See also:
http://www.linuxfromscratch.org/blfs/view/svn/postlfs/profile.html

It could be .login .profile .bashrc .bash_profile
The different files are used in different contexts.

Once you edit your .bashrc file, you either need to login again or:
$ source .bashrc

Which will place those variables in your shell.

You could also try the build sequence as mentioned at:

http://genome-source.cse.ucsc.edu/gitweb/?p=kent.git;a=blob;f=src/userApps/README

which avoids all this rigamarole. There are many paths to the top of the mountain ...

--Hiram

On 8/8/16 3:05 PM, Ronald Cutler wrote:
> "place this setting in your home directory .bashrc or .tcshrc
> environment files so it will be set properly the next time you
> login. Remember to "export" it as show here for the bash shell"
>

> How do I do the above? I can't find the setting file after setting the
> environment variable in the bash shell.

>
> Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile
> <https://www.linkedin.com/pub/ronald-cutler/a5/383/409>
>
> Class of 2018
> Double Major: Biology & Computer Science
>

[Ronald Cutler]

   "place this setting in your home directory .bashrc or .tcshrc
   environment files so it will be set properly the next time you

   login.  Remember to "export" it as show here for the bash shell"

How do I do the above? I can't find the setting file after setting the environment variable in the bash shell. 

Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

Class of 2018

Double Major: Biology & Computer Science 

[Ronald Cutler]

Thanks for your continued help. I'm not sure I understand what the build sequence is for - could you please explain?

So far these are the steps I have followed:

1. Check that the environment variable MACHTYPE exist on your system. $echo $MACHTYPE x86_64-apple-darwin14 2. Determine what system reports itself as: $uname -m x86_64 3. Set the MACHTYPE environment variable in ~/.bash_profile MACHTYPE=x86_64 export MACHTYPE $source ~/.bash_profile 4. Make the directory ~/bin/$MACHTYPE where (non-web) executables will go. $mkdir /usr/local/bin/$MACHTYPE 5. Add the directory made above to the path. In /.bash_profile PATH=$PATH:/usr/local/bin/$MACHTYPE 6. Make the directory $MACHTYPE in the lib directory (if not there already) $mkdir /usr/local/bin/blastsrc/lib/$MACHTYPE 7. create jkweb in lib directory lib$ make HG_WARN="" 8. In the blastSrc directory type 'make' blastSrc$ make HG_WARN=''"

This gives me the error: 

ld: symbol(s) not found for architecture x86_64 

clang: error: linker command failed with exit code 1 (use -v to see invocation) 

make[1]: *** [blat] Error 1 

make: *** [all] Error 2

I have succesfully added x86_64 to my path in the ~/.bash_profile indicated when I get the report from the command:

$echo $MACHTYPE
x86_64

What could be the problem now? I have attached a log file for your reference.

Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

Class of 2018

Double Major: Biology & Computer Science 

[Hiram Clawson]

You can also pick up pre-compiled binaries from:
ftp://hgdownload.cse.ucsc.edu/admin/exe/macOSX.x86_64/blat/

I can't understand the state you are in at this time. I'm very
curious what this command would indicate:
$ file blat.o
blat.o: Mach-O 64-bit object x86_64

> <https://www.linkedin.com/pub/ronald-cutler/a5/383/409>
>
> Class of 2018
> Double Major: Biology & Computer Science
>

[Ronald Cutler]

Thanks for the link, though this seems to be the latest v36 - correct? I am trying to install v34.

To further clarify, I still have no executable blat v34 after attempting to make using the steps I sent in my last message. Please disregard step 5, I meant to take it out since I set the MACHTYPE environment variable in step 3. 

Using the command '$ file blat.o' on the blatSrc/blat/blat.o results in:

blat.o: Mach-O 64-bit object x86_64

Best,

Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

Class of 2018

Double Major: Biology & Computer Science 

[Ronald Cutler]

Thank you so much for all your help! That did the trick! I now have an executable that works. I'm going to post the steps I followed to the seq_answers page for others to reference if they have the same issue. 

I did have one more error after applying the last patch and then executing 'make' in the blastSrc directory, which was:

ld: can't open output file for writing: /Users/confocal/bin/x86_64/blat, errno=2 for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[1]: *** [blat] Error 1 make: *** [all] Error 2

I fixed this by making that directory (Even though this is a weird place for it). Do you know how I could get it to create all the executables in the /usr/local/bin/x86_64 directory?

Thanks again

Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

Class of 2018

Double Major: Biology & Computer Science 

On Tue, Aug 9, 2016 at 1:07 PM, Maximilian Haeussler <m...@soe.ucsc.edu> wrote:

It's very strange that the newer version of BLAT doesn't work... would be good to know why, but well...

Apparently you need to patch jkOwnLib/makefile just like the other makefile, too:

 

 replace this:

ar rcus $(T) $(O)

with this:

libtool -static -o $(T) $(O)

On Mon, Aug 8, 2016 at 5:55 PM, Ronald Cutler <rrcu...@email.wm.edu> wrote:

I need the old version of blat because I am trying to use AlignGraph to align scaffolds from an assembly to a reference. When using AlignGraph with blat v36 I ran into the error: "BLAT CALL FAILED!", in which the fix is: "The current version of BLAT (v35) is not compatible with AlignGraph, so you would have to use an earlier version to avoid this error." This is why I am trying to install blat v34. 

I extracted a fresh blastSrc folder from the blastSrc34.zip file and did as you said above in the lib/makefile so that the line "ar rcus $(MACHTYPE)/jkweb.a $(O)" is now "libtool -static -o $(MACHTYPE)/jkweb.a $(O)". I then ran through the steps I sent you 2 messages ago and got this same error:

clang: error: linker command failed with exit code 1 (use -v to see invocation)

make[1]: *** [blat] Error 1

make: *** [all] Error 2

I have attached a log file for your reference. Really appreciate the help so far. Haven't found myself on what could be wrong, but I don't know much about installation, etc. 

Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

Class of 2018

Double Major: Biology & Computer Science 

On Mon, Aug 8, 2016 at 8:19 PM, Maximilian Haeussler <m...@soe.ucsc.edu> wrote:

I think I have a fix, replace ar rcus in lib/makefile with libtool -static -o no idea why this is necessary suddenly on the newer OSX installs.

Please, can you also tell us why you need an old version of BLAT? We have not modified BLAT, the options are the same, it's almost the same program. Why do you need the old blat version? Why do you think it doesn't work in your pipeline?

[Hiram Clawson]

Good Morning Ronald:

Yes, all binaries produced in the kent source tree will go
into $HOME/bin/$MACHTYPE

If you want them in /usr/local/bin/ then:
sudo cp -p $HOME/bin/$MACHTYPE/* /usr/local/bin/

The problems you encountered are unique to the blat34 source.
I could see the same error here with the blat34 source on Mac OSX 10.11.6 El Capitan.
I tried the blat36 source and it built completely without error
and no alterations to the source code. You may want to consider
moving up to current sources.

--Hiram

On 8/9/16 11:33 AM, Ronald Cutler wrote:
> Thank you so much for all your help! That did the trick! I now have an
> executable that works. I'm going to post the steps I followed to the
> seq_answers page for others to reference if they have the same issue.
>
> I did have one more error after applying the last patch and then executing
> 'make' in the blastSrc directory, which was:
>
> ld: can't open output file for writing: /Users/confocal/bin/x86_64/blat,
> errno=2 for architecture x86_64 clang: error: linker command failed with
> exit code 1 (use -v to see invocation) make[1]: *** [blat] Error 1 make:
> *** [all] Error 2
>
> I fixed this by making that directory (Even though this is a weird place
> for it). Do you know how I could get it to create all the executables in
> the /usr/local/bin/x86_64 directory?
>
> Thanks again
>
> Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

> <https://www.linkedin.com/pub/ronald-cutler/a5/383/409>
>
> Class of 2018
> Double Major: Biology & Computer Science
>

> On Tue, Aug 9, 2016 at 1:07 PM, Maximilian Haeussler <m...@soe.ucsc.edu>
> wrote:
>
>> It's very strange that the newer version of BLAT doesn't work... would be
>> good to know why, but well...
>>
>> Apparently you need to patch jkOwnLib/makefile just like the other
>> makefile, too:
>>
>>
>> replace this:
>>
>> ar rcus $(T) $(O)
>>
>> with this:
>>

>> libtool -static -o *$(T)* *$(O)*

>>
>> On Mon, Aug 8, 2016 at 5:55 PM, Ronald Cutler <rrcu...@email.wm.edu>
>> wrote:
>>
>>> I need the old version of blat because I am trying to use AlignGraph

>>> <https://github.com/baoe/AlignGraph> to align scaffolds from an assembly

>>> to a reference. When using AlignGraph with blat v36 I ran into the
>>> error: "BLAT CALL FAILED!", in which the fix is: "The current version of
>>> BLAT (v35) is not compatible with AlignGraph, so you would have to use an
>>> earlier version to avoid this error." This is why I am trying to install
>>> blat v34.
>>>
>>> I extracted a fresh blastSrc folder from the blastSrc34.zip file and did
>>> as you said above in the lib/makefile so that the line "ar rcus
>>> $(MACHTYPE)/jkweb.a $(O)" is now "libtool -static -o $(MACHTYPE)/jkweb.a
>>> $(O)". I then ran through the steps I sent you 2 messages ago and got this
>>> same error:
>>>
>>> clang: error: linker command failed with exit code 1 (use -v to see
>>> invocation)
>>>
>>> make[1]: *** [blat] Error 1
>>>
>>> make: *** [all] Error 2
>>>
>>>
>>> I have attached a log file for your reference. Really appreciate the help
>>> so far. Haven't found myself on what could be wrong, but I don't know much
>>> about installation, etc.
>>>
>>> Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile

>>> <https://www.linkedin.com/pub/ronald-cutler/a5/383/409>
>>>
>>> Class of 2018
>>> Double Major: Biology & Computer Science
>>>

>>> On Mon, Aug 8, 2016 at 8:19 PM, Maximilian Haeussler <m...@soe.ucsc.edu>
>>> wrote:
>>>
>>>> I think I have a fix, replace ar rcus in lib/makefile with libtool
>>>> -static -o no idea why this is necessary suddenly on the newer OSX installs.
>>>>
>>>> Please, can you also tell us why you need an old version of BLAT? We
>>>> have not modified BLAT, the options are the same, it's almost the same
>>>> program. Why do you need the old blat version? Why do you think it doesn't
>>>> work in your pipeline?
>>>>
>>>> On Mon, Aug 8, 2016 at 5:12 PM, Ronald Cutler <rrcu...@email.wm.edu>
>>>> wrote:
>>>>
>>>>> Thanks for the link, though this seems to be the latest v36 - correct?
>>>>> I am trying to install v34.
>>>>>
>>>>> To further clarify, I still have no executable blat v34 after
>>>>> attempting to make using the steps I sent in my last message. Please
>>>>> disregard step 5, I meant to take it out since I set the MACHTYPE
>>>>> environment variable in step 3.
>>>>>
>>>>> Using the command '$ file blat.o' on the blatSrc/blat/blat.o results
>>>>> in:
>>>>> blat.o: Mach-O 64-bit object x86_64
>>>>>
>>>>> Best,
>>>>>
>>>>>
>>>>> Ronald R. Cutler | Cell: (757) 332-4808 | Linkedin Profile