Any problem with OSX El Capitan and MacQIIME?

[Bahtiyar Yilmaz]

Dear all,

I would like to ask if any of you install latest Mac OSX that became available yesterday and afterwards, experience any problem with Macqiime. My problem is really the beginning and most probably a basic one. Today after installing the OSX v10.11 (El Capitan), I tried to use Macqiime but it didn't work. I was using it until yesterday and today it is not working. Actually it seems like it is not installed at all. Anyway, I started from scratch to install it as it is described in (http://www.wernerlab.org/software/macqiime/macqiime-installation)  but upon installation I got this error:

Creating script /usr/bin/macqiime...

cp: /usr/bin/macqiime: Operation not permitted

chmod: /usr/bin/macqiime: No such file or directory

done.

Because of this, Macqiime is not installed. I can modify /usr/bin/ and I don't know why since I am the admin and only user of this computer. 

Any suggestion regarding what I am doing wrong?

Thank you very much for your all help and suggestions.

All the best,

Bahti

[John Chase]

Hi, 

This is an issue with how OS 10.11 has decided to deal with certain securities. In this situation the install script for MacQIIME is trying to copy files to /usr/bin/ which is not writeable in 10.11 even with root. There are two solutions to get macqiime working until the installation is modified to be compatible with 10.11. The first is to disable specific security features, you can read about that here: http://stackoverflow.com/questions/32590053/copying-file-under-root-got-failed-in-os-x-el-capitan-10-11 Personally I would try to avoid this type of solution even if it seems simple.

Second, and what I would recommend as a workaround, is to modify the installation file to install macqiime in a directory you have access to. The easiest way to do this is to open the install.s and replace all instances of /usr/bin/ with a path to a folder that you have write access to. For instance /Users/YOUR_USERNAME where YOUR_USERNAME is your username. 

I attached an example install.s file so you can see what I mean. 

1. Download the install.s file

2. Replace all of the instances of YOUR_USERNAME and save the file 

3. Run the script as normal. This will install macqiime in your home directory. 

4. In the terminal type: 

    open ~/.bash_profile 

 This will open your bash profile in your default text editor

5. Paste the following at the end of your profile:

    export PATH=/Users/YOUR_USERNAME/macqiime:$PATH

With your username instead of YOUR_USERNAME

6. restart the terminal and you should be good to go

Let me know if you run into any issues.

John

[Bahtiyar Yilmaz]

Dear John,

Thanks a lot for your quick answer and great explanation. I did what you said and worked perfectly. Thanks a lot for your time and help.

All the best,

Bahti

[Jeff Werner]

Great, I'm glad that works!  The problem is due to Apple's new "feature" called System Integrity Protection (SIP). Basically, even the root user is not allowed to install or run any system-level software. This is REALLY annoying, and basically assumes every user is just running graphical software from the Apple app store. I.e. they don't want you to use your computer as a computer.  I don't know what XQuartz or other custom libraries, compilers, etc are going to do, because this probably blocks their installation and use as well.

Bahtiyar, does your macqiime installation really work now? I think SIP may also block the installation of the macqiime folder to /macqiime (or it may block your ability to access or run those files).  If you continue to have problems, the only thing to do is disable SIP and then install macqiime.  Here's how to disable SIP (first section of the article):

http://lifehacker.com/how-to-fix-os-x-el-capitans-annoyances-1733836821

In future versions I guess I'll have to put macqiime into the Applications folder?  Bah.

Cheers,

Jeff

[Jeff Werner]

For those with morbid curiosity, it looks like /usr/local is still an allowed place to make allowed modifications/additions, FWIW. Is the Applications folder even off limits unless you're an authorized installer, or installing from the App Store? This OS X update is going to be a major pain in the butt for command line users.

[Bahtiyar Yilmaz]

Dear Jeff,

Replying to your previous reply and this one together, I can say that yes /usr/local/ is working. Actually when I followed up John's comment, I just made a minor change in the install script and tried to install it into /usr/local/ and the installation worked. I just check now again by generating a simple heat map and worked perfectly. 

Moreover, thanks a lot for all the comments and suggestions -specifically, SIP... That, I believe, we will need to change it a lot when we try to modify/install/add some stuffs. 

Thanks a lot once again you, guys,  for your all help.

All the best,

Bahti

[Jeff Werner]

Thanks Bahtiyar,

I may have over-reacted regarding SIP. I've heard from a few folks now that everything works fine in MacQIIME as long as you put the "macqiime" sourcing script somewhere else. It sounds like SIP is fine with a user running software in custom root folders (such as /macqiime/), as long as you don't try to add to or modify /usr/bin/ or a few other unix system folders like that. I guess that can possibly protect against a subset of viruses.  :)  

Cheers,

Jeff

[Bahtiyar Yilmaz]

Hey Jeff,

I talked to my colleagues and they told me that they just put script somewhere and worked brilliantly. Initially, it is better not to touch SIP, I suppose.

Thanks a lot.

All the best,

Bahti

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/N2aZIPrsb_o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Chase]

Arnold, Try these steps:

1. Open up a text editor (atom or text wrangler, or any other text editor) 

2. file>open 

3. Navigate to your home folder

4. press Command+Shift+Period ("." key)

5. This will display hidden files, open ".bash_profile"

6. make the changes you need

Hope this helps!

John

On Mon, Oct 5, 2015 at 8:45 AM, Arnold Rodríguez-Hilario <argel...@gmail.com> wrote:

Hi John,

While I was following your step-by-step I wrote the wrong username in the install.s file and in the bash_profile file. Now when I open the terminal the first message I receive is:

-bash: export: `Rodriguez/macqiime:/Library/Frameworks/Python.framework/Versions/3.4/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin': not a valid identifier

-bash: touch: command not found

Arnolds-Macbook-Pro:~ argelabert$ 

Is there anyway I can re-open the bash_profile again and rename it? When I try to use open ~/.bash_profile I get another error, I'm guessing it's because the location doesn't exist.

Arnolds-Macbook-Pro:~ argelabert$ open ~/.bash_profile 

-bash: open: command not found

Any help would be greatly appreciated. 

--

---
You received this message because you are subscribed to the Google Groups "Qiime Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qiime-forum...@googlegroups.com.

[Dennis Jakupovic]

Hi John, 

Thanks or your instructions, but upon following them I ran into a problem. 

Between point 2 and 3, do you want us to replace the install.s file found in the unarchived MacQIIME folder (MacQIIME_1.9.1-20150604_OS10.7) with the edited one we've downloaded here, and then run the script ./install.s ?

If so, then for some reason I'm not able to:

 >> ./install.s

     -bash: ./install.s: Permission denied

 >> sudo ./install.s

     Password:

     sudo: ./install.s: command not found

But then when I replace the edited install.s file with the original one, it runs normally.

Any idea how this works?

Thanks in advance, 

Dennis

[John Chase]

Hi, 

Yes, I was thinking of just replacing the install.s file that is downloaded with macqiime with the install.s file that you created manually.

John

--

[Jeff Werner]

Hi Dennis,

Could it be that the new install.s file you added did not have executable permissions? To change this, you could do this first before trying to run it:

chmod a+x ./install.s

I hope that helps.

Cheers,

Jeff

[John Chase]

Thanks Jeff! 

I misunderstood what was being asked here, but following Jeff's instructions should correct the problem.

John

--

[Dennis Jakupovic]

Hi John & Jeff,

Thanks for the tip, it worked!

Ciao, 

Dennis

chmod a+x ./install.s

sudo ./install.s

[Jane]

Hello, 

I'm having trouble with qiime too after updating my OS.

So I followed the workaround you outlined. 

Everything got installed under the USERNAME.

But then when I log in to macqiime, it shows this error:

MacQIIME version:

cat: /macqiime/version.txt: No such file or directory

Sourcing MacQIIME environment variables...

  This is the same as a normal terminal shell, except your default

  python is DIFFERENT (/macqiime/bin/python) and there are other new

  QIIME-related things in your PATH.

  Type "exit" (return) to go back to your normal shell

Any help is greatly appreciated.

[zhiying Guo]

Hello John,

Could you help follow this question? I haven't used Mac and have no experience on Mac QIIME.

Thank you very much 

Zhiying

--

[Dennis Jakupovic]

Hi Jane, 

I experienced this too, and at that time QIIME was able to open in the terminal, but I could not make use of the commands. Is QIIME working for you as it should, despite the version.txt missing?

I also experienced difficulties after I reinstalling it following the workaround, but (what I think) made it work again in the end was just copying the whole macqiime folder in this location ~/Users/YOUR_USERNAME/ to the root (~/). 

Hope it helps!
Dennis

[Jane]

Yeah, the commands are not working for me.

I will try to the root thing. 

I didn't do the root - thanks!!

I will keep you posted.

[Jeff Werner]

Did you get any error messages when running the install.s script?  It looks like the macqiime folder and files did not get moved to their final location.  I can't tell, without an El Capitan system, why this might be. Does El Capitan not have have rsync? Or some other kind of lack of write permissions? If anyone has any insights as to what El Capitan is doing, I'd love to hear.

To fix this, what needs to happen is for the "macqiime" folder to get moved to root (/).  From the install folder, one option to make this happen is:

mv ./macqiime /macqiime

Let me know how that's working, thanks! And sorry for the incompatibility of the installer!  :)

Cheers,

Jeff

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/N2aZIPrsb_o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.

[Jane]

So unfortunately the workaround did not work for me, even putting ~.

I got fed up so I tried temporarily disabling the SIP (http://lifehacker.com/how-to-fix-os-x-el-capitans-annoyances-1733836821), and then I re-installed macqiime. Everything was copied properly except the macqqime script was not copied properly to /usr/bin. 

I forgot the error message but I think it is something about permission denied for copying. So, I just sudo copy the script. After macqiime is working fine. I then went back and enable SIP. I re-checked and it still works. So, now SIP is running on my Mac with macqiime working perfectly.

I did try disabling the SIP and running macqiime (without re-installing it) and it did not work. So, at least for me, I have to make a fresh re-install after disabling SIP to make it all work. 

Anyways, thanks to all your help 

[Yoshiki Vazquez-Baeza]

> I did try disabling the SIP and running macqiime (without re-installing it)
> and it did not work. So, at least for me, I have to make a fresh re-install
> after disabling SIP to make it all work.

I can confirm something similar, at least with another program
(specifically mactex), seems like El Capitan removes things that were
previously installed in any of the protected locations, so I had to
reinstall the software (in that particular instance re-creating the
symbolic links didn't work, so maybe there's something else EC does).

Thanks!

Yoshiki.

> > On Mon, Oct 12, 2015 at 6:49 AM Jane <jane.na...@gmail.com <javascript:>>

> > wrote:
> >
> >> Yeah, the commands are not working for me.
> >> I will try to the root thing.
> >> I didn't do the root - thanks!!
> >> I will keep you posted.
> >>
> >>
> >> On Monday, 12 October 2015 11:46:09 UTC+2, Dennis Jakupovic wrote:
> >>>
> >>> Hi Jane,
> >>>
> >>> I experienced this too, and at that time QIIME was able to open in the
> >>> terminal, but I could not make use of the commands. Is QIIME working for
> >>> you as it should, despite the version.txt missing?
> >>>
> >>> I also experienced difficulties after I reinstalling it following the
> >>> workaround, but (what I think) made it work again in the end was just
> >>> copying the whole macqiime folder in this location ~/Users/YOUR_USERNAME/
> >>> to the root (~/).
> >>>
> >>> Hope it helps!
> >>> Dennis
> >>>
> >> --
> >>
> >> ---
> >> You received this message because you are subscribed to a topic in the
> >> Google Groups "Qiime Forum" group.
> >> To unsubscribe from this topic, visit
> >> https://groups.google.com/d/topic/qiime-forum/N2aZIPrsb_o/unsubscribe.
> >> To unsubscribe from this group and all its topics, send an email to

> >> qiime-forum...@googlegroups.com <javascript:>.

> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
>

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "Qiime Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qiime-forum...@googlegroups.com.

[ms1...@bristol.ac.uk]

Just want to add that I am having exactly the same problem. After upgrading to El Capitan, Macqiime is no longer working on my office computer or on my laptop. I tried Jeff's workaround (thank you very much for it) with changing the /install.s file but it didn't work. I still run into the same errors: bash not found. The "biom" software is not working either and I have a suspicion that quite a few more might not as well. 

[Jeff Werner]

For those still having problems, there are TWO pretty straight-forward things that need to happen to get MacQIIME installed:

 1. The "macqiime" folder HAS to be copied over to root, i.e. it should end up as /macqiime/  The most straight-forward way to make this happen is to do this from in the installation folder:

 

# remove any macqiime folder that's already located in /

sudo rm -R /macqiime

# make a copy of the "macqiime" folder from the installation package, to root 

sudo cp -a ./macqiime /

 2. The "macqiime" script file can be placed anywhere convenient, in your PATH, to call it.  This is what John's instructions at the top of this feed are meant to fix.  This is actually optional. Instead of running the "macqiime" script in the terminal, you *could* just do this each time you wanted to start using MacQIIME in the terminal:

source /macqiime/configs/bash_profile.txt

That will give you the qiime-enabled macqiime prompt as well.  The main requirement is that /macqiime needs to be there at root.  To verify that you have a macqiime folder located at root, you can do this:

cat /macqiime/version.txt

If it's installed, that should 

The error about a missing version.txt file is a problem with issue number 1, needing to have a copy of the macqiime folder located at root (/macqiime).  From what I've heard, El Capitan users have not had problems once they get this folder moved to the correct location.  I am at a loss as to why it is not getting moved properly.  Are there really no other error messages when running the installation?

I HOPE THIS WORKS!!  Sorry for all the inconveniences.

Cheers,

Jeff

[Quinn]

Jeff, this is working for me, thanks for figuring it out!

[Liv Yeh]

Hi Qiimers,

I used to work on QIIME virtual box. This is my first time to use MacQIIME.

I also faced the same problem but still can't figure out how to make everything works.

I deeply appreciate if someone can help me clarify the procedure.

I have modified install.s file to install macqiime in my home directory.

Then, should I directly modify the bash_profile.txt in home directory/macqiime/configs ?

Best,

Liv

Jeff Werner於 2015年10月21日星期三 UTC-7上午8時10分17秒寫道：

[Kyle Bittinger]

Liv,

I imagine that you'll have to modify that file.

Is it an option for you to install MacQIIME in the root directory?

--Kyle

[Jeff Werner]

Hi Liv,

The contents of the "macqiime" folder HAVE to be located at root. I.e., the folder should be /macqiime and you should be able to print the version by doing this:

cat /macqiime/version.txt

MacQIIME will not work from any other location and it cannot function as a subfolder in your home folder. Many of the binaries are compiled with references to libraries, etc, in that /macqiime/ root folder location.  And, there's no need to change anything in the "/macqiime/configs/" folder. The "macqiime" executable script file, however, can be placed anywhere as long as it's in your path.  Alternatively, if you want to source macqiime without the script, you can just do this:

source /macqiime/configs/bash_profile.txt

That will accomplish the same thing as running the "macqiime" script.

I hope that helps!

Cheers,

Jeff

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/N2aZIPrsb_o/unsubscribe.

To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.

[Jeff Werner]

I should also add that if you want to have the macqiime folder physically located somewhere else, for storage logistic reasons or whatever, that is possible, as long as you create a symlink to it at the root location /macqiime/

Cheers,

Jeff

[Liv Yeh]

Hi Jeff,

Sorry that it took me a while to get back to you. It works now!!

Thank you so much!

Cheers,

Liv

[Jarrad Marcell]

I was able to install macqiime using the workaround, but I cannot start macqiime from the command line. Is there any way to fix this issue? I receive the error:
-bash: macqiime: command not found. I've tried to source which boots macqiime, but none of the command work. I'm assuming there is a broken link somewhere.

[Brandi Kiel Reese]

I am having the same issue. I can use 

source /macqiime/configs/bash_profile.txt

and start access macqiime that way.

I also verified that macqiime is at the root through the following command and received the expected return. 

Brandis-MacBook-Pro:~ brandikielreese$ cat /macqiime/version.txt

MacQIIME 1.9.1-20150604

Thanks for the help

[ctri...@mymail.mines.edu]

Hi Jarrad and Brandi, a colleague and I were finally able to get MacQiime installed on our 2011 MBPs and running from the command line. I'm not quite sure how to help exactly, but I thought I would share the instructions we used just in case it might shed some light on your issue. I hope it helps!

1. Removed any macqiime folder already located in /

sudo rm -R /macqiime

2. Downloaded installation package from Qiime website (Version 1.9.1 20150604 OS10.7)

3. Downloaded new install.s file from Thread (https://groups.google.com/forum/#!topic/qiime-forum/N2aZIPrsb_o/discussion)

Modified to include USERNAME: yourusername

Navigate to Downloads folder

chmod a+x ./install.s (to give Executable permissions)

4. MacQIIME Installation Steps

cd ~/Downloads

md5 MacQIIME_1.9.1-20150604_OS10.7.tgz (Check MD5 Sum)

Output should be:   6634cb226d71ecf17d60483e6039f66a  MacQIIME_1.9.1-20150604_OS10.7.tgz

tar -xvf MacQIIME_1.9.1-20150604_OS10.7.tgz (Unarchive file using tar command)

cd MacQIIME_1.9.1-20150604_OS10.7/

Replace install.s file in installation package with new install.s containing YOURUSERNAME

./install.s

5. Open bash profile in default text editor

open ~/.bash_profile

export PATH=/Users/YOURUSERNAME/macqiime:$PATH (Replace ending command in profile)

6. Make a copy of the "macqiime" folder from installation package to root

[Jewelna Osei-Poku]

Hello John,

I am trying to install macqiime on my MacPro, and running into a few issues. I finished flowing the installation guide on Wernerlab (http://www.wernerlab.org/software/macqiime/macqiime-installation) but macqiime didn't start. I came across your recommendations to solve the issue, and now I am stuck at post 4. I got the following message when I tried step 4:

Jewelnas-MBP:MacQIIME_1.9.1-20150604_OS10.7 Jewelna$ open ~/.bash_profile

The file /Users/Jewelna/.bash_profile does not exist.

What do I do? Attached is the "install.s" script I run previously. 

Regards,

Jewelna.

[Adam Martinez]

Jeff,

Thanks for these instructions, now MacQiime is working on my El Capitan MBP. I have a related question... After copying the macqiime folder to root, I now have two "macqiime" folders taking up space, right? One in root and one in /Users/USERNAME. Also, if I'm adding things like usearch, usearch61, greengenes files, etc, which "macqiime" folder should I add them to?

Thanks,

Adam

[Colin Brislawn]

Hello Adam,

Once you load macqiime, you can type 'which print_qiime_config.py' to see which space is active. You can safely delete the other one. 

As for adding other software, say usearch or vsearch or blastall, any of these files will be recognized as long as they are part of your Linux path. For example, you could put them into your home directory and they will be recognized even if macqiime is not loaded, because your home directory is already part of your path.

Here is a good summary of how you can set your path: http://unix.stackexchange.com/a/26059 

Colin Brislawn 

[Georgios Kitsios]

Hi Jeff,

I just installed macqiime and run in the issue mentioned above.

tried to run the commands you recommended and these are the error messages i get: 

Any suggestions would be very welcome!

thanks, 

gk

MacQIIME version:

cat: /macqiime/version.txt: No such file or directory

Sourcing MacQIIME environment variables...

  This is the same as a normal terminal shell, except your default

  python is DIFFERENT (/macqiime/bin/python) and there are other new

  QIIME-related things in your PATH.

  Type "exit" (return) to go back to your normal shell

bash-3.2$ open ~/.bash_profile 

The file /Users/gkmd/.bash_profile does not exist.

bash-3.2$ mv ./macqiime/macqiime

usage: mv [-f | -i | -n] [-v] source target

       mv [-f | -i | -n] [-v] source ... directory

bash-3.2$ macqiime

MacQIIME version:

cat: /macqiime/version.txt: No such file or directory

Sourcing MacQIIME environment variables...

  This is the same as a normal terminal shell, except your default

  python is DIFFERENT (/macqiime/bin/python) and there are other new

  QIIME-related things in your PATH.

  Type "exit" (return) to go back to your normal shell

bash-3.2$ exit

exit

Forgetting MacQIIME environment variable settings...

bash-3.2$ sudo rm -R/macqiime

Password:

rm: illegal option -- /

usage: rm [-f | -i] [-dPRrvW] file ...

       unlink file

bash-3.2$ sudo cp -a./macqiime

cp: illegal option -- .

usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file

       cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file ... target_directory

bash-3.2$ sudo cp -a./macqiime

[Colin Brislawn]

Looks like MacQiime is install, so try loading it and running print_qiime_config.py

macqiime

print_qiime_config.py -t

Let me know how that works,
Colin

[Georgios Kitsios]

thanks  - it worked after reinstalling. 

for some weird reason it worked when I double clicked on the tar file and opened it up that way as opposed to using the terminal and tar command. 

in any case, up and running now - thank you very much

[francesca de martini]

Hi all,

I am trying to install Qiime on my new mac (OS X El Capitan version 10.11.15), and it is not working. I tried the install.s suggested by Jess but this is what is giving me:

This script will REPLACE any existing /macqiime directory.

Any files or data currently in /macqiime will be lost!!!!!

If this is NOT what you want to do, HIT control-C to quit.

Otherwise, press return to continue...

Creating the /macqiime directory. (An error, if it already exists, is okay.)

You may have to enter your sudo password.

mkdir: /Users/francesca/macqiime: File exists

Copying all the files (may take a while)...

rsync: link_stat "/Users/francesca/Downloads/./macqiime/." failed: No such file or directory (2)

rsync error: some files could not be transferred (code 23) at /BuildRoot/Library/Caches/com.apple.xbs/Sources/rsync/rsync-47/rsync/main.c(992) [sender=2.6.9]

Creating script /usr/bin/macqiime...

cp: scripts/macqiime: No such file or directory

done.

I am not sure what I am doing wrong.

Thank you for your help,

Francesca

[Jeff Werner]

Hi Francesca,

It sounds like you're not running the install script from inside the macqiime installation folder. It has to be run from in there.  

I've posted updated instructions to fix the El Capitan problems in the regular MacQIIME installation instructions (http://www.wernerlab.org/software/macqiime/macqiime-installation). Basically, all you have to do is:

1. Download ftp://ftp.microbio.me/pub/macqiime-releases/MacQIIME_1.9.1-20150604_OS10.7.tgz  into your ~/Downloads/ folder

2. Do these commands in the Terminal:

cd ~/Downloads

tar -xvf MacQIIME_1.9.1-20150604_OS10.7.tgz

cd MacQIIME_1.9.1-20150604_OS10.7/

./install.s

sudo cp scripts/macqiime /usr/local/bin/macqiime

sudo chmod a+x /usr/local/bin/macqiime

And then it should work! Please let me know if you're still having problems.

Cheers,

Jeff

[francesca de martini]

Thank you Jeff, it worked well. 

Francesca