Build & Dist

74 views
Skip to first unread message

Anthony Scopatz

unread,
Nov 5, 2013, 1:42:01 PM11/5/13
to PyNE Dev
Hello All, 

Katy had a great idea (yesterday?) that maybe the PyNE v0.4 release could simply fix our build and distribution problems.  I wanted to get folk's opinions on this and also gauge interest in working on this problem.  After the tutorial, I really feel that this is our biggest barrier to adoption and further growth.  Should we hold a sprint to tackle this?

We are not the only ones with scientific Python distribution problems, but we are unique in that we are trying to ship a first class C++ API as well.

Be Well
Anthony

Matthew Gidden

unread,
Nov 5, 2013, 1:48:57 PM11/5/13
to pyne...@googlegroups.com
This was top on my list of after hours action items. I'm interested! I think you could either go v3.1 or v4.0, that just depends on your preference to keeping you release timings =).


--
 
---
You received this message because you are subscribed to the Google Groups "PyNE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyne-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.



--
Matthew Gidden
Ph.D. Candidate, Nuclear Engineering
The University of Wisconsin -- Madison
Ph. 225.892.3192

Katy Huff

unread,
Nov 5, 2013, 1:52:13 PM11/5/13
to pyne...@googlegroups.com
Yay Matt! 

To give some extra motivation for those that weren't at the tutorial, everyone had a little dependency hell. BUT, the main problems came from people on macs, where it can be particularly tricky to keep track of 1) which python is running the show, 2) which dylibs are first in the dyld search path, and 3) where pyne is going to put its --user installation.

That said, not all problems seem to be apple's fault. Recently anaconda seems to be causing mac-independent problems. Personally, I had anaconda running pyne just fine, but updated anaconda a few days ago and everything broke spectacularly in a confusing way. This is yet to be understood. Is anyone running the newest anaconda with the newest pyne on any platform? How about canopy?

The only current consistently successful scheme for getting the newest pyne built on a mac appears to be a piecewise macports install of python and all necessary subpackages, an easy_install of cython (the macports one doesn't set itself as active properly), and a from-source build of the yt-3.0 repo (because the macports install of py27-yt doesn't have the new moab frontend in it). 

A similar strategy with fink was failing at last count.

Katy

Katy Huff

unread,
Nov 5, 2013, 1:53:50 PM11/5/13
to pyne...@googlegroups.com
Also, anaconda's libhdf5 doesn't work with pyne, so, as far as I can tell, any machine using anaconda needs its own independent libhdf5.

Matthew Turk

unread,
Nov 5, 2013, 1:57:19 PM11/5/13
to pyne...@googlegroups.com
Hi Katy,

On Tue, Nov 5, 2013 at 1:53 PM, Katy Huff <katy...@gmail.com> wrote:
> Also, anaconda's libhdf5 doesn't work with pyne, so, as far as I can tell,
> any machine using anaconda needs its own independent libhdf5.

That's what we came to the conclusion of for yt, although we opted
eventually to just remove our compile-time HDF5 linking. Here's a bug
where it got discussed:

https://github.com/ContinuumIO/anaconda-issues/issues/18

Katy Huff

unread,
Nov 5, 2013, 2:01:45 PM11/5/13
to pyne...@googlegroups.com
Hey! We're not alone!
Thanks Matt Turk!

Matthew Gidden

unread,
Nov 5, 2013, 2:02:16 PM11/5/13
to pyne...@googlegroups.com
On Tue, Nov 5, 2013 at 12:52 PM, Katy Huff <katy...@gmail.com> wrote:
Yay Matt! 

To give some extra motivation for those that weren't at the tutorial, everyone had a little dependency hell. BUT, the main problems came from people on macs, where it can be particularly tricky to keep track of 1) which python is running the show, 2) which dylibs are first in the dyld search path, and 3) where pyne is going to put its --user installation.

That said, not all problems seem to be apple's fault. Recently anaconda seems to be causing mac-independent problems. Personally, I had anaconda running pyne just fine, but updated anaconda a few days ago and everything broke spectacularly in a confusing way. This is yet to be understood. Is anyone running the newest anaconda with the newest pyne on any platform? How about canopy?

The only current consistently successful scheme for getting the newest pyne built on a mac appears to be a piecewise macports install of python and all necessary subpackages, an easy_install of cython (the macports one doesn't set itself as active properly), and a from-source build of the yt-3.0 repo (because the macports install of py27-yt doesn't have the new moab frontend in it). 
Oof, ok. I was not aware of these mac-specific issues. That makes the difficulty of this fix much higher..

Katy Huff

unread,
Nov 5, 2013, 2:08:48 PM11/5/13
to pyne...@googlegroups.com
On Tue, Nov 5, 2013 at 11:02 AM, Matthew Gidden <gid...@wisc.edu> wrote:



On Tue, Nov 5, 2013 at 12:52 PM, Katy Huff <katy...@gmail.com> wrote:
Yay Matt! 

To give some extra motivation for those that weren't at the tutorial, everyone had a little dependency hell. BUT, the main problems came from people on macs, where it can be particularly tricky to keep track of 1) which python is running the show, 2) which dylibs are first in the dyld search path, and 3) where pyne is going to put its --user installation.

That said, not all problems seem to be apple's fault. Recently anaconda seems to be causing mac-independent problems. Personally, I had anaconda running pyne just fine, but updated anaconda a few days ago and everything broke spectacularly in a confusing way. This is yet to be understood. Is anyone running the newest anaconda with the newest pyne on any platform? How about canopy?

The only current consistently successful scheme for getting the newest pyne built on a mac appears to be a piecewise macports install of python and all necessary subpackages, an easy_install of cython (the macports one doesn't set itself as active properly), and a from-source build of the yt-3.0 repo (because the macports install of py27-yt doesn't have the new moab frontend in it). 
Oof, ok. I was not aware of these mac-specific issues. That makes the difficulty of this fix much higher..


Probably so. But, never fear. It's mostly a matter of making sure people have their paths set up in bashrc to pick the python and pyne that they truly want. I have faith that a pip install or an easy_install that works for linux is likely to work with a mac with a scientific python environment that has been properly set up. I think, also, that a macports portfile is do-able if we can figure out what's happening with macports cython selection.

Paul Romano

unread,
Nov 5, 2013, 5:07:07 PM11/5/13
to PyNE Dev
Just wanted to chime in with a +1 for fixing build/dist issues. Almost all of the feedback that I get on PyNE is "it's cool, but it's a pain in the ass to install".

Paul

Anthony Scopatz

unread,
Nov 5, 2013, 5:16:44 PM11/5/13
to PyNE Dev
On Tue, Nov 5, 2013 at 2:07 PM, Paul Romano <paul.k...@gmail.com> wrote:
Just wanted to chime in with a +1 for fixing build/dist issues. Almost all of the feedback that I get on PyNE is "it's cool, but it's a pain in the ass to install".

Thanks Paul! There is a grumpy old bearded man somewhere inside me shaking his cane about using linux and having a real package manager....  

I am still not sure how we are going to get to a windows build, but I suppose that is easier to test.

At this point, I think that we have a fair degree of consensus that this will be the major issue for v0.4.

Anthony Scopatz

unread,
Nov 6, 2013, 2:29:09 AM11/6/13
to PyNE Dev
Hello All, 

I know that this is not the answer that anyone wants to hear, but maybe the right thing to do is everything.  Right now -- and most of the solutions that have been proposed -- have been developer friendly.  As such, they have always been somewhat unfriendly to some group of users.  If we really have empathy for our users we will meet them on their terms, as much as this might suck for us.

So rather than tackling just one package manger or platform, maybe it is high time we try to insinuate ourselves into as many as possible.  Luckily I think that the workload here decreases with every extra distribution mechanism supported.  For most of these, we would only support release versions. The following is a list of potential targets that have been thrown out just today:

* PyPI
* conda
* macports
* fink
* brew
* apt
* arch 
* windows binary

I think if we were to carve this up smartly we would be able to tackle these.  Maybe a few of us can sprint at ANS on part of this.

Be Well
Anthony

Paul Romano

unread,
Nov 6, 2013, 5:14:09 AM11/6/13
to PyNE Dev
'apt' could have a few different meanings. Is the thought to just create a .deb or to actually get that .deb into the Debian repositories? Another option for Ubuntu is to make a ppa.

Paul

Anthony Scopatz

unread,
Nov 6, 2013, 9:51:04 AM11/6/13
to PyNE Dev
I think apt means ppa, since that subsumes *.deb

Paul Romano

unread,
Nov 6, 2013, 4:47:20 PM11/6/13
to PyNE Dev
ppa is more Ubuntu specific and requires a little more thinking on the user's part. The ideal thing would be to get PyNE into the Debian repository which then gets downstream into the Ubuntu distribution as well. From what I understand, it would be a bit more work than simply creating a ppa though.

Any thoughts on how we would handle data for a binary release? Since some of our data has changed, it might be good for someone (Anthony?) to remind us of what data we absolutely cannot redistribute.

Anthony Scopatz

unread,
Nov 6, 2013, 7:45:18 PM11/6/13
to PyNE Dev
On Wed, Nov 6, 2013 at 1:47 PM, Paul Romano <paul.k...@gmail.com> wrote:
ppa is more Ubuntu specific and requires a little more thinking on the user's part. The ideal thing would be to get PyNE into the Debian repository which then gets downstream into the Ubuntu distribution as well. From what I understand, it would be a bit more work than simply creating a ppa though.

Hello Paul, 

Yes, getting PyNE in upstream debian would be more work but more valuable.  Note that this probably also requires getting packages setup for MOAB, PyTAPS, yt, and Fudge (eventually) as well.  ppa might be a good stop gap.
 
Any thoughts on how we would handle data for a binary release? Since some of our data has changed, it might be good for someone (Anthony?) to remind us of what data we absolutely cannot redistribute.

Well for *.debs and any other distribution mechanism which has a post-install step, we would simply run nuc_data_make afterwards.  For other distributions, this is something that the user would have to do.

As per the data that we can redistribute, you may find that here [1].  It really isn't a whole lot, namely set(['atomic_weight', 'scattering_lengths', 'simple_xs', 'materials'])  Everything we have been asked not to redistribute, even if it is free, open, and not export controlled.  That is an unfortunate cost of doing business.

Be Well
Anthony

Paul Romano

unread,
Nov 7, 2013, 3:26:27 AM11/7/13
to PyNE Dev
Thanks for the reminder about data redistribution, Anthony. As for ppa, would we not have the same package dependency problems since it is merely a .deb? The only ways I see to do it are to 1) include source for MOAB, PyTAPS, etc. and build/install it from one .deb, or 2) have a .deb for each piece.

Anthony Scopatz

unread,
Nov 7, 2013, 3:41:50 AM11/7/13
to PyNE Dev
On Thu, Nov 7, 2013 at 12:26 AM, Paul Romano <paul.k...@gmail.com> wrote:
Thanks for the reminder about data redistribution, Anthony.

Not a problem :)
 
As for ppa, would we not have the same package dependency problems since it is merely a .deb? The only ways I see to do it are to 1) include source for MOAB, PyTAPS, etc. and build/install it from one .deb, or 2) have a .deb for each piece.

I think we want (2), to have a *.deb for each piece.  Doing this for each piece is more work but is ultimately the right thing to do.  This makes it even harder to add all of these packages to debian...though I am not sure how much harder.  It might be the case that once you have done one then the rest are easy.  I suggested ppa originally only because it seems to have a lower barrier to entry.

Be Well
Anthony
 


On Thu, Nov 7, 2013 at 1:45 AM, Anthony Scopatz <sco...@gmail.com> wrote:
On Wed, Nov 6, 2013 at 1:47 PM, Paul Romano <paul.k...@gmail.com> wrote:
ppa is more Ubuntu specific and requires a little more thinking on the user's part. The ideal thing would be to get PyNE into the Debian repository which then gets downstream into the Ubuntu distribution as well. From what I understand, it would be a bit more work than simply creating a ppa though.
better 

Paul Romano

unread,
Nov 7, 2013, 5:16:46 AM11/7/13
to PyNE Dev
Ok, so I guess either way, we're stuck with making .debs for each piece. In that case I agree that ppa would be a good first step (and one that we have more control over) and then if that is successful, we can pursue getting into Debian upstream. I have a bit of experience with ppa so I can probably help out with that effort, time-permitting.

Anthony Scopatz

unread,
Nov 7, 2013, 10:18:49 AM11/7/13
to PyNE Dev

That would be great!  Thanks Paul.

Matthew Turk

unread,
Nov 7, 2013, 10:27:54 AM11/7/13
to pyne...@googlegroups.com, Kacper Kowalik
Hi Anthony,

On Wed, Nov 6, 2013 at 2:29 AM, Anthony Scopatz <sco...@gmail.com> wrote:
> Hello All,
>
> I know that this is not the answer that anyone wants to hear, but maybe the
> right thing to do is everything. Right now -- and most of the solutions
> that have been proposed -- have been developer friendly. As such, they have
> always been somewhat unfriendly to some group of users. If we really have
> empathy for our users we will meet them on their terms, as much as this
> might suck for us.
>
> So rather than tackling just one package manger or platform, maybe it is
> high time we try to insinuate ourselves into as many as possible. Luckily I
> think that the workload here decreases with every extra distribution
> mechanism supported. For most of these, we would only support release
> versions. The following is a list of potential targets that have been thrown
> out just today:
>
> * PyPI
> * conda
> * macports
> * fink
> * brew
> * apt
> * arch
> * windows binary
>
> I think if we were to carve this up smartly we would be able to tackle
> these. Maybe a few of us can sprint at ANS on part of this.

This is something yt would be interested in collaborating on, as well.
In the past we've had some success at getting PPAs, and being
included in MacPorts, but not with a dedicated effort. I'm not sure
the necessary infrastructure for doing buildbots, but one of our devs
(Kacper Kowalik, CC'd on this email but also out of contact for a few
days) has had success getting VMs set up to build Conda packages in
the past. For Conda, I believe providing conda *recipes* is not too
hard, as they can be submitted via PRs to the conda-recipes repo. And
binstar.org exists to provide conda packages, although I think their
support for "teams" is still being built up.

I've recently been in touch with a MacPorts dev about getting the yt
package there updated (thanks to a heads up from Katy that it's in
there, but is woefully out of date!) and I think that Kurt Schwehr has
done some maintenance of various scientific packages in fink and might
be willing to update those as well. Is there any interest in trying
to have modules for PyNE on XSEDE resources, or is that not a usual
platform for deployment?

Anyway, I'd be interested in seeing if we can pool resources on this
front, whether that be in the form of CI servers, VMs, etc etc.

-Matt

Anthony Scopatz

unread,
Nov 7, 2013, 7:07:55 PM11/7/13
to PyNE Dev, Kacper Kowalik
Katy did some sleuthing yesterday and it seems pretty easy, in theory.
Of course, we have yet to see it work in practice.
 
 I'm not sure
the necessary infrastructure for doing buildbots, but one of our devs
(Kacper Kowalik, CC'd on this email but also out of contact for a few
days) has had success getting VMs set up to build Conda packages in
the past.  For Conda, I believe providing conda *recipes* is not too
hard, as they can be submitted via PRs to the conda-recipes repo.

If only HDF5 weren't broken...
 
 And
binstar.org exists to provide conda packages, although I think their
support for "teams" is still being built up.

I think binstar, or our own alternative would be the best.  Binstar might be
the lowest valence state.
 
I've recently been in touch with a MacPorts dev about getting the yt
package there updated (thanks to a heads up from Katy that it's in
there, but is woefully out of date!) and I think that Kurt Schwehr has
done some maintenance of various scientific packages in fink

That would be awesome
 
and might
be willing to update those as well.  Is there any interest in trying
to have modules for PyNE on XSEDE resources, or is that not a usual
platform for deployment?

I don't think XSEDE is on anyone's critical path, but that isn't to say that 
it shouldn't be.  Paul Romano might have the most experience...
 

Anyway, I'd be interested in seeing if we can pool resources on this
front, whether that be in the form of CI servers, VMs, etc etc.

I, of course, would be super excited to see this happen.  In thinking about 
this last night, I bet the reason that yt and PyNE hit this problem so hard 
is that we are really trying to be Cython projects.  As such, we are stretching
the existing systems to their limits....where nothing works :).

Be Well
Anthony

Matthew Turk

unread,
Nov 8, 2013, 11:28:16 AM11/8/13
to pyne...@googlegroups.com, Kacper Kowalik
Hi Anthony,
Ah, great.

>
>>
>> I'm not sure
>> the necessary infrastructure for doing buildbots, but one of our devs
>> (Kacper Kowalik, CC'd on this email but also out of contact for a few
>> days) has had success getting VMs set up to build Conda packages in
>> the past. For Conda, I believe providing conda *recipes* is not too
>> hard, as they can be submitted via PRs to the conda-recipes repo.
>
>
> If only HDF5 weren't broken...

Indeed, although Kacper (who if I remember correctly should be able to
reply in a few days) did say he figured out a way to build HDF5 in a
cross-platform mechanism that avoids these issues. I do think that
there was some other issue with having to link the other HDF5-using
libraries (PyTables, h5py) against the same one.

>
>>
>> And
>> binstar.org exists to provide conda packages, although I think their
>> support for "teams" is still being built up.
>
>
> I think binstar, or our own alternative would be the best. Binstar might be
> the lowest valence state.

It might be, although it's not difficult to set up static directories
that expose conda indices. The index format is a JSON file, so in
principle this could be part of a CI machine's static content serving.

>
>>
>> I've recently been in touch with a MacPorts dev about getting the yt
>> package there updated (thanks to a heads up from Katy that it's in
>> there, but is woefully out of date!) and I think that Kurt Schwehr has
>> done some maintenance of various scientific packages in fink
>
>
> That would be awesome
>
>>
>> and might
>> be willing to update those as well. Is there any interest in trying
>> to have modules for PyNE on XSEDE resources, or is that not a usual
>> platform for deployment?
>
>
> I don't think XSEDE is on anyone's critical path, but that isn't to say that
> it shouldn't be. Paul Romano might have the most experience...
>
>>
>>
>> Anyway, I'd be interested in seeing if we can pool resources on this
>> front, whether that be in the form of CI servers, VMs, etc etc.
>
>
> I, of course, would be super excited to see this happen. In thinking about
> this last night, I bet the reason that yt and PyNE hit this problem so hard
> is that we are really trying to be Cython projects. As such, we are
> stretching
> the existing systems to their limits....where nothing works :).
>

That sounds exactly right to me. :)

So what can I do to support this effort? I've been attempting to find
CI infrastructure, but those efforts have stalled due to some upcoming
travel.

-Matt

Anthony Scopatz

unread,
Nov 10, 2013, 8:33:13 PM11/10/13
to PyNE Dev, Kacper Kowalik
Good to know.
 

>
>>
>> I've recently been in touch with a MacPorts dev about getting the yt
>> package there updated (thanks to a heads up from Katy that it's in
>> there, but is woefully out of date!) and I think that Kurt Schwehr has
>> done some maintenance of various scientific packages in fink
>
>
> That would be awesome
>
>>
>> and might
>> be willing to update those as well.  Is there any interest in trying
>> to have modules for PyNE on XSEDE resources, or is that not a usual
>> platform for deployment?
>
>
> I don't think XSEDE is on anyone's critical path, but that isn't to say that
> it shouldn't be.  Paul Romano might have the most experience...
>
>>
>>
>> Anyway, I'd be interested in seeing if we can pool resources on this
>> front, whether that be in the form of CI servers, VMs, etc etc.
>
>
> I, of course, would be super excited to see this happen.  In thinking about
> this last night, I bet the reason that yt and PyNE hit this problem so hard
> is that we are really trying to be Cython projects.  As such, we are
> stretching
> the existing systems to their limits....where nothing works :).
>

That sounds exactly right to me.  :)

So what can I do to support this effort?  I've been attempting to find
CI infrastructure, but those efforts have stalled due to some upcoming
travel.

We have a lot of access (as does everyone, actually) to a CI service called 
batlab.  They do not have the prettiest interface but what they do have is an
integrated service with every OS we would conceivably support. Additionally 
I think that we could set something up to manage other package creation, 
distribution, and generation via batlab pretty easily through a micro webapp.
I have recently been doing some experimenting with this in support of cyclus.
 
I am going to suggest a joint yt-pyne call to maybe see what is reasonable 
for us to carve out here and hopefully sketch out an architecture or at least 
come up with deliverables.  

Nathan Goldbaum

unread,
Nov 14, 2013, 4:17:38 AM11/14/13
to pyne...@googlegroups.com
Hi Anthony,

Homebrew uses pip to manage pure python dependencies.  See: https://github.com/mxcl/homebrew/wiki/Homebrew-and-Python

I just tried building pyne on my brew-powered mac laptop using a checkout of the PyNE staging branch on github.  I successfully built it but there were two issues, one of which might require a slight modification of the build instructions.

The first issue is due to the fact that Homebrew builds a framework install of python.  This sort of install is integrated with the OS and is incompatible with the --user flag for setup.py, so the install instructions will not work for Homebrew users.  However, I can still build PyNE after omitting the flag.  The second issue is that I need to set DYLD_FALLBACK_LIBRARY_PATH, as noted in the readme.  However, it's not clear where the dylib lives - in my case it was in the build subfolder of the root of the PyNE repository.

Besides those two minor issue I had no issues building and importing PyNE.  It's largely for this ease of building and linking against libraries that I tend to prefer homebrew on OS X over macports and fink.  Anaconda is also nice, but it's a bit fiddly and as others in this thread have noted there are binary compatibility and testing issues that have not yet been resolved.

Sorry for the bit of pontificating - hope the homebrew advice is useful :)

-Nathan

Katy Huff

unread,
Nov 14, 2013, 10:25:14 AM11/14/13
to PyNE Dev
On Thu, Nov 14, 2013 at 1:17 AM, Nathan Goldbaum <natha...@gmail.com> wrote:
Hi Anthony,

Homebrew uses pip to manage pure python dependencies.  See: https://github.com/mxcl/homebrew/wiki/Homebrew-and-Python

I just tried building pyne on my brew-powered mac laptop using a checkout of the PyNE staging branch on github.  I successfully built it but there were two issues, one of which might require a slight modification of the build instructions.

The first issue is due to the fact that Homebrew builds a framework install of python.  This sort of install is integrated with the OS and is incompatible with the --user flag for setup.py, so the install instructions will not work for Homebrew users.  However, I can still build PyNE after omitting the flag.  The second issue is that I need to set DYLD_FALLBACK_LIBRARY_PATH, as noted in the readme.  However, it's not clear where the dylib lives - in my case it was in the build subfolder of the root of the PyNE repository.

Thinking about it, I'm not sure the build dylib is the one you want to link to. The installed destination of the libpyne.dylib is probably the one you want to link to. But, if it's working, hey, it probably doesn't matter!

Paul Wilson

unread,
Nov 14, 2013, 10:33:27 AM11/14/13
to pyne...@googlegroups.com

On 11/14/2013 10:25 AM, Katy Huff wrote:

On Thu, Nov 14, 2013 at 1:17 AM, Nathan Goldbaum <natha...@gmail.com> wrote:
Hi Anthony,

Homebrew uses pip to manage pure python dependencies.  See: https://github.com/mxcl/homebrew/wiki/Homebrew-and-Python

I just tried building pyne on my brew-powered mac laptop using a checkout of the PyNE staging branch on github.  I successfully built it but there were two issues, one of which might require a slight modification of the build instructions.

The first issue is due to the fact that Homebrew builds a framework install of python.  This sort of install is integrated with the OS and is incompatible with the --user flag for setup.py, so the install instructions will not work for Homebrew users.  However, I can still build PyNE after omitting the flag.  The second issue is that I need to set DYLD_FALLBACK_LIBRARY_PATH, as noted in the readme.  However, it's not clear where the dylib lives - in my case it was in the build subfolder of the root of the PyNE repository.

Thinking about it, I'm not sure the build dylib is the one you want to link to. The installed destination of the libpyne.dylib is probably the one you want to link to. But, if it's working, hey, it probably doesn't matter!
Wouldn't it matter if one were to delete the build directory after installation?

Paul
-- 
-- ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ --
Paul Wilson ~ UW-Madison ~ 608-263-0807 ~ cal: http://bit.ly/pphw-cal
Professor, Engineering Physics. ~ http://cnerg.engr.wisc.edu
Faculty Director, Advanced Computing Infrastructure

Katy Huff

unread,
Nov 14, 2013, 10:40:24 AM11/14/13
to PyNE Dev
On Thu, Nov 14, 2013 at 7:33 AM, Paul Wilson <wil...@engr.wisc.edu> wrote:

On 11/14/2013 10:25 AM, Katy Huff wrote:

On Thu, Nov 14, 2013 at 1:17 AM, Nathan Goldbaum <natha...@gmail.com> wrote:
Hi Anthony,

Homebrew uses pip to manage pure python dependencies.  See: https://github.com/mxcl/homebrew/wiki/Homebrew-and-Python

I just tried building pyne on my brew-powered mac laptop using a checkout of the PyNE staging branch on github.  I successfully built it but there were two issues, one of which might require a slight modification of the build instructions.

The first issue is due to the fact that Homebrew builds a framework install of python.  This sort of install is integrated with the OS and is incompatible with the --user flag for setup.py, so the install instructions will not work for Homebrew users.  However, I can still build PyNE after omitting the flag.  The second issue is that I need to set DYLD_FALLBACK_LIBRARY_PATH, as noted in the readme.  However, it's not clear where the dylib lives - in my case it was in the build subfolder of the root of the PyNE repository.

Thinking about it, I'm not sure the build dylib is the one you want to link to. The installed destination of the libpyne.dylib is probably the one you want to link to. But, if it's working, hey, it probably doesn't matter!
Wouldn't it matter if one were to delete the build directory after installation?

Yes, I agree. Best to direct it to the place it's getting *installed.* That's what I intended to indicate with the instructions, but I see now that the instructions are not explicit in this. I'll fix that. 

Nathan, if you used the --user install the right library is going to be either in something like  "~/Library/Python/2.7/lib/python/site-packages/pyne/lib" or something like ".local/python/site-packages/pyne/lib" 

 


Paul
-- 
-- ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ --
Paul Wilson ~ UW-Madison ~ 608-263-0807 ~ cal: http://bit.ly/pphw-cal
Professor, Engineering Physics. ~ http://cnerg.engr.wisc.edu
Faculty Director, Advanced Computing Infrastructure

Anthony Scopatz

unread,
Nov 14, 2013, 12:06:22 PM11/14/13
to PyNE Dev

Thanks for the info about homebrew and Python Nathan.  I was not aware at all that is how it works. 

Also thanks for hammering on PyNE.  I think that the level of trouble you experienced, even though you were able to overcome it, would have been too much for most of our potential users. 

Also thanks for fixing up the docs Katy.  Do you want to make an issue for this?

--

Nathan Goldbaum

unread,
Nov 14, 2013, 12:35:29 PM11/14/13
to pyne...@googlegroups.com
Ah, yes, I see now they live in /usr/local/lib/python2.7/site-packages/pyne/lib.  This was installed without --user.  All homebrew packages are installed to /usr/local.

Glad this was a helpful exercise :)

Katy Huff

unread,
Nov 14, 2013, 1:09:01 PM11/14/13
to PyNE Dev
I'll make an issue for this and other things related to mac installs. Perhaps for now, I'll separate the linux section from the UNIX section. In this, I'll draw out the macports install process, and rely on nathans homebrew exercise to do the same for homebrew. As is, there's still something funky about the only attempted fink install so far, but I'll put my guess of how to make everything work in fink too, and hope that it was specific to that computer's environment.



--
 
---
You received this message because you are subscribed to the Google Groups "PyNE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyne-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Matt Terry

unread,
Nov 14, 2013, 1:10:03 PM11/14/13
to pyne...@googlegroups.com
I've looked at installing matplotlib on mac and the number of ways it can be done is downright staggering.  I think I got to 17 combinations of python/dependencies and this does not include backends.  I don't have the bandwidth for yet another side project, but I'd love to help out.

Katy Huff

unread,
Nov 14, 2013, 1:18:13 PM11/14/13
to PyNE Dev
Cool. What package manager do you use, Redbeard? I use macports. If you use fink, I'd love to have you walk through a clean install of pyne and all its dependencies. In any case, in the next few months, I think help will be appreciated in developing pyne portfiles/brewfiles/pipfiles/finkfiles/whatever. We'll have to see, but I think the current strategy for v0.4 will be to get a one-step pyne install into all conceivable package managers.


--
 
---
You received this message because you are subscribed to the Google Groups "PyNE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyne-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Matt Terry

unread,
Nov 14, 2013, 1:24:33 PM11/14/13
to pyne...@googlegroups.com
Right now I'm using brew, but for MPL I use them all (much to the chagrin of TravisCI).

https://github.com/matplotlib/mpl_mac_testing
It needs a little polish to get all the wires connected, but I'm close to being able to do installation testing for MPL on a bunch of mac environments.

-matt

Katy Huff

unread,
Nov 14, 2013, 4:27:08 PM11/14/13
to PyNE Dev
On Thu, Nov 14, 2013 at 10:24 AM, Matt Terry <matt....@gmail.com> wrote:
Right now I'm using brew, but for MPL I use them all (much to the chagrin of TravisCI).

ouch.
 

https://github.com/matplotlib/mpl_mac_testing
It needs a little polish to get all the wires connected, but I'm close to being able to do installation testing for MPL on a bunch of mac environments.


cool.  

Kevin Manalo

unread,
Nov 18, 2013, 1:30:26 PM11/18/13
to pyne...@googlegroups.com
Hi all, I just wanted to chime in as a completely new user, reading only from the PyNE documents, for the Linux 64-bit side of things, it appears that if I had chosen to work with my Mac there are quite a few more kinks to work out!  I installed on CentOS 5.9 and Linux Mint 15 systems without too much trouble.

Other than maybe having numpy/scipy, the other prerequisites were quite new to me.  In particular, Issue #190 highlighted a confusion regarding cython, so it would be helpful to let new users know how best to install cython (installation via pip seemed to work best).  Secondly, the PyTAPs requirement for the MCNP module is hard to address.  Would it be more useful to make the related objects that use PyTAPs optional/ not loaded by default?  

I went through banging my head on trying to install PyTAPs and dependencies, and this seems to be a pretty high bar for using the MCNP module; I wanted to use the surface source reader portion.

Overall, everything went smoothly enough, it's just the prereq's making some of the hurdles in installation.

Kevin Manalo (Georgia Tech)


On Tuesday, November 5, 2013 12:42:01 PM UTC-6, Anthony Scopatz wrote:
Hello All, 

Elliott Biondo

unread,
Nov 18, 2013, 1:53:59 PM11/18/13
to pyne...@googlegroups.com
PyTAPS should not be a dependency of the MCNP module. Fixing this now. That being said, PyTAPS is not actually that hard to install (for Linux at least), though the PyTAPS documentation is not the most helpful. I can add some PyNE documentation for this.




--
 
---
You received this message because you are subscribed to the Google Groups "PyNE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyne-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.



--
Elliott D. Biondo
Nuclear Regulatory Commission Fellow
Computational Nuclear Engineering Research Group (CNERG)
Department of Engineering Physics
University of Wisconsin - Madison

434 Engineering Research Building
1500 Engineering Drive
Madison, WI 53706

Anthony Scopatz

unread,
Nov 20, 2013, 8:22:27 AM11/20/13
to pyne...@googlegroups.com
Hello Kevin, 

Thanks a ton for chiming in!  It is precisely this feedback that we need and precisely this feedback that will make PyNE better.  

As Elliott mentions and you saw on GitHub, the  PyTAPS dependency is a tough one.  PyTAPS itself is not a huge pain (use pip) but the PyTAPS dependency on MOAB (+others?) does make things a huge pain, even for me! Unfortunately, we cannot rely on the PyTAPS or MOAB developers to fix this problem.  I still think MOAB is the right choice (other options being libmesh and vtk), but install is the cost we pay for that decision.

If you are willing to more broadly dive in and help fix address our install issues, that'd be great!

I'll be sending out a strategy email shortly.

Be Well
Anthony
Reply all
Reply to author
Forward
0 new messages