Path issues

[Chiil]

In the previous post we had some discussion about path issues. You
guys had some suggestions about putting /opt/arch/bin as the last
directory in the PATH string.

This has as an implication that none of the scripts using sed work
properly. For instance the latex packages cannot be built now, because
Apple's sed cannot handle the command. What needs to be done is either
add the full path to sed or to think of another solution.

Somehow I would prefer to have pacman installed packages as the first
element of the list, especially for those packages which run at more
recent versions as the ones provided by Apple, fink or MacPorts (in my
case svn, git, texlive and gcc). However, this brings up the issue
again of the coreutils package overriding Apple's coreutils.

Maybe it's good to discuss a bit about how to make an install that
should in principle work for everybody and which has not the
potentially dangerous effect of overriding any of Apple's tool.

[Kevin Barry]

(Quick idea to fix your problem is to link /bin/sed to /opt/arch/bin/sed and leave apple sed at /usr/bin. Rant follows.)

Apple's sed is braindead. There is no reason for them to make their sed that is so strongly incompatible with GNU sed when the rest of the world uses GNU sed.. The issue is that sed (GNU and Apple) support the -i argument to mean inplace, as in edit the file that it is reading. Naturally sometimes when you do this you want to have a backup file, so 'sed -i~ foo' is often used, but most build scripts do not want or need such a backup, so they will say 'sed -i foo' which GNU/sed happily understands, however Apple sed thinks the backup should be suffixed with foo now and complains about no input file.
Apple sed happily accepts "sed -i '' foo' to do what you would expect it to, however GNU/sed will not. GNU/sed will accept "sed -i'' foo" (no space) but Apple sed will not.

The only portable solution is to use a backup extension (and maybe delete the backup after sed completes). You will notice that in many PKGBUILDs I did this, so it will run with either sed, however it is hard to catch them all, and I have sense made /bin/sed point to /opt/bin/sed to build other tools more easily (optware for dd-wrt routers comes to mind).

[Loic Nageleisen]

Besides, I noticed that depending of args order you have to give the
expression with -e if you give -i and a file, else it would take one
for the other.

So for a portable way that gives:
sed -i~ -e 's#foo#bar#' qux

By the way they're not alone to use BSD sed (by default): of course
BSD, also AIX, and I don't know for Solaris.

It's not much more braindead that say, ls which won't taker --colors,
or du which won't take --max--depth, unless they're GNU (and GPL). BSD
wants to be BSD licensed, so they have BSD tools. GNU went with their
enhancements, and licensed them GPL. So BSD can't have them without
making their tools GPL (or reimplementing, like they did with -h for
sizes). All of this is fairly historical and I'm fairly sure that
there's more BSD-like implementations of sed in all unix variants than
tere is behaving like GNU sed. We're just biased because there's a lot
of Linux out there.

[Kevin Barry]

When I first ran into the incompatibility I did think it was true on FreeBSD as well (the Mac sed man page is from FreeBSD) however I had since thought that wasn't true for some reason, but I tried on FreeBSD and sure indeed they are the same, so I can't just blame Apple for this. But it is much more frustrating than ls --colors. ls --colors is for a user, but sed is used in scripts all the time. Having that kind of incompatibility in scripts really hurts the cross platform nature.

[Loic Nageleisen]

They're painful indeed but that's upstream (i.e script writer) fault.
This reminds me of that bash/dash as /bin/sh argument, not the fact
that all scripts need to be written as sh, but that if they start
with /bin/sh shebang, then they shall be sh compliant and not use bash
features. If so, then they should shebang to /bin/bash. And so it was
upstream task to either change their script to point to /bin/bash or
make them /bin/sh compatible.

Sadly we are in a different calling case so we can't change shebang
when there's none. My feeling is that if we encounter a script failing
on BSD (really, non-GNU in fact and not only BSD) sed then we should
notify upstream instead of trying to hack things around (as this will
bite us back at some point, and only induce complexity later on).
Notifying upstream will be beneficial to all, even non-ArchOSX users,
and starting with upstream itself. This is also at the core of the
Arch philosophy, where staying as vanilla as possible is a consequence
of KISS.

So we have a problem with scripts depending on GNU sed, that I totally
agree, but hacks are out of question to get out of this. We support
Darwin as a platform, so I feel we should not force GNU tools on
people, the same way upstream should not force them on us. A user
should be able to set his path in the order he wants, prioritizing BSD
or GNU, and it should just work.

BTW I'd like to voice my personal preference of using non-GNU tools on
my machine for some reason, including that I use them regularly on
various platforms at work.

On Aug 6, 7:38 pm, Kevin Barry <bar...@gmail.com> wrote:
> When I first ran into the incompatibility I did think it was true on FreeBSD
> as well (the Mac sed man page is from FreeBSD) however I had since thought
> that wasn't true for some reason, but I tried on FreeBSD and sure indeed
> they are the same, so I can't just blame Apple for this. But it is much more
> frustrating than ls --colors. ls --colors is for a user, but sed is used in
> scripts all the time. Having that kind of incompatibility in scripts really
> hurts the cross platform nature.
>
> On Thu, Aug 6, 2009 at 3:50 AM, Loic Nageleisen

> <loic.nagelei...@gmail.com>wrote:

[Chiil]

For me, adding full paths to the sed command in the makefiles is not
an ugly hack, but a prerequisite for smooth behaviors regardless of
the path order. However, in my impression, to make this project
unique, I would aim on bringing ArchOSX as close as possible to the
Arch Linux experience, without breaking the link to Apple's core
libraries. This means keeping Apple's libc, binutils and gcc, but
switch the user experience to as much GNU as possible, using GNU's
coreutils and streameditor etc.

I would prefer to copy Arch repo style (core, extra, community) and
add an additional one for mac. To me the best approach would be first
to go trough core (a lot has already been done) and strip out all the
hardware related packages (kernel, video drivers) until we have a core
repo that works perfectly on top of OSX but with a nice Linux-like
experience. And the most important thing that we can add compared to
macports, is that we have packages which are mostly up to date.

What is your vision?

[Loic Nageleisen]

By adding full paths to sed in makefiles, you mean hard-pointing to /
opt/arch/bin/sed?
What I dubbed a hack is relying on /bin/sed as a link to /opt/arch/bin/
sed. We should never have to modify anything outside /opt/arch, and
our tools should work with whatever one is set first on the path. If
that means patching to hardcode /opt/arch/bin/sed when they require
GNU sed, then I'm totally fine with that. Ironically I think this
patching is just a sed command away.

My vision is that ArchOSX is just that, Arch on OSX, but neither
ArchLinux or OSX nor GNU/Arch. IOW we're not porting ArchLinux to OSX,
but 'only' its philosophy. This project should really live on its own,
not be somehow a sub-project of AL. This does not mean that people
wanting to use GNU tools cannot, and they should be free to do so.

As for repos, I'm ok with whatever repo layout idea, as long as it is
simple. ArchLinux one suits me, it's just that we're currently not
enough to really warrant such a structure, but why not. By the way, I
regard the move to git and github a nice one, as it makes it a much
improved decentralized alternative to the AUR.

[Kevin Barry]

I disagree with pointing to /opt/arch/bin/sed in Makefiles unless really necessary. Better is to just use the cross-platform method by creating a backup file:

sed -i~ -e 's/foo/bar/g' XXX

I have already done in this in PKGBUILDs where I found the problem (And for a while avoided installing GNU/sed to help catch further problems)

This works with either sed and with the PATH in any order. Having /opt/arch/bin/sed hardcoded prevents our PKGBUILDs from ever workin on archlinux (I know they don't already, but I would like to move in that direction if possible). Having just sed and assuming GNU or BSD is obviously wrong also.

My suggestion for /bin/sed was really just a way to solve the problem, not a solution for arch-osx. I've just found that many softwares I compile outside of arch-osx also require sed hacks and, for example, optware, was a pain to try to fix the sed commands (And wouldn't respect my PATH either) so I just caved and did the /bin/sed hack.

[Loic Nageleisen]

Indeed. The point I was trying to make is that maybe some package
would require a GNU only command (not necessarily sed) for which we
would have no alternative but to use full path to GNU.

By the way, rest assured that I took your workaround for precisely
that, not a permanent fix. I just wanted to clear things out, like
better be thorough than leave place for misunderstandings later on.

On Aug 7, 7:16 pm, Kevin Barry <bar...@gmail.com> wrote:
> I disagree with pointing to /opt/arch/bin/sed in Makefiles unless really
> necessary. Better is to just use the cross-platform method by creating a
> backup file:
>
> sed -i~ -e 's/foo/bar/g' XXX
>
> I have already done in this in PKGBUILDs where I found the problem (And for
> a while avoided installing GNU/sed to help catch further problems)
>
> This works with either sed and with the PATH in any order. Having
> /opt/arch/bin/sed hardcoded prevents our PKGBUILDs from ever workin on
> archlinux (I know they don't already, but I would like to move in that
> direction if possible). Having just sed and assuming GNU or BSD is obviously
> wrong also.
>
> My suggestion for /bin/sed was really just a way to solve the problem, not a
> solution for arch-osx. I've just found that many softwares I compile outside
> of arch-osx also require sed hacks and, for example, optware, was a pain to
> try to fix the sed commands (And wouldn't respect my PATH either) so I just
> caved and did the /bin/sed hack.
>
> On Fri, Aug 7, 2009 at 1:10 PM, Loic Nageleisen

> <loic.nagelei...@gmail.com>wrote:

> > > > > > > > Maybe it's...
>
> read more »