[Bioperl-l] standardize shebang line in bioperl scripts
Carnë Draug
I was wondering if there's any consensus about what should be used in
the shebang line. Looking at the existing scripts, I found:
#!perl
#!/usr/bin/env perl
#!/usr/bin/perl
#!/usr/bin/perl -w
Using the bioperl distzilla plugin bundle, 'perl' and '/usr/bin/perl'
get converted to the path of perl used for the package installation.
However, if '/usr/bin/env perl' it remains unmodified. This behaviour
seems to be a bug in MakeMaker [1].
To me it makes sense to have the shebang line replaced with the perl
used for installation since that's the one that will have the rest of
bioperl modules. But the most important thing, I think, is to be
consistent.
If no one opposes soon, I will change all of them to '/usr/bin/perl'
so that it gets fixed during install.
I will also replace the '-w' flag with the warnings pragma.
Carnë
[1] https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/issues/58
_______________________________________________
Bioperl-l mailing list
Biop...@mailman.open-bio.org
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Fields, Christopher J
Fields, Christopher J
Carnë Draug
>
> I’d go ahead and implement your plan. My suspicion is that this
> will affect very few users.
>
> chris
I have pushed the changes. This required changes to the following
repos:
Bio-Biblio
Bio-EUtilities
Bio-SearchIO-Writer-BSMLResultWriter
bioperl-live
Not sure if the change is worth a new patch release. I also did not
mention the change on the Changelog because again, I'm not sure if
this worth a note.
The issue remains on the Bio-Graphics package but that's not really
part of bioperl anyway and I never get a reply from them about bug
reports and patches anyway.
I have also removed the `-w` flag on all other modules and scripts,
which covered many other repositories, but I done it on the day I sent
the first email.
Carnë
Fields, Christopher J
...
I have pushed the changes. This required changes to the following
repos:
Bio-Biblio
Bio-EUtilities
Bio-SearchIO-Writer-BSMLResultWriter
bioperl-live
Not sure if the change is worth a new patch release. I also did not
mention the change on the Changelog because again, I'm not sure if
this worth a note.
The issue remains on the Bio-Graphics package but that's not really
part of bioperl anyway and I never get a reply from them about bug
reports and patches anyway.
I have also removed the `-w` flag on all other modules and scripts,
which covered many other repositories, but I done it on the day I sent
the first email.
Carnë
We’re probably overdue for a patch release. I still see some odd random travis-ci fails with bioperl but restarting them seems to fix them, not sure why. I do see a more serious fail on Bio-EUtilities that’s worth investigating, something that is going on with espell:
https://travis-ci.org/bioperl/Bio-EUtilities
chris
Carnë Draug
That's not really because of my change. Looking at the build history
[1], it has been failing for 9 months, since this change [2].
I have fixed it now [3].
Carnë
[1] https://travis-ci.org/bioperl/Bio-EUtilities/builds
[2] https://github.com/bioperl/Bio-EUtilities/commit/f5ad9e6c4d95
[3] https://github.com/bioperl/Bio-EUtilities/commit/c723180987c3
Fields, Christopher J
\o/ Thanks for catching that!
chris
Carnë Draug
The fix was based on your previous comments that some errors in the
XML should not be fatal. Can you share such file to be added to the
test suite?
Carnë
Steven Lembark
Carnë Draug <carandr...@gmail.com> wrote:
> Hi
>
> I was wondering if there's any consensus about what should be used in
> the shebang line. Looking at the existing scripts, I found:
>
> #!perl
> #!/usr/bin/env perl
> #!/usr/bin/perl
> #!/usr/bin/perl -w
>
> Using the bioperl distzilla plugin bundle, 'perl' and '/usr/bin/perl'
> get converted to the path of perl used for the package installation.
> However, if '/usr/bin/env perl' it remains unmodified. This behaviour
> seems to be a bug in MakeMaker [1].
>
> To me it makes sense to have the shebang line replaced with the perl
> used for installation since that's the one that will have the rest of
> bioperl modules. But the most important thing, I think, is to be
> consistent.
>
> If no one opposes soon, I will change all of them to '/usr/bin/perl'
> so that it gets fixed during install.
/usr/bin/perl is the last thing you want to standardize on.
POSIX did away with non-absolute path #! lines -- anything with #!perl
is *old* and no longer works on most systems.
#!/usr/bin/perl is largely unusable as it assumes you are using the
O/S installed versin of perl, which you usually aren't (unless you
want to use 5.8 on Centos or 5.10 on Redhat or 5.16 on Amazon linux).
You normally want to use /opt/perl/bin/perl (linux, SysV),
/usr/local/bin/perl (BSD), or ~/.perlbrew/whatever (just about
anything).
There is no good way other than $PATH to determine the preferred
perl to use. For example, you might change your path when testing
a new install to /opt/perl/5.26.1/bin/perl just to test that release
before symlinking it all into /opt/bin.
Net result: "env" is the only sane way to go.
Luvly thing is that env might be installed in /bin/env or /usr/bin/env
depending on Linux vs. BSD vs. SysV roots of whatever you are running
Perl on (let alone Windows...).
The one most likely to work is:
#!/usr/bin/env perl
linux habitually installs env into /bin with a symlink in /usr/bin;
BSD & SysV install it in /usr/bin maybe-without a symlink in /bin.
I any case using "/usr/bin/perl" either ties you to decade(s) out of
date perl with the O/S distro or forces you to update the disro perl,
which may break ancient pieces of the distro itself, and is nearly
always the wrong thing to do.
--
Steven Lembark 3920 10th Ave South
Workhorse Computing Birmingham, AL 35222
lem...@wrkhors.com +1 888 359 3508
Steven Lembark
Carnë Draug <carandr...@gmail.com> wrote:
> Hi
>
> I was wondering if there's any consensus about what should be used in
> the shebang line. Looking at the existing scripts, I found:
>
> #!perl
> #!/usr/bin/env perl
> #!/usr/bin/perl
> #!/usr/bin/perl -w
>
> Using the bioperl distzilla plugin bundle, 'perl' and '/usr/bin/perl'
> get converted to the path of perl used for the package installation.
> However, if '/usr/bin/env perl' it remains unmodified. This behaviour
> seems to be a bug in MakeMaker [1].
>
> To me it makes sense to have the shebang line replaced with the perl
> used for installation since that's the one that will have the rest of
> bioperl modules. But the most important thing, I think, is to be
> consistent.
>
> If no one opposes soon, I will change all of them to '/usr/bin/perl'
> so that it gets fixed during install.
/usr/bin/perl is the last thing you want to standardize on.
#!/usr/bin/env perl
_______________________________________________
Steven Lembark
Carnë Draug <carandr...@gmail.com> wrote:
> Hi
>
> I was wondering if there's any consensus about what should be used in
> the shebang line. Looking at the existing scripts, I found:
>
> #!perl
> #!/usr/bin/env perl
> #!/usr/bin/perl
> #!/usr/bin/perl -w
>
POSIX did away with non-absolute path #! lines.
#!/usr/bin/perl is largely unusable as it assumes you are using the
O/S installed versin of perl, which you usually aren't (unless you
want to use 5.8 on Centos or 5.10 on Redhat or 5.16 on Amazon linux).
Net result: "env" is the only sane way to go.
Luvly thing is that env might be installed in /bin/env or /usr/bin/env
depending on Linux vs. BSD vs. SysV roots of whatever you are running
Perl on (let alone Windows...).
The one most likely to work is:
#!/usr/bin/env perl
linux habitually installs env into /bin with a symlink in /usr/bin;
BSD & SysV install it in /usr/bin maybe-without a symlink in /bin.
I any case using "/usr/bin/perl" either ties you to decade(s) out of
date perl with the O/S distro or forces you to update the disro perl,
which may break ancient pieces of the distro itself.
--
Steven Lembark 3920 10th Ave South
Workhorse Computing Birmingham, AL 35222
lem...@wrkhors.com +1 888 359 3508
_______________________________________________
Mark Jensen
Sent from my iPhone
Fields, Christopher J
Adam Sjøgren
> During installation the shebang line gets replaced with the perl used
> for installation. But if the shebang line has 'env' then this does
> not happen and you may end up with a botched installation.
>
> So we set /usr/bin/perl, not because that is the best perl to be used,
> but because that will be transformed in the only perl that we know for
> sure can use the modules the scripts are dependent on.
+1
... and thanks for packing bioperl for Debian - much appreciated!
Best regards,
Adam
--
"I mean, what a disappointing 21st century this has Adam Sjøgren
been so far, you know?" as...@koldfront.dk
Carnë Draug
> On Wed, 29 Aug 2018 12:19:54 +0100
> Carn? Draug <carandr...@gmail.com> wrote:
>> Hi
>>
>> I was wondering if there's any consensus about what should be used in
>> the shebang line. Looking at the existing scripts, I found:
>>
>> #!perl
>> #!/usr/bin/env perl
>> #!/usr/bin/perl
>> #!/usr/bin/perl -w
>>
>> get converted to the path of perl used for the package installation.
>> However, if '/usr/bin/env perl' it remains unmodified. This behaviour
>> seems to be a bug in MakeMaker [1].
>>
>> To me it makes sense to have the shebang line replaced with the perl
>> used for installation since that's the one that will have the rest of
>> bioperl modules. But the most important thing, I think, is to be
>> consistent.
>>
>> If no one opposes soon, I will change all of them to '/usr/bin/perl'
>> so that it gets fixed during install.
>
> /usr/bin/perl is the last thing you want to standardize on.
>
>
> POSIX did away with non-absolute path #! lines -- anything with #!perl
> is *old* and no longer works on most systems.
>
> O/S installed versin of perl, which you usually aren't (unless you
> want to use 5.8 on Centos or 5.10 on Redhat or 5.16 on Amazon linux).
> /usr/local/bin/perl (BSD), or ~/.perlbrew/whatever (just about
> anything).
>
> There is no good way other than $PATH to determine the preferred
> perl to use. For example, you might change your path when testing
> a new install to /opt/perl/5.26.1/bin/perl just to test that release
> before symlinking it all into /opt/bin.
>
>
> Luvly thing is that env might be installed in /bin/env or /usr/bin/env
> depending on Linux vs. BSD vs. SysV roots of whatever you are running
> Perl on (let alone Windows...).
>
> The one most likely to work is:
>
> #!/usr/bin/env perl
>
> linux habitually installs env into /bin with a symlink in /usr/bin;
> BSD & SysV install it in /usr/bin maybe-without a symlink in /bin.
>
> I any case using "/usr/bin/perl" either ties you to decade(s) out of
> date perl with the O/S distro or forces you to update the disro perl,
> always the wrong thing to do.
You're missing the point. What gets in the shebang line in the source
release is not what will be in the shebang line after installation.
The source code has three states:
1. the code in the git repo
2. the code in the source release
3. the code installed in the system
During installation the shebang line gets replaced with the perl used
for installation. But if the shebang line has 'env' then this does
not happen and you may end up with a botched installation.
So we set /usr/bin/perl, not because that is the best perl to be used,
but because that will be transformed in the only perl that we know for
sure can use the modules the scripts are dependent on.
This is not a theoretical issue. This issue is exactly the reason why
I'm looking into this. One of the bioperl scripts I was using was no
longer working my env changed to use a perl that had not the bioperl
modules. This is also Debian Policy, which I care about, because I've
been packaging this for Debian.
---
This also means that people calling the scripts from the git repo
can't do it. They should have never done it in the first place. If
people want to run development versions, one needs to build and
install them. It's not only because of the changing of the shebang
line. There may be other changes happening to the source during the
release and installation step. At least, the $version value is set
automatically during release.
Carnë
Carnë Draug
>
> On Wed, 12 Sep 2018 17:10:25 +0100
> Carnë Draug <carandr...@gmail.com> wrote:
>
> > You're missing the point. What gets in the shebang line in the source
> > release is not what will be in the shebang line after installation.
>
>
> If anyone sees #!/usr/bin/perl on the disk they are fried.
>
> It is no harder to replace '#!/usr/bin/env perl' with your preferred
> path than it is to replace '#!/usr/bin/perl'. The difference is that
> with a simple shell script (either their own or provided by you) the
> users can have whichever perl they prefer.
>
>
> > So we set /usr/bin/perl, not because that is the best perl to be used,
> > but because that will be transformed in the only perl that we know for
> > sure can use the modules the scripts are dependent on.
>
>
> In that case the link may be replaced without you having any control
> over were it points to. If you readlink the path and hardwire it then
> you prevent anyone from choosing their own version by setting the path
> or a symlink -- both well-proven approaches on *NIX.
>
> > This is not a theoretical issue. This issue is exactly the reason why
> > I'm looking into this. One of the bioperl scripts I was using was no
> > longer working my env changed to use a perl that had not the bioperl
> > modules. This is also Debian Policy, which I care about, because I've
> > been packaging this for Debian.
>
> to determine the executable in use at any time is a practical approach
> that generally works well. In the case you describe above, if the path
> has to be one specific perl then prefix your path with the path to it
> and /usr/bin/env perl will provide the correct results every time.
>
> The problem is that hardwiring the path oviates any number of other
> effective approaches to managing the executable path that work well
> in a variety of situations.
>
> If having the path to a specific perl is that important to you then
> either prefix it to your path, provide a symlink like
> "/opt/bin/bio-perl" that can be used to run things, provide a shell
> script that prefixes speciric perl's path
> (e.g., /opt/bin/run-bioperl ...).
>
> The point is don't assume that you know more about how someone else
> manages their Perl distributions than they do, or that packaging
> approaches for Debian are suitable for all distributions of Linux,
> Unix, and VMS.
We are using MakeMaker for installation and it's MakeMaker that's
setting the shebang. If you want the shebang to be "env", then set
PERL_MM_SHEBANG=relocatable for the installation.
Any change to this logic, can't be unique to the bioperl project and
will be better discussed in MakeMaker and other toolchains used for
installation.
Fields, Christopher J
...
>
> The point is don't assume that you know more about how someone else
> manages their Perl distributions than they do, or that packaging
> approaches for Debian are suitable for all distributions of Linux,
> Unix, and VMS.
We are using MakeMaker for installation and it's MakeMaker that's
setting the shebang. If you want the shebang to be "env", then set
PERL_MM_SHEBANG=relocatable for the installation.
Any change to this logic, can't be unique to the bioperl project and
will be better discussed in MakeMaker and other toolchains used for
installation.
Carnë
Just so it’s noted for the record, I agree with this. I really don’t want to go down the road of forging a bioperl-specific way of doing things, the current bioperl code base has tons of examples of this already.
It’s also worth noting that we’ve very rarely (if ever) had complaints about issues with scripts related to specific perl paths in the past. That doesn’t mean problems have never occurred, but it does suggest that it’s never risen to the point over the years that someone has complained about it. All Carnë wants to do is standardize this.
chris