run.{bash,bat,rc} in Go

[Uriel]

As part of efforts to remove the last remaining bits of shell from the
build system and replacing it with portable dist C and/or Go, i'm
looking into src/run.*

Would a replacement as src/run.go make sense?

Or maybe the functionality should be merged into test/run.go?

The name run.{bash,bat,rc,go} seems a bit confusing for what it does,
maybe is a good chance to rename it to tests.go or something like
that?

Any thoughts before I start to work on it?

Uriel

[Russ Cox]

Please do not. We are comfortable with how little shell script is
left. You are making work for us for no good reason.

Russ

[Uriel]

What is more work? Instead of having stuff duplicated in (ba)sh, rc
and bat (and whatever else future ports might want to use) you would
have a single (more readable and more efficient to run) .go file.

The build system is so close to being free of all this non-portable
cruft that it is really a shame to lave it like this when it would
take so little to fix it properly.

As it stands the Go distribution can't be included in the default
installs of systems like (to pick a random example) OpenBSD.

And really, what is the downside of having Go code instead of (ba)sh,
rc and bat code?

[hstimer]

For what it is worth, I give my vote for getting rid of all the shell
scripts. Go is superior to any shell
script language out there, and much more portable; I can't imagine why
anyone would want to keep it around,
especially since someone is volunteering to do the work.

I've used a lot of sh with c/c++ projects, but that is because
everything is painful in c/c++. Go is easy, fun, fast,
portable, and expressive, and it does all this without libc! So why
leave old crufty junk around?

[Jason Catena]

This attitude is exactly why I stopped bothering trying to add things to Go (or use it, actually), which probably is just fine with its powers that be. I'm not even really sure why they bothered to open source it, since it's obviously just their language, and not necessarily intended to be useful or improved by anyone else.

[Steven Degutis]

I've noticed that the majority of people with this kind of response just like yours are usually the same kind of people who wish to change Go to allow every feature that goes against its principles and are upset that they can't, for instance "Go shouldn't complain about my unused variables/imports" or "Go really needs generics and is unusable without it" or "Go should let me obtain function values from a given string which I can derive at runtime" or "Go needs to have an interpreter" or "Go is useless because it doesn't have real exceptions". And that's cool. Hey, fork it and add those features man, make the world a better place. It's open source :)

-Steven

[Uriel]

On Sat, Sep 1, 2012 at 4:04 AM, Steven Degutis <sdeg...@8thlight.com> wrote:
> I've noticed that the majority of people with this kind of response just
> like yours are usually the same kind of people who wish to change Go to
> allow every feature that goes against its principles and are upset that they
> can't, for instance "Go shouldn't complain about my unused
> variables/imports" or "Go really needs generics and is unusable without it"
> or "Go should let me obtain function values from a given string which I can
> derive at runtime" or "Go needs to have an interpreter" or "Go is useless
> because it doesn't have real exceptions". And that's cool. Hey, fork it and
> add those features man, make the world a better place. It's open source :)

I think you are confusing two very different groups of people.

First there are those who want Go to have all the bells and whistles
of other mainstream systems, and miss that the main point of Go (as
Rob put it) is that it has so much less, but offers so much more. And
that if Go added all those "features", then Go would become precisely
what it was meant to avoid.

But there is another (much smaller) camp. I know it is difficult for
some to understand, but Go's minimalism (both as language and as
implementation) is not radical enough to please some fundamentalist
followers of the Unix/C/Plan 9 traditions.

The Go maintainers have the difficult task of handling all this
contradictory external expectations while trying to preserve their own
view of what Go should be.

My aim was to propose a change as minimal and uncontroversial as
possible (there would be no functional changes, and code would be
simpler and more maintainable) that would help make Go more palatable
to this second camp. (There are also functional benefits, but not very
significant beyond convenience for people running rather marginal
platforms).

I believe the change I proposed had no downsides, yet, it is the job
of the Go maintainers (as in any other large project) so say "No!" by
default unless they can be convinced that there is enough
justification for a change.

Where to set this bar is a tricky question, and the rather
conservative approach by the maintainers, combined with a perhaps not
very transparent decision making process, has discouraged people like
Jason from being involved, which is extremely unfortunate.

Anyway, sorry for turning this into a philosophical dissertation about
software design an development.

Uriel

[Hans Stimer]

The topic is the pros and cons of getting rid of shell script from the build system:

Pros:

* Potentially more portable

* Improved maintainability

* Shows off Go's flexibility

Cons:

* Bootstrapping more complicated?

* Replacing working code can lead to a temporary regression in stability and functionality

[Uriel]

On Sat, Sep 1, 2012 at 5:49 AM, Hans Stimer <hans....@gmail.com> wrote:
> The topic is the pros and cons of getting rid of shell script from the build
> system:
>
> Pros:
> * Potentially more portable
> * Improved maintainability
> * Shows off Go's flexibility

* (Marginally) more efficient
* Less hassle for people in non-mainstream/standard setups who might
have technical or philosophical objections to installing bash
* Makes it possible for Go to be included as part of the standard
distribution of systems that don't include bash by default (eg.,
OpenBSD).

> Cons:
> * Bootstrapping more complicated?

Not the case at all. Bootstrapping is already handled by the dist command.

Easier perhaps, less system-specific stuff to fix.

> * Replacing working code can lead to a temporary regression in stability and
> functionality

This is the only real downside.

But the amount of code involved is minimal enough that I'd hope the
risks are insignificant.

[Peter S]

I'd add to Cons:
* Time spent by the Go team working on (reviewing, testing, merging etc) functionally equivalent changes is time spent not working on bug fixes, performance optimizations, and feature additions

Just looking at this single change, I might actually personally prefer Go versions over bash scripts, but on the other hand I think there are many other higher priority things to be done with Go (e.g. most issues marked ASAP/Soon/Later), so I prefer if the Go team works on those.

Peter

[Uriel]

On Sat, Sep 1, 2012 at 6:02 AM, Peter S <spete...@gmail.com> wrote:
> I'd add to Cons:
> * Time spent by the Go team working on (reviewing, testing, merging etc)
> functionally equivalent changes is time spent not working on bug fixes,
> performance optimizations, and feature additions
>
> Just looking at this single change, I might actually personally prefer Go
> versions over bash scripts, but on the other hand I think there are many
> other higher priority things to be done with Go (e.g. most issues marked
> ASAP/Soon/Later), so I prefer if the Go team works on those.

Is not a request for work, this is an offer of work (already got a
patch in the works, but the contribution guidelines say one should
discuss changes before submitting patches).

Is clear the codereview process is already strained, so is fine if
such low priority changes are left at the bottom of the review pile to
wait until there is nothing more important to look into.

That obviously is different from an outright rejection of the change.

There are two questions:

1) Is it an improvement over the current code?
2) If so, what priority does it have?

I understand if this is considered extremely low priority by most
people (on the other hand, some people care enough for whatever
reasons to the point of rejecting Go as long as this is not changed).

But I think it is undisputably a positive change (and I have my
reasons to want to do it, among others that I personally know several
people who fall in the "if it depends on bash I refuse to use it"
camp).

Uriel

[Uriel]

On Sat, Sep 1, 2012 at 6:01 AM, Uriel <ur...@berlinblue.org> wrote:
> On Sat, Sep 1, 2012 at 5:49 AM, Hans Stimer <hans....@gmail.com> wrote:
>> The topic is the pros and cons of getting rid of shell script from the build
>> system:
>>
>> Pros:
>> * Potentially more portable
>> * Improved maintainability
>> * Shows off Go's flexibility
>
> * (Marginally) more efficient
> * Less hassle for people in non-mainstream/standard setups who might
> have technical or philosophical objections to installing bash
> * Makes it possible for Go to be included as part of the standard
> distribution of systems that don't include bash by default (eg.,
> OpenBSD).

Forgot another one (which IMHO should be enough justification):

* Dogfooding

[Rémy Oudompheng]

On 2012/9/1 Uriel <ur...@berlinblue.org> wrote:
> What is more work? Instead of having stuff duplicated in (ba)sh, rc
> and bat (and whatever else future ports might want to use) you would
> have a single (more readable and more efficient to run) .go file.

It is compatible with Go habits to have a few duplicated lines.

> The build system is so close to being free of all this non-portable
> cruft that it is really a shame to lave it like this when it would
> take so little to fix it properly.
>
> As it stands the Go distribution can't be included in the default
> installs of systems like (to pick a random example) OpenBSD.
>
> And really, what is the downside of having Go code instead of (ba)sh,
> rc and bat code?

I doubt you would be able to use a single file in an elegant way.
Notably if we have to introduce ulimit hacks for NetBSD/OpenBSD in the
scripts.

Rémy.

[Archos]

When you compile Go in NetBSD/OpenBSD you have to configure it anyway (manually), so it would be better if it were handled from run.go.

Uriel, if you need a tester for it, tell me, since I've all systems installed where Go works.

[Archos]

El sábado, 1 de septiembre de 2012 02:57:32 UTC+1, Jason Catena escribió:

This attitude is exactly why I stopped bothering trying to add things to Go (or use it, actually), which probably is just fine with its powers that be. I'm not even really sure why they bothered to open source it, since it's obviously just their language, and not necessarily intended to be useful or improved by anyone else.

I'm totally agree. But thanks to it, I've found that Rust community (through its main designer, Graydon Hoare) is more friendly on that sense.
i.e., it has been added the she-bang to the language so Rust can be run directly like a shell language.
In addition, the Rust community brings closer to Python community while Go community is going closer to Ruby one.

So I'm not putting all eggs in the same basket. ;)

[Larry Clapp]

On Friday, August 31, 2012 12:38:45 PM UTC-4, Uriel DeLarge wrote:

As part of efforts to remove the last remaining bits of shell from the
build system and replacing it with portable dist C and/or Go, i'm
looking into src/run.*

Would a replacement as src/run.go make sense?

So how would you bootstrap on a new OS or architecture?  You said in another subthread that "go tool dist" solves this problem, but I don't see how.  (But I'm open to enlightenment.  Cross-compile?)

-- Larry

 

[Uriel]

On Sat, Sep 1, 2012 at 3:04 PM, Larry Clapp <la...@theclapp.org> wrote:
> So how would you bootstrap on a new OS or architecture? You said in another
> subthread that "go tool dist" solves this problem, but I don't see how.
> (But I'm open to enlightenment. Cross-compile?)

Look at src/cmd/dist/* and look at run.bash, none of the proposed
changes have any impact whatsoever to bootstrapping, to the contrary,
the reason src/cmd/dist was created was to make bootstrapping in
places like Plan 9 easier, the proposed changes would push further in
that direction.

Uriel

[Larry Clapp]

Ah, thanks.

I see that dist is written in C and not Go.  I guess my point was, don't you invoke dist via "go tools dist", where "go" is, in fact, written in Go?  I'm a newbie at Go; it's probable that the answer is "no, you're wrong, please go away".  :)

Regardless, my point is wrong either way.  As you say, run.X doesn't impact bootstrapping: it's not used to compile Go, but to run tests on a Go tree, where you have a release candidate compiler already compiled.

So if I understand correctly: On a new system, you compile using dist, which is in C, then you use run.bash (et al) to test the new compilers.  You're proposing that you re-write run.bash in Go and get rid of run.bash (et al).

So I guess then the question is (IF I understand this all correctly :), is it valid to test the Go compilers with the Go compilers?  Is it possible (and likely enough to matter :) that they could fail in such a way that the test fails but run.go does not detect the failure?

Again, it's possible that the answer is "please go away".  I'm not challenging you or saying you're wrong, I'm just trying to understand.

-- Larry

[Uriel]

On Sat, Sep 1, 2012 at 5:31 PM, Larry Clapp <la...@theclapp.org> wrote:
> So I guess then the question is (IF I understand this all correctly :), is
> it valid to test the Go compilers with the Go compilers? Is it possible
> (and likely enough to matter :) that they could fail in such a way that the
> test fails but run.go does not detect the failure?

run.bash (et al) already call the go tool, so again, replacing
run.bash (et al) with a (very small) Go 'script' makes no difference.

Uriel

[Rob Pike]

I would also like to ask you to drop this.

1) Only people developing from source use make.bash and all.bash. It
has no bearing on the released installation and so affects very few
people.

2) We still need a way to bootstrap. Yes, the dist command is written
in C, but something needs to be written to configure, to build it, and
to run it. A shell script of some kind is the simplest, easiest, most
direct way of doing that. It doesn't matter much that we need several
such scripts for different systems.

3) Environment variables need to be set, so the shell environment is
already involved anyway.

4) Bootstrapping does not just mean the first compilation on a
machine. It means any fresh build, including cross-compilations and
some debugging runs. You need some rock to hold on to, and the shell
(or a set of shells) is a fine rock. The only real alternative is some
kind of Makefile, but that's jumping from the pan to the fire.

5) The current setup is not a problem to the people working on the
system. It's easy to use and works well and, the occasional tweak
aside (which will be necessary with any setup), it's close to
maintenance-free.

6) Changing the build setup will be a costly undertaking. Even given a
new one to deploy, there will be a lot of testing and vetting and
concomitant fixes to make. That's a lot of time and energy for little
or no gain.

7) Your offer to write the code does not mitigate number 6 at all. It
will likely cost as much time in code reviews and debugging as it does
to write, and then we would still have the issues listed in point 6.

In short, it's not important, it's not broken, it doesn't need fixing,
and it's expensive to do. It's already cost more time than it's worth.
Please drop it.

-rob

[Lucio]

On Saturday, 1 September 2012 18:43:17 UTC+2, Rob Pike wrote:

I would also like to ask you to drop this.

[ ... ]

 

6) Changing the build setup will be a costly undertaking. Even given a
new one to deploy, there will be a lot of testing and vetting and
concomitant fixes to make. That's a lot of time and energy for little
or no gain.

Uriel makes the point that anti-bash developers don't like the existing build
system. I understand that though I don't agree with the sentiment; rather, I
prefer the Go Team view of it.

That said, Go is an odd bod, using an unconventional source installation
approach that for all its good points, will find resistance in the conventional
development community. To be specific, I personally know one NetBSD
developer that would not touch Go unless (until? I can't remember) it could
be installed using the "package" system. I don't agree with him, but I
understand where he is coming from. Having been involved in (pointless)
discussions about the wisdom of using Bash, I have additional cause to
direct this discussion in hat I hope will be a more productive direction.

Perhaps we ought to look at the Go distribution(s) more carefully. Firstly,
we have a good product in hand: it works well in Plan 9, which is
admirable; the only fly in the ointment is that there is no equivalent facility
for Windows in that it requires MinGW and that is an issue in my opinion,
others will no doubt disagree - it's a chapter of its own and here may not
be the best place to discuss it. So we have a great starting point, let's
make sure it does not get disturbed, the price to pay may be as high as
Rob suggests and if it isn't, the comfort of the Go Team is still best decided
by its members.

Now let's look at the downsides, see where the contributors can help. Here I
speak as one of those contributors: in the days when "dist" wasn't available,
I had to shoehorn the Go distribution into the Plan 9 system and it wasn't
always fun to do. To me, there are two paths ahead for those who would like
Go to reach the traditional community: (a) distil the essence from the Go
distribution and pour it into a ".configure"-like environment for the public at
large (what I did and others did for Plan 9) or, more plausibly (b) distil the
essence of the "dist" toolset and make it available to users of the ".configure"
(autoconf, I suppose) approach as an alternative or additional installation tool.

Given (b), I believe that the broader Go community is better placed to produce
such a tool than the Go Team, whose focus has already wandered off
unnecessarily to produce "dist" in the first place. The Go Team, on the other
hand, are going to be good judges as to whether an enhanced version of the
"dist" tool is adequate to their need and is able to replace the original, to the
extent that the original can (once!) be removed from the Go distribution and
released as a separate entity, or re-incorporated into the Go distribution.

Here, we'd have the game changer that has long been advocated by the
anti-autoconf community, but right now, that is a project that should not impact
the development of the Go language. How such a development will proceed is
not really a matter for discussion here.

Just a few thoughts, I reserve the right to adjust my opinions :-)