new builders for Solaris

[shawn....@oracle.com]

Hi,

I have some new builders for Solaris I'd like to add.

These systems are hosted at Oracle and I'm managing them and the golang buildlet:

* 2 Oracle Solaris 11.3 amd64 systems
  host-solaris-oracle-amd64-release-oracle

* 2 Oracle Solaris 11.3 sparc64 systems
  host-solaris-oracle-sparc64-release-oracle

These systems are graciously hosted at Bielefeld university in Germany, although I don't maintain the server, I will be managing the golang buildlet:

* 1 Oracle Solaris 11.3 amd64 system
  host-solaris-oracle-amd64-release-bielefeld

* 1 Oracle Solaris 11.3 sparc64 system
  host-solaris-oracle-sparc64-release-bielefeld

Although they're all running Solaris 11.3 today, whenever the next supported release of Solaris is available, they'll be upgraded, hence the generic naming.

Is this agreeable?  Is there any other information required?

The naming patterns I chose above is: host-<goos>-<goos vendor>-<goarch>-<goos version>-<organization>

Thanks,
-Shawn

[Brad Fitzpatrick]

And what builder names do you plan to use?

Remember, the builder configs are the names that shows up at build.golang.org and represent a machine + Go config. (e.g. builder name "linux-386-387" runs with GOARCH=386 + GO386=387 but runs on host type "host-linux-amd64-kubestd").

Do you intend to load balance builder "solaris-amd64" builds across the two sites? If so, they need the same host type.

If not, do we really need two Oracle Solaris 11.3 builders? I don't think so.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[shawn....@oracle.com]

On Thursday, July 6, 2017 at 2:00:41 PM UTC-7, Brad Fitzpatrick wrote:

And what builder names do you plan to use?

Remember, the builder configs are the names that shows up at build.golang.org and represent a machine + Go config. (e.g. builder name "linux-386-387" runs with GOARCH=386 + GO386=387 but runs on host type "host-linux-amd64-kubestd").

Then, I would assume the two builder config names would be:

solaris-amd64-oraclerel
solaris-sparc64-oraclerel
 

Do you intend to load balance builder "solaris-amd64" builds across the two sites? If so, they need the same host type.

By load balance, what do you mean?  Does that play into numTryTestHelpers?

The test sharding?
 

If not, do we really need two Oracle Solaris 11.3 builders? I don't think so. 

My belief was that the additional builders could be configured as trybots.

If they must all have the same host type, then perhaps these names?

host-solaris-oracle-amd64-oraclerel
host-solaris-oracle-sparc64-oraclerel

-Shawn

[Brad Fitzpatrick]

On Thu, Jul 6, 2017 at 2:12 PM, <shawn....@oracle.com> wrote:

On Thursday, July 6, 2017 at 2:00:41 PM UTC-7, Brad Fitzpatrick wrote:

And what builder names do you plan to use?

Remember, the builder configs are the names that shows up at build.golang.org and represent a machine + Go config. (e.g. builder name "linux-386-387" runs with GOARCH=386 + GO386=387 but runs on host type "host-linux-amd64-kubestd").

Then, I would assume the two builder config names would be:

solaris-amd64-oraclerel
solaris-sparc64-oraclerel

Those look fine.

 

 

Do you intend to load balance builder "solaris-amd64" builds across the two sites? If so, they need the same host type.

By load balance, what do you mean?  Does that play into numTryTestHelpers?

The test sharding?

I don't think we have the Solaris machine capacity on your side or the human maintenance capacity on our side to add more trybots right now.

You'd need more than 2 or 3 machines, and we'd need to react to changes in their availability.

Let's start with just regular post-submit builders for now.

 

If not, do we really need two Oracle Solaris 11.3 builders? I don't think so. 

My belief was that the additional builders could be configured as trybots.

If they must all have the same host type, then perhaps these names?

host-solaris-oracle-amd64-oraclerel
host-solaris-oracle-sparc64-oraclerel

Those look fine.

I'll email you keys.

[shawn....@oracle.com]

On Thursday, July 6, 2017 at 2:19:43 PM UTC-7, Brad Fitzpatrick wrote:

On Thu, Jul 6, 2017 at 2:12 PM, <shawn....@oracle.com> wrote:

On Thursday, July 6, 2017 at 2:00:41 PM UTC-7, Brad Fitzpatrick wrote:

And what builder names do you plan to use?

Remember, the builder configs are the names that shows up at build.golang.org and represent a machine + Go config. (e.g. builder name "linux-386-387" runs with GOARCH=386 + GO386=387 but runs on host type "host-linux-amd64-kubestd").

Then, I would assume the two builder config names would be:

solaris-amd64-oraclerel
solaris-sparc64-oraclerel

Those look fine.

 

 

Do you intend to load balance builder "solaris-amd64" builds across the two sites? If so, they need the same host type.

By load balance, what do you mean?  Does that play into numTryTestHelpers?

The test sharding?

I don't think we have the Solaris machine capacity on your side or the human maintenance capacity on our side to add more trybots right now.

You'd need more than 2 or 3 machines, and we'd need to react to changes in their availability.

Let's start with just regular post-submit builders for now.

Well, the eventual goal is to certainly have them available for TryBots, so when possible, please let me know what I need to do to make that happen.  The builders are fairly over-provisioned relative to what an actual build needs.  As far as I know, at most, maybe a plugin to start Solaris Zones (containers) on demand would be required.
 

 

If not, do we really need two Oracle Solaris 11.3 builders? I don't think so. 

My belief was that the additional builders could be configured as trybots.

If they must all have the same host type, then perhaps these names?

host-solaris-oracle-amd64-oraclerel
host-solaris-oracle-sparc64-oraclerel

Those look fine.

I'll email you keys.

Thanks, I wasn't planning on starting the sparc64 builders until after 1.9 was released since they'll just repeatedly fail right now.  Is that ok?

-Shawn

[Brad Fitzpatrick]

On Thu, Jul 6, 2017 at 2:28 PM, <shawn....@oracle.com> wrote:

On Thursday, July 6, 2017 at 2:19:43 PM UTC-7, Brad Fitzpatrick wrote:

On Thu, Jul 6, 2017 at 2:12 PM, <shawn....@oracle.com> wrote:

On Thursday, July 6, 2017 at 2:00:41 PM UTC-7, Brad Fitzpatrick wrote:

And what builder names do you plan to use?

Remember, the builder configs are the names that shows up at build.golang.org and represent a machine + Go config. (e.g. builder name "linux-386-387" runs with GOARCH=386 + GO386=387 but runs on host type "host-linux-amd64-kubestd").

Then, I would assume the two builder config names would be:

solaris-amd64-oraclerel
solaris-sparc64-oraclerel

Those look fine.

 

 

Do you intend to load balance builder "solaris-amd64" builds across the two sites? If so, they need the same host type.

By load balance, what do you mean?  Does that play into numTryTestHelpers?

The test sharding?

I don't think we have the Solaris machine capacity on your side or the human maintenance capacity on our side to add more trybots right now.

You'd need more than 2 or 3 machines, and we'd need to react to changes in their availability.

Let's start with just regular post-submit builders for now.

Well, the eventual goal is to certainly have them available for TryBots, so when possible, please let me know what I need to do to make that happen.  The builders are fairly over-provisioned relative to what an actual build needs.  As far as I know, at most, maybe a plugin to start Solaris Zones (containers) on demand would be required.

Sure, if you want to work on that.

Or you could do it all on your side, too.

You can write a thing on your side that polls http://farmer.golang.org/status/reverse.json and if it sees the "Waiters" count for your host type go up, you create more containers in Zones. If you see them idle, kill them off.

e.g. currently I see:

"host-linux-arm5spacemonkey": {

"HostType": "host-linux-arm5spacemonkey",

"Connected": 2,

"Expect": 3,

"Idle": 0,

"Busy": 2,

"Waiters": 42,

"Machines": {

"go-builder-2": {

"Name": "go-builder-2",

"HostType": "host-linux-arm5spacemonkey",

"ConnectedSec": 2694.805564621,

"BusySec": 2694.805521922,

"Version": "11",

"Busy": true

},

"go-builder-3": {

"Name": "go-builder-3",

"HostType": "host-linux-arm5spacemonkey",

"ConnectedSec": 2699.065982825,

"BusySec": 2699.065924534,

"Version": "11",

"Busy": true

}

}

},

... it has 42 builds waiting to run, if only there were more than two ARM5 machines.

[shawn....@oracle.com]

On Thursday, July 6, 2017 at 2:32:26 PM UTC-7, Brad Fitzpatrick wrote:

On Thu, Jul 6, 2017 at 2:28 PM, <shawn....@oracle.com> wrote:

On Thursday, July 6, 2017 at 2:19:43 PM UTC-7, Brad Fitzpatrick wrote:

On Thu, Jul 6, 2017 at 2:12 PM, <shawn....@oracle.com> wrote:

On Thursday, July 6, 2017 at 2:00:41 PM UTC-7, Brad Fitzpatrick wrote:

And what builder names do you plan to use?

Remember, the builder configs are the names that shows up at build.golang.org and represent a machine + Go config. (e.g. builder name "linux-386-387" runs with GOARCH=386 + GO386=387 but runs on host type "host-linux-amd64-kubestd").

Then, I would assume the two builder config names would be:

solaris-amd64-oraclerel
solaris-sparc64-oraclerel

Those look fine.

 

 

Do you intend to load balance builder "solaris-amd64" builds across the two sites? If so, they need the same host type.

By load balance, what do you mean?  Does that play into numTryTestHelpers?

The test sharding?

I don't think we have the Solaris machine capacity on your side or the human maintenance capacity on our side to add more trybots right now.

You'd need more than 2 or 3 machines, and we'd need to react to changes in their availability.

Let's start with just regular post-submit builders for now.

Well, the eventual goal is to certainly have them available for TryBots, so when possible, please let me know what I need to do to make that happen.  The builders are fairly over-provisioned relative to what an actual build needs.  As far as I know, at most, maybe a plugin to start Solaris Zones (containers) on demand would be required.

Sure, if you want to work on that.

Yes, to be perfectly clear, I was volunteering for that work, not expecting anyone else to do it.

 

Or you could do it all on your side, too.

My understanding was that this was a "simple" plugin to the buildlet/builder system.  Is there any disadvantage/advantage to doing it one way or another?

I'm willing to implement whatever the preferred solution is.
 

You can write a thing on your side that polls http://farmer.golang.org/status/reverse.json and if it sees the "Waiters" count for your host type go up, you create more containers in Zones. If you see them idle, kill them off.

Yes, that's possible as well.  The only question I have is what the requirements are for the container in this situation.  For example, can I reuse a previously created container again and again?  Do I have to wipe it between each reuse?  I ask because currently, Solaris Zone creation is not a lightweight process, so my preference would be to simply pre-install a pool of them and then start and stop them on demand in such a scheme.

My assumption was that, in this case, there weren't any special requirements -- if it's good enough to run a buildlet, that's enough.

Ah, if you're looking at a queue depth of 50, then I understand why a handful of builders isn't enough.  Now things make a lot more sense.

Thanks,
-Shawn

[Brad Fitzpatrick]

On Thu, Jul 6, 2017 at 2:43 PM, <shawn....@oracle.com> wrote:

On Thursday, July 6, 2017 at 2:32:26 PM UTC-7, Brad Fitzpatrick wrote:

On Thu, Jul 6, 2017 at 2:28 PM, <shawn....@oracle.com> wrote:

On Thursday, July 6, 2017 at 2:19:43 PM UTC-7, Brad Fitzpatrick wrote:

On Thu, Jul 6, 2017 at 2:12 PM, <shawn....@oracle.com> wrote:

On Thursday, July 6, 2017 at 2:00:41 PM UTC-7, Brad Fitzpatrick wrote:

And what builder names do you plan to use?

Remember, the builder configs are the names that shows up at build.golang.org and represent a machine + Go config. (e.g. builder name "linux-386-387" runs with GOARCH=386 + GO386=387 but runs on host type "host-linux-amd64-kubestd").

Then, I would assume the two builder config names would be:

solaris-amd64-oraclerel
solaris-sparc64-oraclerel

Those look fine.

 

 

Do you intend to load balance builder "solaris-amd64" builds across the two sites? If so, they need the same host type.

By load balance, what do you mean?  Does that play into numTryTestHelpers?

The test sharding?

I don't think we have the Solaris machine capacity on your side or the human maintenance capacity on our side to add more trybots right now.

You'd need more than 2 or 3 machines, and we'd need to react to changes in their availability.

Let's start with just regular post-submit builders for now.

Well, the eventual goal is to certainly have them available for TryBots, so when possible, please let me know what I need to do to make that happen.  The builders are fairly over-provisioned relative to what an actual build needs.  As far as I know, at most, maybe a plugin to start Solaris Zones (containers) on demand would be required.

Sure, if you want to work on that.

Yes, to be perfectly clear, I was volunteering for that work, not expecting anyone else to do it.

 

Or you could do it all on your side, too.

My understanding was that this was a "simple" plugin to the buildlet/builder system.  Is there any disadvantage/advantage to doing it one way or another?

If you put it in the coordinator that'd be ideal, but more work. You'd need a little protocol it could speak to something on your end to create & destroy things and get state, etc.

 

 

I'm willing to implement whatever the preferred solution is.
 

You can write a thing on your side that polls http://farmer.golang.org/status/reverse.json and if it sees the "Waiters" count for your host type go up, you create more containers in Zones. If you see them idle, kill them off.

Yes, that's possible as well.  The only question I have is what the requirements are for the container in this situation.  For example, can I reuse a previously created container again and again?  Do I have to wipe it between each reuse?  I ask because currently, Solaris Zone creation is not a lightweight process, so my preference would be to simply pre-install a pool of them and then start and stop them on demand in such a scheme.

My assumption was that, in this case, there weren't any special requirements -- if it's good enough to run a buildlet, that's enough.

They should be new worlds between invocations. One buggy CL shouldn't impact a future non-buggy CL's test results.

But if you destroy all the processes and just want to reuse the filesystem, the buildlet will create a temp dir and/or wipe a directory on startup.

[shawn....@oracle.com]

 
Ok, I will consider both options.  Thanks for the suggestions and explanation.

 

 

I'm willing to implement whatever the preferred solution is.
 

You can write a thing on your side that polls http://farmer.golang.org/status/reverse.json and if it sees the "Waiters" count for your host type go up, you create more containers in Zones. If you see them idle, kill them off.

Yes, that's possible as well.  The only question I have is what the requirements are for the container in this situation.  For example, can I reuse a previously created container again and again?  Do I have to wipe it between each reuse?  I ask because currently, Solaris Zone creation is not a lightweight process, so my preference would be to simply pre-install a pool of them and then start and stop them on demand in such a scheme.

My assumption was that, in this case, there weren't any special requirements -- if it's good enough to run a buildlet, that's enough.

They should be new worlds between invocations. One buggy CL shouldn't impact a future non-buggy CL's test results.

But if you destroy all the processes and just want to reuse the filesystem, the buildlet will create a temp dir and/or wipe a directory on startup.

These are essentially VMs, so yes, every time I stop one, all processes will disappear.  The only thing that would be reused is the filesystem on startup.

-Shawn
 

[shawn....@oracle.com]

Ok, I have the amd64 release builder running.  Would you be so kind as to generate two additional keys for me for the development builders?

The builder config names will be:

solaris-amd64-oracledev
solaris-sparc64-oracledev

The host types will be:

host-solaris-oracle-amd64-oracledev

host-solaris-oracle-sparc64-oracledev

Thanks,
-Shawn

[heng....@gmail.com]

Hi Shawn:

Is there any plans to build on Solaris 10 amd64? 

在 2017年7月7日星期五 UTC-4下午2:26:59，shawn....@oracle.com写道：

[Aram Hăvărneanu]

No, The go port uses APIs that are only available in Solaris 11 and
later (and derivatives).