build-env vs. build.rxt

[Brent Villalobos]

I see that rez-build bez plugin will produce a build-env script and a build.rxt file for each variant.  I'm wondering what the difference is in the environment created running build-env vs. rez-env -i build.rxt?  At a cursory glance, the shells look that same.  In fact, looking at the bez build plugin, the build-env script is really just loading the build.rxt under the hood.  Is the build-env script more of a convenience or is there more going on under the hood?  Thanks.

-Brent

[Allan Johns]

The build-env script exists only so that you can enter an interactive shell within the build env, so you can repeatedly run make/whatever without needing to re-resolve the environment. Basically this is the way you repeatedly perform builds as quickly as possible, given that build settings/requirements aren't changed.

The differences between this and "rez-env -i build.rxt" are:

* The rex "building" var is set to True - however you can get the same result using the rez-env -b flag;

* The relevant build-related env-vars (see https://github.com/nerdvegas/rez/wiki/Environment-Variables#resolved-build-environment-variables) are set. Rez-env won't set these, it has no concept of a current package/variant being built.

Hth

A

ps - Use of bez is discouraged, in favor of build_command, see:
https://github.com/nerdvegas/rez/wiki/Package-Definition-Guide#preprocess
https://github.com/nerdvegas/rez/wiki/Building-Packages#custom-build-commands

Note though that I don't think I have the build-env script support done for custom build commands yet..

On Sat, Jun 3, 2017 at 6:54 AM, Brent Villalobos <brent...@gmail.com> wrote:

I see that rez-build bez plugin will produce a build-env script and a build.rxt file for each variant.  I'm wondering what the difference is in the environment created running build-env vs. rez-env -i build.rxt?  At a cursory glance, the shells look that same.  In fact, looking at the bez build plugin, the build-env script is really just loading the build.rxt under the hood.  Is the build-env script more of a convenience or is there more going on under the hood?  Thanks.

-Brent

--
You received this message because you are subscribed to the Google Groups "rez-config" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+unsubscribe@googlegroups.com.
To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.

[Brent Villalobos]

Thanks Allan for shedding light between the two approaches.  One suggestion for the build-env functionality is to have a "batch" mode for them.  I can envision a scenario for build infrastructures distributed across multiple hosts where a master process bakes out the build-env scripts for each variant and then different hosts pick up and run a build after initializing their environment running build env.  Something like:

[path to variant build dir]/build-env -- scons @install --foo --bar

Maybe that's getting too redundant since the remote hosts could easily just run this:

rez-build --variants ## -- @install --foo --bar

But using build-env does save some time by not re-resolving the environment.  I noticed that there is no --timestamp flag to rez-build or rez-release anymore which also means without the pre-resolved environments you run the risk of different variants resolving differently if they run over a long period of time (such as the case when you may have builds waiting for remote machines to become available).  So having a baked out build environment in a batch environment would be handy.

PS: Thanks for the tip about not using bez.  I only used it as an example in the post since it's something that this community might better understand rather than my custom scons rez plugin.

-Brent

On Friday, June 2, 2017 at 5:42:36 PM UTC-7, allan.johns wrote:

The build-env script exists only so that you can enter an interactive shell within the build env, so you can repeatedly run make/whatever without needing to re-resolve the environment. Basically this is the way you repeatedly perform builds as quickly as possible, given that build settings/requirements aren't changed.

The differences between this and "rez-env -i build.rxt" are:

* The rex "building" var is set to True - however you can get the same result using the rez-env -b flag;

* The relevant build-related env-vars (see https://github.com/nerdvegas/rez/wiki/Environment-Variables#resolved-build-environment-variables) are set. Rez-env won't set these, it has no concept of a current package/variant being built.

Hth

A

ps - Use of bez is discouraged, in favor of build_command, see:
https://github.com/nerdvegas/rez/wiki/Package-Definition-Guide#preprocess
https://github.com/nerdvegas/rez/wiki/Building-Packages#custom-build-commands

Note though that I don't think I have the build-env script support done for custom build commands yet..

On Sat, Jun 3, 2017 at 6:54 AM, Brent Villalobos <brent...@gmail.com> wrote:

I see that rez-build bez plugin will produce a build-env script and a build.rxt file for each variant.  I'm wondering what the difference is in the environment created running build-env vs. rez-env -i build.rxt?  At a cursory glance, the shells look that same.  In fact, looking at the bez build plugin, the build-env script is really just loading the build.rxt under the hood.  Is the build-env script more of a convenience or is there more going on under the hood?  Thanks.

-Brent

--
You received this message because you are subscribed to the Google Groups "rez-config" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.

[Allan Johns]

Heya,

Oh the build env isn't timestamped any more? I don't think that was intentional, could you put in a ticket for this? I agree that variants of a package should be built to the same timestamp.

RE remote builds, there's a skeleton BuildProcess class that is intended to provide the functionality you describe, it just hasn't been done yet. My thoughts on this are to basically dispatch the build to various configured hosts, probably just based on packages present in each variant initially. For eg you could configure so that variants containing 'os-centos' go to ['host-cent-01', 'host-cent-2'], etc. There'd probably also be an option to explicitly specify the build host for each variant.

It's on the (rather long) todo list... ;)

A

To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+unsubscribe@googlegroups.com.

[Brent Villalobos]

To clarify, build-env environments are timestamped.  But it looks like build-env can only be used to instantiate an interactive shell and can't be used for non-interactive batch processing.  That means for automated build and CI systems I need to run rez-build which does not have a --timestamp flag or a way to pass in the build.rxt file as far as I can tell. That means that any type of automated build system will always need to resolve to the latest as far as I can tell.  But I'm very new to rez-build so maybe I'm wrong.  But I can submit a ticket to either have build-env support non-interactive workflows or for rez-build to accept timstamps.

As far as rez managing remote hosts for builds, that sounds a bit like feature or domain creep to me.  It may be helpful for an individual that has a handful of machines under his/her desk.  But for larger scale build farms, those are usually managed by frameworks like Jenkins, Bamboo, ElectricFlow, etc.  I think it's best that rez-build runs within those frameworks and let them handle the resource allocation.  In other words, no worries if remote builds are a really low priority.

-Brent