cmake.py and MSVC nmake -J option

[sergei nazarenko]

Hi!

nmake refused compile with -j option.

I added condition to file rez/rezplugins/build_system/cmake.py 

181  if not platform_.name == 'windows':

182           if not any(x.startswith("-j") for x in (self.child_build_args or [])):

...                n = variant.config.build_thread_count or cpu_count()

...                cmd.append("-j%d" % n)

Has rez got an option to override plugins? For example I would like to create my own plugin build_system/cmake.py. 

Right now I have to remove rez/rezplugins/build_system/cmake.py to force rez to see my build_system/cmake.py plugin from my package

Thanks,

Sergiy

[allan.johns]

Hey Sergiy,

You're probably going to have problems overriding the builtin plugins, and I don't think it'd be a good thing to have anyway.

But please do submit your update via a PR. Have any other Windows users run into this problem?

Cheers
A

[Fabio Piparo]

Yeah, more or less.

nmake doesn't have a -j option, as it handles this at the compiler stage, and passes the flag /MP to cl.exe.

I have run into a few issues and I have this branch where I have added a few things to work, and basically that is:

-bind plugin for msvc, 

-build_system/cmake.py some back/forward slash replacement, removed -j when on windows 

-added a target package to manage 32 vs 64 bit builds on windows

rez-bind msvc will try to install any versions found by looking up where vc is installed from registry, but when using python 64 bit, that doesn't work, but you can pass the flag --root where vcvarsall.bat is and install them one-by-one.

There is also a change that will install a bunch of .bat files on windows, because this is the only way I can get rez to work - still don't know it works for everybody else :( 

@sergei something like this from my branch should get you there:

cd rez_src
python setup.py install
rez-bind os
rez-bind arch
rez-bind platform
rez-bind target
rez-bind python
rez-bind msvc
rez-env msvc-10 -- cl.exe
Microsoft (R) C/C++ Optimizing Compiler Version 16.00.40219.01 for x64
...
rez-env target-AMD32 msvc-10 -- cl.exe
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.40219.01 for 80x86

Btw, you don't have to require the target package, as by default it will be part of the the implicit package arch, so "rez-env msvc-10" will resolve in target-%REZ_ARCH_VER%, which will most likely be always 64 bit.

Also, I've been able to use the cmake plugin to build a few packages (ilmbase, boost, pyqt, sip) from the repository on windows, but if you want to use your own version, you should only have one cmake.py file visible in the REZ_PLUGIN_PATH.

@Allan, do you think the target package is the best approach for this? ie: this is how I install the different variants, and this is how I request a specific variant.

[Seth Lippman]

Fabio,
  Thank you very much for posting this. I think Rez is a great toolset, yet I too am having trouble getting it to work on Windows. I have used Rez at a studio I worked for previously and I would very much like to continue to use it, but my current situation requires that I am able get it to function on all three of the major platforms. Linux? Check. Darwin? Check. Windows? Not yet. But your branch seems to have got me a few steps closer, so thank you for that!  Let me rewind a bit and give you a bit of history. I initially was working on the resources2 branch, but when I started getting errors I noticed that Allan had merged to, and continued to update, the master branch. I pulled all the latest from the master, and tried a very simple build but was immediately hit with errors that looked alot like windows pathing issues:

(I replaced some paths / names with XXX for security reasons)

>rez-build

--------------------------------------------------------------------------------
Building os-windows-6.1.7601.SP1...
--------------------------------------------------------------------------------
Resolving build environment:
resolved by XXX, on Sat Nov 14 23:39:40 2015, using Rez v2.0.rc1.33

requested packages:
~platform==windows         (implicit)
~arch==AMD64               (implicit)
~os==windows-6.1.7601.SP1  (implicit)

resolved packages:

Invoking cmake build system...
Executing: c:\program files (x86)\cmake\bin\cmake.exe -d XXX\code\os\windows-6.1.7601.SP1 -Wno-dev -DCMAKE_ECLIPSE_GENERATE_SOURCE_PROJECT=TRUE -D_ECLIPSE_VERSION=4.3 --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX=~/packages\os\windows-6.1.7601.SP1 -DCMAKE_MODULE_PATH=%CMAKE_MODULE_PATH% -DCMAKE_BUILD_TYPE=Release -DREZ_BUILD_TYPE=local -DREZ_BUILD_INSTALL=0 -G NMake Makefiles
23:39:41 ERROR    Error detecting physical core count, defaulting to 1: invalid literal for int() with base 10: '8\r\r\n\r\r\n\r\r\nNumberOfCores'
Not searching for unused variables given on the command line.
-- The C compiler identification is MSVC 19.0.23026.0
-- The CXX compiler identification is MSVC 19.0.23026.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe
CMake Error at CMakeLists.txt:2 (set):
  Syntax error in cmake code at

    XXX/code/os/windows-6.1.7601.SP1/build/CMakeFiles/CMakeTmp/CMakeLists.txt:2

  when parsing string

    c:\temp\rez\lib\site-packages\rez-2.0.rc1.33-py2.7.egg\rezplugins\build_system\cmake_files

  Invalid character escape '\l'.


CMake Error: Internal CMake error, TryCompile configure of cmake failed
-- Configuring incomplete, errors occurred!
See also "XXX/code/os/windows-6.1.7601.SP1/build/CMakeFiles/CMakeOutput.log".
rez: BuildError: The cmake build system failed

As Windows is the platform I have the least amount of production experience with, I was glad to find your thread noting that you had made some adjustments to the cmake process regarding windows back/forward slash replacement, because I was suspicious that this was at least one of the issues causing me trouble.

I pulled your branch and tried again to rez-build, this time getting closer:
>rez-build

--------------------------------------------------------------------------------
Building os-windows-6.1.7601.SP1...
--------------------------------------------------------------------------------
Resolving build environment:
resolved by XXXX, on Sat Nov 14 23:52:29 2015, using Rez v2.0.rc1.28

requested packages:
~platform==windows         (implicit)
~arch==AMD64               (implicit)
~os==windows-6.1.7601.SP1  (implicit)

resolved packages:

Invoking cmake build system...
Executing: c:\program files (x86)\cmake\bin\cmake.exe -d XXX\code\os\windows-6.1.7601.SP1 -Wno-dev -DCMAKE_ECLIPSE_GENERATE_SOURCE_PROJECT=TRUE -D_ECLIPSE_VERSION=4.3 --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX=XXX/packages\os\windows-6.1.7601.SP1 -DCMAKE_MODULE_PATH=%CMAKE_MODULE_PATH% -DCMAKE_BUILD_TYPE=Release -DREZ_BUILD_TYPE=local -DREZ_BUILD_INSTALL=0 -G NMake Makefiles
Not searching for unused variables given on the command line.
-- The C compiler identification is MSVC 19.0.23026.0
-- The CXX compiler identification is MSVC 19.0.23026.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe -- broken
CMake Error at c:/Program Files (x86)/CMake/share/cmake-3.3/Modules/CMakeTestCCompiler.cmake:61 (message):
  The C compiler "C:/Program Files (x86)/Microsoft Visual Studio
  14.0/VC/bin/cl.exe" is not able to compile a simple test program.

  It fails with the following output:

   Change Dir: XXX/code/os/windows-6.1.7601.SP1/build/CMakeFiles/CMakeTmp



  Run Build Command:"nmake" "/NOLOGO" "cmTC_be62c\fast"

        "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\nmake.exe" -f
  CMakeFiles\cmTC_be62c.dir\build.make /nologo -L
  CMakeFiles\cmTC_be62c.dir\build

  Building C object CMakeFiles/cmTC_be62c.dir/testCCompiler.c.obj

        C:\PROGRA~2\MICROS~1.0\VC\bin\cl.exe
  @C:\Users\XXX\AppData\Local\Temp\nmA310.tmp

  testCCompiler.c

  Linking C executable cmTC_be62c.exe

        "c:\Program Files (x86)\CMake\bin\cmake.exe" -E vs_link_exe
  C:\PROGRA~2\MICROS~1.0\VC\bin\link.exe /nologo
  @CMakeFiles\cmTC_be62c.dir\objects1.rsp
  @C:\Users\XXX\AppData\Local\Temp\nmA3EB.tmp

  The parameter is incorrectLINK : fatal error LNK1104: cannot open file
  'cmTC_be62c.exe.embed.manifest.res'

  LINK Pass 1 failed.  with 1104

  NMAKE : fatal error U1077: '"c:\Program Files (x86)\CMake\bin\cmake.exe"' :
  return code '0xffffffff'

  Stop.

  NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio
  14.0\VC\BIN\nmake.exe"' : return code '0x2'

  Stop.


  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):

-- Configuring incomplete, errors occurred!
See also "XXX/code/os/windows-6.1.7601.SP1/build/CMakeFiles/CMakeOutput.log".
See also "XXX/code/os/windows-6.1.7601.SP1/build/CMakeFiles/CMakeError.log".
rez: BuildError: The cmake build system failed

I noticed this error:

CMake Error at c:/Program Files (x86)/CMake/share/cmake-3.3/Modules/CMakeTestCCompiler.cmake:61 (message):
  The C compiler "C:/Program Files (x86)/Microsoft Visual Studio
  14.0/VC/bin/cl.exe" is not able to compile a simple test program.

and wanted to make sure that I indeed had a working version of cl.exe and cmake installed, so I created a simple hello_world.c program and successfully compiled it with both cl.exe and by using cmake --build . --config release. At this point I do not actually plan on compiling any C based code as I work primarily in Python, but I did just want to ensure that cmake itself was working properly. At this point I am not sure the best way to proceed so I figured I would post my findings and then reach out to Allan and yourself to see if either of you had any ideas for the best way to continue to debug this.

As I said, I have this working at my current location on linux and mac, and I very much want to roll it out site wide - so any additional help in getting the Windows part rolling would be super appreciated!

Thanks!

on a side note - I've not used rez-bind before, I was actually building each of the os, arch, platform and python packages manually, which seemed to work fine. What does bind do that build does not?

[Allan Johns]

Hey Seth,

1) Freaking Windows, amiright? I'm not going to be much help here - I'm not running it, I'm not very interested in it, and I am definitely leaning on outside contributors to invest in this.

2) rez-bind isn't used very much, it's primarily for creating rez packages for rez itself and for rez-gui. The intent of rez-bind is to create a rez package for something already installed on your system. Rez-build on the other hand is intended to build specifically what you're giving it.

hth

A

ps - that ERROR about not being able to detect physical core count should probably be a WARNING. Don't worry about it in any case, it recovers and just defaults to 1. Not something that'll matter much right now.

--
You received this message because you are subscribed to the Google Groups "rez-config" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rez-config+...@googlegroups.com.
To post to this group, send email to rez-c...@googlegroups.com.
Visit this group at http://groups.google.com/group/rez-config.
For more options, visit https://groups.google.com/d/optout.

[Fabio Piparo]

Hey Seth!

I remember getting the same error before, and in my case I looked at the cmake log (XXX/code/os/windows-6.1.7601.SP1/build/CMakeFiles/CMakeOutput.log) where i had some errors about x86 vs x64 architecture.

I jsut updated my branch, and merged the latest master from Allan.

In the windows-fixs branch I added/modify a few rez-bind plugins to help in bootstrapping a build environment for windows, so you can try something like this:

/c

holofermes@HG $ mkdir packages

/c

holofermes@HG $ rez-bind -i "C:/packages" os

created package 'os-windows-6.2.9200' in C:/packages

/c

holofermes@HG $ rez-bind -i "C:/packages" arch

created package 'arch-AMD64' in C:/packages

/c

holofermes@HG $ rez-bind -i "C:/packages" platform

created package 'platform-windows' in C:/packages

/c

holofermes@HG $ rez-bind -i "C:/packages" target

created package 'target-['AMD64', 'AMD32']' in C:/packages

/c

holofermes@HG $ rez-bind -i "C:/packages" python

created package 'python-2.7.10' in C:/packages

/c

holofermes@HG $ rez-bind -i "C:/packages" cmake

created package 'cmake-3.3.0' in C:/packages

/c

holofermes@HG $ rez-bind -i "C:/packages" msvc

can't find msvc root, use --root and pass the directory to find vcvarsall.bat

/c

holofermes@HG $ rez-bind -i "C:/packages" msvc --root "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC"

created package 'msvc-['14.0']' in C:/packages

holofermes@HG $ tree packages/

packages/

|-- arch

|   `-- AMD64

|       `-- package.py

|-- cmake

|   `-- 3.3.0

|       |-- package.py

|       `-- platform-windows

|           `-- arch-AMD64

|               `-- os-windows-6.2.9200

|                   `-- bin

|                       `-- cmake

|-- msvc

|   `-- 14.0

|       |-- package.py

|       `-- platform-windows

|           `-- arch-AMD64

|               `-- os-windows-6.2.9200

|                   |-- target-AMD32

|                   |   `-- bin

|                   |       `-- setupmsvc.bat

|                   `-- target-AMD64

|                       `-- bin

|                           `-- setupmsvc.bat

|-- os

|   `-- windows-6.2.9200

|       `-- package.py

|-- platform

|   `-- windows

|       `-- package.py

|-- python

|   `-- 2.7.10

|       |-- package.py

|       `-- platform-windows

|           `-- arch-AMD64

|               `-- os-windows-6.2.9200

|                   `-- bin

|                       `-- python

`-- target

    |-- AMD32

    |   `-- package.py

    `-- AMD64

        `-- package.py

30 directories, 12 files

A few notes:

-this worked from a git bash, msys2, and of course cmd.exe.

-if you bind python, and cmake you might need to use an admin shell as rez will create a symlink to the executables (and apparently on windows only admin can do so)

-if you have/install (http://www.lfd.uci.edu/~gohlke/pythonlibs/#pywin32) pywin32, then rez-bind msvc will lookup the rely on reading the registry to get the installed versions of visual studio, ie:

holofermes@HG $ rez-bind -i "C:\packages" msvc

created package 'msvc-['10.0', '12.0', '11.0', '14.0']' in C:\packages

/c

holofermes@HG $ tree packages/msvc/

packages/msvc/
...
...
36 directories, 16 files

 otherwise you have to pass the --root argument for every version you want to bind.

-you need to bind the target package for the current msvc package to work. Its purpose of for differentiating between the x86 vs x64 target architectures of the visual studio tool chain, so that you can do something like this:

holofermes@HG $ rez-env --path "C:/packages" msvc -c "rez context && cl.exe"

resolved by holofermes@hg, on Mon Nov 16 21:53:19 2015, using Rez v2.0.rc1.28

requested packages:

msvc

~platform==windows     (implicit)

~arch==AMD64           (implicit)

~os==windows-6.2.9200  (implicit)

resolved packages:

arch-AMD64           C:\packages\arch\AMD64

msvc-14.0            C:\packages\msvc\14.0\platform-windows\arch-AMD64\os-windows-6.2.9200\target-AMD64

os-windows-6.2.9200  C:\packages\os\windows-6.2.9200

platform-windows     C:\packages\platform\windows

target-AMD64         C:\packages\target\AMD64

Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23026 for x64

Copyright (C) Microsoft Corporation.  All rights reserved.

by default you will get target-{os} most likely target-AMD64. But if you want to build a x86 thing just request target-AMD86.:

holofermes@HG $ rez-env --path "C:/packages" target-AMD86 msvc -c "rez context && cl.exe"

resolved by holofermes@hg, on Mon Nov 16 22:00:03 2015, using Rez v2.0.rc1.28

requested packages:

target-AMD86

msvc

~platform==windows     (implicit)

~arch==AMD64           (implicit)

~os==windows-6.2.9200  (implicit)

resolved packages:

arch-AMD64           C:\packages\arch\AMD64

msvc-14.0            C:\packages\msvc\14.0\platform-windows\arch-AMD64\os-windows-6.2.9200\target-AMD86

os-windows-6.2.9200  C:\packages\os\windows-6.2.9200

platform-windows     C:\packages\platform\windows

target-AMD86         C:\packages\target\AMD86

Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23026 for x86

Copyright (C) Microsoft Corporation.  All rights reserved.

The msvc package will Popen vcvarsall.bat, and pass the argument x86_amd64 for AMD64, or x86 for anything else, and it will essentially merge the evaluated environment variables into the evaluated context.

Here is the list of the vcvarsall.bat arguments.

With this setup I am able to rez-build boost, ilmbase, openexr, pyqt5, sip, tbb, and zlib from source, and I am keeping this in this separate branch.

I hope this helps.

[Mark Streatfield]

Hello,

181  if not platform_.name == 'windows':
182           if not any(x.startswith("-j") for x in (self.child_build_args or [])):
...                n = variant.config.build_thread_count or cpu_count()
...                cmd.append("-j%d" % n)

In this case, the -j argument is child build process specific (make), not platform. Therefore, the conditional should (I think) be based on whether the child build process is make, and not the platform.

There is also a change that will install a bunch of .bat files on windows, because this is the only way I can get rez to work - still don't know it works for everybody else :(

When I install rez on windows, the executables in the Scripts/rez folder are all .exe files. .exe is then set as part of the list in the PATHEXT variable. I also then make sure that Scripts/rez (and not the Scripts folder) is in PATH. I believe this combination is what allows us to run the executables without .bat files.

The fork you link to includes updates to the CMake plugin to call cmake.win.bat (instead of just cmake) - is this file included by the CMake install on windows or part of the rez-bind step?  This should be (and I think in the current HEAD of rez is) configurable - you can see he re that the CMake executable is coming from the config file.  This would keep it more easily compatible across facilities using rez (we don't use rez-bind for example and so presumably don't have cmake.win.bat, so if your branch is merged it would cause a problem for us).

I also have a long-running pull request (6 months old) with some misc. improvements which affect the use of rez on Windows.  Nothing substantial, but maybe worth checking out as there are some small changes that helped us.

Mark.

[Allan Johns]

I've finally had some spare time to start chipping away at rez outside of work, which is why the wiki has finally started taking shape (and there's a new logo! You can thank procrastination for that). The next time I get some time (perhaps this weekend) I'll address some of these older PRs.

Thx

A

--

[Fabio Piparo]

In this case, the -j argument is child build process specific (make), not platform. Therefore, the conditional should (I think) be based on whether the child build process is make, and not the platform. 

Yep, makes more sense.

When I install rez on windows, the executables in the Scripts/rez folder are all .exe files. .exe is then set as part of the list in the PATHEXT variable. I also then make sure that Scripts/rez (and not the Scripts folder) is in PATH. I believe this combination is what allows us to run the executables without .bat files.

When I first run into this issue back I installed rez in with "python setup.py install" and it didn't create .exe files, only when rez is installed with "python ./install.py -v DEST_DIR" they are executed. Despite that both way works just fine, no need for bats!

The fork you link to includes updates to the CMake plugin to call cmake.win.bat (instead of just cmake) - is this file included by the CMake install on windows or part of the rez-bind step?  This should be (and I think in the current HEAD of rez is) configurable - you can see he re that the CMake executable is coming from the config file.  This would keep it more easily compatible across facilities using rez (we don't use rez-bind for example and so presumably don't have cmake.win.bat, so if your branch is merged it would cause a problem for us).

I used cmake.win.bat initially, but just for testing, and should not be in the latest branch.

--
You received this message because you are subscribed to a topic in the Google Groups "rez-config" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rez-config/pAQKq3sTpz0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rez-config+...@googlegroups.com.

[Mark Streatfield]

Ah cool, Fabio - glad to hear you got it working without the .bat files.  Sorry it took me so long to reply on the other thread.

[Seth Lippman]

Fabio,
 I went through your process and it helped me get farther, so thanks for that! That being said, I would rather rez-build python instead of using rez-bind. Not only is it tricky for me to get admin access, I want to avoid using symlinks as well.

A successful rez-build of python also seems like a good basic benchmark to test if REZ, and my Windows build environment are configured correctly before moving forward. Have you been able to rez-build python on windows? I have been able to build it successfully with rez-build on both osx and multiple flavors of linux, but on windows I am getting the same error (from my previous post) at build time.  For a simpler test, I tried to rez-build "platform" on windows, and again, I get the failure with cl.exe.

I am launching my rez commands inside a "Developer Command Prompt for VS2015" shell, in which I have confirmed that I am able to compile a simple c++ program:

C:\Users\XXXX\Documents\Visual Studio 2015\Projects>cl.exe hello_world.c

Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23026 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

hello_world.c
Microsoft (R) Incremental Linker Version 14.00.23026.0

Copyright (C) Microsoft Corporation.  All rights reserved.

/out:hello_world.exe
hello_world.obj

I also confirmed that I can use cmake on the command line to successfully build the same c program:

C:\Users\XXXX\Documents\Visual Studio 2015\Projects>cmake CMakeLists.txt
-- Building for: Visual Studio 14 2015

-- The C compiler identification is MSVC 19.0.23026.0
-- The CXX compiler identification is MSVC 19.0.23026.0

-- Check for working C compiler using: Visual Studio 14 2015
-- Check for working C compiler using: Visual Studio 14 2015 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler using: Visual Studio 14 2015
-- Check for working CXX compiler using: Visual Studio 14 2015 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/XXXX/Documents/Visual Studio 2015/Projects

As you suggested, I took a look at C:\Users\XXXX\code\platform\windows\build\CMakeFiles\CMakeError.log, and this is what I see:

Determining if the C compiler works failed with the following output:
Change Dir: C:/Users/XXXX/code/platform/windows/build/CMakeFiles/CMakeTmp

Run Build Command:"nmake" "/NOLOGO" "cmTC_d0f6f\fast"
    "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\nmake.exe" -f CMakeFiles\cmTC_d0f6f.dir\build.make /nologo -L                  CMakeFiles\cmTC_d0f6f.dir\build

Building C object CMakeFiles/cmTC_d0f6f.dir/testCCompiler.c.obj

    C:\PROGRA~2\MICROS~1.0\VC\bin\cl.exe  @C:\Users\LIPPS004\AppData\Local\Temp\nmE18.tmp

testCCompiler.c

Linking C executable cmTC_d0f6f.exe

    "c:\Program Files (x86)\CMake\bin\cmake.exe" -E vs_link_exe C:\PROGRA~2\MICROS~1.0\VC\bin\link.exe /nologo @CMakeFiles\cmTC_d0f6f.dir\objects1.rsp @C:\Users\XXXX\AppData\Local\Temp\nmEB5.tmp

The parameter is incorrectLINK : fatal error LNK1104: cannot open file 'cmTC_d0f6f.exe.embed.manifest.res'

LINK Pass 1 failed. with 1104

NMAKE : fatal error U1077: '"c:\Program Files (x86)\CMake\bin\cmake.exe"' : return code '0xffffffff'

Stop.

NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\nmake.exe"' : return code '0x2'

Stop.

I have very limited experience compiling on windows, and my experience with cmake is limited to the instances where I have used it with REZ, so I am a bit at a loss as to where else to look to debug this. If you have any other ideas of things I can try, please let me know!  Your help is appreciated!

[Seth Lippman]

On a positive note, I forgot to demonstrate that your updates DID allow for me to rez-env msvc!

C:\Users\XXXX\code\platform\windows>rez-env msvc -c "rez context && cl.exe"
resolved by XXXX, on Thu Nov 19 23:38:46 2015, using Rez v2.0.rc1.35

requested packages:
msvc
~platform==windows         (implicit)
~arch==AMD64               (implicit)

~os==windows-6.1.7601.SP1  (implicit)

resolved packages:
arch-AMD64               XXXX\packages\versioned\init\arch\AMD64
msvc-14.0                C:\Users\XXXX\packages\msvc\14.0\platform-windows\arch-AMD64\os-windows-6.1.7601.SP1\target-AMD86  (local)
os-windows-6.1.7601.SP1  XXXX\packages\versioned\init\os\windows-6.1.7601.SP1
platform-windows        XXXX\packages\versioned\init\platform\windows
target-AMD86             C:\Users\XXXX\packages\target\AMD86                                                                (local)

Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23026 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

usage: cl [ option... ] filename... [ /link linkoption... ]

C:\Users\XXXX\code\platform\windows>rez-env msvc -- cl.exe "c:\Users\XXXX\Documents\Visual Studio 2015\Projects\hello_world.c"
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23026 for x86

[Fabio Piparo]

Hey Seth,

A successful rez-build of python also seems like a good basic benchmark to test if REZ, and my Windows build environment are configured correctly before moving forward. Have you been able to rez-build python on windows? I have been able to build it successfully with rez-build on both osx and multiple flavors of linux, but on windows I am getting the same error (from my previous post) at build time.

I think there are a couple of distinctions between rez-build, and rez-bind:

rez-bind:

-it's a rez plugin that lets you define in a {plugin}.py file how to install existing software

-package.py is created on the fly

rez-build:

-it's a plugin which knows how to execute builds from external build systems: cmake, make, bez

-you need to create a package.py which will be installed by rez-build

When you say that you can rez-build python it implies you are doing so using your own build system, with your own logic, and your own package.py. If you were using cmake as a build system, you could do anything cmake can do which could be from simply coping files, to  compile python from source. With rez-bind the logic is a little bit different, as it's a plugin system. By default rez ships with a set of bindable software that should work on all platforms, so "rez-bind python" means there is a python.py file exposed to the REZ_PLUGIN_PATH.

I went through your process and it helped me get farther, so thanks for that! That being said, I would rather rez-build python instead of using rez-bind. Not only is it tricky for me to get admin access, I want to avoid using symlinks as well.

the python rez-bind plugin (as well as cmake) are using symlinks only for the executable, see here<https://github.com/holofermes/rez/blob/resources2-windowsfixs/src/rez/bind/python.py#L43>. I think maybe an option like --no-sym-links can be added that will allow to do a full copy rather than a symlink.

For a simpler test, I tried to rez-build "platform" on windows, and again, I get the failure with cl.exe.

you mean rez-bind platform?

In short, I would much rather use rez-bind to bootstrap rez. Hope this helps.

[allan.johns]

Hey Fabio,

Sorry I missed your question in this post.

In terms of modelling target as a package - right now that is the best way yes. The best way to do that long term involves a new rez feature that I've been thinking about for a while, but it isn't trivial. The idea is that a package should be able to have "features", and that these features can be requested in a resolve. A package should also be able to provide variant-specific features. So in your case you might have a 'target' feature whos value would be AMD64 in one variant and AMD32 in another, and someone might be able to do something like: rez-env PyQt@target:AMD64. This is long term though, there are a few issues that need to be solved before I could even have this working.

Thx
A

[allan.johns]

Hey all,

RE rez-bind vs rez-build: Rez-bind gets you working packages quicker, but you have less control over them and you're limited to whatever is currently installed on the system. Rez-build builds and installs software from source, so you have the most control.

Rez is a package manager, but unlike aptitude and its ilk, it is not intended to manage *all* packages on your system. So inevitably you hit a point where there is a rez-system boundary. Where that boundary is is up to the user. Some may decide that they don't want to build python from source, and they're happy to use the system python and assume that it won't be upgraded. This user would rez-bind python. Another may want to be running multiple versions of python, in their case they would rather rez-build python packages. Further still, some users might want to compile their compilers from source, while others may be content using the system gcc installation. So rez-bind acts as the bridge between rez and system software.

Seth is correct that rez-BUILDing python is a good test of general rez health on Windows, because if that's not building then probably a rez-build of a simply C++ package will fail also. I can't be of more help RE the specific build errors you're getting though Seth, as I'm not running Windows.

Fabio I think mentioned earlier that rez-build could have a --no-symlinks option to avoid some issues related to symlinking. This would be easy to do and I'll aim to add that soon. I'm currently doing some work on rez-bind actually. When I'm able to tackle some of these PRs I'll also make it priority to pull the misc-fixes branch Mark mentioned... and Fabio do you think your windows fixes branch is in a state to merge also? It sounds like there's been some progress.

In any case Seth, while it does skirt the issue, if you just need rez running on Windows in order to manage python packages then rez-binding everything is probably going to work fine. But yes it does sound as though there's an issue going on there. Have you tried from a bash shell?

A

To unsubscribe from this group and all its topics, send an email to rez-config+unsubscribe@googlegroups.com.