How to debug PEX?

[Jin Feng]

Hi,

I'm trying to add a pythong_requirement_library for pantsbuild.pants.testinfra (again, this work is pants_internal but still most the failure came from the os pants stack so I'm trying here...)

-----------------------

diff --git a/3rdparty/python/pantsbuild/pants/testinfra/BUILD b/3rdparty/python/pantsbuild/pants/testinfra/BUILD

new file mode 100644

index 0000000..54a3aff

--- /dev/null

+++ b/3rdparty/python/pantsbuild/pants/testinfra/BUILD

@@ -0,0 +1,6 @@

+python_requirement_library(

+  name='testinfra',

+  requirements=[

+    python_requirement('pantsbuild.pants.testinfra')

+  ]

+)

-----------------------

The I'm simply adding this to pants python_binary:

-----------------------

diff --git a/src/python/twitter/pants_internal/BUILD b/src/python/twitter/pants_internal/BUILD

index f138cfa..2022552 100644

--- a/src/python/twitter/pants_internal/BUILD

+++ b/src/python/twitter/pants_internal/BUILD

@@ -3,6 +3,7 @@ python_library(

   sources = ['__init__.py', 'register.py'],

   dependencies = [

     '3rdparty/python/pantsbuild/pants',

+    '3rdparty/python/pantsbuild/pants/testinfra',

     'src/python/twitter/pants_internal/build_helpers',

     'src/python/twitter/pants_internal/targets',

     'src/python/twitter/pants_internal/tasks',

-----------------------

Now when I'm running:

$ PANTS_DEV=1 ./pants src/python/twitter/pants_internal:pants

I'm getting:

-----------------------

==================================

OrderedSet([PythonRequirement(pantsbuild.pants), PythonRequirement(pantsbuild.pants.testinfra), PythonRequirement(twitter.common.collections), PythonRequirement(twitter.git-review.owners-util), PythonRequirement(twitter.common.config), PythonRequirement(twitter.common.contextutil), PythonRequirement(twitter.common.dirutil), PythonRequirement(twitter.common.lang), PythonRequirement(twitter.common.pants>=0.5.0,<0.6)])

<pants.backend.python.resolver.PantsObtainer object at 0x10e6eb250>

PythonInterpreter(u'/System/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6', PythonIdentity('CPython', 2, 6, 8), {(u'wheel', u'0.23.0'): u'/Users/jinfeng/workspace/pants_internal/.pants.d/python/interpreters/CPython-2.6.8/wheel-0.23.0-py2.6.egg', (u'setuptools', u'5.4.1'): u'/Users/jinfeng/workspace/pants_internal/.pants.d/python/interpreters/CPython-2.6.8/setuptools-5.4.1-py2.6.egg'})

linux-x86_64

==================================

Problem executing PythonBuilder for targets OrderedSet([PythonBinary(BuildFileAddress(/Users/jinfeng/workspace/pants_internal/src/python/twitter/pants_internal/BUILD, pants)), PythonLibrary(BuildFileAddress(/Users/jinfeng/workspace/pants_internal/src/python/twitter/pants_internal/BUILD, pants_internal-plugin)), PythonLibrary(BuildFileAddress(/Users/jinfeng/workspace/pants_internal/src/python/twitter/pants_internal/build_helpers/BUILD, build_helpers)), PythonLibrary(BuildFileAddress(/Users/jinfeng/workspace/pants_internal/src/python/twitter/pants_internal/targets/BUILD, targets)), PythonLibrary(BuildFileAddress(/Users/jinfeng/workspace/pants_internal/src/python/twitter/pants_internal/tasks/BUILD, tasks)), PythonLibrary(BuildFileAddress(/Users/jinfeng/workspace/pants_internal/src/python/twitter/pants_internal/tasks/resource_preprocessors/BUILD, resource-preprocessors)), PythonLibrary(BuildFileAddress(/Users/jinfeng/workspace/pants_internal/src/python/twitter/pants_internal/tasks/idea/BUILD, idea))]): Traceback (most recent call last):

  File "/Users/jinfeng/.pants.dev/os_pants/src/python/pants/backend/python/commands/build.py", line 127, in _python_build

    fast_tests=self.options.fast)

  File "/Users/jinfeng/.pants.dev/os_pants/src/python/pants/backend/python/python_builder.py", line 49, in build

    conn_timeout=conn_timeout).run()

  File "/Users/jinfeng/.pants.dev/os_pants/src/python/pants/backend/python/binary_builder.py", line 53, in run

    env = self.chroot.dump()

  File "/Users/jinfeng/.pants.dev/os_pants/src/python/pants/backend/python/python_chroot.py", line 187, in dump

    conn_timeout=self._conn_timeout)

  File "/Users/jinfeng/.pants.dev/os_pants/src/python/pants/backend/python/resolver.py", line 108, in resolve_multi

    platform=platform)

  File "build/bdist.macosx-10.9-x86_64/egg/pex/resolver.py", line 101, in resolve

    raise Unsatisfiable('Cannot satisfy requirements: %s' % requirement_set[requirement.key])

Unsatisfiable: Cannot satisfy requirements: [PythonRequirement(pantsbuild.pants.testinfra)]

-----------------------

1) As you can see, I put some print/debug code into OS pants python/resolver.py, but I couldn't figure out how to debug the pex/resolver.py.

2) Is there some magic place I need to add the pantsbuild.pants.testinfra to some requirements.txt whatnot to help the resolution?

Or it's still a my laptop is corrupted/dirty issue?

Thanks a lot!

Thanks

   Jin Feng

[John Sirois]

Last I knew, the twitter/pants_internal build used purely internal pypi repos served up by svn.  You'd need to ensure your release process was capturing the correct full dep set to that svn repo.  Twitter folks should be able to help.

See pants.ini for the list of

[python-repos]

repos: [...]

[Jin Feng]

Thanks for the pointer!

And to the question on how to live debug the PEX?

[Brian Wickman]

To get information about the environment bootstrapping

PEX_VERBOSE=1 ./foo.pex

To drop into an interpreter w/ the environment bootstrapped:

PEX_INTERPRETER=1 ./foo.pex

There's a few other environment variables you can set but those are two handy ones.

[Joe Smith]

I've also found that sometimes it's not pex, but a distutils-related error, in which case DISTUTILS_DEBUG=1 is very helpful as well.