python can't getpwuid(1000) under spk dev

[Dan Connolly]

I'm trying to do a sort of hello world sandstorm app in python:

from BaseHttpServer import (BaseHTTPRequestHandler,

                            HTTPServer)

def main(port=10000,

         HandlerClass=BaseHTTPRequestHandler,

         ServerClass=HTTPServer,

         protocol="HTTP/1.0"):

    server_address = ('', port)

    HandlerClass.protocol_version = protocol

    httpd = ServerClass(server_address, HandlerClass)

    sa = httpd.socket.getsockname()

    print "Serving HTTP on", sa[0], "port", sa[1], "..."

    httpd.serve_forever()

main()

I get this far OK:

$ spk init -p 10000 -- python hi.py

$ spk dev

App is now available from Sandstorm server. Ctrl+C to disconnect.

but when I go into sandstorm and open a new Example document, I get a blank page; the debug log says:

** SANDSTORM SUPERVISOR: Starting up grain.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site.py", line 563, in <module>
    main()
  File "/usr/lib/python2.7/site.py", line 545, in main
    known_paths = addusersitepackages(known_paths)
  File "/usr/lib/python2.7/site.py", line 272, in addusersitepackages
    user_site = getusersitepackages()
  File "/usr/lib/python2.7/site.py", line 247, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/usr/lib/python2.7/site.py", line 237, in getuserbase
    USER_BASE = get_config_var('userbase')
  File "/usr/lib/python2.7/sysconfig.py", line 578, in get_config_var
    return get_config_vars().get(name)
  File "/usr/lib/python2.7/sysconfig.py", line 529, in get_config_vars
    _CONFIG_VARS['userbase'] = _getuserbase()
  File "/usr/lib/python2.7/sysconfig.py", line 210, in _getuserbase
    return env_base if env_base else joinuser("~", ".local")
  File "/usr/lib/python2.7/sysconfig.py", line 196, in joinuser
    return os.path.expanduser(os.path.join(*args))
  File "/usr/lib/python2.7/posixpath.py", line 269, in expanduser
    userhome = pwd.getpwuid(os.getuid()).pw_dir
KeyError: 'getpwuid(): uid not found: 1000'

I looked at the port of ipython to sandstorm, and I couldn't find anything relevant. Clues?

Where is the relevant /etc/passwd file? The normal one has an entry for 1000:

$ grep 1000 /etc/passwd|wc

      1       3      70

 

-- 

Dan Connolly

http://www.madmode.com/

[Jason Paryani]

This can be worked around by defining the HOME environment variable, since all it's trying to do is find the user's home directory from its uid.

 environ = [

      # Note that this defines the *entire* environment seen by your app.

      (key = "PATH", value = "/usr/local/bin:/usr/bin:/bin"),

      (key = "HOME", value = "/var")

    ]

Kenton, should we add this by default in the `spk init` template?

--
You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jason Paryani]

Sorry, I meant to say that you need to change the `environ` definition in sandstorm-pkgdef.capnp to define HOME like so:

 environ = [

      # Note that this defines the *entire* environment seen by your app.

      (key = "PATH", value = "/usr/local/bin:/usr/bin:/bin"),

      (key = "HOME", value = "/var")

    ]

[David Renshaw]

It should also work to unhide "/etc/passwd" from in your pkgdef's
searchPath, or to add a local "etc/passwd" file in the project
directory.

[Kenton Varda]

Instead of automatically adding hacks to the default template, I'd rather build an index of "things that go wrong with various frameworks" that people can refer to (and, longer-term, specialized tools for each framework that automate these things).

In this case, it seems like a bug in Python that it refuses to run in a userless environment. At some point someone ought to fix that. :)

-Kenton

[Asheesh Laroia]

Hi Kenton & all,

While it's hypothetically that it'd be nice if someone would change Python 2.7 so that os.path.expanduser() can work in the absence of either of the following:

* /etc/passwd file available

* HOME environment variable available,

I don't really see how os.path.expanduser() can function in the absence of either of those, except perhaps with the support of a libnss-sandstorm. It seems that Python wants to call that function at start-up.

In principle, I'm at least +0 (on a range of -1 "I hate it" to +1 "I love it") to someone making an index of "things that can go wrong with various frameworks", but when the list starts to grow, and developers like Dan start to wonder if porting to Sandstorm is unnecessarily difficult, I'm hopeful that generally Sandstorm can change when we realize that certain difficulties are unnecessary.

Also, Python 2.7 is in maintenance mode, so changing it to operate in a world where os.path.expanduser() crashes is probably not going to happen, alas.

So I support the idea of $HOME or a working /etc/passwd being available by default. These are just my early-morning thoughts, so I might have missed something important.

[Kenton Varda]

On Mon, Dec 22, 2014 at 8:02 AM, Asheesh Laroia <ash...@asheesh.org> wrote:

I don't really see how os.path.expanduser() can function in the absence of either of those, except perhaps with the support of a libnss-sandstorm. It seems that Python wants to call that function at start-up.

Right. I don't think the bug is in expanduser(). Rather, I think that site.py should be fixed so that it doesn't search for user packages in a userless environment. It's possible that this is a bug not in site.py but in our usage.

 

In principle, I'm at least +0 (on a range of -1 "I hate it" to +1 "I love it") to someone making an index of "things that can go wrong with various frameworks", but when the list starts to grow, and developers like Dan start to wonder if porting to Sandstorm is unnecessarily difficult, I'm hopeful that generally Sandstorm can change when we realize that certain difficulties are unnecessary.

In most cases, the solutions are one-liners.

But I agree, it should be automated. In general I think the right place for automating framework-specific workarounds is in tools specific to each framework. E.g. we have the meteor-spk tool for Meteor today which works around several nitpicky issues with packaging a Meteor app. I want to be cautious about adding such workarounds to the base tools because any hack could just break something else.

 

So I support the idea of $HOME or a working /etc/passwd being available by default. These are just my early-morning thoughts, so I might have missed something important.

Actually, /etc/passwd may be a general enough requirement to be worth solving directly. Although passwd files don't actually contain passwords or anything sensitive these days, it is nevertheless pretty weird to map your own passwd file into a package.

Perhaps we should similarly fill in things like /etc/resolv.conf and a few other common /etc files. Many of these have clear correct answers for Sandstorm and those correct answers often aren't what would be present on the user's system. (For example, the UID in the sandbox is always 1000, so /etc/passwd should just list that.)

[Kenton Varda]

Thanks, Dan.

It felt a little funny to have one specific Python issue listed at the top of our porting guide, before we've even told people how to get started, so I created a new stub page for Python-specific stuff and moved this tip there, and then linked it next to meteor-spk. We should add more pages like this for other frameworks.

-Kenton

On Fri, Jan 9, 2015 at 7:42 PM, Dan Connolly <dc...@madmode.com> wrote:

On Sun, Dec 21, 2014 at 10:47 PM, Kenton Varda <ken...@sandstorm.io> wrote:
>
> Instead of automatically adding hacks to the default template, I'd rather build an index of "things that go wrong with various frameworks" that people can refer to (and, longer-term, specialized tools for each framework that automate these things).

I added a "Framework-specific gotchas" section under
"Framework-specific tools" and noted this there...
https://github.com/sandstorm-io/sandstorm/wiki/Porting-Guide

It could probably be tidied up a bit.

> In this case, it seems like a bug in Python that it refuses to run in a userless environment. At some point someone ought to fix that. :)

Yes, it's a bug in python.
http://bugs.python.org/issue10496