Python does not get environment variable when using cron.
Stephen Cattaneo
I am attempting to execute an automated test (written in Python) via
cron. I have to check the HOSTNAME variable as part of the test, oddly
under cron the HOSTNAME environment variable is not in the os.environ
dictionary. I know that cron runs in a subshell that does not have all
of the normally set environment variables. HOSTNAME is not one of those
variables, it is set even in cron's subshell. Why doesn't python get
this variable? Is this a bug in python2.4?
>From a cronjob to check environment variables in cron's shell vs
python's os.environ (please excuse my lack of creativity in naming
convention)-
BASH=/bin/bash
BASH_ARGC=()
BASH_ARGV=()
BASH_EXECUTION_STRING='set; echo "-----------------"; echo "import os;
print os.environ" | python'
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="3" [1]="1" [2]="17" [3]="1" [4]="release"
[5]="i686-redhat-linux-gnu")
BASH_VERSION='3.1.17(1)-release'
DIRSTACK=()
EUID=501
GROUPS=()
HOME=/home/regression
HOSTNAME=regress5
HOSTTYPE=i686
IFS=$' \t\n'
LOGNAME=regression
MACHTYPE=i686-redhat-linux-gnu
OPTERR=1
OPTIND=1
OSTYPE=linux-gnu
PATH=/usr/local/bin:/bin:/usr/bin:/home/regression/bin
PPID=819
PS4='+ '
PWD=/home/regression
PYTHONPATH=/home/regression/lib
SHELL=/bin/bash
SHELLOPTS=braceexpand:hashall:interactive-comments
SHLVL=1
TERM=xterm
UID=501
USER=regression
_=/bin/bash
-----------------
{'TERM': 'xterm', 'SHELL': '/bin/bash', 'SHLVL': '1', 'PYTHONPATH':
'/home/regression/lib', 'PWD': '/home/regression', 'LOGNAME':
'regression', 'USER': 'regression', 'HOME': '/home/regression', 'PATH':
'/usr/local/bin:/bin:/usr/bin:/home/regression/bin', '_':
'/usr/bin/python'}
Thanks,
Steve
Eric Wertman
give you what you need otherwise.
John Machin
> I'm not sure about the environment variable, but os.uname() should
> give you what you need otherwise.
As documented, it should but may not ... consider using
socket.gethost*
Asun Friere
What did you run in the cronjob to get back all those variables?
I just ran /usr/bin/env (on a 2.6.x linux box) as a cronjob and only
got back SHELL, USER, PATH, PWD, SHLVL, HOME, LOGNAME. This is the
same set of variables os.environ (python2.4.4) gives me when run as a
cronjob. I tried the same thing on a bladeserver running SunOS5.9 and
got a smaller set of variables, again identical between /usr/bin/env
and os.environ.
John Nagle
> Hello all,
>
> I am attempting to execute an automated test (written in Python) via
> cron. I have to check the HOSTNAME variable as part of the test, oddly
> under cron the HOSTNAME environment variable is not in the os.environ
> dictionary. I know that cron runs in a subshell that does not have all
> of the normally set environment variables. HOSTNAME is not one of those
> variables, it is set even in cron's subshell. Why doesn't python get
> this variable? Is this a bug in python2.4?
Cron doesn't normally use a shell at all. It just runs the
requested program in a subprocess. So there's no shell involved,
and you don't get a shell-type user environment. If you have
a crontab line like
10 3 * * * /usr/bin/python someprogram.py
there's no shell. You can try
10 3 * * * /bin/sh /usr/bin/python someprogram.py
which will load a shell, which in turn will load Python.
Or, in Python, you can use "socket.gethostname()", which will
get you the host name used for networking purposes.
John Nagle
Cameron Simpson
> Stephen Cattaneo wrote:
>> I am attempting to execute an automated test (written in Python) via
>> cron. I have to check the HOSTNAME variable as part of the test, oddly
>> under cron the HOSTNAME environment variable is not in the os.environ
>> dictionary. I know that cron runs in a subshell that does not have all
>> of the normally set environment variables. HOSTNAME is not one of those
>> variables, it is set even in cron's subshell. Why doesn't python get
>> this variable? Is this a bug in python2.4?
Because $HOSTNAME is a bash specific variable, set by bash but NOT
EXPORTED! Like $0 and a bunch of other "private" variables, subprocesses
do not inherit this value. From "man bash":
Shell Variables
The following variables are set by the shell:
[...]
HOSTNAME
Automatically set to the name of the current host.
Note that "set" does not imply "exported". Only exported vairables are
seen by subprocesses.
> Cron doesn't normally use a shell at all. It just runs the
> requested program in a subprocess. So there's no shell involved,
> and you don't get a shell-type user environment.
This statement is false.
Cron hands _all_ jobs to a shell. From "man 5 crontab":
The ``sixth'' field (the rest of the line) specifies the command to be
run. The entire command portion of the line, up to a newline or % char-
acter, will be executed by /bin/sh or by the shell specified in the SHELL
variable of the cronfile.
That's from Vixie cron, but all UNIX crons hand the command part of the
cron line to a shell, usually /bin/sh.
You're probably confused by the fact that cron does not invoke "login" shells
(with their associated initialisation from /etc/profile and $HOME/.profile).
> If you have
> a crontab line like
> 10 3 * * * /usr/bin/python someprogram.py
> there's no shell.
Not so. A shell will be interpreting the string "/usr/bin/python
someprogram.py", mch as happens when you type this on the command line.
> You can try
> 10 3 * * * /bin/sh /usr/bin/python someprogram.py
> which will load a shell, which in turn will load Python.
Even more not so. The command "/bin/sh /usr/bin/python someprogram.py"
will attempt to parse the file "/usr/bin/python" as though it were a
shell script. That's almost certainly not what you wanted.
To do what you describe requires the line:
/bin/sh -c '/usr/bin/python someprogram.py'
which hands the command string "/usr/bin/python someprogram.py" to
/bin/sh for interpretation. Were you to do this as a cron line, the
string: "/bin/sh -c '/usr/bin/python someprogram.py'" would be handed
to /bin/sh for interpretation. That interpreter in turn then hands
"/usr/bin/python someprogram.py" to /bin/sh for interpretation.
A parent-child nested process listing (with some fake quoting tossed in
to show the strings) would look like this:
/bin/sh -c "/bin/sh -c '/usr/bin/python someprogram.py'"
/bin/sh -c '/usr/bin/python someprogram.py'
/usr/bin/python someprogram.py
i.e. three process alive.
> Or, in Python, you can use "socket.gethostname()", which will
> get you the host name used for networking purposes.
Or, on a cron line (after the time fields, omitted here):
HOSTNAME=`hostname`; export HOSTNAME; python someprogram.py
Cheers,
--
Cameron Simpson <c...@zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/
186,282 miles per second - Not just a good idea, It's the Law!
Maric Michaud
> You're probably confused by the fact that cron does not invoke "login"
> shells (with their associated initialisation from /etc/profile and
> $HOME/.profile).
This can be solved by invoking /bin/bash -l -c ..., the -l option forces a
login shell even in non-interactive session.
--
_____________
Maric Michaud
Edwin....@verizonwireless.com
hope that helps.
Edwin
-----Original Message-----
From: python-list-bounces+edwin.madari=verizonwi...@python.org
[mailto:python-list-bounces+edwin.madari=verizonwi...@python.org]
On Behalf Of Eric Wertman
Sent: Sunday, August 17, 2008 9:39 PM
To: pytho...@python.org
Subject: Re: Python does not get environment variable when using cron.
I'm not sure about the environment variable, but os.uname() should
give you what you need otherwise.
--
http://mail.python.org/mailman/listinfo/python-list
The information contained in this message and any attachment may be
proprietary, confidential, and privileged or subject to the work
product doctrine and thus protected from disclosure. If the reader
of this message is not the intended recipient, or an employee or
agent responsible for delivering this message to the intended
recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please notify me
immediately by replying to this message and deleting it and all
copies and backups thereof. Thank you.
Shawn Milochik
When you log into a Unix session, certain files in your home directory
are read and add environment variables to your session. When you run a
cron job, it does not do this. It still runs as "you" as far as
permissions go, but it is not identical to you typing the command in
an interactive session.
The easiest solution (in my opinion) is to write a bash script to
execute your Python script, and use that bash script to add those
environment variables. The most likely file you'll want to run is
.bashrc in your home directory. If you're on a Mac, it's .bash_login
instead.
Example:
#/usr/bin/env bash
source ~/.bashrc
path/my_script.py
Something like that should take care of it. If not, get creative --
add the "env" command to your bash script and have it send the output
to a file: env > cron_env.txt
Then run env in your interactive session and look for differences.
Shawn
Stephen Cattaneo
<cron date / time> set; echo "-----------------"; echo "import os; print
os.environ" | python
Cheers,
S
- -----Original Message-----
- From: Asun Friere [mailto:afr...@yahoo.co.uk]
- Sent: Sunday, August 17, 2008 7:55 PM
- To: pytho...@python.org
- Subject: Re: Python does not get environment variable when using cron.
-
- On Aug 18, 11:15 am, "Stephen Cattaneo"
- <Stephen.Catta...@u4eatech.com> wrote:
- > Hello all,
- >
- > I am attempting to execute an automated test (written in Python) via
- > cron. I have to check the HOSTNAME variable as part of the test,
oddly
- > under cron the HOSTNAME environment variable is not in the
os.environ
- > dictionary. I know that cron runs in a subshell that does not have
all
- > of the normally set environment variables. HOSTNAME is not one of
those
- > variables, it is set even in cron's subshell. Why doesn't python
get
- > this variable? Is this a bug in python2.4?
- >
- > >From a cronjob to check environment variables in cron's shell vs
- >
- > python's os.environ (please excuse my lack of creativity in naming
- > convention)-
- >
- > BASH=/bin/bash
- > BASH_ARGC=()
- > BASH_ARGV=()
- > BASH_EXECUTION_STRING='set; echo "-----------------"; echo "import
os;
- > print os.environ" | python'
- > BASH_LINENO=()
- > BASH_SOURCE=()
- > BASH_VERSINFO=([0]="3" [1]="1" [2]="17" [3]="1" [4]="release"
- > [5]="i686-redhat-linux-gnu")
- > BASH_VERSION='3.1.17(1)-release'
- > DIRSTACK=()
- > EUID=501
- > GROUPS=()
- > HOME=/home/regression
- > HOSTNAME=regress5
- > HOSTTYPE=i686
- > IFS=$' \t\n'
- > LOGNAME=regression
- > MACHTYPE=i686-redhat-linux-gnu
- > OPTERR=1
- > OPTIND=1
- > OSTYPE=linux-gnu
- > PATH=/usr/local/bin:/bin:/usr/bin:/home/regression/bin
- > PPID=819
- > PS4='+ '
- > PWD=/home/regression
- > PYTHONPATH=/home/regression/lib
- > SHELL=/bin/bash
- > SHELLOPTS=braceexpand:hashall:interactive-comments
- > SHLVL=1
- > TERM=xterm
- > UID=501
- > USER=regression
- > _=/bin/bash
- > -----------------
- > {'TERM': 'xterm', 'SHELL': '/bin/bash', 'SHLVL': '1', 'PYTHONPATH':
- > '/home/regression/lib', 'PWD': '/home/regression', 'LOGNAME':
- > 'regression', 'USER': 'regression', 'HOME': '/home/regression',
'PATH':
- > '/usr/local/bin:/bin:/usr/bin:/home/regression/bin', '_':
- > '/usr/bin/python'}
- >
- > Thanks,
- >
- > Steve
-
- What did you run in the cronjob to get back all those variables?
-
- I just ran /usr/bin/env (on a 2.6.x linux box) as a cronjob and only
- got back SHELL, USER, PATH, PWD, SHLVL, HOME, LOGNAME. This is the
- same set of variables os.environ (python2.4.4) gives me when run as a
- cronjob. I tried the same thing on a bladeserver running SunOS5.9 and
- got a smaller set of variables, again identical between /usr/bin/env
- and os.environ.
Asun Friere
wrote:
> - What did you run in the cronjob to get back all those variables?
>
> <cron date / time> set; echo "-----------------"; echo "import os; print
> os.environ" | python
>
> Cheers,
>
> S
As I should have noted from $BASH_EXECUTION_STRING. I'd be half
dangerous if I just ... paid ... ATTENTION!
OK, the difference between using bash's builtin 'set' and '/usr/bin/
env' is that 'env' looks at ENVIRONMENT variables, whereas 'set' looks
at SHELL variables (which in bash includes HOSTNAMES). Since shell
variables are private to the shell, 'os.environ' can only see
environment variables.
In other words if you want 'os.environ' to see a variable 'set' sees,
you have to export it from the shell to the environment, eg.
* * * * * export HOSTNAME;echo "import os;print
os.environ['HOSTNAME']" | /usr/bin/python
But note this solution depends on the particular predilections of the
bash shell. If you are solely concerned with finding the hostname,
the more generic solution would be the one using
'socket.gethostname' (as John Machin suggested above), which
presumably calls the C library routine of the same name.
cheers
Asun Friere
<snip>
>
> The easiest solution (in my opinion) is to write a bash script to
> execute your Python script, and use that bash script to add those
> environment variables.
Agreed. Wrap it in a shell script, easier to read and grow than a
oneliner on the crontab.
> The most likely file you'll want to run is
> .bashrc in your home directory. If you're on a Mac, it's .bash_login
> instead.
>
> Example:
>
> #/usr/bin/env bash
>
> source ~/.bashrc
> path/my_script.py
>
I for one don't have $HOSTNAME defined in my .bashrc file. I doubt
this is likely to give him much joy.
> Something like that should take care of it. If not, get creative --
> add the "env" command to your bash script and have it send the output
> to a file: env > cron_env.txt
>
Again no. The reason os.environ can't find HOSTNAME is that it is NOT
defined in the environment, if env can find it os.environ should be
able to as well.