How to install local module other than in "site-packages"?
Jive Dadson
while ago, and it's yet to show up.)
This is no doubt a beginner's question, but I've searched for the
answer for quite a while, to no avail. I'm running Python 2.6 under
Windows XP.
How do I install a module that I wrote, without putting it in the
site-packages directory for a specific release? I have stuff that, to
the best of my knowledge and belief, ought to work under any release. I
do not want to have multiple copies of it. When I edit one of the
modules, I want it to "take" for every release. Is that clear? Hope so.
Diez B. Roggisch
Setting the PYTHONPATH environment variable should work.
Diez
Jive Dadson
I was trying that just now. So far, no joy. Can you type up a DUMMIES
version?
There was no environment variable PYTHONPATH. I added one to my "User
Variables." It's still not finding anything. And yes, I closed and
re-opened my Python shell before I tested it.
Ben Finney
> How do I install a module that I wrote, without putting it in the
> site-packages directory for a specific release? I have stuff that, to
> the best of my knowledge and belief, ought to work under any release.
Nevertheless, the compiled byte-code version will be specific to the
Python version that compiled it. You need to install the module to a
version-specific directory for that reason at least.
> I do not want to have multiple copies of it. When I edit one of the
> modules, I want it to "take" for every release. Is that clear? Hope
> so.
You'll need (I'd love to be corrected on this) to come up with a
deployment discipline: a build process that deploys the module from its
(single, canonical) source location to the appropriate locations when
you decide to deploy it.
--
\ “It is well to remember that the entire universe, with one |
`\ trifling exception, is composed of others.” —John Andrew Holmes |
_o__) |
Ben Finney
Diez B. Roggisch
Did "echo %PYTHONPATH%" yield anything? Or is it part of
>>> import sys
>>> sys.path
?
Diez
Jive Dadson
> Jive Dadson <noton...@noisp.com> writes:
>
>> How do I install a module that I wrote, without putting it in the
>> site-packages directory for a specific release? I have stuff that, to
>> the best of my knowledge and belief, ought to work under any release.
>
> Nevertheless, the compiled byte-code version will be specific to the
> Python version that compiled it. You need to install the module to a
> version-specific directory for that reason at least.
>
>> I do not want to have multiple copies of it. When I edit one of the
>> modules, I want it to "take" for every release. Is that clear? Hope
>> so.
>
> You'll need (I'd love to be corrected on this) to come up with a
> deployment discipline: a build process that deploys the module from its
> (single, canonical) source location to the appropriate locations when
> you decide to deploy it.
>
Thanks for your help.
Well that majorly sucks. I was hoping the "foo.pth" thing would save
me. There ought to be a way. But even if they did invent one, it
wouldn't help me, because I am stuck at version 2.6. Don't even get me
started on that. I really like Python, but the way every release breaks
everything is a royal pain. The biggest problem is C-extensions of stuff
that does not come with the Python release. I've complained about it
before, but it didn't help. I guess I got myself started on that. :-)
Since I cannot imagine facing the challenge of going to 3.0, I guess it
doesn't matter if I have my stuff in a folder that's specific to 2.6.
Jive Dadson
>
> Did "echo %PYTHONPATH%" yield anything? Or is it part of
>
> >>> import sys
> >>> sys.path
>
>
> ?
>
> Diez
Yes and no in that order. Never mind. Ben Fenny talked me out of it
anyway. Grrrrr.
Jive Dadson
>
> Did "echo %PYTHONPATH%" yield anything? Or is it part of
>
> >>> import sys
> >>> sys.path
>
>
> ?
>
> Diez
Update: It's working now. I guess I hadn't reloaded something that I
need to. Thanks for your help.
Jive Dadson
I created a directory (folder in MS-speak) named Modules, and put its
path in the PYTHONPATH env variable.
I can now put a file foo.py into the directory Modules, and it will load
foo.py when I say "import foo."
Now I put a folder into Modules called myModule and in that I put
bar.py. It does not find bar.py when I say either import bar or import
myModule.bar. I tried the myModule.pth trick but it didn't help. Maybe
I did it wrong.
Benjamin Kaplan
Just put a blank file called __init__.py inside myModule
In Python, packages have to have an __init__.py file in them, even if
it's empty. Whatever you put in __init__.py gets imported when you
"import myModule".
Jive Dadson
message titled "The answer," but I accidentally created a new thread
with it. I'll post it in this thread.
Jive Dadson
read the caveat by Ben Fenny in this thread. All this stuff is good for
one default version of Python only. The PYTHONPATH described below, for
example, cannot specify a version number. Yes, that's a pain in the
butt, but there's no way around it. If you switch versions, you may
have to delete all the .pyc files that will show up in the module
folders. Python ought to check them to see if they are valid, but I do
not know if it does so.
These instructions are for MS Windows.
1) Create your modules folder. Let's say it's named "Modules." The
documentation calls it a "package."
2) In an explorer window or on the desktop, right click on My Computer,
and select Properties.
3) Select the Advanced tab, and click on Environment Variables near the
bottom.
4) Look for an environment variable named PYTHONPATH.
a) If you do not find one, create one using the New button(s). I
don't know if it has to be in User Variables or System Variables. To
save time experimenting, I just put one in both. For the value, put the
full path of the folder Modules.
b) If there's already a PYTHONPATH, Edit it, adding a semi-colon
and the full path of folder Module to the end.
5) Put your module folders into the folder Module.
6) (Here's a really arcane bit.) Into each module folder, put a file
named __init__.py. It will be executed when you load the module. It
can be empty, but it has to be there or else the module folder will be
ignored.
Alf P. Steinbach
> Okay, with your help I've figured it out. Instructions are below, but
> read the caveat by Ben Fenny in this thread. All this stuff is good for
> one default version of Python only. The PYTHONPATH described below, for
> example, cannot specify a version number. Yes, that's a pain in the
> butt, but there's no way around it. If you switch versions, you may
> have to delete all the .pyc files that will show up in the module
> folders. Python ought to check them to see if they are valid, but I do
> not know if it does so.
>
> These instructions are for MS Windows.
>
> 1) Create your modules folder. Let's say it's named "Modules." The
> documentation calls it a "package."
>
> 2) In an explorer window or on the desktop, right click on My Computer,
> and select Properties.
>
> 3) Select the Advanced tab, and click on Environment Variables near the
> bottom.
>
> 4) Look for an environment variable named PYTHONPATH.
>
> a) If you do not find one, create one using the New button(s). I
> don't know if it has to be in User Variables or System Variables. To
> save time experimenting, I just put one in both. For the value, put the
> full path of the folder Modules.
The User variables /override/ the System variables for a given user. The System
variables provide defaults for all users. One notable exception is PATH, where
the User PATH items are appended to the System PATH.
Anyways, with many items in a variable like PATH it can be impractical to use
Windows' functionality, which presents it all on one line, which for a PATH with
many items can be exceeding long -- like a many thousand characters[1] line.
So for your convenience, below is a little Windows app (as an HTML application,
it's just some HTML and VBScript and CSS) that presents the user PATH items one
per line in a Notepad-like window. It should be no problem modifying it to
instead edit PYTHONPATH, or even a choice of environment variable. As my late
father used to say, when you don't have a tool, you make it.
Maybe now I should recode this in Python. But on the other hand, one shouldn't
spend time fixing what works. So ... enjoy! :-)
Note: it's crucial to use [.HTA] filename extension.
Also, it's quite possible that this doesn't work in Windows Vista (at least not
without configuration of Windows), but it works in XP and earlier.
<code file="userpath.hta">
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Strict//EN">
<!-- If this works, then it was written by Alf P. Steinbach.
Otherwise it's someone impersonating me.
And yes, if you've seen a comment like this before: I/he stole it. :-) -->
<html>
<head>
<meta name=vs_targetSchema
content="http://schemas.microsoft.com/intellisense/ie5">
<meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
<meta http-equiv="MSThemeCompatible" content="yes">
<hta:application scroll="no"/>
<title>Simple user-environment %path% editor</title>
<style>
body { margin: 0px; padding: 0px; }
#menuArea
{
width: 100%; height: 1.8em; white-space: nowrap; padding-top: 2px;
background: #E0E0E0;
}
#clientArea
{
width: 100%; position: absolute; top: 1.8em; bottom: 0;
}
#pathEditor
{
display: inline-block;
width: 100%; height: 100%;
border: 0;
position: absolute; top: 0;
overflow: scroll;
}
</style>
<script type="text/vbscript" language="vbscript">
option explicit
dim wshShell
dim wshUserEnv
dim originalText
sub loadText
pathEditor.innerText = replace( wshUserEnv( "PATH" ), ";",
vbNewline)
end sub
sub saveText
dim text
dim button
text = pathEditor.innerText
text = replace( text, vbNewLine, ";" )
button = MsgBox( _
text, vbYesNo + vbQuestion, "Save this as new %path% value?" _
)
if button = vbYes then wshUserEnv( "PATH" ) = text
end sub
sub onBtnLoad
loadText
end sub
sub onBtnLoadOriginal
pathEditor.innerText = originalText
end sub
sub onBtnSave
saveText
end sub
sub onLoaded
set wshShell = createObject( "WScript.Shell" )
set wshUserEnv = wshShell.environment( "USER" )
loadText
originalText = pathEditor.innerText
MsgBox _
"Type one path per line (no semicolons)", _
vbInformation, _
"How to use:"
end sub
</script>
</head>
<body onload="onLoaded">
<div id="menuArea">
<button onClick="onBtnLoad">Load current</button>
<button onClick="onBtnLoadOriginal">Reload original</button>
<button onClick="onBtnSave">Save as current ...</button>
</div>
<div id="clientArea">
<textarea id="pathEditor" wrap="off">
</textarea>
</div>
</body>
</html>
</code>
> b) If there's already a PYTHONPATH, Edit it, adding a semi-colon and
> the full path of folder Module to the end.
>
> 5) Put your module folders into the folder Module.
>
> 6) (Here's a really arcane bit.) Into each module folder, put a file
> named __init__.py. It will be executed when you load the module. It
> can be empty, but it has to be there or else the module folder will be
> ignored.
Cheers & hth.,
- Alf
Notes:
[1] Windows XP and I believe also Vista places a limit of about 8 KiB on the
length of any environment variable.
Jive Dadson
>
> Actually, if you're using Python 2.6+/3.x, you can effectively skip
> steps 1-5, as these versions now support user site-packages.
>
> Rather than create a Module folder and modify your PYTHONPATH, add (if
> it doesn't exist already) the following folder:
> %APPDATA%/Python/Python26/site-packages
>
> Modules can sit directly in the folder, or within packages.
>
> For more details: http://www.python.org/dev/peps/pep-0370/
That requires a directory whose name embeds the Python version number,
which is the evil from which I flee, or rather sought to flee. Imagine
if all your C++ code had to go into directories that were named for some
specific C++ compiler. It's just WRONG. It's a maintenance nightmare
to have a bunch of different source files that all have to be updated
whenever you fix a bug or add a feature.
alex23
> That requires a directory whose name embeds the Python version number,
> which is the evil from which I flee, or rather sought to flee. Imagine
> if all your C++ code had to go into directories that were named for some
> specific C++ compiler. It's just WRONG. It's a maintenance nightmare
> to have a bunch of different source files that all have to be updated
> whenever you fix a bug or add a feature.
With the PEP 370 approach, you can just designate a folder the core
one and symlink all other versions to it. Or better yet, use version
control to push updates to each folder.
With all of the versions sharing one folder, as you'd prefer, it would
be a lot more difficult to actually achieve version-level granularity
if and when you need it. It's not WRONG, it's just providing a level
of control that you don't need at this point.
Dave Angel
Jive Dadson wrote:
> <div class="moz-text-flowed" style="font-family: -moz-fixed">alex23
As others have pointed out, you need a "deploy" script, which in your
case would copy the files from source control to the appropriate
production folder. That's analogous to the compile, link and deploy
steps of C++. And if you want to be even more analogous, copy just the
.pyc files, after building them.
Certainly you have lots more files in your version control system which
are not intended to be copied to the "Modules" folder, such as the test
suite.
DaveA