Hello world and Starter Pyramid on AWS

320 views
Skip to first unread message

Dan

unread,
Mar 11, 2017, 5:50:22 PM3/11/17
to pylons-discuss
Hi all,

I spent some time over the last week or so trying to get some pyramid apps on AWS via Elastic Beanstalk. I had some trouble because unlike flask and django, there aren't any step-by-step docs or templates. I also had a requirement, keep it as simple as possible. This meant no EB CLI. Luckily, with a hint from stackoverflow's webjunkie (also thanks to Michael Merickel for some input) I was able to get it working. 

After it was working I was logging into the EC2 instances to install my starter application, but I eventually tweaked my requirements.txt file so that I only had to upload my project as a compressed zip file.

That said, I have two articles on how to do these things on my new AWS site, http://www.bruisedthumb.com. I appreciate any feedback. Yes I know the site is ridiculously simple. I'm a dad, so I have to do this stuff before 7AM before the kids wake up.

Thanks,
--Dan

Steve Piercy

unread,
Mar 11, 2017, 6:05:05 PM3/11/17
to pylons-...@googlegroups.com
Dan,

Would you consider adding a link to your articles by adding
another page on the Pyramid Community Cookbook:
http://docs.pylonsproject.org/projects/pyramid_cookbook/en/latest/deployment/index.html

...and under Publications:
https://trypyramid.com/community.html

I can also publicize via our Twitter feed.

I've seen a few requests for tutorials/how-tos for AWS
deployment come through the #pyramid IRC channel.

--steve


On 3/11/17 at 2:50 PM, wpu....@gmail.com (Dan) pronounced:
------------------------
Steve Piercy, Soquel, CA

Dan Clark

unread,
Mar 11, 2017, 6:31:58 PM3/11/17
to pylons-...@googlegroups.com
Sure, I just need to figure it out. I am an svn guy, so git is a bit foreign to me.

--Dan





--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/r473Ps-10123i-DAE8335B3C284D64850AFE30ADDE458D%40Steves-iMac.local.

For more options, visit https://groups.google.com/d/optout.

Dan

unread,
Mar 11, 2017, 7:16:48 PM3/11/17
to pylons-discuss
Got the pull request out.

Do you mind if I skip the trypyramid.com pub section? I am pretty new at all of this, and my site needs quite a bit of work before I want to hang it out among those other sites. I definitely appreciate it though I just don't feel worthy! I need to keep working and writing!

I am very intrigued by all of this.

Thanks,
--Dan
To post to this group, send email to pylons-discuss@googlegroups.com.

Michael Merickel

unread,
Mar 11, 2017, 7:53:30 PM3/11/17
to Pylons
Hey Dan!

I'm really happy you decided to write up your experience and I'm glad you got it working. I have one small request and that would be to figure out why zope.deprecation was causing you issues and add an update mentioning that it's fixed for <insert reason here>. Putting a blog post out there saying "I don't know why - but you need to use this old version" isn't great for other people learning. Obviously no one wants that workflow to be broken so if it is it'd be good to figure out what's going on. If you drop into #pyramid on freenode we can even debug it together but I'm never happy with settling for hacks. That being said the post looks great and we'll be happy to link to it - you should be proud of it!

- Michael


To post to this group, send email to pylons-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscribe@googlegroups.com.

To post to this group, send email to pylons-discuss@googlegroups.com.

Dan Clark

unread,
Mar 11, 2017, 8:08:10 PM3/11/17
to pylons-...@googlegroups.com
Certainly, that was my intention. As I get some time I will look into it. I still have work to do on my site as well.

Thanks so much for the feedback! All of this has been an exhilarating change of pace for me.

--Dan

--


--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.

Steve Piercy

unread,
Mar 12, 2017, 12:45:49 AM3/12/17
to pylons-...@googlegroups.com
Hi Dan,

Thanks for taking the time to publish what you learned.

I publish because the feedback from others helps me learn from
and correct my mistakes, and my future self appreciates the
reference material. It's a continual process. Practice makes a
little better.

For learning git, I found that visual tools and guides helped me
much more than narrative documentation with commands. GUI tools
like Sourcetree[1] and PyCharm's VCS integration[2] helped me
see what goes on when issuing commands. GitHub provides a
visual guide of the GitHub Flow process[3].

[1] https://www.sourcetreeapp.com/
[2] https://www.youtube.com/watch?v=jFnYQbUZQlA
[2] https://www.youtube.com/watch?v=_w9XWHDSSa4
[2] https://www.youtube.com/watch?v=AHkiCKG-JhM
[3] https://guides.github.com/introduction/flow/index.html

--steve


On 3/11/17 at 8:08 PM, wpu....@gmail.com (Dan Clark) pronounced:
>>>> pylons-discus...@googlegroups.com.
>>>> To post to this group, send email to pylons-...@googlegroups.com.
>>>> To view this discussion on the web visit https://groups.google.com/d/ms
>>>> gid/pylons-discuss/r473Ps-10123i-DAE8335B3C284D64850AFE30ADD
>>>> E458D%40Steves-iMac.local.
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>>You received this message because you are subscribed to the Google Groups
>>"pylons-discuss" group.
>>To unsubscribe from this group and stop receiving emails from it, send an
>>email to pylons-discus...@googlegroups.com.
>>
>>To post to this group, send email to pylons-...@googlegroups.com.
>>To view this discussion on the web visit https://groups.google.com/d/ms
>>gid/pylons-discuss/a52e9324-14e9-44ef-b602-5b9331bc1c26%40googlegroups.com
>>
><https://groups.google.com/d/msgid/pylons-discuss/a52e9324-14e9-44ef-b602-5b9331bc1c26%
>40googlegroups.com?utm_medium=email&utm_source=footer>
>>.
>>
>>For more options, visit https://groups.google.com/d/optout.
>>
>
>--
>You received this message because you are subscribed to a topic in the
>Google Groups "pylons-discuss" group.
>To unsubscribe from this topic, visit https://groups.google.com/d/
>topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
>To unsubscribe from this group and all its topics, send an email to
>pylons-discus...@googlegroups.com.
>To post to this group, send email to pylons-...@googlegroups.com.
>To view this discussion on the web visit https://groups.google.com/d/
>msgid/pylons-discuss/CAKdhhwFFMRZLPjwKn6ZTvs8Qary8N
>oRNp9P-m9Kizs2Gkpk8qw%40mail.gmail.com
><https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwFFMRZLPjwKn6ZTvs8Qary8NoRNp9P-
>m9Kizs2Gkpk8qw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>.
>
>For more options, visit https://groups.google.com/d/optout.
>

Tres Seaver

unread,
Mar 14, 2017, 10:42:13 PM3/14/17
to pylons-...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/11/2017 07:52 PM, Michael Merickel wrote:
> I have one small request and that would be to figure out why
> zope.deprecation was causing you issues and add an update mentioning
> that it's fixed for <insert reason here>.

I've just looked at the diff between zope.deprecation 4.2.1 and 4.2.0:
the only meaningful changes are to docs, dropping support for Python 2.6
and 3.2, adding support for Python 3.5, and adding "universal wheel"
support. I would need to see actual tracebacks / build failures to help
diagnose that further.



Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tse...@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJYyKnsAAoJEPKpaDSJE9HYNiwP/0/Ag5TT9yURXgV0bD91tu7k
FBNKRP3IfGLTjEAPzVbgjZ2EZW6x2OBg4ut2lCcIZ5y9cRlTMu/wBgvbgtnWtsfo
NwMkr76DZg+5/MJ65V6lB1TfWnUD3SP4CMRXZnmg57lsirUSuGDRvcPxUPUMwZ7o
Pl6znA7RqWPu/j2OEx8sPNWexsMN7GCr0NUJ9A7QAkz/jGMTjZDL0n2fz//ErtPI
HWMqRC36ks5ik9uBsKjWUsOY/kfgEbOmn72Wkeqe8Vt0SAkeQgE9VBleRGhmnWD9
YOAO1Z/EjJH/BwUi2p/Da2edVpgX3NStnOi71XiFrhMK7m1aMFjRYZRUd0pUsQEf
0IrrjhbSNzmezA1fT/vk+T7I3lcH6IrUtDJgr2ZaLCpET7qeDTLufxHIjuMRgva/
G/5lIljvfG0uHAQzKaASJps0gE8v5Ti/QPGVjG8WYXxYZFpBluY8AhJxzLr6bagO
dkYHRg9KvRuZ8OI3rIzCPPPdblxWkVjqmLYGrjwoEaIP8nAB8rDc2kZ6GY27gU2A
ArqwAf2Mu0i7Uapu7pbO+yOAosZbtA3vUQFCTsLZzvW4cMvzi050Mwk9VOgKFxXT
Omj5bb09H7aDrVxrOLemp1OQNYm81+YqXIbmGo/yXt1yhapxWZJgDUA06hcFqlUm
EYX2OY1GCqtpcLqGxuOO
=gZmy
-----END PGP SIGNATURE-----

Dan Clark

unread,
Mar 15, 2017, 7:23:04 AM3/15/17
to pylons-...@googlegroups.com
Tres,

I just committed the hello pyramid project to https://github.com/adidas/aws_hello_pyramid. It's a simple script (with requirements.txt) that can be tweaked to recreate the issue on AWS. Just need to change the requirements.txt to use 4.2.

Anyway, I ran out of time today, but if you give me another morning I can upload the project with 4.2 and get the complete traceback.

Thanks,
--Dan

--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.

Tres Seaver

unread,
Mar 15, 2017, 3:18:17 PM3/15/17
to pylons-...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/15/2017 07:23 AM, Dan Clark wrote:
> Tres,
>
> I just committed the hello pyramid project to
> https://github.com/adidas/aws_hello_pyramid. It's a simple script
> (with requirements.txt) that can be tweaked to recreate the issue on
> AWS. Just need to change the requirements.txt to use 4.2.
>
> Anyway, I ran out of time today, but if you give me another morning I
> can upload the project with 4.2 and get the complete traceback.

Dan,

Thanks for working on it, and for documenting your path. If you figure
something out about why you are needing to pin zope.deprecation, please
create an issue for either pyramid[1] or zope.deprecation[2] and mentaion
me (@tseaver) in the description.

[1] https://github.com/Pylons/pyramid/issues/
[2] https://github.com/zopefoundation/zope.deprecation/issues


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tse...@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJYyZNYAAoJEPKpaDSJE9HYwwsQALUqMf+LmYVHq6m8+A1nBCTT
M9ZPkb6eXYd+LaBTeCt0ah6/vtpbdsYeiDW/xKHWaJUZKq7v0CUvDd+MBaH9wmyc
vIKDwMm7NLdPHmke2W+RhseqRzRZlEi8fb12LXHH2wNJSWOiCXQqjSVH3I2ZszNI
oZI+nshAV/Yieu6zS3TvcjinW7uKZKc6D68UKYwgA8KAW2UK62t3TGdnojpEnNXX
nJI9ytUCFJLX1qGMM5DLF1uzKXqpjVUSyuw3MJcL6fncAlLNRndML51uLkYj2aiT
iI4g4HdptW46peqdPJJRh4tjL+6/87ZMgwsvoeRjmwSVEWV2VuYU4KuThxuRR96D
lmxsJf8HgdoU1ojVgZkLeaufpgHbLqcdqwwKFvQDHPM66UZ7s1oPSAS6nc8RvZLi
WSMb5g5LRty2QIctCywgiez9CeO6BKeaQdPGDtearXXYnzawyMwxKJDaMCGDdKpa
1nYE6Rz/MRz8Pox1uTzUsTN3UooZ+7K6c7tybBH8Jy7TuhI1whraIribkKvdWTWl
DZ/YDzoAFuLDe1oy1Ekm4hT93AFWACizE2OXDiJD3tGUi67V0IABniM/s27pb0uC
asQDEAeW21Eln7AD+fn3+/cA8ppsCRXPkfzMALItATjBeyKxXvA848yq32XCweny
Ntghx0YOXesn4ix5Oa0u
=soKF
-----END PGP SIGNATURE-----

Dan Clark

unread,
Mar 15, 2017, 9:38:06 PM3/15/17
to pylons-...@googlegroups.com
Here's the traceback

[Thu Mar 16 01:36:38.429962 2017] [:error] [pid 2187] [remote 69.127.251.49:50376] mod_wsgi (pid=2187): Target WSGI script '/opt/python/current/app/application.py' cannot be loaded as Python module.
[Thu Mar 16 01:36:38.430078 2017] [:error] [pid 2187] [remote 69.127.251.49:50376] mod_wsgi (pid=2187): Exception occurred processing WSGI script '/opt/python/current/app/application.py'.
[Thu Mar 16 01:36:38.430145 2017] [:error] [pid 2187] [remote 69.127.251.49:50376] Traceback (most recent call last):
[Thu Mar 16 01:36:38.430237 2017] [:error] [pid 2187] [remote 69.127.251.49:50376]   File "/opt/python/current/app/application.py", line 2, in <module>
[Thu Mar 16 01:36:38.430255 2017] [:error] [pid 2187] [remote 69.127.251.49:50376]     from pyramid.config import Configurator
[Thu Mar 16 01:36:38.430314 2017] [:error] [pid 2187] [remote 69.127.251.49:50376]   File "/opt/python/run/venv/lib/python3.4/site-packages/pyramid/config/__init__.py", line 12, in <module>
[Thu Mar 16 01:36:38.430329 2017] [:error] [pid 2187] [remote 69.127.251.49:50376]     from pyramid.interfaces import (
[Thu Mar 16 01:36:38.430383 2017] [:error] [pid 2187] [remote 69.127.251.49:50376]   File "/opt/python/run/venv/lib/python3.4/site-packages/pyramid/interfaces.py", line 1, in <module>
[Thu Mar 16 01:36:38.430399 2017] [:error] [pid 2187] [remote 69.127.251.49:50376]     from zope.deprecation import deprecated
[Thu Mar 16 01:36:38.430444 2017] [:error] [pid 2187] [remote 69.127.251.49:50376] ImportError: No module named 'zope.deprecation'
[Thu Mar 16 01:36:38.569219 2017] [:error] [pid 2187] [remote 69.127.251.49:54728] mod_wsgi (pid=2187): Target WSGI script '/opt/python/current/app/application.py' cannot be loaded as Python module.
[Thu Mar 16 01:36:38.569285 2017] [:error] [pid 2187] [remote 69.127.251.49:54728] mod_wsgi (pid=2187): Exception occurred processing WSGI script '/opt/python/current/app/application.py'.
[Thu Mar 16 01:36:38.569339 2017] [:error] [pid 2187] [remote 69.127.251.49:54728] Traceback (most recent call last):
[Thu Mar 16 01:36:38.569418 2017] [:error] [pid 2187] [remote 69.127.251.49:54728]   File "/opt/python/current/app/application.py", line 2, in <module>
[Thu Mar 16 01:36:38.569435 2017] [:error] [pid 2187] [remote 69.127.251.49:54728]     from pyramid.config import Configurator
[Thu Mar 16 01:36:38.569493 2017] [:error] [pid 2187] [remote 69.127.251.49:54728]   File "/opt/python/run/venv/lib/python3.4/site-packages/pyramid/config/__init__.py", line 12, in <module>
[Thu Mar 16 01:36:38.569509 2017] [:error] [pid 2187] [remote 69.127.251.49:54728]     from pyramid.interfaces import (
[Thu Mar 16 01:36:38.569563 2017] [:error] [pid 2187] [remote 69.127.251.49:54728]   File "/opt/python/run/venv/lib/python3.4/site-packages/pyramid/interfaces.py", line 1, in <module>
[Thu Mar 16 01:36:38.569595 2017] [:error] [pid 2187] [remote 69.127.251.49:54728]     from zope.deprecation import deprecated
[Thu Mar 16 01:36:38.569638 2017] [:error] [pid 2187] [remote 69.127.251.49:54728] ImportError: No module named 'zope.deprecation'

-----END PGP SIGNATURE-----

--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.

Dan Clark

unread,
Mar 15, 2017, 11:03:18 PM3/15/17
to pylons-...@googlegroups.com
I think I figured it out. go to the bottom for the actual issue. Hint it's not a permission issue. If you're good you might be able to figure it out before the end.

I decided to get as basic as possible. I SSH'ed into the EC2 VM and with in my home directory I used virtualenv to create a venv.

(venv)[ec2-user@ip-172-31-13-110 ~]$ virtualenv -p python3 deprecation
Running virtualenv with interpreter /opt/python/run/venv/bin/python3
Using real prefix '/usr'
New python executable in deprecation/bin/python3
Also creating executable in deprecation/bin/python
Installing setuptools, pip...done.


Then I installed pyramid into that venv

(deprecation)[ec2-user@ip-172-31-13-110 ~]$ pip install pyramid
You are using pip version 6.0.8, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pyramid
  Downloading pyramid-1.8.3-py2.py3-none-any.whl (574kB)
    100% |################################| 577kB 511kB/s 
Collecting PasteDeploy>=1.5.0 (from pyramid)
  Downloading PasteDeploy-1.5.2-py2.py3-none-any.whl
Collecting zope.deprecation>=3.5.0 (from pyramid)
  Downloading zope.deprecation-4.2.0-py2.py3-none-any.whl
Collecting repoze.lru>=0.4 (from pyramid)
  Downloading repoze.lru-0.6.tar.gz
Collecting venusian>=1.0a3 (from pyramid)
  Downloading venusian-1.0.tar.gz (45kB)
    100% |################################| 49kB 3.5MB/s 
Collecting WebOb>=1.7.0rc2 (from pyramid)
  Downloading WebOb-1.7.1-py2.py3-none-any.whl (83kB)
    100% |################################| 86kB 2.6MB/s 
Collecting hupper (from pyramid)
  Downloading hupper-0.4.4-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./deprecation/lib/python3.4/site-packages (from pyramid)
Collecting translationstring>=0.4 (from pyramid)
  Downloading translationstring-1.3-py2.py3-none-any.whl
Collecting zope.interface>=3.8.0 (from pyramid)
  Downloading zope.interface-4.3.3.tar.gz (150kB)
    100% |################################| 151kB 1.7MB/s 
Installing collected packages: zope.interface, translationstring, hupper, WebOb, venusian, repoze.lru, zope.deprecation, PasteDeploy, pyramid
  Running setup.py install for zope.interface
    building 'zope.interface._zope_interface_coptimizations' extension
    gcc -pthread -Wno-unused-result -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.4m -c src/zope/interface/_zope_interface_coptimizations.c -o build/temp.linux-x86_64-3.4/src/zope/interface/_zope_interface_coptimizations.o
    src/zope/interface/_zope_interface_coptimizations.c:34:1: warning: "METH_KEYWORDS" redefined
    In file included from /usr/include/python3.4m/Python.h:90,
                     from src/zope/interface/_zope_interface_coptimizations.c:15:
    /usr/include/python3.4m/methodobject.h:56:1: warning: this is the location of the previous definition
    gcc -pthread -shared build/temp.linux-x86_64-3.4/src/zope/interface/_zope_interface_coptimizations.o -L/usr/lib64 -lpython3.4m -o build/lib.linux-x86_64-3.4/zope/interface/_zope_interface_coptimizations.cpython-34m.so
    Skipping installation of /home/ec2-user/deprecation/lib64/python3.4/site-packages/zope/__init__.py (namespace package)
    Installing /home/ec2-user/deprecation/lib64/python3.4/site-packages/zope.interface-4.3.3-py3.4-nspkg.pth



  Running setup.py install for venusian
  Running setup.py install for repoze.lru
    Skipping installation of /home/ec2-user/deprecation/lib/python3.4/site-packages/repoze/__init__.py (namespace package)
    Installing /home/ec2-user/deprecation/lib/python3.4/site-packages/repoze.lru-0.6-py3.4-nspkg.pth



Successfully installed PasteDeploy-1.5.2 WebOb-1.7.1 hupper-0.4.4 pyramid-1.8.3 repoze.lru-0.6 translationstring-1.3 venusian-1.0 zope.deprecation-4.2.0 zope.interface-4.3.3

And then checked for zope.deprecation

(deprecation)[ec2-user@ip-172-31-13-110 ~]$ pip freeze
You are using pip version 6.0.8, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
hupper==0.4.4
PasteDeploy==1.5.2
pyramid==1.8.3
repoze.lru==0.6
translationstring==1.3
venusian==1.0
WebOb==1.7.1
zope.deprecation==4.2.0
zope.interface==4.3.3

Then I jumped into python and imported pip and made sure I still could see it.

(deprecation)[ec2-user@ip-172-31-13-110 deprecation]$ python
Python 3.4.3 (default, Sep  1 2016, 23:33:38) 
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pip #needed to use the pip functions
>>> for i in pip.get_installed_distributions(local_only=True):
...     print(i)
... 
zope.interface 4.3.3
pip 6.0.8
pyramid 1.8.3
zope.deprecation 4.2.0
translationstring 1.3
venusian 1.0
PasteDeploy 1.5.2
repoze.lru 0.6
setuptools 12.0.5
WebOb 1.7.1
hupper 0.4.4
>>> 

Then I tried to import zope.deprecation, but ng

>>> from zope.deprecation import deprecated
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'zope.deprecation'

From here it didn't make sense. I went into site-packages and zope.deprecation was there, BUT zope.interfaces was missing, but it was still working? Pip happily reported that zope.interface was still there

Then as I was uninstalling/installing with various options I noticed the paths. zope.deprecation was in lib and zope.interface was in lib64

(deprecation)[ec2-user@ip-172-31-13-110 site-packages]$ pwd
/home/ec2-user/deprecation/lib/python3.4/site-packages
(deprecation)[ec2-user@ip-172-31-13-110 site-packages]$ ls -l
total 412
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:34 appdirs-1.4.3.dist-info
-rw-rw-r--  1 ec2-user ec2-user  24701 Mar 16 02:34 appdirs.py
-rw-rw-r--  1 ec2-user ec2-user    126 Mar 16 02:34 easy_install.py
drwxrwxr-x  3 ec2-user ec2-user   4096 Mar 16 02:13 hupper
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:13 hupper-0.4.4.dist-info
drwxrwxr-x  3 ec2-user ec2-user   4096 Mar 16 02:34 packaging
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:34 packaging-16.8.dist-info
drwxrwxr-x  3 ec2-user ec2-user   4096 Mar 16 02:13 paste
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:13 PasteDeploy-1.5.2.dist-info
-rw-rw-r--  1 ec2-user ec2-user    304 Mar 16 02:13 PasteDeploy-1.5.2-py2.6-nspkg.pth
drwxrwxr-x 11 ec2-user ec2-user   4096 Mar 16 02:33 pip
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:33 pip-9.0.1.dist-info
drwxrwxr-x  3 ec2-user ec2-user   4096 Mar 16 02:34 pkg_resources
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:34 __pycache__
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:34 pyparsing-2.2.0.dist-info
-rw-rw-r--  1 ec2-user ec2-user 231039 Mar 16 02:34 pyparsing.py
drwxrwxr-x  7 ec2-user ec2-user   4096 Mar 16 02:13 pyramid
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:13 pyramid-1.8.3.dist-info
drwxrwxr-x  3 ec2-user ec2-user   4096 Mar 16 02:13 repoze
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:13 repoze.lru-0.6-py3.4.egg-info
-rw-rw-r--  1 ec2-user ec2-user    305 Mar 16 02:13 repoze.lru-0.6-py3.4-nspkg.pth
drwxrwxr-x  4 ec2-user ec2-user   4096 Mar 16 02:34 setuptools
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:34 setuptools-34.3.2.dist-info
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:34 six-1.10.0.dist-info
-rw-rw-r--  1 ec2-user ec2-user  30098 Mar 16 02:34 six.py
drwxrwxr-x  4 ec2-user ec2-user   4096 Mar 16 02:13 translationstring
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:13 translationstring-1.3.dist-info
drwxrwxr-x  5 ec2-user ec2-user   4096 Mar 16 02:13 venusian
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:13 venusian-1.0-py3.4.egg-info
drwxrwxr-x  3 ec2-user ec2-user   4096 Mar 16 02:13 webob
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:13 WebOb-1.7.1.dist-info
drwxrwxr-x  3 ec2-user ec2-user   4096 Mar 16 02:42 zope
drwxrwxr-x  2 ec2-user ec2-user   4096 Mar 16 02:42 zope.deprecation-4.2.0.dist-info
-rw-rw-r--  1 ec2-user ec2-user    349 Mar 16 02:42 zope.deprecation-4.2.0-py2.7-nspkg.pth
(deprecation)[ec2-user@ip-172-31-13-110 site-packages]$ cd zope
(deprecation)[ec2-user@ip-172-31-13-110 zope]$ ls -l
total 4
drwxrwxr-x 3 ec2-user ec2-user 4096 Mar 16 02:42 deprecation
(deprecation)[ec2-user@ip-172-31-13-110 zope]$ 

And zope.interface...

(deprecation)[ec2-user@ip-172-31-13-110 site-packages]$ pwd
/home/ec2-user/deprecation/lib64/python3.4/site-packages
(deprecation)[ec2-user@ip-172-31-13-110 site-packages]$ ls -l
total 12
drwxrwxr-x 3 ec2-user ec2-user 4096 Mar 16 02:42 zope
drwxrwxr-x 2 ec2-user ec2-user 4096 Mar 16 02:42 zope.interface-4.3.3-py3.4.egg-info
-rw-rw-r-- 1 ec2-user ec2-user  529 Mar 16 02:42 zope.interface-4.3.3-py3.4-nspkg.pth
(deprecation)[ec2-user@ip-172-31-13-110 site-packages]$ cd zope
(deprecation)[ec2-user@ip-172-31-13-110 zope]$ ls -l
total 4
drwxrwxr-x 5 ec2-user ec2-user 4096 Mar 16 02:42 interface
(deprecation)[ec2-user@ip-172-31-13-110 zope]$ 

My thought was that perhaps the interpreter was getting confused because it found zope under lib64 but not deprecate and just gave up.To test this out I tried uninstalling zope.interface, and I was now able to import zope.deprecate!

SO NOW WHAT??

Anyway, I am going to sleep. I doubt I'll work on this tomorrow morning, but if anyone can tell me how to trick the pip into installing zope.interface into lib/site-packages instead of lib64/site-packages, I'd appreciate it.

Thanks,
--Dan


To post to this group, send email to pylons-...@googlegroups.com.

Steve Piercy

unread,
Mar 16, 2017, 12:09:07 AM3/16/17
to pylons-...@googlegroups.com
When I see things like the following, I smell trouble:

(venv)[ec2-user@ip-172-31-13-110 ~]$ virtualenv -p python3 deprecation
(deprecation)[ec2-user@ip-172-31-13-110 ~]$ pip install pyramid
^^^ activated virtual environments, but where am I on the
system? ¯\_(ツ)_/¯

We used to get lots of questions about Pyramid installation
problems in IRC, most of which had nothing to do with Pyramid.
Almost always it boiled down to either misuse of activate,
mixing easy_install with pip, or a lack of understanding of a
Python virtual environment. We straightened it all out in our docs.

AWS can give you Python 3.4, which includes the minimum
requirements for installing packages. After that, you can pick
up from this point in the Pyramid docs, and Bob's your uncle:

http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/install.html#installing-pyramid-on-a-unix-system

Let me point out that the following commands are almost the same thing.

# Use explicit path to executable
# (this command will also upgrade pip and setuptools, ending the
nanny warning, too)
$ /path/to/virtual/environment/bin/pip install --upgrade pip setuptools

# Use explicit path to executable via an environment variable
set by
# $ export VENV=/path/to/virtual/environment
$ $VENV/bin/pip install --upgrade pip setuptools

# while in the virtual environment directory:
# $ source bin/activate
$ (environment)$ pip install --upgrade pip setuptools

We recommend the second method, then the first, and avoid the
third unless you know the implications of activate.

At this point, I'd recommend blowing away the virtual
environment, and trying this documented method instead of the
activate stuff, and see if that gets you further along.

--steve


On 3/15/17 at 11:03 PM, wpu....@gmail.com (Dan Clark) pronounced:
>>> pylons-discus...@googlegroups.com.
>>> To post to this group, send email to pylons-...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/ms
>>> gid/pylons-discuss/oac412%24p8k%241%40blaine.gmane.org.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>

Bert JW Regeer

unread,
Mar 16, 2017, 1:07:43 AM3/16/17
to pylons-...@googlegroups.com
Could you try using:

python3 -mvenv deprecation

Please?

I am wondering if this an issue with virtualenv using Py2 (32 bit) and Py3 (64 bit)? I am grasping at straws here, but the whole lib64 thing is interesting.

While using the python in the venv, could you also run:

import sys
print('\n'.join(sys.path))

I’d be interested in seeing what your sys.path is.

What version of Linux are you running? CentOS 7?

I am not sure why the version of zope.interface would change anything though.

Thanks,
Bert JW Regeer
> To unsubscribe from this group and all its topics, send an email to pylons-discus...@googlegroups.com.
> To post to this group, send email to pylons-...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/oac412%24p8k%241%40blaine.gmane.org.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
> To post to this group, send email to pylons-...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAKbG-MjGQe327AQGoPqYXR76f5uDJ7dMmG0Zu2%3D_TztuTJcwAw%40mail.gmail.com.

Bert JW Regeer

unread,
Mar 16, 2017, 3:58:03 PM3/16/17
to pylons-...@googlegroups.com
I did notice this on a Ubuntu 16.04 system that is using python3 -mvenv:

total 32K
drwxr-xr-x 6 root root 4.0K Mar 16 19:52 .
drwx------ 7 root root 4.0K Mar 16 19:54 ..
drwxr-xr-x 3 root root 4.0K Mar 16 19:54 bin
drwxr-xr-x 2 root root 4.0K Mar 16 19:52 include
drwxr-xr-x 3 root root 4.0K Mar 16 19:52 lib
lrwxrwxrwx 1 root root    3 Mar 16 19:52 lib64 -> lib
-rw-r--r-- 1 root root   60 Mar 16 19:52 pip-selfcheck.json
-rw-r--r-- 1 root root   69 Mar 16 19:52 pyvenv.cfg
drwxr-xr-x 3 root root 4.0K Mar 16 19:52 share

So it looks like lib64 should be symlinked to lib…

Bert

Dan Clark

unread,
Mar 17, 2017, 5:53:33 AM3/17/17
to pylons-...@googlegroups.com
Got stuck and then unstuck, but there are questions.

First, AWS's EB linux instances are redhat-ish

[ec2-user@ip-172-31-4-9 ~]$ cat /etc/*-release
NAME="Amazon Linux AMI"
VERSION="2016.09"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2016.09"
PRETTY_NAME="Amazon Linux AMI 2016.09"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2016.09:ga"
Amazon Linux AMI release 2016.09

Also trying to use python3 -m venv resulted in the following

[ec2-user@ip-172-31-4-9 ~]$ python3 -m venv /home/ec2-user/dep2
Error: Command '['/home/ec2-user/dep2/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

However, I was able to do the following

[ec2-user@ip-172-31-4-9 ~]$ python3 -m venv --without-pip $VENV
[ec2-user@ip-172-31-4-9 ~]$ ls dep
bin  include  lib  lib64  pyvenv.cfg
[ec2-user@ip-172-31-4-9 ~]$ ls dep/bin/
activate  activate.csh  activate.fish  python  python3

So this means no pip. To get past that I downloaded and installed pip

[ec2-user@ip-172-31-4-9 ~]$ curl https://bootstrap.pypa.io/get-pip.py | $VENV/bin/python3
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1558k  100 1558k    0     0  2817k      0 --:--:-- --:--:-- --:--:-- 2817k
Collecting pip
  Using cached pip-9.0.1-py2.py3-none-any.whl
Collecting setuptools
  Using cached setuptools-34.3.2-py2.py3-none-any.whl
Collecting wheel
  Using cached wheel-0.29.0-py2.py3-none-any.whl
Collecting six>=1.6.0 (from setuptools)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting appdirs>=1.4.0 (from setuptools)
  Using cached appdirs-1.4.3-py2.py3-none-any.whl
Collecting packaging>=16.8 (from setuptools)
  Using cached packaging-16.8-py2.py3-none-any.whl
Collecting pyparsing (from packaging>=16.8->setuptools)
  Using cached pyparsing-2.2.0-py2.py3-none-any.whl
Installing collected packages: pip, six, appdirs, pyparsing, packaging, setuptools, wheel
Successfully installed appdirs-1.4.3 packaging-16.8 pip-9.0.1 pyparsing-2.2.0 setuptools-34.3.2 six-1.10.0 wheel-0.29.0
[ec2-user@ip-172-31-4-9 ~]$ ls dep/bin/
activate      activate.fish  easy_install-3.4  pip3    python   wheel
activate.csh  easy_install   pip               pip3.4  python3

Then upgraded pip for good measure. This was pointless, but I wanted to make sure.

[ec2-user@ip-172-31-4-9 ~]$ $VENV/bin/pip install --upgrade pip setuptools
Requirement already up-to-date: pip in ./dep/lib/python3.4/site-packages
Requirement already up-to-date: setuptools in ./dep/lib/python3.4/site-packages
Requirement already up-to-date: packaging>=16.8 in ./dep/lib/python3.4/site-packages (from setuptools)
Requirement already up-to-date: six>=1.6.0 in ./dep/lib/python3.4/site-packages (from setuptools)
Requirement already up-to-date: appdirs>=1.4.0 in ./dep/lib/python3.4/site-packages (from setuptools)
Requirement already up-to-date: pyparsing in ./dep/lib/python3.4/site-packages (from packaging>=16.8->setuptools)

I will note that I feel ridiculous at this point because I can't find (or understand that I might be seeing) a proper solution for this. Everything I find is related to pyenv or virutalenv or a ubuntu OS. I assume I need to install something like python3-venv to get this working out of the box. It also doesn't help that I am more comfortable in a Ubuntu environment.

Anyway, seeing that everything was installed I went ahead and installed pyramid.

[ec2-user@ip-172-31-4-9 ~]$ $VENV/bin/pip install pyramid
Collecting pyramid
  Downloading pyramid-1.8.3-py2.py3-none-any.whl (574kB)
    100% |████████████████████████████████| 583kB 1.2MB/s 
Collecting PasteDeploy>=1.5.0 (from pyramid)
  Downloading PasteDeploy-1.5.2-py2.py3-none-any.whl
Collecting venusian>=1.0a3 (from pyramid)
  Downloading venusian-1.0.tar.gz (45kB)
    100% |████████████████████████████████| 51kB 5.0MB/s 
Requirement already satisfied: setuptools in ./dep/lib/python3.4/site-packages (from pyramid)
Collecting zope.deprecation>=3.5.0 (from pyramid)
  Downloading zope.deprecation-4.2.0-py2.py3-none-any.whl
Collecting repoze.lru>=0.4 (from pyramid)
  Downloading repoze.lru-0.6.tar.gz
Collecting WebOb>=1.7.0rc2 (from pyramid)
  Downloading WebOb-1.7.2-py2.py3-none-any.whl (83kB)
    100% |████████████████████████████████| 92kB 3.4MB/s 
Collecting hupper (from pyramid)
  Downloading hupper-0.4.4-py2.py3-none-any.whl
Collecting translationstring>=0.4 (from pyramid)
  Downloading translationstring-1.3-py2.py3-none-any.whl
Collecting zope.interface>=3.8.0 (from pyramid)
  Downloading zope.interface-4.3.3.tar.gz (150kB)
    100% |████████████████████████████████| 153kB 3.2MB/s 
Requirement already satisfied: six>=1.6.0 in ./dep/lib/python3.4/site-packages (from setuptools->pyramid)
Requirement already satisfied: appdirs>=1.4.0 in ./dep/lib/python3.4/site-packages (from setuptools->pyramid)
Requirement already satisfied: packaging>=16.8 in ./dep/lib/python3.4/site-packages (from setuptools->pyramid)
Requirement already satisfied: pyparsing in ./dep/lib/python3.4/site-packages (from packaging>=16.8->setuptools->pyramid)
Building wheels for collected packages: venusian, repoze.lru, zope.interface
  Running setup.py bdist_wheel for venusian ... done
  Stored in directory: /home/ec2-user/.cache/pip/wheels/5c/84/95/93b9705c1e71faa11c58e4e62f35664e6b9eab97ef8dffa56b
  Running setup.py bdist_wheel for repoze.lru ... done
  Stored in directory: /home/ec2-user/.cache/pip/wheels/b2/cd/b3/7e24400bff83325a01d492940eff6e9579f553f33348323d79
  Running setup.py bdist_wheel for zope.interface ... done
  Stored in directory: /home/ec2-user/.cache/pip/wheels/00/aa/8b/f1d1eb398423e59894b45ee151344e243808156c2d182c9f4e
Successfully built venusian repoze.lru zope.interface
Installing collected packages: PasteDeploy, venusian, zope.deprecation, repoze.lru, WebOb, hupper, translationstring, zope.interface, pyramid
Successfully installed PasteDeploy-1.5.2 WebOb-1.7.2 hupper-0.4.4 pyramid-1.8.3 repoze.lru-0.6 translationstring-1.3 venusian-1.0 zope.deprecation-4.2.0 zope.interface-4.3.3

And we're in a much better spot

[ec2-user@ip-172-31-4-9 dep]$ pwd
/home/ec2-user/dep
[ec2-user@ip-172-31-4-9 dep]$ ls -la
total 28
drwxrwxr-x 5 ec2-user ec2-user 4096 Mar 17 09:30 .
drwx------ 6 ec2-user ec2-user 4096 Mar 17 09:28 ..
drwxrwxr-x 2 ec2-user ec2-user 4096 Mar 17 09:34 bin
drwxrwxr-x 2 ec2-user ec2-user 4096 Mar 17 09:28 include
drwxrwxr-x 3 ec2-user ec2-user 4096 Mar 17 09:28 lib
lrwxrwxrwx 1 ec2-user ec2-user    3 Mar 17 09:28 lib64 -> lib
-rw-rw-r-- 1 ec2-user ec2-user   60 Mar 17 09:30 pip-selfcheck.json
-rw-rw-r-- 1 ec2-user ec2-user   69 Mar 17 09:28 pyvenv.cfg
[ec2-user@ip-172-31-4-9 dep]$ $VENV/bin/python
Python 3.4.3 (default, Sep  1 2016, 23:33:38) 
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import zope.deprecation
>>> import zope.interface
>>>  YAY NO ERRORS!!!
  File "<stdin>", line 1
    YAY NO ERRORS!!!
    ^
IndentationError: unexpected indent

This all leaves me with questions.

* What is the best way forward then? 
* How do we turn this into an easy to deploy boilerplate? 
* Am I knucklehead for downloading get-pip.py?
* Do I need to get AWS staff involved to get something fixed since their default install process isn't working?
* Should I become a plumber and forget all of this?
* What does 42 mean?

Seriously though, am I doing this right? I know I am a bit of a hacker when it comes to all of this, but it's how I operate.

To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/oac412%24p8k%241%40blaine.gmane.org.
For more options, visit https://groups.google.com/d/optout.



--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/36BE72D0-1224-4E06-81A7-EE5822959639%400x58.com.

Dan Clark

unread,
Mar 17, 2017, 6:18:56 AM3/17/17
to pylons-...@googlegroups.com
Thanks for the heads up on activate. I relied on that script quite a bit.

--Dan


To post to this group, send email to pylons-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/pylons-discuss/oac412%24p8k%241%40blaine.gmane.org.
For more options, visit https://groups.google.com/d/optout.

------------------------
Steve Piercy, Soquel, CA

--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.

To post to this group, send email to pylons-...@googlegroups.com.

Bert JW Regeer

unread,
Mar 17, 2017, 12:18:22 PM3/17/17
to pylons-...@googlegroups.com
On Mar 17, 2017, at 03:53, Dan Clark <wpu....@gmail.com> wrote:

Got stuck and then unstuck, but there are questions.

First, AWS's EB linux instances are redhat-ish

[ec2-user@ip-172-31-4-9 ~]$ cat /etc/*-release
NAME="Amazon Linux AMI"
VERSION="2016.09"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2016.09"
PRETTY_NAME="Amazon Linux AMI 2016.09"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2016.09:ga"
Amazon Linux AMI release 2016.09

Also trying to use python3 -m venv resulted in the following

[ec2-user@ip-172-31-4-9 ~]$ python3 -m venv /home/ec2-user/dep2
Error: Command '['/home/ec2-user/dep2/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

I hate distributions that do this…

But you are going to need to figure out what package you need to install to have ensurepip available. On Ubuntu this would python3-venv. 

Maybe this:

sudo yum install python35 python35-virtualenv python35-pip

and then using virtualenv3 instead? Either way, you know what the issue is now, and there is a work-around available.
Figure out why Amazon’s image doesn’t provide stock Python 3 functionality (why is -mvenv broken?)

 
* How do we turn this into an easy to deploy boilerplate? 

python3 -mvenv should just work… then there is no needing to do all the extra steps.

* Am I knucklehead for downloading get-pip.py?

It’s one way to make it work, so no?

* Do I need to get AWS staff involved to get something fixed since their default install process isn't working?

Maybe, find out if there is a package you can install that provides ensurepip

* Should I become a plumber and forget all of this?

Yes. I’m buying a parcel of land and keeping cows, and chickens.

* What does 42 mean?

It’s the answer.

To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAKbG-Mh5WWQRpehzdsk53K7aFi0p%3D2SOw8MTrvufEoWZBpP__w%40mail.gmail.com.

Steve Piercy

unread,
Mar 17, 2017, 6:40:15 PM3/17/17
to pylons-...@googlegroups.com
On 3/17/17 at 5:53 AM, wpu....@gmail.com (Dan Clark) pronounced:

>Got stuck and then unstuck, but there are questions.
>
>First, AWS's EB linux instances are redhat-ish
>
>[ec2-user@ip-172-31-4-9 ~]$ cat /etc/*-release
>NAME="Amazon Linux AMI"
>VERSION="2016.09"
>ID="amzn"
>ID_LIKE="rhel fedora"
>VERSION_ID="2016.09"
>PRETTY_NAME="Amazon Linux AMI 2016.09"
>ANSI_COLOR="0;33"
>CPE_NAME="cpe:/o:amazon:linux:2016.09:ga"
>HOME_URL="http://aws.amazon.com/amazon-linux-ami/"
>Amazon Linux AMI release 2016.09

The ID_LIKE attribute could be misleading.

Part of the battle might be understanding where everything is
located and their names. I don't have an AWS playground, but
maybe this will help.

Here is Python support information for AWS EB:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts.platforms.html#concepts.platforms.python

Let's see where and which version of Python 3 you really have,
then look in the same directory for other Python packaging tools.

$ python3 --version
$ which python3

You might have a pip3, easy_install-3.4, and virtualenv. Check
the versions and whether they symlink elsewhere. Report back
and maybe we can figure out what's next.

Also I recall that you had to use `sudo` at one point, which
could be a sign of system hosing. If you suspect something is
terribly wrong, then there's no harm in blowing away your AWS
instance and starting clean, or rolling back to the initial
clean version (if rollbacks are possible with AWS).

--steve

Dan Clark

unread,
Mar 18, 2017, 7:37:14 PM3/18/17
to pylons-...@googlegroups.com
Steve/Bert

> Let's see where and which version of Python 3 you really have, then look in the same directory for other Python packaging tools.

[ec2-user@ip-172-31-4-9 ~]$ python3 --version
Python 3.4.3
[ec2-user@ip-172-31-4-9 ~]$ which python3
/usr/bin/python3

However, I am not sure what you want me to look for. If I go into /usr/bin here are the ones I think might be relevant

lrwxrwxrwx 1 root root   24 Mar 17 09:22 python -> /etc/alternatives/python
-rwxr-xr-x 1 root root 5120 Sep  1  2016 python27
-rwxr-xr-x 1 root root 5120 Sep  1  2016 python2.7
-rwxr-xr-x 1 root root 1846 Sep  1  2016 python2.7-config
lrwxrwxrwx 1 root root   25 Mar  7 19:40 python3 -> /etc/alternatives/python3
-rwxr-xr-x 3 root root 6864 Sep  1  2016 python34
-rwxr-xr-x 3 root root 6864 Sep  1  2016 python3.4
lrwxrwxrwx 1 root root   17 Mar  7 19:40 python3.4-config -> python3.4m-config
-rwxr-xr-x 3 root root 6864 Sep  1  2016 python3.4m
-rwxr-xr-x 1 root root  173 Sep  1  2016 python3.4m-config
-rwxr-xr-x 1 root root 3288 Sep  1  2016 python3.4m-x86_64-config
lrwxrwxrwx 1 root root   32 Mar  7 19:40 python3-config -> /etc/alternatives/python3-config
lrwxrwxrwx 1 root root   31 Mar 17 09:22 python-config -> /etc/alternatives/python-config
lrwxrwxrwx 1 root root   25 Mar  7 19:40 python-config2 -> /usr/bin/python3.4-config
lrwxrwxrwx 1 root root   25 Mar  7 19:40 pyvenv3 -> /etc/alternatives/pyvenv3
-rwxr-xr-x 1 root root  230 Sep  1  2016 pyvenv3.4

lrwxrwxrwx 1 root root     21 Mar 17 09:22 pip -> /etc/alternatives/pip
-rwxr-xr-x 1 root root    290 Aug 10  2016 pip-2.7

lrwxrwxrwx 1 root root  30 Mar 17 09:22 easy_install -> /etc/alternatives/easy_install
-rwxr-xr-x 1 root root 328 Aug 10  2016 easy_install-2.7
-rwxr-xr-x 1 root root 328 Aug 10  2016 easy_install-3.4

in /etc/alternatives we have

lrwxrwxrwx 1 root root 18 Mar 17 09:22 python -> /usr/bin/python2.7
lrwxrwxrwx 1 root root 34 Mar 17 09:22 python.1.gz -> /usr/share/man/man1/python2.7.1.gz
lrwxrwxrwx 1 root root 18 Mar  7 19:40 python3 -> /usr/bin/python3.4
lrwxrwxrwx 1 root root 34 Mar  7 19:40 python3.1.gz -> /usr/share/man/man1/python3.4.1.gz
lrwxrwxrwx 1 root root 25 Mar  7 19:40 python3-config -> /usr/bin/python3.4-config
lrwxrwxrwx 1 root root 25 Mar 17 09:22 python-config -> /usr/bin/python2.7-config
lrwxrwxrwx 1 root root 18 Mar  7 19:40 pyvenv3 -> /usr/bin/pyvenv3.4

lrwxrwxrwx 1 root root 16 Mar 17 09:22 pip -> /usr/bin/pip-2.7

lrwxrwxrwx 1 root root 25 Mar 17 09:22 easy_install -> /usr/bin/easy_install-2.7

Versions:

[ec2-user@ip-172-31-4-9 bin]$ /usr/bin/pip-2.7 --version
pip 6.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)

[ec2-user@ip-172-31-4-9 bin]$ /usr/bin/easy_install-3.4 --version
setuptools 12.2
[ec2-user@ip-172-31-4-9 bin]$ /usr/bin/easy_install-2.7 --version
setuptools 12.2

> Also I recall that you had to use `sudo` at one point

I have been starting fresh with new EB instances. Haven't used sudo recently.

Either way it sounds like pip for 3.4 isn't installed by default. I hope to spend some time tonight to figure out how python -m venv works. Anyway, I then followed Bert's advice and tried installing the packages.

Now I have

lrwxrwxrwx 1 root root     21 Mar 18 23:17 pip -> /etc/alternatives/pip
-rwxr-xr-x 1 root root    290 Aug 10  2016 pip-2.7
-rwxr-xr-x 1 root root    290 Aug 10  2016 pip-3.5

However, I am still getting the same error. I noticed that the /etc/alternatives link was still pointing to 2.7, so I changed that with

sudo alternatives --install /usr/bin/pip pip /usr/bin/pip-3.5 1

and now pip's version is

[ec2-user@ip-172-31-4-9 alternatives]$ pip --version
pip 6.1.1 from /usr/lib/python3.5/dist-packages (python 3.5)

but I still get the same error when installing a new env. Not sure what's next.

I need to feed the kids now, but on the next go around I am going to blow everything away and start from scratch.

Anyway, I do appreciate the help.

Thanks,
--Dan



--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.

Steve Piercy

unread,
Mar 19, 2017, 2:58:22 AM3/19/17
to pylons-...@googlegroups.com
On 3/18/17 at 7:37 PM, wpu....@gmail.com (Dan Clark) pronounced:
Interesting. I've never seen such a structure.

Following the symlink chain:

/usr/bin/python3 -> /etc/alternatives/python3 -> /usr/bin/python3.4
/usr/bin/pip -> /etc/alternatives/pip -> /usr/bin/pip-2.7
/usr/bin/easy_install -> /etc/alternatives/easy_install -> /usr/bin/easy_install-2.7

I don't think it matters that pip and setuptools point to a 2.7
package because I assume that both pip and setuptools work
cross-platform for Python 2.7.(something) and 3.4.3. Let's
leave that alone for now.

I would ignore pyvenv for creating virtual environments, as it
eventually became deprecated in 3.6 and it invokes the venv
module anyway. The venv module has been included with Python
since v3.3, and is preferred for creating virtual environments.
Python 3.4.3 should include pip and setuptools, too.
easy_install is part of setuptools.

When you start clean, here's what I recommend.

* Use your *system* package manager to update the system (sudo
yum update).
* Verify that you have Python 3.4.3, pip, and setuptools
(easy_install) by checking their --version.
* Do not update your system Python 3.4.3, pip, or setuptools,
except through the system package manager. Do not install any
of these if they appear missing.
* Create a virtual environment inside your user home directory.

$ cd ~
$ export VENV=${PWD}/env
$ python3 -m venv $VENV

* Upgrade pip and setuptools inside your virtual environment,
which is isolated from your system.

$ $VENV/bin/pip install --upgrade pip setuptools

* Check versions of Python, pip, and setuptools inside your
virtual environment.

$ $VENV/bin/python --version
$ $VENV/bin/pip --version
$ $VENV/bin/easy_install --version

* Now install Pyramid into your virtual environment.

$ $VENV/bin/pip install pyramid

* Create a project.

$ mkdir myproject
$ cd myproject
... create and edit a Hello World app ...
... prepare your app for AWS deployment ...

I left the rest of the details up to you.

--steve
>>pylons-discus...@googlegroups.com.
>>To post to this group, send email to pylons-...@googlegroups.com.
>>To view this discussion on the web visit https://groups.google.com/d/ms
>>gid/pylons-discuss/r473Ps-10123i-C625127BE1D54D76855C51B3331
>>2A5AC%40Steves-iMac.local.
>>
>>For more options, visit https://groups.google.com/d/optout.
>>
>

Dan Clark

unread,
Mar 19, 2017, 7:52:12 AM3/19/17
to pylons-...@googlegroups.com
Thanks Steve,

Below is the results from following your steps. I started clean with the sample application, not my app.

> * Use your *system* package manager to update the system (sudo yum update).

[ec2-user@ip-172-31-28-79 ~]$ sudo yum update
... Omitted checks 
Running transaction
  Updating   : 1:openssl-1.0.1k-15.99.amzn1.x86_64                                                1/11 
  Updating   : 1:openssl-devel-1.0.1k-15.99.amzn1.x86_64                                          2/11 
  Updating   : kernel-headers-4.4.51-40.58.amzn1.x86_64                                           3/11 
  Updating   : kernel-tools-4.4.51-40.58.amzn1.x86_64                                             4/11 
  Installing : kernel-4.4.51-40.58.amzn1.x86_64                                                   5/11 
  Updating   : python27-crypto-2.6.1-1.14.amzn1.x86_64                                            6/11 
  Cleanup    : 1:openssl-devel-1.0.1k-15.96.amzn1.x86_64                                          7/11 
  Cleanup    : kernel-headers-4.4.44-39.55.amzn1.x86_64                                           8/11 
  Cleanup    : 1:openssl-1.0.1k-15.96.amzn1.x86_64                                                9/11 
  Cleanup    : kernel-tools-4.4.44-39.55.amzn1.x86_64                                            10/11 
  Cleanup    : python27-crypto-2.6.1-1.12.amzn1.x86_64                                           11/11 


  Verifying  : python27-crypto-2.6.1-1.14.amzn1.x86_64                                            1/11 
  Verifying  : kernel-4.4.51-40.58.amzn1.x86_64                                                   2/11 
  Verifying  : kernel-tools-4.4.51-40.58.amzn1.x86_64                                             3/11 
  Verifying  : kernel-headers-4.4.51-40.58.amzn1.x86_64                                           4/11 
  Verifying  : 1:openssl-devel-1.0.1k-15.99.amzn1.x86_64                                          5/11 
  Verifying  : 1:openssl-1.0.1k-15.99.amzn1.x86_64                                                6/11 
  Verifying  : 1:openssl-1.0.1k-15.96.amzn1.x86_64                                                7/11 
  Verifying  : 1:openssl-devel-1.0.1k-15.96.amzn1.x86_64                                          8/11 
  Verifying  : kernel-tools-4.4.44-39.55.amzn1.x86_64                                             9/11 
  Verifying  : kernel-headers-4.4.44-39.55.amzn1.x86_64                                          10/11 
  Verifying  : python27-crypto-2.6.1-1.12.amzn1.x86_64                                           11/11 

Installed:
  kernel.x86_64 0:4.4.51-40.58.amzn1                                                                   

Updated:
  kernel-headers.x86_64 0:4.4.51-40.58.amzn1         kernel-tools.x86_64 0:4.4.51-40.58.amzn1         
  openssl.x86_64 1:1.0.1k-15.99.amzn1                openssl-devel.x86_64 1:1.0.1k-15.99.amzn1        
  python27-crypto.x86_64 0:2.6.1-1.14.amzn1         

Complete!

> * Verify that you have Python 3.4.3, pip, and setuptools (easy_install) by checking their --version.

[ec2-user@ip-172-31-28-79 bin]$ /usr/bin/python --version
Python 2.7.12
[ec2-user@ip-172-31-28-79 bin]$ /usr/bin/python3 --version
Python 3.4.3
[ec2-user@ip-172-31-28-79 bin]$ /usr/bin/pip --version
pip 6.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)
[ec2-user@ip-172-31-28-79 bin]$ /usr/bin/easy_install --version
setuptools 12.2

> * Create a virtual environment inside your user home directory.


[ec2-user@ip-172-31-28-79 bin]$ cd
[ec2-user@ip-172-31-28-79 ~]$ export VENV=${PWD}/env
[ec2-user@ip-172-31-28-79 ~]$ python3 -m venv $VENV
Error: Command '['/home/ec2-user/env/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

Again, no pip, so executing

$VENV/bin/pip install --upgrade pip setuptools 

yields the following

[ec2-user@ip-172-31-28-79 ~]$ $VENV/bin/pip install --upgrade pip setuptools
-bash: /home/ec2-user/env/bin/pip: No such file or directory

However, I will note that we can create custom environments, so maybe this is the key. Whatever AWS is using for python (which I imagine is for flask and django) has been tailored for that. They also say you can also use Ubuntu or RHEL. I am thinking that a custom environment is needed with a proper python3 install.


Anyway, I will try to work on this a bit later. I just wanted to report back asap.

Thanks
--Dan


To post to this group, send email to pylons-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/pylons-discuss/r473Ps-10123i-C625127BE1D54D76855C51B3331
2A5AC%40Steves-iMac.local.

For more options, visit https://groups.google.com/d/optout.


------------------------
Steve Piercy, Soquel, CA

--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.

To post to this group, send email to pylons-...@googlegroups.com.

Steve Piercy

unread,
Mar 19, 2017, 8:23:13 AM3/19/17
to pylons-...@googlegroups.com
On 3/19/17 at 7:52 AM, wpu....@gmail.com (Dan Clark) pronounced:

> [ec2-user@ip-172-31-28-79 ~]$ python3 -m venv $VENV
> Error: Command '['/home/ec2-user/env/bin/python3', '-Im', 'ensurepip',
> '--upgrade', '--default-pip']' returned non-zero exit status 1

Read this:
https://docs.python.org/3.4/library/ensurepip.html

Then try this:

$ python3 -m ensurepip

Then:

$ python3 -m venv $VENV

Dan Clark

unread,
Mar 19, 2017, 9:20:09 AM3/19/17
to pylons-...@googlegroups.com
This makes sense, but it seems like the AWS installation of python3 isn't playing nice

[ec2-user@ip-172-31-28-79 dist-packages]$ which python3
/usr/bin/python3
[ec2-user@ip-172-31-28-79 dist-packages]$ python3 -m ensurepip
Traceback (most recent call last):
  File "/usr/lib64/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib64/python3.4/ensurepip/__main__.py", line 4, in <module>
    ensurepip._main()
  File "/usr/lib64/python3.4/ensurepip/__init__.py", line 209, in _main
    default_pip=args.default_pip,
  File "/usr/lib64/python3.4/ensurepip/__init__.py", line 98, in bootstrap
    "_bundled/{}".format(wheel_name),
  File "/usr/lib64/python3.4/pkgutil.py", line 629, in get_data
    return loader.get_data(resource_name)
  File "<frozen importlib._bootstrap>", line 1623, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib64/python3.4/ensurepip/_bundled/setuptools-12.0.5-py2.py3-none-any.whl'
[ec2-user@ip-172-31-28-79 dist-packages]$

I decided to search if ensurepip is anywhere. First I checked /usr/lib and /usr/lib64.

[ec2-user@ip-172-31-28-79 bin]$ cd /usr/lib/python3.4/
[ec2-user@ip-172-31-28-79 python3.4]$ ls
dist-packages
[ec2-user@ip-172-31-28-79 python3.4]$ cd dist-packages/
[ec2-user@ip-172-31-28-79 dist-packages]$ ls
__pycache__  README

...

[ec2-user@ip-172-31-28-79 python3.4]$ cd /usr/lib64/python3.4
[ec2-user@ip-172-31-28-79 python3.4]$ ls -la | grep ensu
drwxr-xr-x  3 root root   4096 Mar  7 19:40 ensurepip
[ec2-user@ip-172-31-28-79 python3.4]$ cd ensurepip/
[ec2-user@ip-172-31-28-79 ensurepip]$ ls -la
total 28
drwxr-xr-x  3 root root 4096 Mar  7 19:40 .
drwxr-xr-x 33 root root 4096 Mar  7 19:40 ..
-rw-r--r--  1 root root 6391 Sep  1  2016 __init__.py
-rw-r--r--  1 root root   67 Sep  1  2016 __main__.py
drwxr-xr-x  2 root root 4096 Mar  7 19:40 __pycache__
-rw-r--r--  1 root root  780 Sep  1  2016 _uninstall.py

I am not sure what this means. There isn't a _bundled directory to get the wheel from. It sounds like some sort of custom/old python install that is missing things.

In my searches I did find https://bugs.launchpad.net/ubuntu/+source/python3.4/+bug/1290847. It sounds like the issue is with the OS package for 3.4.

Still digging.


--Dan







--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.

Dan Clark

unread,
Mar 19, 2017, 10:03:28 AM3/19/17
to pylons-...@googlegroups.com
This is a bit above my skill level, but it sounds like there was a decision to intentionally exlude _bundled from RHEL distros.


It sounds like there was a patch to sttop the exclusion. I suspect that AWS hasn't worked necessary patches into their distro.

--Dan

To post to this group, send email to pylons-...@googlegroups.com.

Dan Clark

unread,
Mar 19, 2017, 12:20:13 PM3/19/17
to pylons-...@googlegroups.com
UGHHHHH!!!

I decided to try out one of the other environments. It turns out that that the preconfigured docker python environment is based off of debian instead of Amazon Linux. I deployed the "Hello Pyramid" app and it worked. The requirements.txt file just had a single entry "pyramid".

I also tried deploying my own app to it, but I am new to docker, so there's that. Once I get up to speed on docker I might be able get a bit further.

--Dan

Dan Clark

unread,
Mar 19, 2017, 6:54:30 PM3/19/17
to pylons-...@googlegroups.com
So the answer for Amazon Linux is that the setuptools and pip wheels are missing in /usr/lib64/python3.4/ensurepip/_bundled.


shows how another way to get around this. It also has more info on the state of the situation. My guess is that python34 needs to be rebuilt so that it isn't missing those wheels. Then amazon needs to update their images to make this all work.

--Dan

Steve Piercy

unread,
Mar 19, 2017, 8:04:50 PM3/19/17
to pylons-...@googlegroups.com
Can you file an issue with AWS?

Thank you for being persistent. This was quite the... uh... "learning experience".

--steve



On 3/19/17 at 6:54 PM, wpu....@gmail.com (Dan Clark) pronounced:
> >>>> pylons-discus...@googlegroups.com.
> >>>> To post to this group, send email to pylons-...@googlegroups.com.
> >>>> To view this discussion on the web visit https://groups.google.com/d/ms
> >>>> gid/pylons-discuss/r473Ps-10123i-5BE56888D60A4E4F98BD8088625
> >>>> 7387E%40Steves-iMac.local.
> >>>> For more options, visit https://groups.google.com/d/optout.
> >>>>
> >>>
> >>>
> >>
> >
>

Dan Clark

unread,
Mar 20, 2017, 6:06:14 AM3/20/17
to pylons-...@googlegroups.com
Looks like I need a support account in order to open a ticket. I posted to their support forum for now

(https://forums.aws.amazon.com/thread.jspa?threadID=251803&tstart=0)

Hopefully, that's enough. If it isn't maybe I'll sign up for the dev support for a month or something.

Either way, thanks everyone. You all have definitely helped through this. I have definitely learned quite a bit. Hopefully this gets resolved so that the process to create a pyramid app on AWS is as simple as it is for django or flask.

Thanks again,
--Dan

> >>>> pylons-discuss+unsubscribe@googlegroups.com.
> >>>> To post to this group, send email to pylons-discuss@googlegroups.com.

> >>>> To view this discussion on the web visit https://groups.google.com/d/ms
> >>>> gid/pylons-discuss/r473Ps-10123i-5BE56888D60A4E4F98BD8088625
> >>>> 7387E%40Steves-iMac.local.
> >>>> For more options, visit https://groups.google.com/d/optout.
> >>>>
> >>>
> >>>
> >>
> >
>

------------------------
Steve Piercy, Soquel, CA

--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/MSAMBzwx7aQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/r473Ps-10123i-78241046F47C4A5382EE723BD4AB1E40%40Steves-iMac.local.
Reply all
Reply to author
Forward
0 new messages