Raspberry Pi build of Daily Snap Shots

[James Key]

Has anyone successfully built one on the daily snapshots on Raspberry Pi. I'm currently running 'Jessie' and attempted to build 4.0.0a4.dev3185. I was unable to compile the DEBUG version due to what appeared to be a memory issue, so I compiled the RELEASE version. It compiles but when it attempts to link it runs out of memory. The Pi has 256MB of memory so I thought I'd try a virtual QEMU install. It's hard coded with the same memory cap in the kernel. 

Are there some switches I'm missing? or is it possible to use another ARM Debian kernel to compile it on QEMU? 

[Tim Roberts]

James Key wrote:
> Has anyone successfully built one on the daily snapshots on Raspberry
> Pi. I'm currently running 'Jessie' and attempted to build
> 4.0.0a4.dev3185. I was unable to compile the DEBUG version due to what
> appeared to be a memory issue, so I compiled the RELEASE version. It
> compiles but when it attempts to link it runs out of memory. The Pi
> has 256MB of memory so I thought I'd try a virtual QEMU install. It's
> hard coded with the same memory cap in the kernel.

So, the question is not "has anyone RUN wxPython on a Pi", the question
is "has anyone BUILT wxPython on a Pi", right?

I'm not terribly surprised that 256MB is not enough to run the
development tools. You may need to cross-compile this from an x86
system, using something like buildroot.

--
Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.

[James Key]

Oh yes, it's about the build process, I'm trying to develop an app for the Pi and decided on wxPython. I'm not familiar with build root, but I will do some investigation.

[nepix32]

On Monday, July 10, 2017 at 6:12:46 PM UTC+2, Tim Roberts wrote:

I'm not terribly surprised that 256MB is not enough to run the

Maybe it would be cheaper (spending less hours) for the original poster to spend 40 Bucks on a raspberry Pi 3 with 1 GB RAM, compile there and transfer the binary to the smaller machine?

[James Key]

Not a bad idea!!! Sometimes the obvious escapes me!

On Sunday, July 9, 2017 at 5:30:41 PM UTC-4, James Key wrote:

[James Key]

The new Pi has arrived and wxWidgets will build on it both shared and monolithic, but when I attempt to build wxPython Phoenix 4.0.0b1. I keep getting this error:

Running command: dox

bash ./regen.sh xml

Finished command: dox (1m58.67s)

Running command: etg

"/usr/bin/python3.4" etg/image.py --sip --nodoc

Unable to find xml file for ITEM: wxTIFFHandler

Tried: /home/pi/Phoenix-wxPython-4.0.0b1/ext/wxWidgets/docs/doxygen/out/xml/classwx_t_i_f_f_handler.xml

       /home/pi/Phoenix-wxPython-4.0.0b1/ext/wxWidgets/docs/doxygen/out/xml/structwx_t_i_f_f_handler.xml

       /home/pi/Phoenix-wxPython-4.0.0b1/ext/wxWidgets/docs/doxygen/out/xml/wxTIFFHandler

Traceback (most recent call last):

  File "etg/image.py", line 681, in <module>

    run()

  File "etg/image.py", line 43, in run

    etgtools.parseDoxyXML(module, ITEMS)

  File "/home/pi/Phoenix-wxPython-4.0.0b1/etgtools/__init__.py", line 82, in parseDoxyXML

    raise DoxyXMLError(msg)

etgtools.DoxyXMLError: Unable to find xml file for ITEM: wxTIFFHandler

Command '"/usr/bin/python3.4" etg/image.py --sip --nodoc' failed with exit code 1.

Finished command: etg (0.814s)

I have installed and build sip and doxygen for the arm on the Pi.

Any thoughts?

On Sunday, July 9, 2017 at 5:30:41 PM UTC-4, James Key wrote:

[Robin Dunn]

Is it doxygen version 1.8.8? If not, then they may have changed how the
xml files are named or something. Do a grep in
ext/wxWidgets/docs/doxygen/out/xml for wxTIFFHandler to see if the
class' definition is in a different file.

On the other hand, if you are using the source tarball then you do not
need to do the dox, etg, or sip commands. The generated files should
already be present. Just start with the build command followed by
whatever packaging or installation commands you want to use.


--
Robin Dunn
Software Craftsman
http://wxPython.org

[James Key]

SUCCESS!!!

I have attached a Markdown document for those of you who wish to try this.

[Claudia Frank]

Thank you with this guide we were able to compile wxpython on the new rock64 sbc.

One additional step we needed to do was to replace the config.sub and config.guess 

within ext/widgets with a more recent version.

First test:

rock64@therock:~$ python -c "import wx; import platform; print 'wx:{}\narch:{}'.format(wx.version(),platform.machine())"

wx:4.0.0b1 gtk2 (phoenix)

arch:aarch64

Cheers

Claudia

[James Key]

I'm very happy that it help you. It's also available on the wxPython wiki for future reference.

[Daniel McBrearty]

Thank you for this James. I have followed it closely, but I get core dumps when trying to build. Very odd. I have posted the output in another post. If any ideas, most grateful.

[James Key]

Check out the wxpython wiki, there are some minor changes that occurred when the newest version was released.

https://wiki.wxpython.org/BuildWxPythonOnRaspberryPi

There was a change with gtk2 to gtk3.

On Dec 6, 2017 15:56, "Daniel McBrearty" <danielm...@gmail.com> wrote:

Thank you for this James. I have followed it closely, but I get core dumps when trying to build. Very odd. I have posted the output in another post. If any ideas, most grateful.

--
You received this message because you are subscribed to a topic in the Google Groups "wxPython-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wxpython-users/gJlZW56ZG44/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wxpython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel McBrearty]

yes, that is the page I have been using. Still don't have a successful build on 3.6 (I did get past compilation, but pip then hung on last install step - appeared to be trying to compile prerequsites AGAIN ... when all it really needed to do was move the compiled files to the right place and clean up ... ), trying again for 3.4 now. See comments in my thread above. Thanks.

On Wednesday, 6 December 2017 22:01:28 UTC+1, James Key wrote:

Check out the wxpython wiki, there are some minor changes that occurred when the newest version was released.

https://wiki.wxpython.org/BuildWxPythonOnRaspberryPi

There was a change with gtk2 to gtk3.

On Dec 6, 2017 15:56, "Daniel McBrearty" <danielm...@gmail.com> wrote:

Thank you for this James. I have followed it closely, but I get core dumps when trying to build. Very odd. I have posted the output in another post. If any ideas, most grateful.

--
You received this message because you are subscribed to a topic in the Google Groups "wxPython-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wxpython-users/gJlZW56ZG44/unsubscribe.

To unsubscribe from this group and all its topics, send an email to wxpython-user...@googlegroups.com.

[Daniel McBrearty]

There seems to me to be an issue most  likely caused by concurrency to the sdCard -  the build uses four concurrent jobs on an RPi, but it seems that some kind of corruption can occur - perhaps due to caching to the card? I don't know, but I've been now almost a week trying to get a clean build on an RPi 3, and I have succeeded exactly once. I have seen a lot of random seeming hangs, core dumps and crashes, and I am pretty sure the problem is not me.

For the compilation step, recommend to use the --jobs=1 flag (that seems to give a stable  build) but - unfortunately - the final "pip install" step *also* kicks off a round of multithread compilation which can also hang.

I don't know why I am the only person reporting this problem - perhaps it is only on a Pi 3 (quad core uP) that the issue gets serious, or perhaps not many people are trying to use wx on a Pi, or perhaps others have tried and failed silently. But I do know it is a serious issue for me. I am trying to use this platform for a fairly serious app, and I *need* reliable and repeatable behaviour. Corrupt (perhaps silently so) sdCards are not really compatible with that goal.

I am hoping that maybe Robin can throw some light on what is going on, perhaps tell me how I can do the final "pip install" step without concurrency, or whatever. For now, I have a few more things to try that I hope might mean I don't have to think about recoding my project on some other GX library:

1. use an industrial quality sdCard (I have a couple on order).

2. use wx without the final " install" step - I think that that is possible (at least according to the text that happens at the end of teh compilation step)

cheers

Daniel

[James Key]

I used a pi 3 myself, I had initially thought my difficulties were based on memory issues with my Pi zero. Though you will note that my notes at the bottom talk about if it hangs (esp. the pip install) to just restart the process. Maybe these two items are the same issue? I will up date the FAQ with the –jobs=1 flag just incase.

 

Sent from Mail for Windows 10

--

[James Key]

If you get a chance take a look at the wiki page and tell me if this is what you meant with the “–jobs=1”. I would, obviously, like to accurate. Also I’m wondering if something similar could be passed to pip or if the package could be edited to enable a similar restraint.

 

Sent from Mail for Windows 10

 

From: Daniel McBrearty
Sent: Monday, December 11, 2017 11:55
To: wxPython-users
Subject: [wxPython-users] Re: Raspberry Pi build of Daily Snap Shots

 

There seems to me to be an issue most  likely caused by concurrency to the sdCard -  the build uses four concurrent jobs on an RPi, but it seems that some kind of corruption can occur - perhaps due to caching to the card? I don't know, but I've been now almost a week trying to get a clean build on an RPi 3, and I have succeeded exactly once. I have seen a lot of random seeming hangs, core dumps and crashes, and I am pretty sure the problem is not me.

--

[James Key]

I investigating a “–skip-build” in the setup.py. This may be your fix.

 

Sent from Mail for Windows 10

 

From: Daniel McBrearty
Sent: Monday, December 11, 2017 11:55
To: wxPython-users
Subject: [wxPython-users] Re: Raspberry Pi build of Daily Snap Shots

 

There seems to me to be an issue most  likely caused by concurrency to the sdCard -  the build uses four concurrent jobs on an RPi, but it seems that some kind of corruption can occur - perhaps due to caching to the card? I don't know, but I've been now almost a week trying to get a clean build on an RPi 3, and I have succeeded exactly once. I have seen a lot of random seeming hangs, core dumps and crashes, and I am pretty sure the problem is not me.

--

[Chris Barker]

On Mon, Dec 11, 2017 at 8:55 AM, Daniel McBrearty <danielm...@gmail.com> wrote:

For the compilation step, recommend to use the --jobs=1 flag (that seems to give a stable  build) but - unfortunately - the final "pip install" step *also* kicks off a round of multithread compilation which can also hang.

pip often tries to be too "smart" -- a pip install will try to build from source if it thinks it needs to.

but what you want is for the build step to build a binary wheel, and then pip can simply install that.

Sorry, no time to look at the build scripts, but maybe with this hint you can figure out how to get it to use that process.

-CHB

 

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

[Daniel McBrearty]

Hi James, Thanks for this.

Yes, I read your excellent and very helpful page quite carefully (several times) and I did note your comments about hanging and so on.

The --jobs=1 is correct. I think that you should also change this:

$ pip3 install -r requirements.txt
    $ cd wxPython-4.0.0b2

These need to be in reverse order. And I actually found something similar with pip - at least once using "-r requirements" caused hangs and crashes, so I just installed the file contents one by one. i don't know how to make pip run single threaded but that is clearly advisable.

Another thing to note is DON'T do "make clean" (not sure what it does but it's not what you expect as it seems to result in a build with core package missing) and if you hit problems - start again, even with a fresh install because you might have corrupted the card (this happened at least once to me). Setting the boot options to ALWAYS run fsck is also a good idea.

For my part - I'm afraid that I was able to install and run Qt in a matter of minutes on the RPi (OK, not in a virtualenv, but that is not a prerequisite for me on this project - I am quite happy for my code to all use the installed python3, that should not cause problems unless python libraries are somewhere mutually incompatible, god forbid) so I am more than half convinced to just cut my losses and migrate now. For me, stability and repeatability are not really optional, and when I see that something in a build is resulting in this kind of flaky behaviour - I get a bit concerned.

All the best and many thanks for the help,

Daniel

On Sunday, 9 July 2017 23:30:41 UTC+2, James Key wrote:

[Chris Barker - NOAA Federal]

 > I don't know how to make pip run single threaded but that is clearly advisable.

I’m pretty  sure pip IS single threaded— which is why you can’t find a flag about it.

https://github.com/pypa/pip/issues/825

However, pip will simply invoke setuptools’ build, so THAT may not be single threaded.

But if this is all done right, if you’ve Already built it, pip shouldn’t build again.

So something is off.

-CHB

Another thing to note is DON'T do "make clean" (not sure what it does but it's not what you expect as it seems to result in a build with core package missing) and if you hit problems - start again, even with a fresh install because you might have corrupted the card (this happened at least once to me). Setting the boot options to ALWAYS run fsck is also a good idea.

For my part - I'm afraid that I was able to install and run Qt in a matter of minutes on the RPi (OK, not in a virtualenv, but that is not a prerequisite for me on this project - I am quite happy for my code to all use the installed python3, that should not cause problems unless python libraries are somewhere mutually incompatible, god forbid) so I am more than half convinced to just cut my losses and migrate now. For me, stability and repeatability are not really optional, and when I see that something in a build is resulting in this kind of flaky behaviour - I get a bit concerned.

All the best and many thanks for the help,

Daniel

On Sunday, 9 July 2017 23:30:41 UTC+2, James Key wrote:

Has anyone successfully built one on the daily snapshots on Raspberry Pi. I'm currently running 'Jessie' and attempted to build 4.0.0a4.dev3185. I was unable to compile the DEBUG version due to what appeared to be a memory issue, so I compiled the RELEASE version. It compiles but when it attempts to link it runs out of memory. The Pi has 256MB of memory so I thought I'd try a virtual QEMU install. It's hard coded with the same memory cap in the kernel. 

Are there some switches I'm missing? or is it possible to use another ARM Debian kernel to compile it on QEMU? 

--
You received this message because you are subscribed to the Google Groups "wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-user...@googlegroups.com.

[Robin Dunn]

On Tuesday, December 12, 2017 at 3:52:42 PM UTC-8, Chris Barker - NOAA Federal wrote:

 > I don't know how to make pip run single threaded but that is clearly advisable.

I’m pretty  sure pip IS single threaded— which is why you can’t find a flag about it.

https://github.com/pypa/pip/issues/825

However, pip will simply invoke setuptools’ build, so THAT may not be single threaded.

But if this is all done right, if you’ve Already built it, pip shouldn’t build again.

So something is off.

In our case there is a custom build command set in setup.py that gets invoked instead of setuptools default build, which uses build.py to do the actual building, and by default that will use the number of available CPU cores as the number of simultaneous build jobs to run. If you need to limit the number of jobs then my suggestion is to not use pip to do the build, just the install.  Instead, fetch the source tarball and run build.py something like this:

    python build.py --jobs=1 build bdist_wheel

And then use pip to install the wheel file produced by that command.

[James Key]

Thanks Robin!

 

Sent from Mail for Windows 10

 

--
You received this message because you are subscribed to a topic in the Google Groups "wxPython-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wxpython-users/gJlZW56ZG44/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wxpython-user...@googlegroups.com.