Re: Issue 186 in bpbible: Running BPBible 0.5 on Linux
bpb...@googlecode.com
Summary: Running BPBible 0.5 on Linux
Status: Accepted
Owner: jonmmorgan
Labels: Type-Defect Priority-Medium Milestone-Release0.5
Comment #1 on issue 186 by jonmmorgan: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
1. Up till now I have only tested BPBible 0.5 on Ubuntu, and I don't think
that is going to change. However, I'm happy to receive comments about
other distros and I have hopefully added support for xulrunner-stub in
r1168 and r1169 (though I haven't had a chance to test it).
2. As for wx.wc, that is the wxWebConnect library, which exists in a
somewhat patched form at https://github.com/jonmmorgan/wxwebconnect and
https://github.com/jonmmorgan/pywebconnect. It can cause quite a bit of
trouble getting it working, and I haven't really managed to make it
reliable to build. I have captured a brief dump of the build process at
https://github.com/jonmmorgan/pywebconnect/wiki/Build-instructions, but I'm
still not entirely sure that will work. I might be able to get a better
source distribution for it in the next week or so, which would not require
SWIG and could just be run directly. It's probably not worth trying until
I do that (though you are welcome to try).
bpb...@googlecode.com
Comment #2 on issue 186 by warkot: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Thanks for your detailed response, Jon.
1. Just tested, this works now. And I can always test things on Debian.
2. I did check the links you provided. Some things in the build sequence I
didn't completely understand (like what's a "matching" copy of
wxWebConnect, does this imply there's version compatibility issues between
wxWebConnect and wxWidgets? You have to get THE right version?)
The reason I didn't try building this thing (besides lack of building info)
is because I only have access to a netbook right now, not suited for
compiling wxWidgets on it. I think I'm gonna wait for "a better source
distribution" for now.
3. On a separate note, even though there's a YouTube video, running the
Linux version of BPBible is a bit of a pain for an average Joe, I think. I
have some experience in building Debian packages, and I'd like to change
that.
For starters, it wouldn't be hard to make a Debian/Ubuntu package that
would ship with _Sword.so, _wc.so, etc. Eventually, I think we could
package the python sword bindings and python wxWebConnect bindings as well.
A new DEB package for every minor/major release like 0.5.x would not take a
lot of my time.
What do you guys think about this? If I did it, would you be able to test
it on Ubuntu, and host it here?
bpb...@googlecode.com
Comment #3 on issue 186 by jonmmorgan: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
With respect to packaging, I agree it is not easy to set up and could be
improved. A month or so ago one of the Debian CrossWire Packaging Team
said he was intending to package BPBible for Ubuntu (the discussion is at
http://lists.alioth.debian.org/pipermail/pkg-crosswire-devel/2010-December/001372.html).
You might want to look at this first. Based on that email, the Sword
bindings have already been packaged. I think packaging as part of the
general Debian and Ubuntu repositories would be more useful than hosting
packages here.
The matching copy of wxWebConnect just means a copy of wxWebConnect that
matches the version of the Python bindings [it doesn't refer to wxWidgets -
any version of 2.8 should work, and I know it has also been used with 2.9
on Mac]. Taking the latest of each should probably be OK, but the Python
bindings will definitely not work with the stock version of wxWebConnect
from Kirix. (I would want to include both wxWebConnect and the Python
bindings in any source distribution of the bindings, to make sure there is
no such mismatch).
bpb...@googlecode.com
Comment #4 on issue 186 by warkot: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
> I think packaging as part of the general Debian and Ubuntu repositories
> would be more useful than hosting packages here.
No doubt about it, but after reading the correspondence at the
pkg-crosswire-devel mailing list I got the impression that the BPBible
v0.5.x-specific changes introduced in the wxWebConnect and its Python
bindings are not mature enough, which might make it very hard for those
guys to push these changes upstream, and hence, get these libs packaged in
Debian, am I right?
So yes, packaging as part of Debian/Ubuntu would be ideal, but that might
take ages to happen, in light of the above. Therefore I still think a
hackish DEB package that ships its own wxWebConnect libs pre-compiled would
be tons better than just a zipped source code file for people to tinker
with. Not ideal, no, but way better.
Personally, I could do with a zipped source archive, I don't need the DEB
package myself. So if I make one, will you host it until a better solution
like official Debian packages comes up?
bpb...@googlecode.com
Comment #5 on issue 186 by jonmmor...@gmail.com: Running BPBible 0.5 on
Linux
http://code.google.com/p/bpbible/issues/detail?id=186
I'm not sure whether hosting a package here would be better than anywhere
else (e.g. an Ubuntu PPU, Google Sites download, ...). If it is then we
would certainly consider doing it.
I haven't yet had time to make the changes necessary to have a zipped
source code archive. I don't know when I will do it. However, I have
uploaded the binaries (which I should have done earlier) to
https://sites.google.com/site/jmmorganswordmodules/. You could try that.
If you want to, download _wc.so and wc.py and put them in your Python
site-packages directory. I got it from
/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode/wx. Maybe you could
try putting it there, but I don't know whether Debian puts things in the
same places.
bpb...@googlecode.com
Comment #6 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Thanks for the binaries, Jon.
Here's my experience so far --
1. Debian looks for wc.py and _wc.so in the same location as Ubuntu, so no
trouble here.
2. Then things go bad, though. BPBible loads and then (when I think it
tries to render the content) it throws an ugly error box with gibberish
instead of proper characters and complains it can't find libxul.so. I
hit "Save" in that error window, the result is log1.txt
3. I have both xulrunner-1.9.1 (for xiphos) and xulrunner-1.9.2 (for
firefox) installed. There's no conflict between them, they both work fine
on my box. It's all within Debian package management.
The libraries are there:
% dlocate libxul.so
xulrunner-1.9.2: /usr/lib/xulrunner-1.9.2/libxul.so
xulrunner-1.9.1: /usr/lib/xulrunner-1.9.1/libxul.so
I assumed on Ubuntu they belong in /usr/lib, so I tried symlinking from
/usr/lib/libxul.so to /usr/lib/xulrunner-1.9.2/libxul.so or to the 1.9.1
version.
The error is gone (does this confirm my theory about Ubuntu putting
libxul.so in /usr/lib?), but BPBible won't even load then and zsh tells me
it segfaults, although no such message is in the logs. I saved a log to
log2.txt
Just a wild guess: Did you compile smth against a specific version of
xulrunner? If so, what's the exact xulrunner version you're running? and
your Ubuntu version?
Also, where's libxul.so on your system? What does "find /usr
-name 'libxul.so'" say?
I'd like to bug-test but can't get it to work so far.
Attachments:
log1.txt 3.3 KB
log2.txt 3.8 KB
bpb...@googlecode.com
Comment #7 on issue 186 by jonmmor...@gmail.com: Running BPBible 0.5 on
Linux
http://code.google.com/p/bpbible/issues/detail?id=186
It shouldn't be compiled against an exact version of XULRunner. However,
it does need to run against some version of XULRunner 1.9.2. In Ubuntu
they are not in /usr/lib, but it finds the XULRunner directory in
/usr/lib/xulrunner-1.9.2.13. The version I am running has changed from XR
1.9.2.11 to 1.9.2.12 to 1.9.2.13 without it stopping working. It should
link to libxul.so in the directory specified as /usr/lib/xulrunner-1.9.2.
It looks like it has detected XULRunner 1.9.2.13 OK. I'm a little
concerned about "Couldn't set locale or even english!!!! Bad things may
happen!!!" Don't know, though.
bpb...@googlecode.com
Comment #8 on issue 186 by jonmmor...@gmail.com: Running BPBible 0.5 on
Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Are you running 32-bit or 64-bit? Those binaries are for 32-bit, so I
assume you will be using 32-bit. It's possible there are some differences
in alignment of the standard structures, but that isn't supposed to
happen...
Up until the Pango warning about invalid UTF-8 (which I would guess is the
problem, though I don't know what causes it) it looks like a fairly normal
log. From the web, it seems like such errors can be to do with having an
invalid locale, but obviously that wasn't a problem with 0.4.7. Do you get
any similar Pango warnings from 0.4.7?
bpb...@googlecode.com
Comment #9 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
The "Couldn't set locale or even english!!!! Bad things may happen!!!" was
always there, it's a long-standing bug on linux (issue 123, issue 159).
Here's a log from r1077:
13:57:29 Couldn't set at all, smudging...
13:57:29 Couldn't set locale or even english!!!! Bad things may happen!!!
13:57:29 Couldn't add wx catalog 'en'
r1077 doesn't throw pango warnings, no, this is a new issue with r1171. I'm
not sure, but these pango warnings seem to be connected with the gibberish
in the message about not finding the libxul.so library. This is another
issue, it may even be pretty harmless, or not visible to the user. The
primary issue that the library is not found, no idea why :-(
bpb...@googlecode.com
Comment #10 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Oh, I forgot -- I'm running a 32-bit system.
bpb...@googlecode.com
Comment #11 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Is there any way to turn on extra debugging on this?
bpb...@googlecode.com
Comment #12 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
OK, here's an update --
I installed xulrunner from luvid-updates
(http://packages.ubuntu.com/lucid-updates/i386/xulrunner-1.9.2/download).
BPBible loads now (and the 0.5.b1 looks very nice, too :)
It does display another error about some other missing libs (log.txt) But
this may have smth to do with the Ubuntu/Debian xulrunner mismatch. Dunno.
I guess I do have to compile my own libs against a Debian xulrunner to
settle this, what do you think?
Attachments:
log.txt 3.3 KB
bpb...@googlecode.com
Comment #13 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
OK, I symlinked the two missing libs to there proper Debian locations and
now BPBible starts without throwing visible errors.
Here's what I did --
cd /usr/lib/ && ln -s libsqlite3.so.0 libsqlite3.so
cd /usr/lib/ && ln -s libsoftokn3.so nss/libsoftokn3.so
Now, of course iceweasel/firefox complains about missing libs and would't
start :)
I can at least start testing now, though. I will open issues per every bug.
I'm not sure about this specific issue, though. As far as I understand,
it's an issue of wc.so lib that needs to be compiled on Debian Squeeze
against current Debian xulrunner. I might find time to try it, but I might
need help as the procedure is tricky.
Then I could try and package a BPBible Lucid package with your binaries as
well as a Debian Squeeze package with mine.
Just thinking out loud here, maybe there's a better course of action?
bpb...@googlecode.com
Comment #14 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
OK, I symlinked the two missing libs to their proper Debian locations and
bpb...@googlecode.com
Comment #15 on issue 186 by jonmmor...@gmail.com: Running BPBible 0.5 on
Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Congratulations for persisting and getting that far. It's unfortunately
not easy.
Obviously there is a problem finding libraries. However, I'm not sure that
it is due to the Debian / Ubuntu mismatch, since I have experienced at
least some of them on Ubuntu. Ultimately, I think they are problems with
how wxWebConnect loads libraries:
1. It seems when looking for "libsqlite3.so" it does not
see "libsqlite3.so.0" as a match. I don't know enough about dynamic
linking on Linux to know whether this is a problem with how it links with
dlopen(), or whether it should also be searching for
libsqlite3.so.<sonumber>. I'll have to look into it.
2. libsoftokn3.so cannot be found: This is actually also a problem with my
Ubuntu system. However, I found that it threw up an error message which I
could then dismiss, and after that BPBible seemed to work OK. The reason
libsoftokn3.so is loaded is because it is listed in the dependent libraries
list for XULRunner (dependentlibs.list in the XULRunner directory).
However, unlike all the other libraries in that list, it was not in the
XULRunner directory. I don't know whether that is a problem with Ubuntu /
Debian packaging or with wxWebConnect's expectations. That is another
thing I will need to look into. Certainly on Windows both sqlite3.dll and
softkon3.dll are in the XULRunner directory as well as being in
dependentlibs.list. Intriguingly, I found that on my Ubuntu machine
libsoftokn3.so was in my Firefox-3.6.13 directory but not in my XULRunner
1.9.2.13 directory. No idea why yet.
bpb...@googlecode.com
Comment #16 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Well, I don't know about Linux dll linking either, but yeah, some part of
BPBible (whether xulrunner or wxWebConnect) is just looking for a specific
*name* of the libs, like libsqlite3.so and not libsqlite3.so.0, for example.
When packaging for Debian/Ubuntu it'd be obviously a very, very bad idea to
alter a file in another package (like
/usr/lib/xulrunner-1.9.2/dependentlibs.list), but making a symlink
automatically when a package is installed is totally OK. So this is not a
big issue, IMHO, but I'm no expert.
All I know is if I do --
cd /usr/lib/ && ln -s libsqlite3.so.0 libsqlite3.so
cd /usr/lib/ && ln -s libsoftokn3.so nss/libsoftokn3.so
bpbible loads all the libs just fine, no complains, and we could easily
make this symlinking part of Debian/Ubnuntu package.
Just to clarify: When I said Debian/Ubuntu mismatch I meant just name, not
code, mismatches. Like on Ubuntu a file is called libsqlite3.so.0 and not
libsqlite3.so, or vice versa, so you get fewer DLL load warnings on Ubuntu
than I do on Debian. Like I said, no big deal.
There's almost certainly a code mismatch/incompatibility between Ubuntu and
Debian xulrunners/wc.so's. That's because on Debian your wc.so makes
bpbible segfault. Now *that's* an issue, as it means I probably need to
recompile wxWidgets et al.
bpb...@googlecode.com
Comment #17 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Jon,
I'm trying to compile my own wc.so by following your build instructions.
According to the README of wxWidgets 2.8.10, the latest available SWIG
patch is swig-1.3.29.patch. But SWIG version 1.3.29 seems to only support
Python 2.4, not even Python 2.5, which is way too low.
Question: Which SWIG version did you use? And which Python version?
bpb...@googlecode.com
Comment #18 on issue 186 by jonmmor...@gmail.com: Running BPBible 0.5 on
Linux
http://code.google.com/p/bpbible/issues/detail?id=186
My understanding is that XULRunner and wxWebConnect are intended to be able
to load from the XULRunner directory without needing any globally defined
libraries (e.g. in /usr/lib). Obviously it does work loading from the
standard library path, but if I can find a way of avoiding that I would
prefer it. I'm inclined to think that making changes to /usr/lib should
also be avoided if it is possible.
I think SWIG should support Python 2.4 and greater. I used it with Python
2.6. I used the pre-patched source package from
http://wxpython.wxcommunity.com/tools/SWIG-1.3.29-wx.tar.gz.
bpb...@googlecode.com
Comment #19 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Thanks for the info, but where should I run your setup.py form, exactly?
wxwidgets2.8-2.8.10.1/wxPython/contrib?
bpb...@googlecode.com
Comment #20 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Incidentally, if I run it from wxwidgets2.8-2.8.10.1/wxPython/contrib, I
get this error:
Traceback (most recent call last):
File "./setup-wc.py", line 42, in <module>
from config import *
File "/home/*/Apps/bpbible/config.py", line 1, in <module>
from backend.verse_template import VerseTemplate, SmartVerseTemplate
File "/home/*/Apps/bpbible/backend/verse_template.py", line 3, in <module>
from swlib.pysw import process_digits
File "/home/*/Apps/bpbible/swlib/pysw.py", line 1499, in <module>
change_locale("bpbible", "abbr")
File "/home/*/Apps/bpbible/swlib/pysw.py", line 1453, in change_locale
worked, locale, locale_encoding = get_locale(lang,
additional=additional)
File "/home/*/Apps/bpbible/swlib/pysw.py", line 1444, in get_locale
assert locale.getEncoding() == "UTF-8", "Only UTF-8 locales supported
(locale was %s)" % locale.getName()
AssertionError: Only UTF-8 locales supported (locale was en)
Is setup-wc.py supposed to import stuff my BPBible installation directory?
A little confused here... Would appreciate your help.
bpb...@googlecode.com
Comment #21 on issue 186 by jonmmor...@gmail.com: Running BPBible 0.5 on
Linux
http://code.google.com/p/bpbible/issues/detail?id=186
It should be run from the base directory (e.g. wxWidgets2.8-2.8.10.1).
As for the config error, presumably that shows your BPBible directory is in
your Python path. There is a config module in this directory. It is
probably actually looking for a config module used in setuptools. I would
remove the BPBible path from your PYTHONPATH if it is there, then try again.
bpb...@googlecode.com
Comment #22 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Yes, I adjusted my PYTHONPATH now, thanks.
OK, here's what I got so far:
1. Currently I'm using Debian's wxwidgets2.8_2.8.10.1.orig.tar.gz source
package, and it doesn't have a config.py in the base directory. But it does
have one in the wxPython directory, which also includes setup.py (on which
setup-wc.py is based). So I assume wxPython is the directory I should run
setup-wc.py from (it doesn't work from other dirs, anyway.)
2. I've build a DEB of SWIG 1.3.29 with the proper patch applied, and
installed it. So I'm using the right version of SWIG.
3. After I follow your build steps, and run ./setup-wc.py USE_SWIG=1 from
wxPython directory, a directory at wxPython/contrib/wc/gtk is created with
these files in it:
wc.py
wc_wrap.cpp
That's it, no _wc.so.
There's no README on how to build wxPython, but by googling I discovered
that you do smth like "python2.5 setup.py build_ext --inplace". I tried
build_ext, build, and other commands, but none of this gets me _wc.so.
I feel like I'm missing smth here, any pointers would be very much
appreciated.
bpb...@googlecode.com
Comment #23 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
OK, this is very weird.
1. I got my _wc.so to build (I had to adjust your build instructions in
certain places), but it still doesn't work!
Traceback (most recent call last):
File "./bpbible.py", line 53, in <module>
import wx.wc
File "/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode/wx/wc.py",
line 8, in <module>
import _wc
ImportError:
/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode/wx/_wc.so: undefined
symbol: _ZN22wxWebControlXmlHandlerC1Ev
Does this mean anything to you?
2. for some obscure reason your wc.so started working! I'll explore
possible reasons for it, and test it on another Debian box.
Anyway, your _wc.so seems to work on my box... This probably needs more
testing.
bpb...@googlecode.com
Comment #24 on issue 186 by jonmmor...@gmail.com: Running BPBible 0.5 on
Linux
http://code.google.com/p/bpbible/issues/detail?id=186
1. Sorry, what it means is that not only did I give you wrong build
instructions (which I have just fixed: thanks for the comment), but I also
uploaded an old version of the setup-wc.py which does not include
xh_webcontrol.cpp. To fix this, you can either download a new version of
setup-wc.py or just add the following line into your setup-wc.py below the
line for webprefs.cpp:
'%s/webconnect/xh_webcontrol.cpp' % location,
2. I still have no idea why it fluctuates from non-working to working and
vice versa.
bpb...@googlecode.com
Comment #25 on issue 186 by war...@gmail.com: Running BPBible 0.5 on Linux
http://code.google.com/p/bpbible/issues/detail?id=186
1. Good news, with the new setup-wc.py, it compiles and SEEMS to work fine
on my box. No errors or warnings so far. Hooray, I got me my very own
_wc.so ;)
I'll test it some more. I still want to make a DEB for 0.5, for the release.
2. It's not anymore, I think.
Thanks for all the help.
bpb...@googlecode.com
Comment #26 on issue 186 by jonmmor...@gmail.com: Running BPBible 0.5 on
Linux
http://code.google.com/p/bpbible/issues/detail?id=186
Issue 231 has been merged into this issue.