Adding support for installing libmapper via package managers
29 views
Skip to first unread message
Matt Peachey
Jul 12, 2021, 12:38:58 PM7/12/21
to dot_mapper
Hi everyone,
I think it would be great if libmapper was able to be installed and used directly from package managers. I have been working on trying to create a PyPi package that uses the existing python bindings (found in the /swig directory) that would hopefully allow a user to install them to their machine via "pip install libmapper" or a similar command.
This type of work is far from my expertise however, and I have made only small amounts of progress. I have been able to build a wheel via the existing setup.py script and "python setup.py bdist_wheel". I also installed the .whl file on a separate linux machine (AWS EC2), using "pip install <wheel file name>" which doesn't report any errors. However when I try to "import mapper as mpr" in a python script, I get an import error (ImportError: libmapper.so.8: cannot open shared object file: No such file or directory) which has lead me to believe that the underlying C code and/or .so files are not being packaged properly (or at all) with the python wheel.
I'm curious if anyone in this group has tried building a PyPi package for distributing libmapper or has done this type of thing with other projects? If so, I would appreciate some guidance on the issue as online documentation seems to be a bit thin for distributing SWIG bindings.
It would also be great if the existing libmapper bindings for other languages could be installed via their respective package managers as well, so I'd be interested to know if anyone has worked in that area!!
I'm looking forward to hearing your thoughts! Thanks!
Edu Meneses
Jul 14, 2021, 3:36:33 AM7/14/21
to dot_m...@googlegroups.com
+1 for this idea!
On a similar topic, I've mentioned to Joe and Johnty the need for a binary distribution for webmapper. I needed it a couple of weeks ago for a VPN test, and stopping everything to compile libmapper/webmapper remotely is not the best option (it was technically challenging for the remote user).
It's great to start a discussion on how we can maintain libmapper/webmapper on package managers: Max/MSP package manager, PD Deken, PyPi, Arduino (Mathias B. is already on that), SC (Mathias is also on that one), snap (?)
Cheers,
Edu Meneses
McGill University | Ph.D. Researcher
--
You received this message because you are subscribed to the Google Groups "dot_mapper" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dot_mapper+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dot_mapper/5d63170a-9295-4277-b7bd-9aa7d302e303n%40googlegroups.com.
Stephen Sinclair
Jul 14, 2021, 3:44:03 AM7/14/21
to dot_m...@googlegroups.com
I have done some work in this direction (making a multiplatform PyPI
package) but dropped it as I didn't have time. I can try to work on
it a bit more, I still have it on a local branch I think.
One thing that stopped me was realizing there was already a PyPI
package called "mapper" so I wasn't sure what to do name-wise. Maybe
simply "libmapper" is fine. (libmapper's Python module is called
"mapper" however, so it could get a bit confusing)
We could also try to rekindle efforts to get it packaged for Debian.
Steve
> To view this discussion on the web visit https://groups.google.com/d/msgid/dot_mapper/CAGg7NZ7b5n7S2FFd0adx96T-KvW7XOf_OMryTrgTZX6ggsJb7g%40mail.gmail.com.
package) but dropped it as I didn't have time. I can try to work on
it a bit more, I still have it on a local branch I think.
One thing that stopped me was realizing there was already a PyPI
package called "mapper" so I wasn't sure what to do name-wise. Maybe
simply "libmapper" is fine. (libmapper's Python module is called
"mapper" however, so it could get a bit confusing)
We could also try to rekindle efforts to get it packaged for Debian.
Steve
Stephen Sinclair
Jul 21, 2021, 5:33:00 PM7/21/21
to dot_m...@googlegroups.com
Apologies if anyone was getting lots of github actions notification
emails today, I was taking a shot at this.
I almost got it working, building the python wheels using the package
cibuildwheel, but not quite there yet.
It actually works and builds the wheels locally if I do:
python -m pip wheel build/swig
--wheel-dir=/tmp/cibuildwheel/built_wheel --no-deps -v
which is the same command that cibuildwheel uses. But, so far on
github actions I am stuck at a complaint about the .so already
existing, even though I have a command to rm it first. So a bit
confused, but it's a complicated rube goldberg stack of dockers.
https://github.com/libmapper/libmapper/runs/3125446217
In the meantime I *am* able to produce wheels locally, at least for
Linux, so theoretically we could manually upload them to pypi if we
can figure out what to do about the name conflict. I have no idea
what would be involved in doing the same thing for Mac and Windows --
to be investigated.
regards,
Steve
ps, technical details:
Basically the complicating aspect is that cibuildwheel (and "pip
wheel") is set up to handle projects where "setup.py" is in the main
directory and is used as the main build system. Since we have it in
the "swig" directory and it's only produced after running configure,
things get weird, because you need to run make first, then clean, then
delete some other things, and make mapper_wrap.c, all to prepare to
tell "pip wheel" to run. Even more confusing is that it appears that
"pip wheel" first makes a copy of the "swig" directory under "/tmp"
and tries to build it from there, making it not be able to find
mapper_internal.h. There is an experimental branch of "cibuildwheel"
that tells "pip wheel" not to do this by giving it the in-tree-build
feature flag, so that is what I am currently using. Very messy.
and on and on.. anyways, I'll try to give it another try soon. If
anyone wants to try, please be my guest ;)
I've been doing poorly-organized quick commits to the "wheels" branch,
I would plan to squash this eventually. Maybe use a different branch
if you want to try some things.
emails today, I was taking a shot at this.
I almost got it working, building the python wheels using the package
cibuildwheel, but not quite there yet.
It actually works and builds the wheels locally if I do:
python -m pip wheel build/swig
--wheel-dir=/tmp/cibuildwheel/built_wheel --no-deps -v
which is the same command that cibuildwheel uses. But, so far on
github actions I am stuck at a complaint about the .so already
existing, even though I have a command to rm it first. So a bit
confused, but it's a complicated rube goldberg stack of dockers.
https://github.com/libmapper/libmapper/runs/3125446217
In the meantime I *am* able to produce wheels locally, at least for
Linux, so theoretically we could manually upload them to pypi if we
can figure out what to do about the name conflict. I have no idea
what would be involved in doing the same thing for Mac and Windows --
to be investigated.
regards,
Steve
ps, technical details:
Basically the complicating aspect is that cibuildwheel (and "pip
wheel") is set up to handle projects where "setup.py" is in the main
directory and is used as the main build system. Since we have it in
the "swig" directory and it's only produced after running configure,
things get weird, because you need to run make first, then clean, then
delete some other things, and make mapper_wrap.c, all to prepare to
tell "pip wheel" to run. Even more confusing is that it appears that
"pip wheel" first makes a copy of the "swig" directory under "/tmp"
and tries to build it from there, making it not be able to find
mapper_internal.h. There is an experimental branch of "cibuildwheel"
that tells "pip wheel" not to do this by giving it the in-tree-build
feature flag, so that is what I am currently using. Very messy.
and on and on.. anyways, I'll try to give it another try soon. If
anyone wants to try, please be my guest ;)
I've been doing poorly-organized quick commits to the "wheels" branch,
I would plan to squash this eventually. Maybe use a different branch
if you want to try some things.
Joseph Malloch
Jul 23, 2021, 9:44:15 PM7/23/21
to dot_m...@googlegroups.com
Nice progress :)
Regarding the package name conflict, we have been tentatively using the name IOMapper for the Godot module to clarify that it concerns input-output mapping rather than game terrain. I’m personally ok with calling the PyPI package libmapper just to get it out there.
+1 for looking at the Debian package again!
Cheers,
Joe
To view this discussion on the web visit https://groups.google.com/d/msgid/dot_mapper/CADopUHasDpAS9jO-h9ZbZ9QD7kaSTRR5bG3SvPL5tKK2zVOQmA%40mail.gmail.com.
Stephen Sinclair
Jul 28, 2021, 5:11:07 AM7/28/21
to dot_m...@googlegroups.com
I think I got something working on Linux and Mac, not sure the process
is as clean as it could be, but please download artifacts.zip from
here and, in a python virtual env, try:
$ pip install <appropriate .whl file for your python version>
$ python3 -c 'import libmapper; print(libmapper.device(name="test"))'
Have not actually tested this on mac, so ymmv. Seems to work for me on Ubuntu.
https://github.com/libmapper/libmapper/actions/runs/1064941291
I notice there is no ARM build.
Steve
> To view this discussion on the web visit https://groups.google.com/d/msgid/dot_mapper/FDD3A3CD-CDEA-4577-86E1-8320B53C216C%40gmail.com.
is as clean as it could be, but please download artifacts.zip from
here and, in a python virtual env, try:
$ pip install <appropriate .whl file for your python version>
$ python3 -c 'import libmapper; print(libmapper.device(name="test"))'
Have not actually tested this on mac, so ymmv. Seems to work for me on Ubuntu.
https://github.com/libmapper/libmapper/actions/runs/1064941291
I notice there is no ARM build.
Steve
Reply all
Reply to author
Forward
0 new messages