Help wanted: debian package for plover

[Hesky Fisher]

Hi Everyone,

It would be fantastic if we could get a debian package made for plover. From the lively discussions going on, it seems like we have some engaged programmers on the list. If any of you want to volunteer to take this on then that would be much appreciated.

Hesky

[Daniel Langlois]

Plover has no Makefile on Github. I've attached one. Probably, a debian folder also, could be created at https://github.com/plover/plover, and would serve as a place for a rules file. A note, that I assume we want to build a source package, not a binary package. Now about that rules file. This is the debian Makefile used to generate your .deb package. I've attached one. 

I think the thing to consider, here, might be, that, using a Personal Package Archive (PPA), you can distribute software and updates directly to Ubuntu users. Create your source package, upload it and Launchpad will build binaries and then host them in your own apt repository. That means Ubuntu users can install our packages in just the same way they install standard Ubuntu packages and they'll

automatically receive updates as and when we make them. 

When Launchpad builds a source package in a PPA, it creates binaries for:

x86

AMD64

For Ubuntu versions prior to 10.04 (Lucid): LPIA.

If this sounds like the thing, then, well, you can only activate a PPA if you have signed the Ubuntu code of conduct. Launchpad generates a unique key for each PPA and uses it to sign any packages built in that PPA.

I get into all of this, because Ubuntu uses Debian's system of packaging software. To get software into a PPA, you need

to build a source package. So I figure, that's the idea. 

Here is the Ubuntu Packaging Guide:

http://developer.ubuntu.com/packaging/html/

It says for example: 'The rules file--The last file we need to look at is rules. This does all the work for creating our package. 

It is a Makefile with targets to compile and install the application, then create the .deb file from the 

installed files. It also has a target to clean up all the build files so you end up with just a source package again.'

I've tossed out a few points, because I'm curious what we are getting ourselves into. Once you have the source package branch in a shared repository, you’ll want to create additional branches for the fixes or other work you plan to do. Normally, even before looking into the whole rules file aspect, there would be the matter of creating a copyright file. https://github.com/plover/plover has

a LICENSE.txt, but this is different. Who holds the copyright? And, who is the upstream author(s). 

I'm familiar with this:

https://launchpad.net/plover

Plover used to be on Launchpad. I'm inclined to broach the subject. I have to admit that Github is pretty cool. Github does not have a build system integrated with it, like PPA..Github's Bug Tracker is next to useless. Launchpad does a lot of stuff that other systems don't. Including, a tool to manage the package archive and build binary packages for the various supported architectures.

So, code hosting is not the only important feature of Launchpad. I guess I might seem to care, in a way that really I'm not bothered.

Although, Launchpad's bug tracker is far, far superior to Github's. Launchpad will track project forks just like GitHub does, but it also allows you to attach a branch to milestones, bug tracking, blueprints, questions and answers, and translations. I claim not to be bothered, because I understand that Launchpad is a single vendor, and could provide some kind of standardized way of having Launchpad and sites like GitHub communicate.

Anyways, if it's just about getting somebody to create the files that are useful for the debian packaging, then I'll add that setup.py does a big part of the installing job, and you will just have to edit/create the mandatory files needed. 

- debian/changelog

- debian/compat

- debian/control

- debian/copyright

- debian/rules

- debian/watch

Some of these, I've discussed (I've attached a rules file). I'd like to suggest that an up-to-date Plover launchpad site, might be a good sandbox for this.

[Hesky Fisher]

Hi Daniel,

Thanks for taking the time to respond and for all the information.

Unfortunately, this is information overload for me right now. What I'm asking for is a volunteer who will create the source package start to finish, to build and test it as well to make sure it works as expected. Then send a pull request with all that's necessary to easily create packages for each new version just like there are for windows and osx.

Launchpad PPA is certainly an attractive option and we can explore using it. But using it comes strictly after creating the source package.

Hesky

--
You received this message because you are subscribed to the Google Groups "Plover" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ploversteno...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Daniel Langlois]

I note this, from https://github.com/plover/plover:

---

These installation notes are for Debian-like Linux systems. From the directory in which this README file is located, run the following commands:

sudo apt-get install python-xlib python-serial python-wxgtk2.8 appdirs

sudo python setup.py install

---

And, appdirs is not in Debian. 

This has come up, as here:

https://groups.google.com/forum/?fromgroups=#!topic/ploversteno/7Vfb_3ixaqg

---

Hesky Fisher  

Apr 7

Hi Varid and Brent,

I'm sorry you are having trouble installing plover. The real solution to this problem will be to have a debian package that people can use to install plover. Something like: apt-get install plover. Hopefully we will have that soon.

Until then, appdirs is a python package that plover depends on. Many python packages are not available via apt-get and need to be installed via pip or something similar. We'll need to update the instructions. We will also need to update the install guide because the version of plover linked to on launchpad is out of date.

Hesky

---

I'd like to construe the issue, as one of automatically find/download/install/upgrading dependencies at build time. More generally, our goal is to distribute a Python package that has several other widely used Python packages as dependencies.

What about pip/virtualenv. That is, installing pip and virtualenv, and then figuring out how these tools work, and how they are different. 

Here:

https://groups.google.com/forum/?fromgroups=#!searchin/ploversteno/pip/ploversteno/O8eUhHVegEk/zu-7TMUbl-EJ

---

I grabbed the zip from the dictupdate branch, and had some difficulties installing.  Thought you might want to know.

1) Appdirs.  Several people have mentioned this as being a problem, but just for clarity, the solution on Ubuntu is to install pip "sudo apt-get install python-pip", and then use pip to install appdirs "sudo pip install appdirs".

---

Is there a way to distribute software in a way that doesn't depend on users having to chase down all these dependencies and versions? To, perhaps, compile all the relevant packages into something self-contained, that can just be downloaded and used as a binary? This issue happens even when all of our Python package dependencies are well-known, well-written and well-maintained Pypi-available packages.

And, I get it, that the idea you have in mind, is that for the end user, the installation process will be sudo apt-get install some_python_package or sudo pacman -S some_python_package, however their package manager works. This is sensible. But there will need to be a bootstrap script, or something. This could download something for managing remote installations, and then build, and install, our package + deps.

I have created a little Debian package, a 'single binary'. I've run the Debian package checker on the package, it shows me a list of errors and warnings, such as, I didn't create a man page. But the last test of the package is to install it and run it. Which, it asks for appdirs :)

[Daniel Langlois]

To sum up, in my previous post I described building a .deb, and doing so, I got a fresh perspective on the problem of more easily building and distributing Python packages, especially ones that have dependencies on other packages. Here are some further notes:

Here is the idea, can we produce eggs, upload to PyPI, and automatically include all packages in the directory where the setup.py lives. Answer: yes. 

I mention releasing Plover to PyPI, has this been considered? Yes.

https://groups.google.com/forum/?fromgroups=#!searchin/ploversteno/pypi/ploversteno/ASNmkVTeqtQ/T8DBcb8EC7EJ

Kevin Tran  

11/8/11

Will Plover be uploaded to PyPI eventually?

Is there something that supports automatically installing dependencies when a package is installed, and including information about dependencies in Python Eggs (so that package management tools like EasyInstall can use the information)? Answer: Yes. 

I think we might download 'distribute_setup.py':

http://nightly.ziade.org/distribute_setup.py

And, put it in the same directory as your setup.py script.

That means, it literally goes into Github here: 

https://github.com/plover/plover

Then add these two lines to the very top of your setup script:

import distribute_setup

distribute_setup.use_setuptools()

Then, in the case of users not having setuptools installed on their machines, the distribute_setup module will automatically download a matching version of setuptools from PyPI. 

There is also a PyPI 'upload' command--you can use 'setup.py sdist upload' or 'setup.py bdist_egg upload' to upload your source or egg distributions respectively.

I'll mention setup.py register, and then I'll mention:

setup.py register sdist bdist_egg upload

This will register the package, build source and egg distributions, and then upload them both to PyPI. 

These are just notes, at this point, had a little progress..

[Thomas Thurman]

On 28/04/2013, Hesky Fisher <hesky....@gmail.com> wrote:
> Hi Daniel,
>
> Thanks for taking the time to respond and for all the information.
>
> Unfortunately, this is information overload for me right now. What I'm
> asking for is a volunteer who will create the source package start to
> finish, to build and test it as well to make sure it works as expected.
> Then send a pull request with all that's necessary to easily create
> packages for each new version just like there are for

> windows<https://github.com/plover/plover/tree/master/windows>and
> osx <https://github.com/plover/plover/tree/master/osx>.

>
> Launchpad PPA is certainly an attractive option and we can explore using
> it. But using it comes strictly after creating the source package.

I made the original ITP, but work got in the way. Then another person
on this list (I forget who) said he'd rather make the .deb, but that
seemed to come to nothing. I'm happy to give it another go this week,
if people would like. If anyone would like to help, all the better!

T

[Hesky Fisher]

Hi Daniel,

I'm not sure what the right answer is for appdirs. It is a dependent package that does not have a debian package. I'm hoping there is a standard method for dealing with this kind of problem somewhere.

I believe that debian frowns on downloads being needed as part of the installation process of packages.

Hesky

[Hesky Fisher]

Hi Thomas,

If you could work on this then that would be much appreciated.

Thanks,

Hesky

T

[Jeremy Kahn]

I believe the recommended approach would be to help "appdir" release their package to Debian or at least a PPA.

Less preferred, but still workable, would be to bundle the appropriate version of appdir into the deb package for plover.

[Hesky Fisher]

Hi Jeremy,

Thanks for the response!

Since option one is preferred I opened an issue with them to see if they are interested: https://github.com/ActiveState/appdirs/issues/29

In the meantime, I think we should push ahead with option two and bundle appdirs in our package so that we are not slowed down.

Hesky

[Daniel Langlois]

For what it's worth, here's a .deb, taking the approach of automatically installing PYPI dependencies. Anyways, it's a .deb, what one looks like, and it's something to test on somebody else's machine. A .deb file is installed with this: sudo dpkg -i plover_1.1-3_i386.deb

There's then a script at /usr/local/bin/plover

There are a couple of threads to disentangle, here. There's the matter of having a debian package, including having a make target for that..also, there are the modifications to automatically install dependencies (what I got into discussing for this, above, I have implemented, but I'll come back around w/maybe a simple pull request).

[Hesky Fisher]

Thanks Daniel!

Could you also share the source package that you used to make this?

Right now setup.py manages the installation of the application. It seems like setup.py should handle installing the python code as a python module but something else should actually do the OS parts.

[Daniel Langlois]

https://github.com/DanLanglois/PloverDebianPackage

This is my source package--it's 7 megs. If you compare it to Plover as represented in Github, there are some new folders, there is the debian folder, and I've discussed, a bit, the contents of this. At the moment, it's got lots of default configurations. there is a 'copyright' file, for example, and a 'changelog'. The files are normal to see in this 'debian' directory, and you have a 'debian' directory, for making a debian package. The debian make file is in there, it's called 'rules'.

Besides the 'debian' directory, there's a 'Makefile'. If you do 'make all', you'll see my 'make buildsrc' listed. 'sudo make buildsrc' will make a debian .deb. A build folder will appear here, in that case, as well. The .deb, is dropped up one directory, along with a .build file, a .changes file. Also in here, is a distribute-0.6.36-py2.7.egg, and distribute-0.6.36.tar.gz. Also, running make will download these files, if they are missing. I've added lines to setup.py, calling distribute_setup.py (also a new file).

There's a plover.egg-info directory. this directory is also created by the debian make target. So, in sum, there's something to peruse here. It works 'on my machine', but I made a number of provisional decisions.

What is it--it has a Maketarget that creates a Debian package, basically. There's a 'dist' folder,   that contains a plover egg. This also is created.

There are, then, multiple moving new parts, here. This may be perused for reference, I hope it's progress, towards adding debian packaging to plover.

[Hesky Fisher]

Hi Daniel,

Thanks again for sharing.

I think something went wrong in the build process though. I extracted the .deb file you sent with ar but there doesn't seem to be any of plover in there. I think the only files that made it into the .deb are the debian specific files.

Hesky

[Daniel Langlois]

Yes, oops. It turns out, that though the extraction procedure is trivial, you copy three files, and run 'dpkg-source -x plove_1.1-1.dsc'. This is a summary of the contents of the source code.

The .deb is, it seems, just a completed binary package. plover_1.1-1.debian.tar.gz contains our debian directory contents. There is also an original upstream source code tarball. I've attached this, 'plover-1.1.orig.tar.gz'.

So actually, nothing went wrong with the build process, but w/my general inexperience, I can't say much of anything more about the details of fixing upstream sources. There is a program that offers a basic method for recording modifications to the upstream source for Debian packages. One thing that I have encountered, is that while I thought most third-party software installed itself in the /usr/local directory hierarchy, and same w/Plover, on Debian this is reserved for private use by the system administrator. Rather than, as they say, obeying the Filesystem Hierarchy Standard, we can always sudo (it seems that actually this is the m.o.).

Beyond knowing something of the required files, and other files, under the debian directory, I haven't read much. Funny thing, I knew to check the package for errors, it wasn't actually *missing the full source!* I was running debian test scripts that were green :)

[Daniel Langlois]

Is this an interesting question-->whether a single file deployment with all binaries (extension modules and their dependencies) packed into the executable, is possible?

[Hesky Fisher]

It's certainly possible. That's how we do our mac and windows releases. But I'm not sure if that's the way debian releases are supposed to work.

[Jeremy Kahn]

See the legalistic (some might say Talmudic!) Debian Python Policy for answers, especially chapters two and three.

http://www.debian.org/doc/packaging-manuals/python-policy/

(The analogy is imperfect, because the DPP needs its own Talmud: commentary, tutorials and subtitles.)

[Thomas Thurman]

I'll work on it and report back before the end of the week. T

On 29 April 2013 18:33, Daniel Langlois <danlan...@gmail.com> wrote:

On Monday, April 29, 2013 11:16:20 AM UTC-5, Hesky Fisher wrote:

[Thomas Thurman]

On 30 April 2013 11:24, Thomas Thurman <tho...@thurman.org.uk> wrote:
> I'll work on it and report back before the end of the week. T

Still working, but closer. Sorry, some things got in the way. T

[Thomas Thurman]

Right, I have a version that builds; now to try uploading it, etc. T

--
Thomas Thurman, poet and programmer - tho...@thurman.org.uk
My poems, mostly formalist, one each day: http://thomasthurman.org/poem-a-day/
What do you do when a dragon steals your library books? http://borrowable.net

[Daniel Langlois]

I found your ITP (Intent To Package) bug report against WNPP (on the WNPP bugs page).

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=654659

ITP: plover -- stenotype input method

Also, although I (basically) put a package together, it wasn't build against a current version of sid--which simple sentence hides a multitude of sins -- it can be a complex process.. But surely there is a more gentle introduction to the sport. A simple step is to provide a publically accessable place where all the files relating to your package can be downloaded from. I was considering the 'pretend' upload queue at http://mentors.debian.net/, which can be used to host packages if we have nowhere better to put them. 

Also, the mailing list <debian-...@lists.debian.org> has been set up for novice maintainers who seek help with initial packaging and other developer-related issues. It appears, that every new developer is invited to subscribe to that list, which is for 'helping newbie developers', and newbie Debian developers can seek help with packaging and other developer-related issues here. I've subscribed..

[Thomas Thurman]

Thanks, Daniel. I've uploaded the Plover package to https://mentors.debian.net/package/plover now; you see I have a few things still to work on, and I'll fix them in the next couple of days.

I am ccing Paul Wise, who offered to consider sponsoring the package about a year and a half ago! Paul: Sorry for the delay; this is just to let you know that the package should be ready to be looked at soon. Is the offer still open? (If not, I understand entirely.)

Thomas

[Daniel Langlois]

CCing Paul's reply below. Also, there's a wealth of QA information, minor things like 'no homepage field present', and showstoppers like: 

The /usr/share/doc/pkg/copyright file still contains template markers from a packaging helper.

There's only one other 'showstopper': 

The common licenses (GPL, BSD, Artistic, etc) have been moved from /usr/doc/copyright to /usr/share/common-licenses.

I'm pulling this info from https://mentors.debian.net/package/plover

With my own packaging experiment, one thing I ran across, is that most third-party software installs itself in the /usr/local directory hierarchy, but, On Debian, this is reserved for private use by the system administrator, so packages must not use directories such as /usr/local/bin, but should instead use system directories such as /usr/bin, obeying the Filesystem Hierarchy Standard (FHS). That is, I observed that the source does not seem to follow the FileSystem Hierarchy Standard (FHS). So here, we're maybe contempating making an intrusive change to our build system. 

In general, I cling to the hope that packaging is not as complicated as it may seem!

On Wed, May 8, 2013 at 5:11 AM, Paul Wise <pa...@debian.org> wrote:

On Wed, 2013-05-08 at 12:13 +0100, Thomas Thurman wrote:

> I am ccing Paul Wise, who offered to consider sponsoring the package
> about a year and a half ago! Paul: Sorry for the delay; this is just
> to let you know that the package should be ready to be looked at soon.
> Is the offer still open? (If not, I understand entirely.)

Sure...

I don't have any keyboards that support steno so I would be relying on
you to test the software correctly.

In general I prefer to do package reviews in public, so please send an
RFS mail to the debian-mentors list.

Before you do that, please check the package with lintian and any other
applicable package checkers. Here are some tips for that:

http://wiki.debian.org/HowToPackageForDebian#Check_points_for_any_package

--
bye,
pabs

http://wiki.debian.org/PaulWise

[Thomas Thurman]

I know, I can read that too! This is the result of a first try; I'll be fixing those over the next few days.

--
You received this message because you are subscribed to the Google Groups "Plover" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ploversteno...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Daniel Langlois]

Could you upload the Debian make files/etc. to Github, (such as my attempt: https://github.com/DanLanglois/PloverDebianPackage).

I/others might as easily fix them. The point about changing the build system maybe requires discussion though. 

[Hesky Fisher]

Plover doesn't need to use /usr/local/bin. If it is currently using it then it can easily be changed.