Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
GUI tool for packaging
13 views
Skip to first unread message
Thomas Kluyver
Nov 9, 2012, 8:30:02 AM11/9/12
to
This is an idea I've had knocking around for a while. Packaging is complex - there are lots of different tools and syntaxes you have to understand to do a good job of it - quilt, debhelper, watch files, etc. - along with specialist terminology. I know various CLI tools aim to simplify things, but not everything can be automated, and the tools end up with lots of options to learn about.
The upshot is that most open source developers rely on a relatively small number of specialist packagers to do the rather esoteric work of preparing Debian packages. To get this to scale, I think we need to encourage more upstreams to provide packaging - whether it's for inclusion in Debian, or to provide .deb packages themselves, like Google Chrome, MongoDB and Dropbox do.
In my opinion, the best way to do that is to build a GUI that holds the user's hand through the process of packaging, showing them the options available. It should be particularly useful for occasional packagers who don't want to remember a load of commands and options.
Ultimately, I envisage a kind of packaging IDE. But the bit I picked to implement first is a wizard to create a watch file. The user can select popular sites like Github or PyPI, enter a couple of details, and a (hopefully correct) watchfile is generated.
Screenshot: http://i.imgur.com/YM2LT.png
Code: https://bitbucket.org/takluyver/packagebench/src
I imagine wizards like this could be a large part of this tool, for tasks like "Add a documentation package" or "Make DEP-8 tests".
So, I'd like to know:
- Do you think this is worth spending time on? Bear in mind it's primarily aimed at new & occasional packagers, not experts who already know the tools.
- Is there already anything like this?
- Would you be interested in helping to develop it?
Of course, this should be useful for any packages, not just Python-based ones. But I thought I'd float the idea here first, to get some feedback before I take it any further.
Thanks,
Thomas
The upshot is that most open source developers rely on a relatively small number of specialist packagers to do the rather esoteric work of preparing Debian packages. To get this to scale, I think we need to encourage more upstreams to provide packaging - whether it's for inclusion in Debian, or to provide .deb packages themselves, like Google Chrome, MongoDB and Dropbox do.
In my opinion, the best way to do that is to build a GUI that holds the user's hand through the process of packaging, showing them the options available. It should be particularly useful for occasional packagers who don't want to remember a load of commands and options.
Ultimately, I envisage a kind of packaging IDE. But the bit I picked to implement first is a wizard to create a watch file. The user can select popular sites like Github or PyPI, enter a couple of details, and a (hopefully correct) watchfile is generated.
Screenshot: http://i.imgur.com/YM2LT.png
Code: https://bitbucket.org/takluyver/packagebench/src
I imagine wizards like this could be a large part of this tool, for tasks like "Add a documentation package" or "Make DEP-8 tests".
So, I'd like to know:
- Do you think this is worth spending time on? Bear in mind it's primarily aimed at new & occasional packagers, not experts who already know the tools.
- Is there already anything like this?
- Would you be interested in helping to develop it?
Of course, this should be useful for any packages, not just Python-based ones. But I thought I'd float the idea here first, to get some feedback before I take it any further.
Thanks,
Thomas
Omer Zak
Nov 9, 2012, 9:50:01 AM11/9/12
to
On Fri, 2012-11-09 at 13:19 +0000, Thomas Kluyver wrote:
> In my opinion, the best way to do that is to build a GUI that holds
> the user's hand through the process of packaging, showing them the
> options available. It should be particularly useful for occasional
> packagers who don't want to remember a load of commands and options.
[... snipped ...]
> In my opinion, the best way to do that is to build a GUI that holds
> the user's hand through the process of packaging, showing them the
> options available. It should be particularly useful for occasional
> packagers who don't want to remember a load of commands and options.
> So, I'd like to know:
> - Do you think this is worth spending time on? Bear in mind it's
> primarily aimed at new & occasional packagers, not experts who already
> know the tools.
> - Is there already anything like this?
> - Would you be interested in helping to develop it?
>
> Of course, this should be useful for any packages, not just
> Python-based ones. But I thought I'd float the idea here first, to get
> some feedback before I take it any further.
There is a general problem of discovering rarely-used commands and/or
options and using them to do something one does not need to do
frequently. Existing documentation and help information are geared
toward novices who will become experts, who make frequent use of the
software. Such people eventually remember all the options of the
software.
What is missing is help for experts, who learned the software sometime
ago but use it rarely and therefore with time they forget the minutiae
and the details of using the software's options.
--- Omer
--
Your liberty to swing your fist ends just where my nose begins.
Your freedom of expression ends where my freedom of expression begins.
Your freedom of religion ends where my rights for equality and
accessibility begin.
My own blog is at http://www.zak.co.il/tddpirate/
My opinions, as expressed in this E-mail message, are mine alone.
They do not represent the official policy of any organization with which
I may be affiliated in any way.
WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html
--
To UNSUBSCRIBE, email to debian-pyt...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/1352468465.31981.74.camel@c4
Jakub Wilk
Nov 14, 2012, 6:50:02 AM11/14/12
to
* Thomas Kluyver <tho...@kluyver.me.uk>, 2012-11-09, 13:19:
--
Jakub Wilk
--
To UNSUBSCRIBE, email to debian-pyt...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/2012111411...@jwilk.net
>- Do you think this is worth spending time on?
I don't think so, sorry.
--
Jakub Wilk
--
To UNSUBSCRIBE, email to debian-pyt...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Thomas Kluyver
Nov 14, 2012, 6:50:02 AM11/14/12
to
Could you expand on this at all? Do you think that packaging should be left to the experts? Or that the existing systems are easy enough for newcomers to learn?
Thomas
Thomas
Barry Warsaw
Nov 14, 2012, 8:20:03 AM11/14/12
to
a look at pkgme as the back-end basis for such a tool.
https://launchpad.net/pkgme
Cheers,
-Barry
--
To UNSUBSCRIBE, email to debian-pyt...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Clint Byrum
Nov 14, 2012, 11:00:01 PM11/14/12
to
Excerpts from Thomas Kluyver's message of 2012-11-09 05:19:03 -0800:
pkgme was created.
The idea was to generate *full packages* based only on the usual metadata
provided to most build systems or language-specific packaging systems.
https://launchpad.net/pkgme
At one point I was interested in writing a ruby backend for this, but
got distracted and moved to other focus, but I think it solves what you
are talking about, without need to develop a large project like a GUI.
--
To UNSUBSCRIBE, email to debian-pyt...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/135295143...@fewbar.com
> This is an idea I've had knocking around for a while. Packaging is complex
> - there are lots of different tools and syntaxes you have to understand to
> do a good job of it - quilt, debhelper, watch files, etc. - along with
> specialist terminology. I know various CLI tools aim to simplify things,
> but not everything can be automated, and the tools end up with lots of
> options to learn about.
>
> The upshot is that most open source developers rely on a relatively small
> number of specialist packagers to do the rather esoteric work of preparing
> Debian packages. To get this to scale, I think we need to encourage more
> upstreams to provide packaging - whether it's for inclusion in Debian, or
> to provide .deb packages themselves, like Google Chrome, MongoDB and
> Dropbox do.
As part of the effort to encourage development of Applications for Ubuntu,
> - there are lots of different tools and syntaxes you have to understand to
> do a good job of it - quilt, debhelper, watch files, etc. - along with
> specialist terminology. I know various CLI tools aim to simplify things,
> but not everything can be automated, and the tools end up with lots of
> options to learn about.
>
> The upshot is that most open source developers rely on a relatively small
> number of specialist packagers to do the rather esoteric work of preparing
> Debian packages. To get this to scale, I think we need to encourage more
> upstreams to provide packaging - whether it's for inclusion in Debian, or
> to provide .deb packages themselves, like Google Chrome, MongoDB and
> Dropbox do.
pkgme was created.
The idea was to generate *full packages* based only on the usual metadata
provided to most build systems or language-specific packaging systems.
https://launchpad.net/pkgme
At one point I was interested in writing a ruby backend for this, but
got distracted and moved to other focus, but I think it solves what you
are talking about, without need to develop a large project like a GUI.
--
To UNSUBSCRIBE, email to debian-pyt...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Thomas Kluyver
Nov 15, 2012, 6:20:02 AM11/15/12
to
Hi Clint,
On 15 November 2012 03:53, Clint Byrum <cl...@ubuntu.com> wrote:
https://launchpad.net/pkgme
At one point I was interested in writing a ruby backend for this, but
got distracted and moved to other focus, but I think it solves what you
are talking about, without need to develop a large project like a GUI.
I have been keeping an eye on pkgme, but I'm not sure it solves the problem. My concern with automated tools is that they tend to to work for about 75% of stuff, but there's always a substantial proportion of things that just do something a little bit odd, and the automatic tool can't handle them. So I have to understand what the automation is doing, to deal with the things it can't do for me.
To give more concrete examples, we have stdeb for Python packages, and debhelper, which can guess much of the standard process of building a package (such as running 'make' or 'setup.py'). But neither can handle enough real-world cases that new packagers can use it without thinking about what's happening.
I tried writing an application with Quickly a couple of months ago, and I was impressed with how well 'quickly package' (which uses pkgme) worked. But a lot of things that I'd like to see packaged aren't developed with Quickly, and I'm not confident that pkgme can do such a good job with them.
Thanks,
Thomas
Andrey Rahmatullin
Nov 15, 2012, 10:20:02 AM11/15/12
to
On Thu, Nov 15, 2012 at 11:17:40AM +0000, Thomas Kluyver wrote:
> I have been keeping an eye on pkgme, but I'm not sure it solves the
> problem. My concern with automated tools is that they tend to to work for
> about 75% of stuff, but there's always a substantial proportion of things
> that just do something a little bit odd, and the automatic tool can't
> handle them.
Of course. That's still applicable to your collection of GUI wizards.
> I have been keeping an eye on pkgme, but I'm not sure it solves the
> problem. My concern with automated tools is that they tend to to work for
> about 75% of stuff, but there's always a substantial proportion of things
> that just do something a little bit odd, and the automatic tool can't
> handle them.
> But neither can handle enough real-world cases that new packagers can
> use it without thinking about what's happening.
It's good to have tools that do the right thing without you intervening
(like debhelper). It's good to have tools that generate some
configs/instructions based on analysis of sources and/or user input
(needed when the first kind of tools doesn't work). In most cases
packaging can be done with a combination of those and probably some small
manual work, but I don't see how some big non-flexible GUI app can help
users get a working package without doing manual checks and fixes.
Bottom line: if you want to get a good package, it's not always possible
to fully automate that, especially in cases of complex (and proprietary)
software like that you mentioned, and so a GUI wizard can't do everything
needed.
--
WBR, wRAR
Thomas Kluyver
Nov 15, 2012, 11:20:02 AM11/15/12
to
On 15 November 2012 15:18, Andrey Rahmatullin <wr...@wrar.name> wrote:
Bottom line: if you want to get a good package, it's not always possible
to fully automate that, especially in cases of complex (and proprietary)
software like that you mentioned, and so a GUI wizard can't do everything
needed.
I absolutely agree, but as I see it, I'm not trying to automate the process. I'm trying to make it easier for the user to manually intervene in the packaging. The wizards are just one part of that.
Packaging at the moment involves an array of different data files, specialist syntax, and command line tools. There's a lot of detail which is hard to learn and remember. For example:
- uscan puts upstream tarballs into ../, but svn-buildpackage expects them in ../tarballs/
- dh_make is something for the user to run, whereas dh_install is something to be run automatically by the build system.
- When you make a new patch, you have to 'quilt add' any files you want to modify before you can modify them. It's easy, and annoying, to forget that. (I know about 'quilt edit', but for other reasons my $EDITOR is set to an in-terminal editor, which isn't what I prefer for modifying large code files)
- To work on patches, you have the debian/ directory in amongst the upstream codebase. But having the rest of the codebase there clutters up information from the version control system. So I often end up with two copies of the packaging, and copying debian/patches/ between them.
- You call pbuilder with a .dsc file, but dput with a .changes file.
- In the <pkg>.install list, a line with a single entry works quite differently from a line with two entries
To be clear, I'm not asking for solutions or workarounds to these individual issues. I'm sure there are good reasons things work the way they do - but being relatively new to this, the array of not-quite-connected tools is daunting.
I think the advantage of a GUI is that it presents your options, rather than requiring you to already know them. If you realise you need to change some of upstream's code to make the package work... hey, that pane says 'patches', that sounds like the thing. If there are some examples, you can see the binary package target has an entry called 'examples'.
Specifically, some of the things I have in mind:
- Integration with quilt for patches. If you try to edit a source file from the GUI, you're prompted to create a patch.
- A joined-up view of the intended binary packages. At present, the details of a binary package are spread out over part of the control file, and the foo.install, .docs, .examples, etc. files.
- Editing copyright info without having to get the file syntax just right, look up license short names, and so on. Just enter a glob and select the license from a dropdown.
- Wizards for specific things like watch files or DEP-8 tests. The key to this is that GUI wizards have a much lower barrier of entry than new command line tools - no flags to remember.
- A build menu where you can select source package, local build, pbuilder or PPA, and it will take care of any preparation needed for that.
Whew, that's more than I meant to write. Thanks to anyone who's read this far - I hope I've managed to give a clearer picture of what my idea involves.
Thanks,
Thomas
Packaging at the moment involves an array of different data files, specialist syntax, and command line tools. There's a lot of detail which is hard to learn and remember. For example:
- uscan puts upstream tarballs into ../, but svn-buildpackage expects them in ../tarballs/
- dh_make is something for the user to run, whereas dh_install is something to be run automatically by the build system.
- When you make a new patch, you have to 'quilt add' any files you want to modify before you can modify them. It's easy, and annoying, to forget that. (I know about 'quilt edit', but for other reasons my $EDITOR is set to an in-terminal editor, which isn't what I prefer for modifying large code files)
- To work on patches, you have the debian/ directory in amongst the upstream codebase. But having the rest of the codebase there clutters up information from the version control system. So I often end up with two copies of the packaging, and copying debian/patches/ between them.
- You call pbuilder with a .dsc file, but dput with a .changes file.
- In the <pkg>.install list, a line with a single entry works quite differently from a line with two entries
To be clear, I'm not asking for solutions or workarounds to these individual issues. I'm sure there are good reasons things work the way they do - but being relatively new to this, the array of not-quite-connected tools is daunting.
I think the advantage of a GUI is that it presents your options, rather than requiring you to already know them. If you realise you need to change some of upstream's code to make the package work... hey, that pane says 'patches', that sounds like the thing. If there are some examples, you can see the binary package target has an entry called 'examples'.
Specifically, some of the things I have in mind:
- Integration with quilt for patches. If you try to edit a source file from the GUI, you're prompted to create a patch.
- A joined-up view of the intended binary packages. At present, the details of a binary package are spread out over part of the control file, and the foo.install, .docs, .examples, etc. files.
- Editing copyright info without having to get the file syntax just right, look up license short names, and so on. Just enter a glob and select the license from a dropdown.
- Wizards for specific things like watch files or DEP-8 tests. The key to this is that GUI wizards have a much lower barrier of entry than new command line tools - no flags to remember.
- A build menu where you can select source package, local build, pbuilder or PPA, and it will take care of any preparation needed for that.
Whew, that's more than I meant to write. Thanks to anyone who's read this far - I hope I've managed to give a clearer picture of what my idea involves.
Thanks,
Thomas
Barry Warsaw
Nov 15, 2012, 3:20:01 PM11/15/12
to
On Nov 15, 2012, at 04:15 PM, Thomas Kluyver wrote:
>- uscan puts upstream tarballs into ../, but svn-buildpackage expects them
>in ../tarballs/
>- uscan puts upstream tarballs into ../, but svn-buildpackage expects them
>in ../tarballs/
>- To work on patches, you have the debian/ directory in amongst the
>upstream codebase. But having the rest of the codebase there clutters up
>information from the version control system. So I often end up with two
>copies of the packaging, and copying debian/patches/ between them.
Ubuntu Distributed Development (UDD) fixes both these real problems. Ignore
>upstream codebase. But having the rest of the codebase there clutters up
>information from the version control system. So I often end up with two
>copies of the packaging, and copying debian/patches/ between them.
the choice of dvcs for the moment, the fact that you can branch a source
package that gives you upstream + debian, with quilt applied, is a huge win.
Once you've branched, you just start hacking. Then bzr-builddeb does all the
magic of creating the source and/or binary packages. No mucking about with
orig.tar.gz locations, uscan, mergeWithUpstream, etc, etc.
IMHO, UDD makes developing for Ubuntu much easier and faster than developing
for Debian, and a downright joy (usually, at least these days :).
In fact, Launchpad imports Debian source packages too, so even using UDD for
Debian development (modulo updating the svn branches) much easier too.
I think it would be very cool if Debian were to adopt the UDD workflow. I
know there are problems, controversies, and detractors, but I find it so much
more pleasant to develop with.
>- In the <pkg>.install list, a line with a single entry works quite
>differently from a line with two entries
>- When you make a new patch, you have to 'quilt add' any files you want to
>modify before you can modify them. It's easy, and annoying, to forget that.
>(I know about 'quilt edit', but for other reasons my $EDITOR is set to an
>in-terminal editor, which isn't what I prefer for modifying large code
>files)
the file, adding it after the fact doesn't help. So you better have those
files under vcs so you can undo your change, quilt add the file, and then redo
the change. This is another way that UDD helps quite a bit.
>- You call pbuilder with a .dsc file, but dput with a .changes file.
>To be clear, I'm not asking for solutions or workarounds to these
>individual issues. I'm sure there are good reasons things work the way they
>do - but being relatively new to this, the array of not-quite-connected
>tools is daunting.
crufty workarounds and ignore the pain. OTOH, none of these are really
solvable by what *I* think of when I hear "graphical packaging tools". Maybe
I'm just thinking about it too narrowly. I'm also not much of a GUI person,
so I do like the command-line-iness of the UDD workflow. But maybe I'd fire
up the GUI once in a while to inspect a package I'm not familiar with.
Cheers,
-Barry
--
To UNSUBSCRIBE, email to debian-pyt...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
0 new messages