Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Backport of Python 3.6 for Debian Stretch?
1,285 views
Skip to first unread message
Nguyễn Hồng Quân
Apr 24, 2018, 3:00:02 AM4/24/18
to
I'm using Debian 9 on an ARM board (BeagleBone), for our IoT project.
We
write an application which needs Python 3.6. But we are struggling with
packaging Python 3.6 as deb packages for Debian 9. We found the dsc
file for Debian buster. But there is difference in dependency hierarchy
between Stretch and Buster, so the found script produces *.deb files
which are not installable (due to missing dependencies).
Could you please provide an official packaging script for Python 3.6 and Debian 9?
Thank you.
--
W. Martin Borgert
Apr 24, 2018, 4:00:02 AM4/24/18
to
Could you elaborate, why you need Python 3.6 instead of 3.5?
Maybe there is a way to make your application work with 3.5?
E.g. by backporting specific modules?
Maybe there is a way to make your application work with 3.5?
E.g. by backporting specific modules?
Nguyễn Hồng Quân
Apr 24, 2018, 4:00:02 AM4/24/18
to
Python 3.6 has much better performance than Python 3.5, especially on embedded computer (I tried and compared).
I also like new Python 3.6 features: f-string, improved type hint (Python 3.5 has it, but Python 3.6 improved).
On Tue, Apr 24, 2018 at 2:44 PM, W. Martin Borgert <deb...@debian.org> wrote:
Could you elaborate, why you need Python 3.6 instead of 3.5?
Maybe there is a way to make your application work with 3.5?
E.g. by backporting specific modules?
Andrey Rahmatullin
Apr 24, 2018, 4:10:02 AM4/24/18
to
On Tue, Apr 24, 2018 at 01:58:23PM +0700, Nguyễn Hồng Quân wrote:
> Could you please provide an official packaging script for Python 3.6 and
> Debian 9?
It's not enough to backport the interpreter, all binary modules will need
> Could you please provide an official packaging script for Python 3.6 and
> Debian 9?
to be recompiled too.
--
WBR, wRAR
W. Martin Borgert
Apr 24, 2018, 4:20:01 AM4/24/18
to
Quoting Nguyễn Hồng Quân <ng.hon...@gmail.com>:
> Python 3.6 has much better performance than Python 3.5, especially on
> embedded computer (I tried and compared).
This is interesting! Did you also compare Python 2.7 with 3.5/3.6?
> embedded computer (I tried and compared).
I'm looking for more reasons to finally port my embedded software :~)
Nguyễn Hồng Quân
Apr 24, 2018, 6:30:03 AM4/24/18
to
On Tue, Apr 24, 2018 at 3:05 PM, Andrey Rahmatullin <wr...@debian.org> wrote:
It's not enough to backport the interpreter, all binary modules will need
to be recompiled too.
Why?
Just leave Python3.6 exist in parallel to Python3.5. Don't touch Python3.5 package folder. Don't convert python3-xxx to be usable with Python3.6.
Nguyễn Hồng Quân
Apr 24, 2018, 6:30:03 AM4/24/18
to
On Tue, Apr 24, 2018 at 3:12 PM, W. Martin Borgert <deb...@debian.org> wrote:
This is interesting! Did you also compare Python 2.7 with 3.5/3.6?
No, I'm not interested in Python 2. Every my project is started with Python 3.
Andrey Rahmatullin
Apr 24, 2018, 6:50:01 AM4/24/18
to
On Tue, Apr 24, 2018 at 05:27:08PM +0700, Nguyễn Hồng Quân wrote:
> > It's not enough to backport the interpreter, all binary modules will need
> > to be recompiled too.
> >
>
> Why?
To be able to be imported in Python3.6.
> > It's not enough to backport the interpreter, all binary modules will need
> > to be recompiled too.
> >
>
> Why?
--
WBR, wRAR
Nguyễn Hồng Quân
Apr 24, 2018, 9:30:02 AM4/24/18
to
Sorry? As I told, just leave the python3-xxx packages to be used with python3.5 (default Python3 of Debian 9).
I don't need those to be used with Python3.6.
For whatever I need to use with Python3.6, I just install via "pip3.6".
For Python3.6, you just need to provide python3.6 (+ its standard lib), python3.6-dev.
Ludovic Gasc
Apr 24, 2018, 10:50:02 AM4/24/18
to
Hi,
You can use pythonz, conda or pyenv to install the version of Python you want without to disturb your system.
Regards.
--
Ludovic Gasc (GMLudo)
Nguyễn Hồng Quân
Apr 24, 2018, 11:10:02 AM4/24/18
to
On Tue, Apr 24, 2018 at 9:45 PM, Ludovic Gasc <gml...@gmail.com> wrote:
Hi,You can use pythonz, conda or pyenv to install the version of Python you want without to disturb your system.
I don't need to "not disturb" system.
If have to use conda, pyenv, I would rather build Python3.6 from source tarball, not to bring more overhead (conda body, pyenv body), and "Python3.6 from source" still not disturb my system, because it is installed to "/usr/local".
But I don't want any method that requires to build Python from source (tarball, pythonz, conda or alike), because I really need *pre-built binaries*.
As I told in the first email, I use Python in an embedded board (BeagleBone Black), which is very limited in term of disk space and CPU.
Building Python is a very heavy job, taking nights to done, and I also want to save flash space from storing Python source.
So, I try to build Python as deb and use apt to install to my board. I was successful to do so with Python 3.6.1, using the .dsc file found on Debian buster repo. But since Python 3.6.4, the build script has changed drastically that it cannot be use to build for Debian stretch.
Btw, what I need is just a build/packaging script. I don't need you to build and publish Python3.6 to official repository.
Andrey Rahmatullin
Apr 24, 2018, 11:30:03 AM4/24/18
to
On Tue, Apr 24, 2018 at 10:07:03PM +0700, Nguyễn Hồng Quân wrote:
> So, I try to build Python as deb and use apt to install to my board. I was
> successful to do so with Python 3.6.1, using the .dsc file found on Debian
> buster repo. But since Python 3.6.4, the build script has changed
> drastically that it cannot be use to build for Debian stretch.
>
> Btw, what I need is just a build/packaging script. I don't need you to
> build and publish Python3.6 to official repository.
Adapt it from the 3.6.1 one?
> So, I try to build Python as deb and use apt to install to my board. I was
> successful to do so with Python 3.6.1, using the .dsc file found on Debian
> buster repo. But since Python 3.6.4, the build script has changed
> drastically that it cannot be use to build for Debian stretch.
>
> Btw, what I need is just a build/packaging script. I don't need you to
> build and publish Python3.6 to official repository.
--
WBR, wRAR
Jeremy Stanley
Apr 24, 2018, 12:00:03 PM4/24/18
to
On 2018-04-24 22:07:03 +0700 (+0700), Nguyễn Hồng Quân wrote:
[...]
Unless I'm missing something, there's no substantial difference
between building a package of Python3.6 and copying it to the
system, or performing a `make altinstall` and copying the resulting
files (via rsync, tar and scp, whatever) to the target system. If
you're okay with the idea of building packages remotely, then why
not build from source remotely?
--
Jeremy Stanley
[...]
> I don't need to "not disturb" system.
> If have to use conda, pyenv, I would rather build Python3.6 from source
> tarball, not to bring more overhead (conda body, pyenv body), and
> "Python3.6 from source" still not disturb my system, because it is
> installed to "/usr/local".
>
> But I don't want any method that requires to build Python from source
> (tarball, pythonz, conda or alike), because I really need *pre-built
> binaries*.
[...]
> If have to use conda, pyenv, I would rather build Python3.6 from source
> tarball, not to bring more overhead (conda body, pyenv body), and
> "Python3.6 from source" still not disturb my system, because it is
> installed to "/usr/local".
>
> But I don't want any method that requires to build Python from source
> (tarball, pythonz, conda or alike), because I really need *pre-built
> binaries*.
Unless I'm missing something, there's no substantial difference
between building a package of Python3.6 and copying it to the
system, or performing a `make altinstall` and copying the resulting
files (via rsync, tar and scp, whatever) to the target system. If
you're okay with the idea of building packages remotely, then why
not build from source remotely?
--
Jeremy Stanley
Nguyễn Hồng Quân
Apr 24, 2018, 12:50:02 PM4/24/18
to
Then why Debian project invent *.deb file, not just pack binary as tar file and let user to untar it?
I favor building deb file, rather than copying "make altinstall" result, because of the same reason.
Andrey Rahmatullin
Apr 24, 2018, 1:10:02 PM4/24/18
to
On Wed, Apr 25, 2018 at 12:01:45AM +0700, Nguyễn Hồng Quân wrote:
> > Replacing the tarball and changing the version in the changelog, for a
> > start.
> >
>
> As I told: "But there is difference in dependency hierarchy between Stretch
>
> One example:
> From 3.6.4, the build script specifies that python3.6 depends on
> python3-distutils, but python3-distutils doesn't exist on Debian stretch.
> In 3.61, there is no this dependence relationship, so the deb is
> installable on Debian stretch.
>
> "python3-distutils" is just one of problems starting to appear from Python
> 3.6.4 packaging script.
How is this related to my suggestion?
--
WBR, wRAR
> On Tue, Apr 24, 2018 at 11:45 PM, Andrey Rahmatullin <wr...@wrar.name> wrote:
>
> > > > Adapt it from the 3.6.1 one?
> > > >
> > >
> > > Excuse me. What do you mean by "adapt"?
>
> > > > Adapt it from the 3.6.1 one?
> > > >
> > >
> > Replacing the tarball and changing the version in the changelog, for a
> > start.
> >
>
> As I told: "But there is difference in dependency hierarchy between Stretch
> and Buster, so the found script produces *.deb files which are not
> installable (due to missing dependencies)."
>
> Changing changelog file is not enough to make it works.
> installable (due to missing dependencies)."
>
>
> One example:
> From 3.6.4, the build script specifies that python3.6 depends on
> python3-distutils, but python3-distutils doesn't exist on Debian stretch.
> In 3.61, there is no this dependence relationship, so the deb is
> installable on Debian stretch.
>
> "python3-distutils" is just one of problems starting to appear from Python
> 3.6.4 packaging script.
How is this related to my suggestion?
--
WBR, wRAR
Nguyễn Hồng Quân
Apr 24, 2018, 1:10:02 PM4/24/18
to
On Tue, Apr 24, 2018 at 11:45 PM, Andrey Rahmatullin <wr...@wrar.name> wrote:
> > Adapt it from the 3.6.1 one?
> >
>
> Excuse me. What do you mean by "adapt"?
Replacing the tarball and changing the version in the changelog, for a
start.
As I told: "But there is difference in dependency hierarchy between Stretch and Buster, so the found script produces *.deb files which are not installable (due to missing dependencies)."
Changing changelog file is not enough to make it works.
One example:
From 3.6.4, the build script specifies that python3.6 depends on python3-distutils, but python3-distutils doesn't exist on Debian stretch.
In 3.61, there is no this dependence relationship, so the deb is installable on Debian stretch.
"python3-distutils" is just one of problems starting to appear from Python 3.6.4 packaging script.
--
Quân
Nguyễn Hồng Quân
Apr 24, 2018, 1:20:02 PM4/24/18
to
On Wed, Apr 25, 2018 at 12:05 AM, Andrey Rahmatullin <wr...@debian.org> wrote:
How is this related to my suggestion?
Your suggestion is:
"
"
Adapt it from the 3.6.1 one. Replacing the tarball and changing the version in the changelog"And I'm proving that it is not enough.
There are so many change between Debian's Python 3.6.1 & 3.6.4 packaging script that I cannot do it myself and I have to request here. I don't understand the packaging as well as the maintainers here.
--
Quân
Andrey Rahmatullin
Apr 24, 2018, 1:30:02 PM4/24/18
to
On Wed, Apr 25, 2018 at 12:11:03AM +0700, Nguyễn Hồng Quân wrote:
> On Wed, Apr 25, 2018 at 12:05 AM, Andrey Rahmatullin <wr...@debian.org>
> wrote:
>
> >
> > How is this related to my suggestion?
> >
>
>
> Your suggestion is:
>
> "
>
> Adapt it from the 3.6.1 one. Replacing the tarball and changing the
> version in the changelog"
>
> And I'm proving that it is not enough.
No, you are talking about the 3.6.4 packaging files while I'm suggesting
> On Wed, Apr 25, 2018 at 12:05 AM, Andrey Rahmatullin <wr...@debian.org>
> wrote:
>
> >
> > How is this related to my suggestion?
> >
>
>
> Your suggestion is:
>
> "
>
> Adapt it from the 3.6.1 one. Replacing the tarball and changing the
> version in the changelog"
>
> And I'm proving that it is not enough.
to not use them.
--
WBR, wRAR
Jeremy Stanley
Apr 24, 2018, 2:00:02 PM4/24/18
to
On 2018-04-24 23:42:30 +0700 (+0700), Nguyễn Hồng Quân wrote:
[...]
[...]
> Then why Debian project invent *.deb file, not just pack binary as
> tar file and let user to untar it? I favor building deb file,
> rather than copying "make altinstall" result, because of the same
> reason.
I completely understand the reason for using packages, but Debian is
> tar file and let user to untar it? I favor building deb file,
> rather than copying "make altinstall" result, because of the same
> reason.
a volunteer project which does not exist solely to solve your
problems so you can either do something you already know how to do
(build Python 3.6 from source) and move on, or learn to make the
packages you want (including fixing any backporting issues you find
when doing so). Asking others to tell you how to do that is not the
sort of self-directed research expected of participants in a
volunteer project, it's at best begging and at worst disrespectful
of those who have invested the time to learn the things you're not
willing.
https://backports.debian.org/Contribute/
--
Jeremy Stanley
Matthew Woodcraft
Apr 24, 2018, 4:20:01 PM4/24/18
to
Nguyễn Hồng Quân wrote:
> I'm using Debian 9 on an ARM board (BeagleBone), for our IoT project.
> We write an application which needs Python 3.6. But we are struggling with
> packaging Python 3.6 as deb packages for Debian 9. We found the dsc file
> for Debian buster. But there is difference in dependency hierarchy between
> Stretch and Buster, so the found script produces *.deb files which are not
> installable (due to missing dependencies).
You didn't say what dependency problem you're seeing. Is it python3.6
> I'm using Debian 9 on an ARM board (BeagleBone), for our IoT project.
> We write an application which needs Python 3.6. But we are struggling with
> packaging Python 3.6 as deb packages for Debian 9. We found the dsc file
> for Debian buster. But there is difference in dependency hierarchy between
> Stretch and Buster, so the found script produces *.deb files which are not
> installable (due to missing dependencies).
depending on python3-distutils?
If it is, you might try building the 3.6.5-3 source package from sid
rather than the 3.6.5~rc1-1 package from buster, as that dependency has
been removed there.
That should give you a working Python 3.6, but without distutils which
has recently been moved to the python3-stdlib-extensions package. If you
need distutils you might try backporting that too.
-M-
Ludovic Gasc
Apr 24, 2018, 5:00:02 PM4/24/18
to
Upgrade the python system from 3.5 to 3.6 might breaks some python system libraries, no ?
Matthew Woodcraft
Apr 24, 2018, 5:30:02 PM4/24/18
to
should install in parallel and cause no trouble (it would be like installing
the python3.7 packages in Buster today, which would go in parallel to 3.6).
I think rebuilding python3-stdlib-extensions would be OK too if done with
enough care; it would need persuading to build packages including libraries
for both 3.5 and 3.6. But yes, that's more risky as it means replacing Stretch
packages rather than just adding new ones.
-M-
Brian May
Apr 24, 2018, 8:50:02 PM4/24/18
to
Nguyễn Hồng Quân <ng.hon...@gmail.com> writes:
> Sorry? As I told, just leave the python3-xxx packages to be used with
> python3.5 (default Python3 of Debian 9).
> I don't need those to be used with Python3.6.
> For whatever I need to use with Python3.6, I just install via "pip3.6".
> For Python3.6, you just need to provide python3.6 (+ its standard lib),
> python3.6-dev.
If using pip to install packages is fine (which can result in compiling
> Sorry? As I told, just leave the python3-xxx packages to be used with
> python3.5 (default Python3 of Debian 9).
> I don't need those to be used with Python3.6.
> For whatever I need to use with Python3.6, I just install via "pip3.6".
> For Python3.6, you just need to provide python3.6 (+ its standard lib),
> python3.6-dev.
from source although increasing use of wheels helps), then I personally
fail to see the problem with using something like pyenv.
--
Brian May <b...@debian.org>
Nguyễn Hồng Quân
Apr 24, 2018, 11:10:02 PM4/24/18
to
On Wed, Apr 25, 2018 at 12:21 AM, Andrey Rahmatullin <wr...@debian.org> wrote:
No, you are talking about the 3.6.4 packaging files while I'm suggesting
to not use them.
Do you mean to use "
--
python3.6_3.6.1-1.debian.tar.xz" (of 3.6.1) and change its number to "python3.6_3.6.4-4.debian.tar.xz" (of 3.6.4) and change its changelog content?
Unfortunately, the .debian.tar.xz for 3.6.1 already disappear from Debian repository!
Today, even the 3.6.4 disappears, only 3.6.5 stay and the dependencies graph of 3.6.5 is the same as 3.6.4.
Unfortunately, the .debian.tar.xz for 3.6.1 already disappear from Debian repository!
Today, even the 3.6.4 disappears, only 3.6.5 stay and the dependencies graph of 3.6.5 is the same as 3.6.4.
--
Quân
Nguyễn Hồng Quân
Apr 24, 2018, 11:10:02 PM4/24/18
to
On Wed, Apr 25, 2018 at 12:58 AM, Jeremy Stanley <fu...@yuggoth.org> wrote:
On 2018-04-24 23:42:30 +0700 (+0700), Nguyễn Hồng Quân wrote:
[...]
I completely understand the reason for using packages, but Debian is
a volunteer project which does not exist solely to solve your
problems so you can either do something you already know how to do
(build Python 3.6 from source) and move on, or learn to make the
packages you want (including fixing any backporting issues you find
when doing so). Asking others to tell you how to do that is not the
sort of self-directed research expected of participants in a
volunteer project, it's at best begging and at worst disrespectful
of those who have invested the time to learn the things you're not
willing.
https://backports.debian.org/Contribute/
Sorry if you feel disrespectful. But don't you think that this issue is not mine-only, it may be other users as well?
I spent much time to research on it, so that I can tell what difference between 3.6.1 and 3.6.4 packaging.
My voice here may let the packager think again why they make package scripts so different between minor versions of Python. Don't you think that it is good?
--
Quân
Nguyễn Hồng Quân
Apr 24, 2018, 11:20:01 PM4/24/18
to
I use pip to install from pre-built wheel. I don't let pip to compile the source on my embedded computer, because it also takes much time.
So in summary, I tried to avoid doing compilation job directly on my embedded computer, either Python interpreter or Python C libraries.
(Where I get wheel? I rent a bare-metal ARM server from Scaleway to build the wheel there).
--
Quân
Nguyễn Hồng Quân
Apr 24, 2018, 11:20:01 PM4/24/18
to
On Wed, Apr 25, 2018 at 3:16 AM, Matthew Woodcraft <mat...@woodcraft.me.uk> wrote:
Nguyễn Hồng Quân wrote:
You didn't say what dependency problem you're seeing. Is it python3.6
depending on python3-distutils?
Yes, and when I tried to backport python3-disutils, it depends on python 3.7 !!!
--
Quân
Andrey Rahmatullin
Apr 25, 2018, 2:00:03 AM4/25/18
to
On Wed, Apr 25, 2018 at 10:00:51AM +0700, Nguyễn Hồng Quân wrote:
> Do you mean to use "
> python3.6_3.6.1-1.debian.tar.xz" (of 3.6.1) and change its number to
> "python3.6_3.6.4-4.debian.tar.xz" (of 3.6.4) and change its changelog
> content?
No, I meant to use the debian/ directory from that tarball, otherwise the
> Do you mean to use "
> python3.6_3.6.1-1.debian.tar.xz" (of 3.6.1) and change its number to
> "python3.6_3.6.4-4.debian.tar.xz" (of 3.6.4) and change its changelog
> content?
idea is more or less correct.
> Unfortunately, the .debian.tar.xz for 3.6.1 already disappear from Debian
> repository!
don't anymore, there is always http://snapshot.debian.org/
--
WBR, wRAR
Jeremy Stanley
Apr 25, 2018, 10:20:02 AM4/25/18
to
On 2018-04-25 10:06:47 +0700 (+0700), Nguyễn Hồng Quân wrote:
[...]
[...]
> I spent much time to research on it, so that I can tell what
> difference between 3.6.1 and 3.6.4 packaging.
[...]
> difference between 3.6.1 and 3.6.4 packaging.
http://metadata.ftp-master.debian.org/changelogs/main/p/python3.6/python3.6_3.6.5-3_changelog
https://manpages.debian.org/debdiff
http://snapshot.debian.org/package/python3.6/
[also, please don't Cc me, I do already read the mailing list]
--
Jeremy Stanley
Nguyễn Hồng Quân
Apr 25, 2018, 1:20:02 PM4/25/18
to
Thank you, Andrey and Jeremy.
I didn't know these sites. I will try again.
On Wed, Apr 25, 2018 at 9:09 PM, Jeremy Stanley <fu...@yuggoth.org> wrote:
On 2018-04-25 10:06:47 +0700 (+0700), Nguyễn Hồng Quân wrote:
[...]
> I spent much time to research on it, so that I can tell what
> difference between 3.6.1 and 3.6.4 packaging.
[...]
http://metadata.ftp-master.debian.org/changelogs/main/p/python3.6/python3.6_3.6.5-3_changelog
https://manpages.debian.org/debdiff
http://snapshot.debian.org/package/python3.6/
--
Quân
0 new messages