Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
A message from CMake upstream: announcing dh-cmake
149 views
Skip to first unread message
Kyle Edwards
Jul 3, 2018, 4:00:03 PM7/3/18
to
Hello everyone!
My name is Kyle. I work at Kitware, Inc., the upstream maintainer of
the CMake buildsystem (https://www.cmake.org/) and VTK, the
Visualization Toolkit (https://www.vtk.org/). As some of you on the
Debian Science list may have heard, we are making an effort to
officially support our flagship product, VTK, on Debian and its
derivatives. To that end, we have created a new set of Debhelper
utilities to meet the unique challenges of packaging a large CMake-
based project like VTK. We have named this project "dh-cmake". It
allows Debhelper to take advantage of some of the more advanced
capabilities of CMake. For example:
* CMake's install() command takes an optional COMPONENT parameter,
which allows you to break the installation up into multiple
"components", such as "Libraries" and "Development". dh-cmake allows
you to assign these components to separate binary packages, to avoid
having to enumerate every file or file glob in the *.install files.
* Projects that are CTest-aware can optionally have the output of
dh_auto_configure, dh_auto_build, and dh_auto_test captured by CTest
and submitted to a CDash server as part of a continuous integration
process. This is very useful for making sure a large software project
builds properly on Debian.
* CPack includes a mechanism to declare dependencies between
installation components, for example, stating that the "Development"
component depends on the "Libraries" component. dh-cmake can
propagate this information into the output packages, so that
libexample-dev will automatically depend on libexample.
You can download the source code at
https://gitlab.kitware.com/debian/dh-cmake, and read more details about
the rationale and how it works. You can also install the binaries from
our own APT repository. Follow the instructions at
https://apt.kitware.com/ to set up the repository, and then install the
"dh-cmake" package.
Our end goal is to get both dh-cmake and VTK into Debian proper, but it
is still in an experimental state, and there is still a lot of work to
be done yet. We would like to get some feedback on dh-cmake, and we
will eventually file a formal ITP and RFS for it as it becomes more
mature. We would also like to see other CMake-based packages follow our
lead and use these utilities. If you have a package that uses CMake, we
encourage you to give dh-cmake a try.
Thank you in advance for the feedback. We are very excited to venture
into Debian development.
Kyle
My name is Kyle. I work at Kitware, Inc., the upstream maintainer of
the CMake buildsystem (https://www.cmake.org/) and VTK, the
Visualization Toolkit (https://www.vtk.org/). As some of you on the
Debian Science list may have heard, we are making an effort to
officially support our flagship product, VTK, on Debian and its
derivatives. To that end, we have created a new set of Debhelper
utilities to meet the unique challenges of packaging a large CMake-
based project like VTK. We have named this project "dh-cmake". It
allows Debhelper to take advantage of some of the more advanced
capabilities of CMake. For example:
* CMake's install() command takes an optional COMPONENT parameter,
which allows you to break the installation up into multiple
"components", such as "Libraries" and "Development". dh-cmake allows
you to assign these components to separate binary packages, to avoid
having to enumerate every file or file glob in the *.install files.
* Projects that are CTest-aware can optionally have the output of
dh_auto_configure, dh_auto_build, and dh_auto_test captured by CTest
and submitted to a CDash server as part of a continuous integration
process. This is very useful for making sure a large software project
builds properly on Debian.
* CPack includes a mechanism to declare dependencies between
installation components, for example, stating that the "Development"
component depends on the "Libraries" component. dh-cmake can
propagate this information into the output packages, so that
libexample-dev will automatically depend on libexample.
You can download the source code at
https://gitlab.kitware.com/debian/dh-cmake, and read more details about
the rationale and how it works. You can also install the binaries from
our own APT repository. Follow the instructions at
https://apt.kitware.com/ to set up the repository, and then install the
"dh-cmake" package.
Our end goal is to get both dh-cmake and VTK into Debian proper, but it
is still in an experimental state, and there is still a lot of work to
be done yet. We would like to get some feedback on dh-cmake, and we
will eventually file a formal ITP and RFS for it as it becomes more
mature. We would also like to see other CMake-based packages follow our
lead and use these utilities. If you have a package that uses CMake, we
encourage you to give dh-cmake a try.
Thank you in advance for the feedback. We are very excited to venture
into Debian development.
Kyle
Paul Wise
Jul 3, 2018, 10:00:02 PM7/3/18
to
On Wed, Jul 4, 2018 at 3:56 AM, Kyle Edwards wrote:
> Our end goal is to get both dh-cmake and VTK into Debian proper, but it
> is still in an experimental state, and there is still a lot of work to
> be done yet. We would like to get some feedback on dh-cmake, and we
> will eventually file a formal ITP and RFS for it as it becomes more
> mature. We would also like to see other CMake-based packages follow our
> lead and use these utilities. If you have a package that uses CMake, we
> encourage you to give dh-cmake a try.
Once it is available in the archive, I'd suggest announcing it via DevNews:
> Our end goal is to get both dh-cmake and VTK into Debian proper, but it
> is still in an experimental state, and there is still a lot of work to
> be done yet. We would like to get some feedback on dh-cmake, and we
> will eventually file a formal ITP and RFS for it as it becomes more
> mature. We would also like to see other CMake-based packages follow our
> lead and use these utilities. If you have a package that uses CMake, we
> encourage you to give dh-cmake a try.
https://wiki.debian.org/DeveloperNews
--
bye,
pabs
https://wiki.debian.org/PaulWise
Andreas Tille
Jul 4, 2018, 5:40:03 AM7/4/18
to
Hi Kyle,
On Tue, Jul 03, 2018 at 03:56:42PM -0400, Kyle Edwards wrote:
> Hello everyone!
>
> My name is Kyle. I work at Kitware, Inc., the upstream maintainer of
> the CMake buildsystem (https://www.cmake.org/) and VTK, the
> Visualization Toolkit (https://www.vtk.org/). As some of you on the
> Debian Science list may have heard, we are making an effort to
> officially support our flagship product, VTK, on Debian and its
> derivatives. To that end, we have created a new set of Debhelper
> utilities to meet the unique challenges of packaging a large CMake-
> based project like VTK. We have named this project "dh-cmake". It
> allows Debhelper to take advantage of some of the more advanced
> capabilities of CMake.
Thanks a lot for this effort. We really appreciate this kind of
support.
> For example:
>
> * CMake's install() command takes an optional COMPONENT parameter,
> which allows you to break the installation up into multiple
> "components", such as "Libraries" and "Development". dh-cmake allows
> you to assign these components to separate binary packages, to avoid
> having to enumerate every file or file glob in the *.install files.
> * Projects that are CTest-aware can optionally have the output of
> dh_auto_configure, dh_auto_build, and dh_auto_test captured by CTest
> and submitted to a CDash server as part of a continuous integration
> process. This is very useful for making sure a large software project
> builds properly on Debian.
> * CPack includes a mechanism to declare dependencies between
> installation components, for example, stating that the "Development"
> component depends on the "Libraries" component. dh-cmake can
> propagate this information into the output packages, so that
> libexample-dev will automatically depend on libexample.
>
> You can download the source code at
> https://gitlab.kitware.com/debian/dh-cmake
I've cloned this and was able to build the package. I think you can
solve the lintian warning
W: dh-cmake source: ancient-python-version-field x-python3-version 3.2
by simply removing
X-Python3-Version: >= 3.2
from d/control. Manpages for the tools in /usr/bin would be nice.
>, and read more details about
> the rationale and how it works. You can also install the binaries from
> our own APT repository. Follow the instructions at
> https://apt.kitware.com/ to set up the repository, and then install the
> "dh-cmake" package.
>
> Our end goal is to get both dh-cmake and VTK into Debian proper, but it
> is still in an experimental state, and there is still a lot of work to
> be done yet. We would like to get some feedback on dh-cmake, and we
> will eventually file a formal ITP and RFS for it as it becomes more
> mature. We would also like to see other CMake-based packages follow our
> lead and use these utilities. If you have a package that uses CMake, we
> encourage you to give dh-cmake a try.
I'll try with the next cmake package I'll touch. If you can give
some examples this would be helpful.
> Thank you in advance for the feedback. We are very excited to venture
> into Debian development.
Thanks a lot to you
Andreas.
--
http://fam-tille.de
On Tue, Jul 03, 2018 at 03:56:42PM -0400, Kyle Edwards wrote:
> Hello everyone!
>
> My name is Kyle. I work at Kitware, Inc., the upstream maintainer of
> the CMake buildsystem (https://www.cmake.org/) and VTK, the
> Visualization Toolkit (https://www.vtk.org/). As some of you on the
> Debian Science list may have heard, we are making an effort to
> officially support our flagship product, VTK, on Debian and its
> derivatives. To that end, we have created a new set of Debhelper
> utilities to meet the unique challenges of packaging a large CMake-
> based project like VTK. We have named this project "dh-cmake". It
> allows Debhelper to take advantage of some of the more advanced
> capabilities of CMake.
support.
> For example:
>
> * CMake's install() command takes an optional COMPONENT parameter,
> which allows you to break the installation up into multiple
> "components", such as "Libraries" and "Development". dh-cmake allows
> you to assign these components to separate binary packages, to avoid
> having to enumerate every file or file glob in the *.install files.
> * Projects that are CTest-aware can optionally have the output of
> dh_auto_configure, dh_auto_build, and dh_auto_test captured by CTest
> and submitted to a CDash server as part of a continuous integration
> process. This is very useful for making sure a large software project
> builds properly on Debian.
> * CPack includes a mechanism to declare dependencies between
> installation components, for example, stating that the "Development"
> component depends on the "Libraries" component. dh-cmake can
> propagate this information into the output packages, so that
> libexample-dev will automatically depend on libexample.
>
> You can download the source code at
> https://gitlab.kitware.com/debian/dh-cmake
solve the lintian warning
W: dh-cmake source: ancient-python-version-field x-python3-version 3.2
by simply removing
X-Python3-Version: >= 3.2
from d/control. Manpages for the tools in /usr/bin would be nice.
>, and read more details about
> the rationale and how it works. You can also install the binaries from
> our own APT repository. Follow the instructions at
> https://apt.kitware.com/ to set up the repository, and then install the
> "dh-cmake" package.
>
> Our end goal is to get both dh-cmake and VTK into Debian proper, but it
> is still in an experimental state, and there is still a lot of work to
> be done yet. We would like to get some feedback on dh-cmake, and we
> will eventually file a formal ITP and RFS for it as it becomes more
> mature. We would also like to see other CMake-based packages follow our
> lead and use these utilities. If you have a package that uses CMake, we
> encourage you to give dh-cmake a try.
some examples this would be helpful.
> Thank you in advance for the feedback. We are very excited to venture
> into Debian development.
Andreas.
--
http://fam-tille.de
Lisandro Damián Nicanor Pérez Meyer
Jul 4, 2018, 1:50:03 PM7/4/18
to
El martes, 3 de julio de 2018 16:56:42 -03 Kyle Edwards escribió:
> Hello everyone!
Hi Kyle!
> My name is Kyle. I work at Kitware, Inc., the upstream maintainer of
> the CMake buildsystem (https://www.cmake.org/) and VTK, the
> Visualization Toolkit (https://www.vtk.org/).
I'm Lisandro and, even if I'm listed as one of Debian's CMake maintainers I
must admit I barely have put my hands into it's packaging.
But on the other side I happen to maintain Qt (which does not uses CMake) and
a lot of Qt based applications (which *do* use CMake). I even use it for 99%
of my personal/job projects!
> As some of you on the
> Debian Science list may have heard, we are making an effort to
> officially support our flagship product, VTK, on Debian and its
> derivatives. To that end, we have created a new set of Debhelper
> utilities to meet the unique challenges of packaging a large CMake-
> based project like VTK. We have named this project "dh-cmake". It
> allows Debhelper to take advantage of some of the more advanced
> capabilities of CMake. For example:
>
> * CMake's install() command takes an optional COMPONENT parameter,
> which allows you to break the installation up into multiple
> "components", such as "Libraries" and "Development". dh-cmake allows
> you to assign these components to separate binary packages, to avoid
> having to enumerate every file or file glob in the *.install files.
A thing that it's not clear to me: can the Debian maintainer override whatever
upstream has set in there?
If upstream happens to be the Debian maintainer then *maybe* this might be
desirable. But if upstream is *not* the Debian maintainer then the later must
be able to easily override whatever upstream has planned as "packaging".
I mean: upstreams normally know how to develop their code, and maintainers
know how to properly deploy it on their distro. If i as a maintainer need to
hack CMake files in order to make a package install stuff in the right place
then I would simply prefer to override whatever upstream has done and use our
tooling.
> * Projects that are CTest-aware can optionally have the output of
> dh_auto_configure, dh_auto_build, and dh_auto_test captured by CTest
> and submitted to a CDash server as part of a continuous integration
> process. This is very useful for making sure a large software project
> builds properly on Debian.
Debian buildds do not allow network connections. Except maybe if some day we
deploy something specifically for this.
> * CPack includes a mechanism to declare dependencies between
> installation components, for example, stating that the "Development"
> component depends on the "Libraries" component. dh-cmake can
> propagate this information into the output packages, so that
> libexample-dev will automatically depend on libexample.
And we are back to my first comment.
> You can download the source code at
> https://gitlab.kitware.com/debian/dh-cmake, and read more details about
> the rationale and how it works. You can also install the binaries from
> our own APT repository. Follow the instructions at
> https://apt.kitware.com/ to set up the repository, and then install the
> "dh-cmake" package.
>
> Our end goal is to get both dh-cmake and VTK into Debian proper, but it
> is still in an experimental state, and there is still a lot of work to
> be done yet. We would like to get some feedback on dh-cmake, and we
> will eventually file a formal ITP and RFS for it as it becomes more
> mature. We would also like to see other CMake-based packages follow our
> lead and use these utilities. If you have a package that uses CMake, we
> encourage you to give dh-cmake a try.
>
> Thank you in advance for the feedback. We are very excited to venture
> into Debian development.
Well, my feedback is clear: if we maintainers do not have an easy way to
override upstream's *packaging* decisions then I will clearly not suggest
fellow maintainers to use dh-cmake.
All that being said discussing details in this list might be appropiate. We
might find a use for it which suites both sides :-)
Kinds regards, Lisandro.
--
La ciencia sin la religión es renga, la religión sin la ciencia es ciega.
Albert Einstein
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/
> Hello everyone!
Hi Kyle!
> My name is Kyle. I work at Kitware, Inc., the upstream maintainer of
> the CMake buildsystem (https://www.cmake.org/) and VTK, the
> Visualization Toolkit (https://www.vtk.org/).
must admit I barely have put my hands into it's packaging.
But on the other side I happen to maintain Qt (which does not uses CMake) and
a lot of Qt based applications (which *do* use CMake). I even use it for 99%
of my personal/job projects!
> As some of you on the
> Debian Science list may have heard, we are making an effort to
> officially support our flagship product, VTK, on Debian and its
> derivatives. To that end, we have created a new set of Debhelper
> utilities to meet the unique challenges of packaging a large CMake-
> based project like VTK. We have named this project "dh-cmake". It
> allows Debhelper to take advantage of some of the more advanced
> capabilities of CMake. For example:
>
> * CMake's install() command takes an optional COMPONENT parameter,
> which allows you to break the installation up into multiple
> "components", such as "Libraries" and "Development". dh-cmake allows
> you to assign these components to separate binary packages, to avoid
> having to enumerate every file or file glob in the *.install files.
upstream has set in there?
If upstream happens to be the Debian maintainer then *maybe* this might be
desirable. But if upstream is *not* the Debian maintainer then the later must
be able to easily override whatever upstream has planned as "packaging".
I mean: upstreams normally know how to develop their code, and maintainers
know how to properly deploy it on their distro. If i as a maintainer need to
hack CMake files in order to make a package install stuff in the right place
then I would simply prefer to override whatever upstream has done and use our
tooling.
> * Projects that are CTest-aware can optionally have the output of
> dh_auto_configure, dh_auto_build, and dh_auto_test captured by CTest
> and submitted to a CDash server as part of a continuous integration
> process. This is very useful for making sure a large software project
> builds properly on Debian.
deploy something specifically for this.
> * CPack includes a mechanism to declare dependencies between
> installation components, for example, stating that the "Development"
> component depends on the "Libraries" component. dh-cmake can
> propagate this information into the output packages, so that
> libexample-dev will automatically depend on libexample.
> You can download the source code at
> https://gitlab.kitware.com/debian/dh-cmake, and read more details about
> the rationale and how it works. You can also install the binaries from
> our own APT repository. Follow the instructions at
> https://apt.kitware.com/ to set up the repository, and then install the
> "dh-cmake" package.
>
> Our end goal is to get both dh-cmake and VTK into Debian proper, but it
> is still in an experimental state, and there is still a lot of work to
> be done yet. We would like to get some feedback on dh-cmake, and we
> will eventually file a formal ITP and RFS for it as it becomes more
> mature. We would also like to see other CMake-based packages follow our
> lead and use these utilities. If you have a package that uses CMake, we
> encourage you to give dh-cmake a try.
>
> Thank you in advance for the feedback. We are very excited to venture
> into Debian development.
override upstream's *packaging* decisions then I will clearly not suggest
fellow maintainers to use dh-cmake.
All that being said discussing details in this list might be appropiate. We
might find a use for it which suites both sides :-)
Kinds regards, Lisandro.
--
La ciencia sin la religión es renga, la religión sin la ciencia es ciega.
Albert Einstein
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/
Kyle Edwards
Jul 5, 2018, 9:30:02 AM7/5/18
to
Hi Lisandro,
Thank you for expressing your concerns. You bring up some very valid
points, and I will try to address all of them here.
On Wed, 2018-07-04 at 14:40 -0300, Lisandro Damián Nicanor Pérez Meyer
wrote:
> If upstream happens to be the Debian maintainer then *maybe* this
> might be desirable. But if upstream is *not* the Debian maintainer
> then the later must be able to easily override whatever upstream has
> planned as "packaging".
You are correct that this works best if the upstream project has done
its packaging correctly, and/or if the upstream maintainer is also the
Debian maintainer. We would certainly like to see projects use CMake in
a way that makes the life of downstream maintainers easier, and the
install() command's COMPONENT parameter is designed specifically for
this purpose.
If upstream hasn't done its packaging correctly, then we would
certainly advise you to use your own judgment and possibly not use it,
or even try to get your changes upstreamed. (Though, there may be cases
where upstream gets it 99% correct, in which case the solution is
"install the 'Development' component in libexample-dev, then remove
this one file that shouldn't be there.")
In our case, we plan on using this to package VTK. Our plan is to
change VTK's upstream CMake scripts to make it more distro-friendly,
then provide packaging scripts that take advantage of these changes.
(We've already made some of these changes in the latest VTK master - it
now automatically installs its libraries in /usr/lib/<arch> if built as
a Debian package.)
> Debian buildds do not allow network connections. Except maybe if some
> day we deploy something specifically for this.
Not to worry, we've already thought about this. Even if the packaging
turns on the CTest functionality, dh-cmake won't actually try to submit
anything to a server unless the builder/developer gives it explicit
permission to do so, via the DEB_CTEST_OPTIONS environment variable
(this was inspired by DEB_BUILD_OPTIONS). This allows for packages that
simulaneously support two workflows: one for in-house development,
where the build machine gives permission to submit results to CDash,
and one for production (Debian's buildd instances) where it doesn't
submit anything, and just acts as a thin wrapper around the dh_auto_*
commands.
The CTest functionality isn't meant to be turned on in production
buildd instances. It's meant to be used as part of the continuous
integration/nightly build process of an upstream software project,
where a Debian package is one of the supported builds. In our case, we
plan on making Debian one of the supported nightly builds of VTK, and
having this information submitted to CDash is very valuable to us, as
it allows us to spot problems with the Debian build as they occur. VTK
is a very large project that is constantly growing, evolving, and
changing, and supporting it on Debian now and for the rest of the
foreseeable future requires the use of this workflow.
> All that being said discussing details in this list might be
> appropiate. We might find a use for it which suites both sides :-)
Yes, my hope is that we've designed this in a way that works for both
upstream and downstream. I think we've done our best to address all of
your concerns while also supporting upstream's needs.
Thanks again for the feedback!
Kyle
Thank you for expressing your concerns. You bring up some very valid
points, and I will try to address all of them here.
On Wed, 2018-07-04 at 14:40 -0300, Lisandro Damián Nicanor Pérez Meyer
wrote:
> I even use it for 99% of my personal/job projects!
Glad to hear it!
> If upstream happens to be the Debian maintainer then *maybe* this
> might be desirable. But if upstream is *not* the Debian maintainer
> then the later must be able to easily override whatever upstream has
> planned as "packaging".
its packaging correctly, and/or if the upstream maintainer is also the
Debian maintainer. We would certainly like to see projects use CMake in
a way that makes the life of downstream maintainers easier, and the
install() command's COMPONENT parameter is designed specifically for
this purpose.
If upstream hasn't done its packaging correctly, then we would
certainly advise you to use your own judgment and possibly not use it,
or even try to get your changes upstreamed. (Though, there may be cases
where upstream gets it 99% correct, in which case the solution is
"install the 'Development' component in libexample-dev, then remove
this one file that shouldn't be there.")
In our case, we plan on using this to package VTK. Our plan is to
change VTK's upstream CMake scripts to make it more distro-friendly,
then provide packaging scripts that take advantage of these changes.
(We've already made some of these changes in the latest VTK master - it
now automatically installs its libraries in /usr/lib/<arch> if built as
a Debian package.)
> Debian buildds do not allow network connections. Except maybe if some
> day we deploy something specifically for this.
turns on the CTest functionality, dh-cmake won't actually try to submit
anything to a server unless the builder/developer gives it explicit
permission to do so, via the DEB_CTEST_OPTIONS environment variable
(this was inspired by DEB_BUILD_OPTIONS). This allows for packages that
simulaneously support two workflows: one for in-house development,
where the build machine gives permission to submit results to CDash,
and one for production (Debian's buildd instances) where it doesn't
submit anything, and just acts as a thin wrapper around the dh_auto_*
commands.
The CTest functionality isn't meant to be turned on in production
buildd instances. It's meant to be used as part of the continuous
integration/nightly build process of an upstream software project,
where a Debian package is one of the supported builds. In our case, we
plan on making Debian one of the supported nightly builds of VTK, and
having this information submitted to CDash is very valuable to us, as
it allows us to spot problems with the Debian build as they occur. VTK
is a very large project that is constantly growing, evolving, and
changing, and supporting it on Debian now and for the rest of the
foreseeable future requires the use of this workflow.
> All that being said discussing details in this list might be
> appropiate. We might find a use for it which suites both sides :-)
upstream and downstream. I think we've done our best to address all of
your concerns while also supporting upstream's needs.
Thanks again for the feedback!
Kyle
Kyle Edwards
Jul 5, 2018, 9:30:03 AM7/5/18
to
On Wed, 2018-07-04 at 11:30 +0200, Andreas Tille wrote:
> I think you can solve the lintian warning
> W: dh-cmake source: ancient-python-version-field x-python3-version
> 3.2
> by simply removing
> X-Python3-Version: >= 3.2
> from d/control.
Thanks for the tip, we will fix this in the next release.
> I think you can solve the lintian warning
> W: dh-cmake source: ancient-python-version-field x-python3-version
> 3.2
> by simply removing
> X-Python3-Version: >= 3.2
> from d/control.
> Manpages for the tools in /usr/bin would be nice.
> I'll try with the next cmake package I'll touch. If you can give
> some examples this would be helpful.
"examples" folder which has examples that can be built out-of-the-box.
Kyle
Simon McVittie
Jul 5, 2018, 10:40:02 AM7/5/18
to
On Thu, 05 Jul 2018 at 09:20:55 -0400, Kyle Edwards wrote:
> Our plan is to
> change VTK's upstream CMake scripts to make it more distro-friendly,
> then provide packaging scripts that take advantage of these changes.
> (We've already made some of these changes in the latest VTK master - it
> now automatically installs its libraries in /usr/lib/<arch> if built as
> a Debian package.)
debhelper's Debian::Debhelper::BuildSystem::cmake already passes
> Our plan is to
> change VTK's upstream CMake scripts to make it more distro-friendly,
> then provide packaging scripts that take advantage of these changes.
> (We've already made some of these changes in the latest VTK master - it
> now automatically installs its libraries in /usr/lib/<arch> if built as
> a Debian package.)
-DCMAKE_INSTALL_LIBDIR=lib/$DEB_HOST_MULTIARCH (among other options)
to packages built using cmake, although for some reason it only does
this when cross-compiling (this seems sufficiently odd that I've
reported it as a bug).
The most helpful thing that CMake could do here would be to have a
predictable set of conventional installation paths, similar to the
--libdir, --bindir etc. in Autotools and Meson, so that debhelper can
define the same paths for all CMake-built packages and have the right
things happen 99% of the time, as it already does for Autotools and
Meson. If I understand correctly, CMake doesn't *necessarily* provide
anything like this, but individual CMake-using projects can opt-in to it
by using <https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html>?
(debhelper does set many of those variables, in the hope that the project
being built uses GNUInstallDirs.)
In general I think we prefer upstream build systems to do something
predictable rather than something clever, because we're usually going
to be passing options to them anyway (so that the clever thing they
do doesn't change unpredictably when the build system is upgraded;
debhelper's compat levels put a package maintainer in control of when
they deal with potentially incompatible changes). It's best if we can
pass essentially the same options to every package built with a particular
build system, as we do for Autotools and Meson.
smcv
Kyle Edwards
Jul 5, 2018, 11:20:03 AM7/5/18
to
So, to clarify: we've changed VTK to use GNUInstallDirs, which *itself*
sets the proper directories for Debian, as I will explain below.
On Thu, 2018-07-05 at 15:38 +0100, Simon McVittie wrote:
> debhelper's Debian::Debhelper::BuildSystem::cmake already passes
> -DCMAKE_INSTALL_LIBDIR=lib/$DEB_HOST_MULTIARCH (among other options)
> to packages built using cmake, although for some reason it only does
> this when cross-compiling (this seems sufficiently odd that I've
> reported it as a bug).
The CMAKE_INSTALL_LIBDIR variable comes from the GNUInstallDirs module,
which already sets it to lib/$DEB_HOST_MULTIARCH if it detects a Debian
installation and if CMAKE_INSTALL_PREFIX is set to /usr. Though, as you
have pointed out, CMAKE_INSTALL_LIBDIR can be overridden if so desired.
> The most helpful thing that CMake could do here would be to have a
> predictable set of conventional installation paths, similar to the
> --libdir, --bindir etc. in Autotools and Meson, so that debhelper can
> define the same paths for all CMake-built packages and have the right
> things happen 99% of the time, as it already does for Autotools and
> Meson. If I understand correctly, CMake doesn't *necessarily* provide
> anything like this, but individual CMake-using projects can opt-in to
> it
> by using <https://cmake.org/cmake/help/latest/module/GNUInstallDirs.h
> tml>?
> (debhelper does set many of those variables, in the hope that the
> project
> being built uses GNUInstallDirs.)
Yes, we've changed VTK to use GNUInstallDirs for this exact reason.
GNUInstallDirs should be considered the standard set of installation
directories for CMake projects.
> In general I think we prefer upstream build systems to do something
> predictable rather than something clever, because we're usually going
> to be passing options to them anyway
I think we're on the same page here. In fact, another predictable thing
we'd like to see upstream buildsystems do is install their libraries in
the "Libraries" or "Runtime" component (via the install() command's
COMPONENT argument) and install the headers and symlinks in the
"Development" component. Then, Debian maintainers can use dh-cmake with
these standard options and have everything go in the correct packages.
Or, if it's a large project with lots of output packages, it could have
more fine-grained components like "example1-Development", "example2-
Development", etc., and then the Debian packaging can take this into
account. (This is exactly what we plan to do with VTK.)
The whole point of this project is to encourage upstream CMake projects
to follow a set of standards that are Debian-friendly (and distro-
friendly in general), so that they "do something predictable."
Kyle
sets the proper directories for Debian, as I will explain below.
On Thu, 2018-07-05 at 15:38 +0100, Simon McVittie wrote:
> debhelper's Debian::Debhelper::BuildSystem::cmake already passes
> -DCMAKE_INSTALL_LIBDIR=lib/$DEB_HOST_MULTIARCH (among other options)
> to packages built using cmake, although for some reason it only does
> this when cross-compiling (this seems sufficiently odd that I've
> reported it as a bug).
which already sets it to lib/$DEB_HOST_MULTIARCH if it detects a Debian
installation and if CMAKE_INSTALL_PREFIX is set to /usr. Though, as you
have pointed out, CMAKE_INSTALL_LIBDIR can be overridden if so desired.
> The most helpful thing that CMake could do here would be to have a
> predictable set of conventional installation paths, similar to the
> --libdir, --bindir etc. in Autotools and Meson, so that debhelper can
> define the same paths for all CMake-built packages and have the right
> things happen 99% of the time, as it already does for Autotools and
> Meson. If I understand correctly, CMake doesn't *necessarily* provide
> anything like this, but individual CMake-using projects can opt-in to
> it
> by using <https://cmake.org/cmake/help/latest/module/GNUInstallDirs.h
> tml>?
> (debhelper does set many of those variables, in the hope that the
> project
> being built uses GNUInstallDirs.)
GNUInstallDirs should be considered the standard set of installation
directories for CMake projects.
> In general I think we prefer upstream build systems to do something
> predictable rather than something clever, because we're usually going
> to be passing options to them anyway
we'd like to see upstream buildsystems do is install their libraries in
the "Libraries" or "Runtime" component (via the install() command's
COMPONENT argument) and install the headers and symlinks in the
"Development" component. Then, Debian maintainers can use dh-cmake with
these standard options and have everything go in the correct packages.
Or, if it's a large project with lots of output packages, it could have
more fine-grained components like "example1-Development", "example2-
Development", etc., and then the Debian packaging can take this into
account. (This is exactly what we plan to do with VTK.)
The whole point of this project is to encourage upstream CMake projects
to follow a set of standards that are Debian-friendly (and distro-
friendly in general), so that they "do something predictable."
Kyle
Lisandro Damián Nicanor Pérez Meyer
Jul 5, 2018, 12:50:03 PM7/5/18
to
El jueves, 5 de julio de 2018 12:11:36 -03 Kyle Edwards escribió:
> So, to clarify: we've changed VTK to use GNUInstallDirs, which *itself*
> sets the proper directories for Debian, as I will explain below.
>
> On Thu, 2018-07-05 at 15:38 +0100, Simon McVittie wrote:
> > debhelper's Debian::Debhelper::BuildSystem::cmake already passes
> > -DCMAKE_INSTALL_LIBDIR=lib/$DEB_HOST_MULTIARCH (among other options)
> > to packages built using cmake, although for some reason it only does
> > this when cross-compiling (this seems sufficiently odd that I've
> > reported it as a bug).
>
> The CMAKE_INSTALL_LIBDIR variable comes from the GNUInstallDirs module,
> which already sets it to lib/$DEB_HOST_MULTIARCH if it detects a Debian
> installation and if CMAKE_INSTALL_PREFIX is set to /usr. Though, as you
> have pointed out, CMAKE_INSTALL_LIBDIR can be overridden if so desired.
Right, CMake is already doing the right thing while packaging. The exception
> So, to clarify: we've changed VTK to use GNUInstallDirs, which *itself*
> sets the proper directories for Debian, as I will explain below.
>
> On Thu, 2018-07-05 at 15:38 +0100, Simon McVittie wrote:
> > debhelper's Debian::Debhelper::BuildSystem::cmake already passes
> > -DCMAKE_INSTALL_LIBDIR=lib/$DEB_HOST_MULTIARCH (among other options)
> > to packages built using cmake, although for some reason it only does
> > this when cross-compiling (this seems sufficiently odd that I've
> > reported it as a bug).
>
> The CMAKE_INSTALL_LIBDIR variable comes from the GNUInstallDirs module,
> which already sets it to lib/$DEB_HOST_MULTIARCH if it detects a Debian
> installation and if CMAKE_INSTALL_PREFIX is set to /usr. Though, as you
> have pointed out, CMAKE_INSTALL_LIBDIR can be overridden if so desired.
was cross compiling, which was fixed not so long ago.
--
Sólo porque un mensaje pueda no ser recibido no implica que no
valga la pena enviarlo.
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.blogspot.com/
Lisandro Damián Nicanor Pérez Meyer
Jul 5, 2018, 1:10:03 PM7/5/18
to
El jueves, 5 de julio de 2018 10:20:55 -03 Kyle Edwards escribió:
> Hi Lisandro,
>
> Thank you for expressing your concerns. You bring up some very valid
> points, and I will try to address all of them here.
[snip]
> Hi Lisandro,
>
> Thank you for expressing your concerns. You bring up some very valid
> points, and I will try to address all of them here.
> > If upstream happens to be the Debian maintainer then *maybe* this
> > might be desirable. But if upstream is *not* the Debian maintainer
> > then the later must be able to easily override whatever upstream has
> > planned as "packaging".
>
> You are correct that this works best if the upstream project has done
> its packaging correctly, and/or if the upstream maintainer is also the
> Debian maintainer. We would certainly like to see projects use CMake in
> a way that makes the life of downstream maintainers easier, and the
> install() command's COMPONENT parameter is designed specifically for
> this purpose.
>
> If upstream hasn't done its packaging correctly, then we would
> certainly advise you to use your own judgment and possibly not use it,
> or even try to get your changes upstreamed. (Though, there may be cases
> where upstream gets it 99% correct, in which case the solution is
> "install the 'Development' component in libexample-dev, then remove
> this one file that shouldn't be there.")
And that will be the vast majority of cases.
> > might be desirable. But if upstream is *not* the Debian maintainer
> > then the later must be able to easily override whatever upstream has
> > planned as "packaging".
>
> You are correct that this works best if the upstream project has done
> its packaging correctly, and/or if the upstream maintainer is also the
> Debian maintainer. We would certainly like to see projects use CMake in
> a way that makes the life of downstream maintainers easier, and the
> install() command's COMPONENT parameter is designed specifically for
> this purpose.
>
> If upstream hasn't done its packaging correctly, then we would
> certainly advise you to use your own judgment and possibly not use it,
> or even try to get your changes upstreamed. (Though, there may be cases
> where upstream gets it 99% correct, in which case the solution is
> "install the 'Development' component in libexample-dev, then remove
> this one file that shouldn't be there.")
> In our case, we plan on using this to package VTK. Our plan is to
> change VTK's upstream CMake scripts to make it more distro-friendly,
> then provide packaging scripts that take advantage of these changes.
> (We've already made some of these changes in the latest VTK master - it
> now automatically installs its libraries in /usr/lib/<arch> if built as
> a Debian package.)
Debian package, that sounds just right, exactly like in your VTK case. Having
the developer being also the maintainer must be the best experience out there.
From what you write above I tend to think that simply by not using dh-cmake
whatever upstream has defined as packaging it will be simply ignored (ie, it
will become a "standard" CMake project).
If all the above is true I would really love to see this concerns explained in
the man page and/or Readme.Debian, possibly even with a note in the package's
long description noting that the user must read those.
> > Debian buildds do not allow network connections. Except maybe if some
> > day we deploy something specifically for this.
>
> Not to worry, we've already thought about this. Even if the packaging
> turns on the CTest functionality, dh-cmake won't actually try to submit
> anything to a server unless the builder/developer gives it explicit
> permission to do so, via the DEB_CTEST_OPTIONS environment variable
> (this was inspired by DEB_BUILD_OPTIONS). This allows for packages that
> simulaneously support two workflows: one for in-house development,
> where the build machine gives permission to submit results to CDash,
> and one for production (Debian's buildd instances) where it doesn't
> submit anything, and just acts as a thin wrapper around the dh_auto_*
> commands.
> The CTest functionality isn't meant to be turned on in production
> buildd instances. It's meant to be used as part of the continuous
> integration/nightly build process of an upstream software project,
> where a Debian package is one of the supported builds. In our case, we
> plan on making Debian one of the supported nightly builds of VTK, and
> having this information submitted to CDash is very valuable to us, as
> it allows us to spot problems with the Debian build as they occur. VTK
> is a very large project that is constantly growing, evolving, and
> changing, and supporting it on Debian now and for the rest of the
> foreseeable future requires the use of this workflow.
> > All that being said discussing details in this list might be
> > appropiate. We might find a use for it which suites both sides :-)
>
> Yes, my hope is that we've designed this in a way that works for both
> upstream and downstream. I think we've done our best to address all of
> your concerns while also supporting upstream's needs.
But beware, you might need to have a double-versioned project.
Take for example VTK. If I'm not mistaken the current version in Debian is
7.1.1. But currently the tarball needs to be modified to comply with Debian
Free Software Guidelines' [+dfsg]. And there's the Debian revision.
Now let's suppose you fix whatever makes the package not DFSG-compliant and
release version 7.2.0. Then the Debian package would be 7.2.0-1.
Now it turns out that you get a bug report where you need to split the
packaging. It's not an upstream issue per-se, but rather a packaging one.
Following normal Debian workflow that would mean simply creating a new Debian
revision.
What would happen with dh-cmake? Would be a new upstream release required?
Would the package maintainer need to patch the packaging editing a CMake file?
[+dfsg] Maybe you can work out to fix this so a repackaging is not needed?
That would be *really* welcomed.
Kinds regards, Lisandro.
--
<perrito666> SlackDeb: velo como un entrenamiento shaolin para geeks,
en vez de meditación y tortura física, abstinencia de internet y sexo
Horacio Francisco Sebastián "Perrito" Durán Barrionuevo, sobre un
viaje que Federico "SlackDeb" Peretti estaba planeando con su novia.
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.blogspot.com/
Kyle Edwards
Jul 6, 2018, 12:10:02 PM7/6/18
to
On Thu, 2018-07-05 at 14:04 -0300, Lisandro Damián Nicanor Pérez Meyer
wrote:
doesn't take anything away from existing packages.
> Now it turns out that you get a bug report where you need to split
> the packaging. It's not an upstream issue per-se, but rather a
> packaging one. Following normal Debian workflow that would mean
> simply creating a new Debian revision.
In our opinion, this actually *is* an upstream issue. To explain why, I
think it would help if I give a brief explanation of the CMake
philosophy.
CMake is intended to be a high-level, cross-platform buildsystem that
lets the end user use whatever tools they like most. Barring hard
technical limitations (such as functionality that is highly specific to
one operating system), if a CMake project is written properly, the
developer and end user should be able to:
* build and run the software on their operating system of choice,
whether it be Windows, Mac, or Linux (among others),
* generate build files for their favorite build tools (Makefiles, Ninja
files, or project files for various IDEs), and
* through the use of CPack, generate installers in whichever format
they desire, including .deb, .rpm, WiX for Windows, and .dmg for Mac.
Expanding on the third point, ideally, upstream would properly split
the project output into components that have all the files in the right
place. Then, the end user can generate well-formed packages in any
format they want with CPack, or, in this case, with dh-cmake in
cooperation with CPack.
If any of these assumptions don't hold true, then the project isn't
utilizing CMake to its full potential, and it can create a poor
experience for the end user if they can't use the tools they feel most
comfortable with. In this case, if a Debian developer can't generate a
package from it using dh-cmake, then we consider it an upstream bug.
> What would happen with dh-cmake? Would be a new upstream release
> required? Would the package maintainer need to patch the packaging
> editing a CMake file?
In a word, yes. See my comment above about this being an upstream
issue.
Of course, many upstream projects won't do things correctly, and the
Debian maintainer will have to do a lot of work no matter what tools
they use. We realize this is unavoidable. In that case, the solution is
to package it as a "traditional" Debian CMake package, without dh-
cmake, the way it's always been done.
This project is meant for the subset of cases where upstream DOES
package it correctly, leaving almost no work for the Debian maintainer
- just add configuration files for dh-cmake and it's ready to go right
away, complete with upstream's dependency graph and project
description.
> [+dfsg] Maybe you can work out to fix this so a repackaging is not
> needed? That would be *really* welcomed.
I have a question about this (and I apologize if this is slightly off
topic): VTK includes "convenience copies" of third-party libraries it
uses to avoid "dependency hell", and also because we've made
modifications to them that haven't yet been upstreamed. I know that on
Debian it's not allowed to *use* these libraries (and we have flags to
use the system versions instead), but do they have to be removed from
the tarball entirely to comply with DFSG?
If the DFSG modifications are because of a non-free file in VTK, we
would certainly like to know about it, and possibly try to fix it. We
don't intend to distribute anything with VTK that's under a non-free,
restrictive license.
Kyle
wrote:
> From what you write above I tend to think that simply by not using
> dh-cmake whatever upstream has defined as packaging it will be simply
> ignored (ie, it will become a "standard" CMake project).
Yes, this is true. dh-cmake only adds optional new functionality and
> dh-cmake whatever upstream has defined as packaging it will be simply
> ignored (ie, it will become a "standard" CMake project).
doesn't take anything away from existing packages.
> Now it turns out that you get a bug report where you need to split
> the packaging. It's not an upstream issue per-se, but rather a
> packaging one. Following normal Debian workflow that would mean
> simply creating a new Debian revision.
think it would help if I give a brief explanation of the CMake
philosophy.
CMake is intended to be a high-level, cross-platform buildsystem that
lets the end user use whatever tools they like most. Barring hard
technical limitations (such as functionality that is highly specific to
one operating system), if a CMake project is written properly, the
developer and end user should be able to:
* build and run the software on their operating system of choice,
whether it be Windows, Mac, or Linux (among others),
* generate build files for their favorite build tools (Makefiles, Ninja
files, or project files for various IDEs), and
* through the use of CPack, generate installers in whichever format
they desire, including .deb, .rpm, WiX for Windows, and .dmg for Mac.
Expanding on the third point, ideally, upstream would properly split
the project output into components that have all the files in the right
place. Then, the end user can generate well-formed packages in any
format they want with CPack, or, in this case, with dh-cmake in
cooperation with CPack.
If any of these assumptions don't hold true, then the project isn't
utilizing CMake to its full potential, and it can create a poor
experience for the end user if they can't use the tools they feel most
comfortable with. In this case, if a Debian developer can't generate a
package from it using dh-cmake, then we consider it an upstream bug.
> What would happen with dh-cmake? Would be a new upstream release
> required? Would the package maintainer need to patch the packaging
> editing a CMake file?
issue.
Of course, many upstream projects won't do things correctly, and the
Debian maintainer will have to do a lot of work no matter what tools
they use. We realize this is unavoidable. In that case, the solution is
to package it as a "traditional" Debian CMake package, without dh-
cmake, the way it's always been done.
This project is meant for the subset of cases where upstream DOES
package it correctly, leaving almost no work for the Debian maintainer
- just add configuration files for dh-cmake and it's ready to go right
away, complete with upstream's dependency graph and project
description.
> [+dfsg] Maybe you can work out to fix this so a repackaging is not
> needed? That would be *really* welcomed.
topic): VTK includes "convenience copies" of third-party libraries it
uses to avoid "dependency hell", and also because we've made
modifications to them that haven't yet been upstreamed. I know that on
Debian it's not allowed to *use* these libraries (and we have flags to
use the system versions instead), but do they have to be removed from
the tarball entirely to comply with DFSG?
If the DFSG modifications are because of a non-free file in VTK, we
would certainly like to know about it, and possibly try to fix it. We
don't intend to distribute anything with VTK that's under a non-free,
restrictive license.
Kyle
Colin Watson
Jul 6, 2018, 4:10:03 PM7/6/18
to
On Fri, Jul 06, 2018 at 11:59:58AM -0400, Kyle Edwards wrote:
> I have a question about this (and I apologize if this is slightly off
> topic): VTK includes "convenience copies" of third-party libraries it
> uses to avoid "dependency hell", and also because we've made
> modifications to them that haven't yet been upstreamed. I know that on
> Debian it's not allowed to *use* these libraries (and we have flags to
> use the system versions instead), but do they have to be removed from
> the tarball entirely to comply with DFSG?
If the libraries in question are DFSG-free themselves, there's no DFSG
> I have a question about this (and I apologize if this is slightly off
> topic): VTK includes "convenience copies" of third-party libraries it
> uses to avoid "dependency hell", and also because we've made
> modifications to them that haven't yet been upstreamed. I know that on
> Debian it's not allowed to *use* these libraries (and we have flags to
> use the system versions instead), but do they have to be removed from
> the tarball entirely to comply with DFSG?
issue and you don't need to remove them from the tarball (and we'd
generally encourage not modifying the upstream tarball unnecessarily for
upload to Debian). The policy about bundling is separate from the DFSG.
Of course it'd be incumbent on whoever's doing the Debian upload to
actually check the licensing status.
--
Colin Watson [cjwa...@debian.org]
Kyle Edwards
Jul 6, 2018, 4:50:02 PM7/6/18
to
On Fri, 2018-07-06 at 21:00 +0100, Colin Watson wrote:
> If the libraries in question are DFSG-free themselves, there's no
> DFSG issue and you don't need to remove them from the tarball (and
> we'd generally encourage not modifying the upstream tarball
> unnecessarily for upload to Debian). The policy about bundling is
> separate from the DFSG. Of course it'd be incumbent on whoever's
> doing the Debian upload to actually check the licensing status.
Thank you Colin, this is good to know. In that case, I will investigate
> If the libraries in question are DFSG-free themselves, there's no
> DFSG issue and you don't need to remove them from the tarball (and
> we'd generally encourage not modifying the upstream tarball
> unnecessarily for upload to Debian). The policy about bundling is
> separate from the DFSG. Of course it'd be incumbent on whoever's
> doing the Debian upload to actually check the licensing status.
VTK's DFSG issues when I get a chance. If there's something in there
with a licensing issue, then we as upstream would like to fix it.
Kyle
Mattia Rizzolo
Jul 6, 2018, 6:30:02 PM7/6/18
to
do a copyright and license check for every file shipped in the tarball.
If there are too many convenience copies this can easily outgrow the
patience (or more simply the available time) of the maintainer, at which
point repacking the upstream tarball is way simpler, easier and quicker.
It's a convenience call the maintainer decides, but one that IMHO always
ought to be done because with our current tooling removing files from
the upstream tarball is an automated process that takes seconds, where
the license check easily takes much more.
--
regards,
Mattia Rizzolo
GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`.
more about me: https://mapreri.org : :' :
Launchpad user: https://launchpad.net/~mapreri `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia `-
Geert Stappers
Jul 6, 2018, 6:40:02 PM7/6/18
to
On Sat, Jul 07, 2018 at 12:25:15AM +0200, Mattia Rizzolo wrote:
> On Fri, Jul 06, 2018 at 04:40:44PM -0400, Kyle Edwards wrote:
> > On Fri, 2018-07-06 at 21:00 +0100, Colin Watson wrote:
> > > If the libraries in question are DFSG-free themselves, there's no
> > > DFSG issue and you don't need to remove them from the tarball (and
> > > we'd generally encourage not modifying the upstream tarball
> > > unnecessarily for upload to Debian). The policy about bundling is
> > > separate from the DFSG. Of course it'd be incumbent on whoever's
> > > doing the Debian upload to actually check the licensing status.
> >
> > Thank you Colin, this is good to know. In that case, I will investigate
> > VTK's DFSG issues when I get a chance. If there's something in there
> > with a licensing issue, then we as upstream would like to fix it.
>
> Whilst everything Colin wrote is true, there is also the detail that we
> do a copyright and license check for every file shipped in the tarball.
> If there are too many convenience copies this can easily outgrow the
> patience (or more simply the available time) of the maintainer, at which
> point repacking the upstream tarball is way simpler, easier and quicker.
> It's a convenience call the maintainer decides, but one that IMHO always
> ought to be done because with our current tooling removing files from
> the upstream tarball is an automated process that takes seconds, where
> the license check easily takes much more.
FWIW the "tooling removing files from the upstream tarball is an automated process"
> On Fri, Jul 06, 2018 at 04:40:44PM -0400, Kyle Edwards wrote:
> > On Fri, 2018-07-06 at 21:00 +0100, Colin Watson wrote:
> > > If the libraries in question are DFSG-free themselves, there's no
> > > DFSG issue and you don't need to remove them from the tarball (and
> > > we'd generally encourage not modifying the upstream tarball
> > > unnecessarily for upload to Debian). The policy about bundling is
> > > separate from the DFSG. Of course it'd be incumbent on whoever's
> > > doing the Debian upload to actually check the licensing status.
> >
> > Thank you Colin, this is good to know. In that case, I will investigate
> > VTK's DFSG issues when I get a chance. If there's something in there
> > with a licensing issue, then we as upstream would like to fix it.
>
> Whilst everything Colin wrote is true, there is also the detail that we
> do a copyright and license check for every file shipped in the tarball.
> If there are too many convenience copies this can easily outgrow the
> patience (or more simply the available time) of the maintainer, at which
> point repacking the upstream tarball is way simpler, easier and quicker.
> It's a convenience call the maintainer decides, but one that IMHO always
> ought to be done because with our current tooling removing files from
> the upstream tarball is an automated process that takes seconds, where
> the license check easily takes much more.
is `uscan` and debian/copyright having a 'Files-Excluded:' entry
Groeten
Geert Stappers
--
Leven en laten leven
gregor herrmann
Jul 6, 2018, 7:00:03 PM7/6/18
to
On Sat, 07 Jul 2018 00:25:15 +0200, Mattia Rizzolo wrote:
> On Fri, Jul 06, 2018 at 04:40:44PM -0400, Kyle Edwards wrote:
> > On Fri, 2018-07-06 at 21:00 +0100, Colin Watson wrote:
> > > If the libraries in question are DFSG-free themselves, there's no
> > > DFSG issue and you don't need to remove them from the tarball (and
> > > we'd generally encourage not modifying the upstream tarball
> > > unnecessarily for upload to Debian). The policy about bundling is
> > > separate from the DFSG. Of course it'd be incumbent on whoever's
> > > doing the Debian upload to actually check the licensing status.
> > Thank you Colin, this is good to know. In that case, I will investigate
> > VTK's DFSG issues when I get a chance. If there's something in there
> > with a licensing issue, then we as upstream would like to fix it.
> Whilst everything Colin wrote is true, there is also the detail that we
> do a copyright and license check for every file shipped in the tarball.
> If there are too many convenience copies this can easily outgrow the
> patience (or more simply the available time) of the maintainer, at which
> point repacking the upstream tarball is way simpler, easier and quicker.
> It's a convenience call the maintainer decides, but one that IMHO always
> ought to be done because with our current tooling removing files from
> the upstream tarball is an automated process that takes seconds, where
> the license check easily takes much more.
In addition to what Colin and Mattia said, repackaging/removing
> On Fri, Jul 06, 2018 at 04:40:44PM -0400, Kyle Edwards wrote:
> > On Fri, 2018-07-06 at 21:00 +0100, Colin Watson wrote:
> > > If the libraries in question are DFSG-free themselves, there's no
> > > DFSG issue and you don't need to remove them from the tarball (and
> > > we'd generally encourage not modifying the upstream tarball
> > > unnecessarily for upload to Debian). The policy about bundling is
> > > separate from the DFSG. Of course it'd be incumbent on whoever's
> > > doing the Debian upload to actually check the licensing status.
> > Thank you Colin, this is good to know. In that case, I will investigate
> > VTK's DFSG issues when I get a chance. If there's something in there
> > with a licensing issue, then we as upstream would like to fix it.
> Whilst everything Colin wrote is true, there is also the detail that we
> do a copyright and license check for every file shipped in the tarball.
> If there are too many convenience copies this can easily outgrow the
> patience (or more simply the available time) of the maintainer, at which
> point repacking the upstream tarball is way simpler, easier and quicker.
> It's a convenience call the maintainer decides, but one that IMHO always
> ought to be done because with our current tooling removing files from
> the upstream tarball is an automated process that takes seconds, where
> the license check easily takes much more.
convenience copies is sometimes also done for other reasons, e.g.
- when the embedded files take a huge percentage of the package size
- to make sure they are really not used during the build process
In cases where third-party files are not removed for DFSG reasons,
usually the version is not created with a +dfsg suffix but frequently
with +ds (debian source).
Cheers,
gregor
--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`- NP: Bettina Wegner: für klein zaches
Lisandro Damián Nicanor Pérez Meyer
Jul 10, 2018, 12:00:03 PM7/10/18
to
El viernes, 6 de julio de 2018 12:59:58 -03 Kyle Edwards escribió:
> On Thu, 2018-07-05 at 14:04 -0300, Lisandro Damián Nicanor Pérez Meyer
>
> wrote:
> > From what you write above I tend to think that simply by not using
> > dh-cmake whatever upstream has defined as packaging it will be simply
> > ignored (ie, it will become a "standard" CMake project).
>
> Yes, this is true. dh-cmake only adds optional new functionality and
> doesn't take anything away from existing packages.
That's just perfect.
> On Thu, 2018-07-05 at 14:04 -0300, Lisandro Damián Nicanor Pérez Meyer
>
> wrote:
> > From what you write above I tend to think that simply by not using
> > dh-cmake whatever upstream has defined as packaging it will be simply
> > ignored (ie, it will become a "standard" CMake project).
>
> Yes, this is true. dh-cmake only adds optional new functionality and
> doesn't take anything away from existing packages.
> > Now it turns out that you get a bug report where you need to split
> > the packaging. It's not an upstream issue per-se, but rather a
> > packaging one. Following normal Debian workflow that would mean
> > simply creating a new Debian revision.
>
> In our opinion, this actually *is* an upstream issue. To explain why, I
> think it would help if I give a brief explanation of the CMake
> philosophy.
That's just perfect. It really sounds like a tool to use if upstream wants to
provide a non-official Debian package, and believe me that sounds pretty good.
But not for Debian proper, see below.
[snip]
> This project is meant for the subset of cases where upstream DOES
> package it correctly, leaving almost no work for the Debian maintainer
> - just add configuration files for dh-cmake and it's ready to go right
> away, complete with upstream's dependency graph and project
> description.
Well, there are cases when upstream is doing things the right way with respect
> package it correctly, leaving almost no work for the Debian maintainer
> - just add configuration files for dh-cmake and it's ready to go right
> away, complete with upstream's dependency graph and project
> description.
to Debian but... what about derivatives (distributions which base themselves
in Debian)? Sometimes they need something different, and even if the package
fits perfectly well in Debian it might not be true for them. So they need to
either patch CMake files or re do the whole packaging using traditional tools.
That's just an example, but I'm sure other will appear.
To sum it up: I *do* think there is a *huge* potential for this helper, just
not for Debian proper. Of course I would *love* to see it packaged in Debian
because it will help projects trying to create their own Debian packages, but
I would expect a very clear explanation that this is not a suitable way to
maintain packages in Debian proper.
Except we can find smart work arounds for this cases, of course.
Cheers, Lisandro.
--
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/
Kyle Edwards
Jul 10, 2018, 2:50:03 PM7/10/18
to
On Tue, 2018-07-10 at 12:52 -0300, Lisandro Damián Nicanor Pérez Meyer
wrote:
that Debian requires *.so library symlinks to live in the -dev package.
But let's say there's a hypothetical Debian derivative that requires
them to live in the library binary package instead.
If upstream has both the headers and the symlink in the "Development"
component, then this would certainly be a problem. Perhaps the solution
is for upstream to make this more flexible, through one of several
options:
1. Further divide the "Development" component into "Headers" and
"Symlinks", allowing downstream to put the symlink in whichever package
is required to meet the distribution's policy. Remember: dh-cmake
allows you to specify *more than one* component per package. In this
case, erring on the side of smaller components would give downstream
the maximum flexibility for grouping them as needed.
2. Make the component names configurable and intelligent through a new,
standardized CMake module, similar to GNUInstallDirs. For example,
there could be new cache variables such as
CMAKE_INSTALL_LIBRARY_COMPONENT, CMAKE_INSTALL_NAMELINK_COMPONENT,
CMAKE_INSTALL_HEADER_COMPONENT, etc. They could be set to defaults that
make sense for the detected distribution, but also configurable to
allow downstream to override them as needed.
I would also add that while this component system might not be as
valuable for smaller, single-library packages, it makes a lot of sense
for VTK. VTK has a module system, with its own internal dependency
graph, that builds and installs dozens of modules, each with their own
headers and CMake files that have to be installed with their respective
shared libraries.
Without the component-based approach taken by dh-cmake, trying to break
up this installation into multiple packages is very difficult. The VTK7
package divides it up into Qt and non-Qt modules (along with docs,
executables, bindings, etc.) To separate Qt and non-Qt modules, it has
to install everything into the non-Qt package, and then manually remove
the Qt modules from it.
But if VTK's module system was updated to install each module into its
own set of components (vtkCommonCore-Libaries, vtkCommonCore-Headers,
etc.), then the debian/*.cpack-components files used by dh-cmake could
simply list these module components, making it very easy to group the
modules together as needed. (Perhaps we could even add a globbing
mechanism so you can just say "vtkCommon*-Libraries" as a shorthand.)
Then, we could further break the VTK package into smaller packages so
you can install some modules without having to install everything.
Now let's say a problem is encountered with the package later on, and
one of the modules needs to be moved into a different package. No
problem - just move the problem module into the correct *.cpack-
components file, and you're done.
If there's another approach we could take to solve this particular
packaging issue, I would love to hear about it, but this is the best
we've come up with so far.
> To sum it up: I *do* think there is a *huge* potential for this
> helper, just not for Debian proper. Of course I would *love* to see
> it packaged in Debian because it will help projects trying to create
> their own Debian packages, but I would expect a very clear
> explanation that this is not a suitable way to maintain packages in
> Debian proper.
In fact, CPack already provides its own method for maintaining 3rd
party Debian packages - it has its own .deb generator. But dh-cmake is
our attempt to make something that fits better into the Debian
workflow, so that it *can* be used to maintain packages in Debian
proper.
We want CMake packaging to be as easy as Python packaging, where you
just activate dh-python. The end goal of dh-cmake is to make CMake
packaging as easy as writing a few configuration files, and then
declaring "dh $@ --buildsystem=cmake --with cmake --with ctest --with
cpack".
In our case, our goal is to maintain an official VTK package in both
Debian proper and Ubuntu proper. VTK is a huge project which has proven
to be difficult to package, and dh-cmake is being created to solve this
problem. We've also made changes to both VTK and CMake itself to
support the VTK packaging effort, and we can and will make more.
> Except we can find smart work arounds for this cases, of course.
I think making the component names configurable and/or standardized, as
I described above, would help tremendously with this.
Kyle
wrote:
> Well, there are cases when upstream is doing things the right way
> with respect to Debian but... what about derivatives (distributions
> which base themselves in Debian)? Sometimes they need something
> different, and even if the package fits perfectly well in Debian it
> might not be true for them. So they need to either patch CMake files
> or re do the whole packaging using traditional tools.
I understand what you're saying. As a concrete example, we all know
> with respect to Debian but... what about derivatives (distributions
> which base themselves in Debian)? Sometimes they need something
> different, and even if the package fits perfectly well in Debian it
> might not be true for them. So they need to either patch CMake files
> or re do the whole packaging using traditional tools.
that Debian requires *.so library symlinks to live in the -dev package.
But let's say there's a hypothetical Debian derivative that requires
them to live in the library binary package instead.
If upstream has both the headers and the symlink in the "Development"
component, then this would certainly be a problem. Perhaps the solution
is for upstream to make this more flexible, through one of several
options:
1. Further divide the "Development" component into "Headers" and
"Symlinks", allowing downstream to put the symlink in whichever package
is required to meet the distribution's policy. Remember: dh-cmake
allows you to specify *more than one* component per package. In this
case, erring on the side of smaller components would give downstream
the maximum flexibility for grouping them as needed.
2. Make the component names configurable and intelligent through a new,
standardized CMake module, similar to GNUInstallDirs. For example,
there could be new cache variables such as
CMAKE_INSTALL_LIBRARY_COMPONENT, CMAKE_INSTALL_NAMELINK_COMPONENT,
CMAKE_INSTALL_HEADER_COMPONENT, etc. They could be set to defaults that
make sense for the detected distribution, but also configurable to
allow downstream to override them as needed.
I would also add that while this component system might not be as
valuable for smaller, single-library packages, it makes a lot of sense
for VTK. VTK has a module system, with its own internal dependency
graph, that builds and installs dozens of modules, each with their own
headers and CMake files that have to be installed with their respective
shared libraries.
Without the component-based approach taken by dh-cmake, trying to break
up this installation into multiple packages is very difficult. The VTK7
package divides it up into Qt and non-Qt modules (along with docs,
executables, bindings, etc.) To separate Qt and non-Qt modules, it has
to install everything into the non-Qt package, and then manually remove
the Qt modules from it.
But if VTK's module system was updated to install each module into its
own set of components (vtkCommonCore-Libaries, vtkCommonCore-Headers,
etc.), then the debian/*.cpack-components files used by dh-cmake could
simply list these module components, making it very easy to group the
modules together as needed. (Perhaps we could even add a globbing
mechanism so you can just say "vtkCommon*-Libraries" as a shorthand.)
Then, we could further break the VTK package into smaller packages so
you can install some modules without having to install everything.
Now let's say a problem is encountered with the package later on, and
one of the modules needs to be moved into a different package. No
problem - just move the problem module into the correct *.cpack-
components file, and you're done.
If there's another approach we could take to solve this particular
packaging issue, I would love to hear about it, but this is the best
we've come up with so far.
> To sum it up: I *do* think there is a *huge* potential for this
> helper, just not for Debian proper. Of course I would *love* to see
> it packaged in Debian because it will help projects trying to create
> their own Debian packages, but I would expect a very clear
> explanation that this is not a suitable way to maintain packages in
> Debian proper.
party Debian packages - it has its own .deb generator. But dh-cmake is
our attempt to make something that fits better into the Debian
workflow, so that it *can* be used to maintain packages in Debian
proper.
We want CMake packaging to be as easy as Python packaging, where you
just activate dh-python. The end goal of dh-cmake is to make CMake
packaging as easy as writing a few configuration files, and then
declaring "dh $@ --buildsystem=cmake --with cmake --with ctest --with
cpack".
In our case, our goal is to maintain an official VTK package in both
Debian proper and Ubuntu proper. VTK is a huge project which has proven
to be difficult to package, and dh-cmake is being created to solve this
problem. We've also made changes to both VTK and CMake itself to
support the VTK packaging effort, and we can and will make more.
> Except we can find smart work arounds for this cases, of course.
I described above, would help tremendously with this.
Kyle
Ian Jackson
Jul 11, 2018, 6:40:02 AM7/11/18
to
Kyle Edwards writes ("Re: A message from CMake upstream: announcing dh-cmake"):
existing upstream components, and dh-cmake, etc., which put the
symlink in the wrong package for them, *and then to fix up that
wrinkle afterwards*.
This kind of thing is entirely normal in packaging.
I really don't agree with the thrust of Lisandro's comments. AFAICT
what Lisandro is saying is this: because the upstream components may
not always be perfect; and even may be totally inappropriate; they are
useless or harmful for packaging for Debian.
The usual case is that the upstream components are mostly right.
If they are slightly wrong, then the right thing to do is to fix them
(carrying the patch in Debian until it makes it through into the
appropriate upstream release), or if upstream don't want the fix for
any reason, to carry that patch forever, or to write a workaround
which fixes it up and carry that forever in debian/rules.
If there is a systematic mismatch, it is normally possible to invent a
systematic fixup. That is an alternative to adding a feature to the
upstream component system to do things differently.
We have taken all of these approaches in various packages. Which to
choose is a matter of judgement, and of course any one of them may be
inappropriate in particular circumstances. But that does not mean
that upstream component systems are harmful or dangerous or that they
should be discouraged or ignored. The usual case will be that it is
easiest to use the upstream component system and deal with any
problems with it in any one of the usual ways we deal with problems
with upstream functionality.
Given that upstream component systems will often be useful, especially
if they are done with a bit of care, it is a very good thing to have a
build tool which can use them automatically.
So, in summary, if I were packaging one of the pieces of software
which dh-cmake is aimed at, I would probably want to use it. Thanks,
Kyle (and your colleagues) for this work. This work sounds excellent
to me. Please do not be discouraged.
I did have some minor technical question when I read the original
posting. But it's not important. It is more important to applaud
what you are doing.
Thanks,
Ian.
> I understand what you're saying. As a concrete example, we all know
> that Debian requires *.so library symlinks to live in the -dev package.
> But let's say there's a hypothetical Debian derivative that requires
> them to live in the library binary package instead.
>
> If upstream has both the headers and the symlink in the "Development"
> component, then this would certainly be a problem. Perhaps the solution
> is for upstream to make this more flexible, through one of several
> options:
You have missed an option, which is for that derivative to use the
> that Debian requires *.so library symlinks to live in the -dev package.
> But let's say there's a hypothetical Debian derivative that requires
> them to live in the library binary package instead.
>
> If upstream has both the headers and the symlink in the "Development"
> component, then this would certainly be a problem. Perhaps the solution
> is for upstream to make this more flexible, through one of several
> options:
existing upstream components, and dh-cmake, etc., which put the
symlink in the wrong package for them, *and then to fix up that
wrinkle afterwards*.
This kind of thing is entirely normal in packaging.
I really don't agree with the thrust of Lisandro's comments. AFAICT
what Lisandro is saying is this: because the upstream components may
not always be perfect; and even may be totally inappropriate; they are
useless or harmful for packaging for Debian.
The usual case is that the upstream components are mostly right.
If they are slightly wrong, then the right thing to do is to fix them
(carrying the patch in Debian until it makes it through into the
appropriate upstream release), or if upstream don't want the fix for
any reason, to carry that patch forever, or to write a workaround
which fixes it up and carry that forever in debian/rules.
If there is a systematic mismatch, it is normally possible to invent a
systematic fixup. That is an alternative to adding a feature to the
upstream component system to do things differently.
We have taken all of these approaches in various packages. Which to
choose is a matter of judgement, and of course any one of them may be
inappropriate in particular circumstances. But that does not mean
that upstream component systems are harmful or dangerous or that they
should be discouraged or ignored. The usual case will be that it is
easiest to use the upstream component system and deal with any
problems with it in any one of the usual ways we deal with problems
with upstream functionality.
Given that upstream component systems will often be useful, especially
if they are done with a bit of care, it is a very good thing to have a
build tool which can use them automatically.
So, in summary, if I were packaging one of the pieces of software
which dh-cmake is aimed at, I would probably want to use it. Thanks,
Kyle (and your colleagues) for this work. This work sounds excellent
to me. Please do not be discouraged.
I did have some minor technical question when I read the original
posting. But it's not important. It is more important to applaud
what you are doing.
Thanks,
Ian.
Lisandro Damián Nicanor Pérez Meyer
Jul 11, 2018, 8:00:03 AM7/11/18
to
El mar., 10 de jul. de 2018 15:46, Kyle Edwards <kyle.e...@kitware.com> escribió:
On Tue, 2018-07-10 at 12:52 -0300, Lisandro Damián Nicanor Pérez Meyer
wrote:
> Well, there are cases when upstream is doing things the right way
> with respect to Debian but... what about derivatives (distributions
> which base themselves in Debian)? Sometimes they need something
> different, and even if the package fits perfectly well in Debian it
> might not be true for them. So they need to either patch CMake files
> or re do the whole packaging using traditional tools.
I understand what you're saying. As a concrete example, we all know
that Debian requires *.so library symlinks to live in the -dev package.
But let's say there's a hypothetical Debian derivative that requires
them to live in the library binary package instead.
If upstream has both the headers and the symlink in the "Development"
component, then this would certainly be a problem. Perhaps the solution
is for upstream to make this more flexible, through one of several
options:
[Snip]
> To sum it up: I *do* think there is a *huge* potential for this
> helper, just not for Debian proper. Of course I would *love* to see
> it packaged in Debian because it will help projects trying to create
> their own Debian packages, but I would expect a very clear
> explanation that this is not a suitable way to maintain packages in
> Debian proper.
In fact, CPack already provides its own method for maintaining 3rd
party Debian packages - it has its own .deb generator. But dh-cmake is
our attempt to make something that fits better into the Debian
workflow, so that it *can* be used to maintain packages in Debian
proper.
We want CMake packaging to be as easy as Python packaging, where you
just activate dh-python. The end goal of dh-cmake is to make CMake
packaging as easy as writing a few configuration files, and then
declaring "dh $@ --buildsystem=cmake --with cmake --with ctest --with
cpack".
In our case, our goal is to maintain an official VTK package in both
Debian proper and Ubuntu proper. VTK is a huge project which has proven
to be difficult to package, and dh-cmake is being created to solve this
problem. We've also made changes to both VTK and CMake itself to
support the VTK packaging effort, and we can and will make more.
> Except we can find smart work arounds for this cases, of course.
I think making the component names configurable and/or standardized, as
I described above, would help tremendously with this.
I really applaud your effort. It's now clear to me that you are not trying to add just some logic around cpack but really are wanting to create a nice tool.
I would say just keep on with vtk packaging and tackle other cases as they appear.
Regards, Lisandro.
Lisandro Damián Nicanor Pérez Meyer
Jul 11, 2018, 8:00:03 AM7/11/18
to
El mié., 11 de jul. de 2018 07:33, Ian Jackson <ijac...@chiark.greenend.org.uk> escribió:
[snip]
I really don't agree with the thrust of Lisandro's comments. AFAICT
what Lisandro is saying is this: because the upstream components may
not always be perfect; and even may be totally inappropriate; they are
useless or harmful for packaging for Debian.
Now that you mention it I reckon it can be read that way, indeed.
But no, my real fear here was a tool more in the cpack kind. Years ago we had packagers trying to get their stuff in by using cpack. While it might be of some help for non official packages it was not really fit for official ones.
Kyle has made it really clear that they intend to support the various realities they will probably face if people start using it everywhere to create good quality packaging, so yes, I also applaud the effort... now ;-)
Cheers, Lisandro.
Kyle Edwards
Jul 11, 2018, 9:10:02 AM7/11/18
to
On Wed, 2018-07-11 at 08:57 -0300, Lisandro Damián Nicanor Pérez Meyer
wrote:
packaging.
> Kyle has made it really clear that they intend to support the various
> realities they will probably face if people start using it everywhere
> to create good quality packaging, so yes, I also applaud the
> effort... now ;-)
Thank you Lisandro! I'm glad I was able to address your concerns, and I
think this has been a really good discussion.
Ian, thanks for the feedback, and if you have minor technical questions
please feel free to ask them, either on- or off-list :)
I think it makes sense for dh-cmake to live primarily in VTK for now,
and then, as we work out the issues, hopefully we will see other
packages start to adopt it as well.
I will keep everyone updated as both dh-cmake and the VTK packaging are
developed. We are hoping for an official Debian release of both
packages towards the end of the year.
Kyle
wrote:
> But no, my real fear here was a tool more in the cpack kind. Years
> ago we had packagers trying to get their stuff in by using cpack.
> While it might be of some help for non official packages it was not
> really fit for official ones.
Yes, the idea here is to make something that is fit for official Debian
> ago we had packagers trying to get their stuff in by using cpack.
> While it might be of some help for non official packages it was not
> really fit for official ones.
packaging.
> Kyle has made it really clear that they intend to support the various
> realities they will probably face if people start using it everywhere
> to create good quality packaging, so yes, I also applaud the
> effort... now ;-)
think this has been a really good discussion.
Ian, thanks for the feedback, and if you have minor technical questions
please feel free to ask them, either on- or off-list :)
I think it makes sense for dh-cmake to live primarily in VTK for now,
and then, as we work out the issues, hopefully we will see other
packages start to adopt it as well.
I will keep everyone updated as both dh-cmake and the VTK packaging are
developed. We are hoping for an official Debian release of both
packages towards the end of the year.
Kyle
Enrico Weigelt, metux IT consult
Nov 19, 2018, 7:10:03 AM11/19/18
to
On 06.07.18 22:00, Colin Watson wrote:
> If the libraries in question are DFSG-free themselves, there's no DFSG
> issue and you don't need to remove them from the tarball (and we'd
> generally encourage not modifying the upstream tarball unnecessarily for
> upload to Debian). The policy about bundling is separate from the DFSG.
> Of course it'd be incumbent on whoever's doing the Debian upload to
> actually check the licensing status.
last time i've packaged vtk, I removed them (at least those that either
> If the libraries in question are DFSG-free themselves, there's no DFSG
> issue and you don't need to remove them from the tarball (and we'd
> generally encourage not modifying the upstream tarball unnecessarily for
> upload to Debian). The policy about bundling is separate from the DFSG.
> Of course it'd be incumbent on whoever's doing the Debian upload to
> actually check the licensing status.
already had been packaged or easy to do so), in order to make sure that
nothing in that really complex cmake file can even try build/use any
piece of them.
the package was just meant for an inhouse installation for my client,
so i didn't care much about policies and orig tarball handling - I've
just patched directly in the git repo.
--mtx
--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
in...@metux.net -- +49-151-27565287
0 new messages