Bug#995670: ITP: zig -- General-purpose programming language and toolchain for maintaining robust, optimal, and reusable software

[Jason Ernst]

Package: wnpp
Severity: wishlist
Owner: Jason Ernst <ernst...@gmail.com>
X-Debbugs-Cc: debian...@lists.debian.org

* Package name : ziglang
Version : 0.8.1
Upstream Author : Andrew Kelley and...@ziglang.org
* URL : https://ziglang.org
* License : MIT
Programming Lang: Zig
Description : General-purpose programming language and toolchain for maintaining robust, optimal, and reusable software

Zig is quickly become a new hot language, and there is growing demand for it
to be available in mainline repositories.

There is a ticket on the Zig repo: https://github.com/ziglang/zig/issues/7340
which requests for mainline packages to be available, and the upstream
maintainer replied that it is out of scope of their repository. I would like to
volunteer to package this and maintain it so it is available easily. Others are
making similar efforts in other distributions such as Fedora.

I also have a package which pulls the release and packs it using checkinstall
at https://github.com/compscidr/zig-deb which is currently hosted from a
3rd party gemfury repository.

For now I am happy to maintain myself, but would more than welcome anyone who
would like to form a pkg-zig team (unless people feel there is a more
appropriate team this could fit into. Co-maintainers are also welcome. I do
need a sponsor, this is my first time participating.

[Paul Wise]

On Sun, Oct 3, 2021 at 9:27 PM Jason Ernst wrote:

> There is a ticket on the Zig repo: https://github.com/ziglang/zig/issues/7340
> which requests for mainline packages to be available, and the upstream
> maintainer replied that it is out of scope of their repository. I would like to
> volunteer to package this and maintain it so it is available easily. Others are
> making similar efforts in other distributions such as Fedora.

Please make sure the package is built solely from the source from
scratch without any existing binaries using the upstream supported
bootstrap process:

https://github.com/ziglang/zig-bootstrap/

Personally, I think merging zig-bootstrap into the zig source repo
would make it easier for distros to use, but I hear upstream isn't
interested in that.

--
bye,
pabs

https://wiki.debian.org/PaulWise

[Andrew Kelley]

On Tue, 5 Oct 2021 00:20:09 +0000 Paul Wise <pa...@debian.org> wrote:
> Please make sure the package is built solely from the source from
> scratch without any existing binaries using the upstream supported
> bootstrap process:
>
> https://github.com/ziglang/zig-bootstrap/
>
> Personally, I think merging zig-bootstrap into the zig source repo
> would make it easier for distros to use, but I hear upstream isn't
> interested in that.

Hi pabs,

Upstream is definitely interested in cooperating with Debian maintainers
to the benefit of our shared users :-)

I'm happy to discuss this suggestion. Here is some information to help
us sort this out:

* zig-bootstrap contains copy+pasted upstream sources from
- LLVM, LLD, Clang
- There is currently 1 tiny patch to LLD's build script to adjust
an include directory to depend on something inside zig-bootstrap rather
than to an external directory
- There are also some deleted files which is merely an attempt to
reduce tarball size; these could be restored to no harmful effect.
- Zig (no patches)
- zlib (no patches)

* Apart from this, the *only* utility that zig-bootstrap provides is a
short build script, which does the following process:
- Build LLVM, Clang, LLD, zlib, zig from source, for the host system
- Using freshly built native zig, use `zig cc`/`zig c++` to rebuild
LLVM, Clang, LLD, zlib, and zig again, from source, for the target system

This is useful for upstream to provide portable binaries of Zig,
however, I suspect that there is a better strategy which is more
compatible with Debian's guidelines. In fact, I have been keeping
Debian-friendliness in mind since the very beginning. I suspect you will
actually find the main build process more amenable to packaging.

In the main upstream repository (https://github.com/ziglang/zig/), it is
a standard cmake build that I believe is already Debian-compatible. I am
guessing you found the zig-bootstrap repository because you wanted to
avoid depending on a zig binary to build zig; what you may not realize
is that the main zig repository *does not utilize a zig binary* to be
built. In fact, the main cmake build process does the following things:

* Use the system C++ compiler to build zig0 executable using system
LLVM, LLD, Clang, zlib libraries
* Use the freshly built zig0 executable to build zig1.o
* Re-link some of the build artifacts, swapping in zig1.o, producing
the zig binary

I believe this is exactly what a Debian package wants, because you will
end up with a binary that
* Uses the system LLVM, LLD, Clang, zlib libraries
* Was built with the system C/C++ compiler and linker

I suggest to try out the main, standard way of building Zig and let me
know if you run into any trouble. I suspect the main issues will be:

* deps/SoftFloat-3e is a vendored library. I would be happy to improve
the cmake build script to enable an option to prefer the system
SoftFloat library instead.

* lib/libcxx, lib/libcxxabi, lib/libunwind, lib/tsan and lib/include
are copy+pasted (MIT-compatibly licensed) from other upstreams, and
there are quite a few patches and preprocessing which is part of the zig
development process. I am hoping that Debian can grant zig an exception
for these files and not require them to be built by a debian package.
These files are part of the "magic" that makes zig attractive in the
first place, and although these files may look like they should be
provided by other debian packages, I think if you examine closely you
will find that they are really derived works that are part of the zig
project.

I hope this helps. I am happy to work together on this :-)

Warm regards,
Andrew

[Jason Ernst]

So far I have a working build from the main zig repo for amd64, using the llvm, clang and lld from the debian repos (although this is only possible in sid because version 12 isn't present in the stable releases).

I've been doing it via docker to ensure a clean build environment:

https://github.com/compscidr/zig-deb/tree/debian-pkg/debian-pkg

It generates all of the needed files, and sings the package according to guides I've been following from the mentors / maintainers websites.

I think I have figured out how to also produce cross-compiled packages for other arches using pbuilder, but I'll have to abandon my docker approach since it doesn't seem to work well with it. Planning on abandoning the docker approach and using a debian VM in virtualbox (I typically use Ubuntu).

I'm guessing in order to work in non-sid debian, I'll have to pull in the actual llvm, lld, and clang sources like the bootstrap repo does. I'm guessing these also need to be packaged in the debian source package too?

One thing I haven't tried is just using zig itself to do the cross compiling.

Andrew, happy to work with you on this, if you like instead of dev-ing this in my own repo, I can make a branch in either the main zig repo, or the bootstrap repo - whichever you'd prefer. Also happy to jump on a quick call to discuss.

Jason

[Nick Hastings]

Hi,

I just discovered this ITP bug report. I have also started working on
making a zig Debian package. So far I've been able to generate a zig
0.9.0 package on bullseye after backporting llvm-13 from testing.

I had not originally planned for this to be an official package for
Debian, I just wanted a working zig on my system so that I could compile
(and package) the river wayland compositor.

I thought I should speak up in case I can be of some help with the
official packaging efforts.

You can find my package at https://github.com/NickHastings/zig-debian
and https://mentors.debian.net/package/zig/

Regards,

Nick.

[Bastian Germann]

X-Debbugs-Cc: nicholas...@gmail.com

Hi,

Jason, you would have to provide a source package (.dsc + referenced files) somewhere,
which is a complete thing for Debian folks. This is the condition to find a sponsor.
I have not looked at your approach because it is unfamiliar.

I have had a look at what Nick provided and this seems to be reasonable enough to take a closer look.
The build log looks like you took the approach that Andrew described.

Jason, would you be willing to hand over the ITP to Nick so that we can make zig happen for bookworm?
Nick, in case Jason does not reply in three weeks please take this over and address this ITP with your changelog.

Thanks,
Bastian

[Jason Ernst]

Yes no problem!

[Bastian Germann]

Control: owner -1 Nick Hastings <nicholas...@gmail.com>

Okay then; Nick, please file a RFS on sponsorship-requests when you have provided an upload with the changelog addition.

[Nick Hastings]

Hi Bastian,

thanks for taking an interest in this and getting it moving along.

* Bastian Germann <ba...@debian.org> [220603 05:47]:

> Control: owner -1 Nick Hastings <nicholas...@gmail.com>
>
> Okay then; Nick, please file a RFS on sponsorship-requests when you
> have provided an upload with the changelog addition.

I've updated the changelog and uploaded again to m.d.n.
https://mentors.debian.net/package/zig/

I can file an RFS on the sponsorship-requests list, but I'm a slightly
unsure because https://mentors.debian.net/sponsors/rfs-howto/ describes
a different way to do this. It says to file a new bug against the
sponsorship-requests pseudo-package.

Can you please confirm which is the correct or best way to do this?

Thanks,

Nick.

[Bastian Germann]

Am 03.06.22 um 02:06 schrieb Nick Hastings:

>> Okay then; Nick, please file a RFS on sponsorship-requests when you
>> have provided an upload with the changelog addition.
> I've updated the changelog and uploaded again to m.d.n.
> https://mentors.debian.net/package/zig/
>
> I can file an RFS on the sponsorship-requests list, but I'm a slightly

> unsure becausehttps://mentors.debian.net/sponsors/rfs-howto/ describes

> a different way to do this. It says to file a new bug against the
> sponsorship-requests pseudo-package.
>
> Can you please confirm which is the correct or best way to do this?

There is no difference between my suggestion and the mentors template.
Yes, sponsorship-requests is a pseudo package and I meant exactly that.

[Henrique Almeida]

Hello, what's the current status of the Zig package for the Debian
official repository ?

--
Henrique Dante de Almeida
hda...@gmail.com

[Nick Hastings]

Hi,

* Henrique Almeida <hda...@gmail.com> [220811 03:57]:

> Hello, what's the current status of the Zig package for the Debian
> official repository ?

I've filed a request for a sponsor
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1012286

Other than not have a sponsor, the two main issues I see are:

1. Some of the post build tests fail.
Checking on zig issues and IRC it seems not so unusual for some tests to
fail in certain situations. I'm not sure how worthwhile it is chasing
down these issues before zig reaches 1.0.0.

2. The copyright file.
With so much bundled source code from different projects it's quite
challenging to write a correct copyright file. I've put considerable
time into this, but it is horribly tedious and boring work. Without a
sponsor to get zig into Debian I could just be completely wasting my
time.

Cheers,

Nick.

[Bastian Germann]

The package got removed from mentors.
Nick, do you still have time to work on it? If not please make this a RFP.

[Nick Hastings]

Hi Bastian,

* Bastian Germann <ba...@debian.org> [221203 02:45]:

> The package got removed from mentors.

Yes, no one sponsored it.

> Nick, do you still have time to work on it? If not please make this a
> RFP.

I think I have time. Zig 0.10.0 was just released and I'm updating the
package for it. Hope to upload to mentors soon.

I think the biggest problem is still the copyright file. I tried using
scancode but it seems that d/copyright output option is still a work in
progress.

Without a sponsor the package can't enter Debian so I don't want to
spend the many tedious hours required to manually produce a correct
copyright file if it will not be used. If/when there is indication that
someone may sponsor this package I can look to put more time/effort into
the d/copyright.

Regards,

Nick.

[Bastian Germann]

Am 04.12.22 um 00:55 schrieb Nick Hastings:

> Without a sponsor the package can't enter Debian so I don't want to
> spend the many tedious hours required to manually produce a correct
> copyright file if it will not be used. If/when there is indication that
> someone may sponsor this package I can look to put more time/effort into
> the d/copyright.

With the copyright in place I will sponsor it. Please notify me when you think you have represented the applicable
licenses completely and I will review it.

[Nick Hastings]

Hi Bastian,

* Bastian Germann <ba...@debian.org> [221204 09:02]:

>
> With the copyright in place I will sponsor it. Please notify me when you
> think you have represented the applicable licenses completely and I will
> review it.

Many thanks for stepping forward for this.

I started looking at the copyright again and have some questions. I'm
not sure if the discussion should continue here in this bug report or if
it is better to take it "offline" to be just with you as the potential
sponsor.

Thanks,

Nick.

[Nick Hastings]

Hi,

I did some more work on the copyright file, I think it may be ready to
be reviewed. The package is now again at
https://mentors.debian.net/package/zig/

I looked at updating to Zig 0.10.0 (released Oct 31). However, there are
significant changes to the build between 0.9.1 and 0.10.0. Release
notes are at https://ziglang.org/download/0.10.0/release-notes.html

Zig 0.9.1 simply built a Zig compiler using the system provided compiler
and libraries. This new release attempts to do a self hosted build.
Details are at
https://ziglang.org/download/0.10.0/release-notes.html#Self-Hosted-Compiler

There are three stages to the 0.10.0 build. As I understand them, they
are:

1. Build a zig compiler using system compilers and libraries (similar to
0.9.1).

2. Use the zig compiler from stage 1 to build a zig compiler from zig
bundled libraries.

3. Repeat stage 2 using the zig compiler in produced in stage 2.

Attempting this on Debian unstable (and also on bullseye with backported
llvm-15) resulted in a failure at stage 2. It seems the build was not
able to find a particular header file. There may be other problems too,
but I did not investigate further.

The release notes actually suggest waiting until Zig 0.10.1 or Zig
0.11.0 before upgrading from 0.9.1. It seems there will be even larger
changes to the build system for Zig 0.11.0 so I'm not sure how much
effort upstream will be put into fixing build related issues in 0.10.0.

So, at this point I think it may be best to wait for the next release
before attempting to update this package.

Regards,

Nick.

[Abraham Raji]

Hey folks o/,

What's the status of this ITP? It seems like Nick is pulling a
considerable amount of work himself. I would personally love to see Zig
in the Debian archives so I am ready to volunteer some of my time
towards the effort.

I feel we can move the work for this package to salsa if no one has any
issues.

Best,
Abraham

[Nick Hastings]

Hi,

* Abraham Raji <wo...@abrahamr.in> [230620 06:38]:

>
> What's the status of this ITP? It seems like Nick is pulling a
> considerable amount of work himself.

I filed an RFS bug
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1012286
There was some progress and some interest from a potential sponsor but
it seems that the copyright file has become a road block.

> I would personally love to see Zig in the Debian archives so I am
> ready to volunteer some of my time towards the effort.

My zig package is up to date with the current version of zig 0.10.1 and
this version of zig can bootstrap. However, it seems that the next
planned release of zig will not be able to bootstrap (in the usual
sense). See for example https://ziglang.org/news/goodbye-cpp/

> I feel we can move the work for this package to salsa if no one has any
> issues.

My package is on salsa: https://github.com/NickHastings/zig-debian

Cheers,

Nick.

[Nick Hastings]

* Nick Hastings <nicholas...@gmail.com> [230620 08:18]:

>
> My package is on salsa: https://github.com/NickHastings/zig-debian

While that statement is true, the link is clearly not the intended one.
https://salsa.debian.org/nickh/zig

Nick.

[Abraham Raji]

On 20/06/23 04:48, Nick Hastings wrote:
> There was some progress and some interest from a potential sponsor but
> it seems that the copyright file has become a road block.

What exactly is the roadblock? Is it just a matter of going through each
of the files and adding the entries? Or is there a licensing issue here?
Asking because for the first we can always try and get more people to
help out.

> My package is on salsa: https://github.com/NickHastings/zig-debian

I have created a zig-team namespace on salsa and I've invited you there.
We can move the packaging work there as it will make it easier for
potential contributors to find it.

Also is there any particular reason you are only committing the debian
directory?

- Abraham

[Nick Hastings]

Hi,

* Abraham Raji <wo...@abrahamr.in> [230620 16:53]:

>
> On 20/06/23 04:48, Nick Hastings wrote:
> > There was some progress and some interest from a potential sponsor but
> > it seems that the copyright file has become a road block.
>
> What exactly is the roadblock? Is it just a matter of going through each
> of the files and adding the entries?

As far as I know the d/copyright file covers everything.

> Or is there a licensing issue here?

I don't think there is a licensing issue.

It is the specifics of the d/copyright file I produced. Please see the
RFS bug for details.

> Asking because for the first we can always try and get more people to
> help out.

That would be nice.

> > My package is on salsa: https://github.com/NickHastings/zig-debian

Adding the correction here too: https://salsa.debian.org/nickh/zig

> I have created a zig-team namespace on salsa and I've invited you there.
> We can move the packaging work there as it will make it easier for
> potential contributors to find it.

I joined it.

> Also is there any particular reason you are only committing the debian
> directory?

That is all that exists in the repo. Builds are done by downloading the
source with uscan with the info from the d/watch file. I did try to look
into keeping upstream in the same repo but I didn't find a clear path
forward. So I just stuck with what I am currently doing since it works
and from the documentation I have read is not "incorrect". If you could
recommend specific documentation for this I can have a look.

Cheers,

Nick.

[Abraham Raji]

On 20/06/23 13:54, Nick Hastings wrote:

> As far as I know the d/copyright file covers everything.
>
>> Or is there a licensing issue here?
>
> I don't think there is a licensing issue.
>
> It is the specifics of the d/copyright file I produced. Please see the
> RFS bug for details.

Will take a look. Thanks.

>> I have created a zig-team namespace on salsa and I've invited you there.
>> We can move the packaging work there as it will make it easier for
>> potential contributors to find it.
>
> I joined it.

Great let's get the zig package there.

>> Also is there any particular reason you are only committing the debian
>> directory?
>
> That is all that exists in the repo. Builds are done by downloading the
> source with uscan with the info from the d/watch file. I did try to look
> into keeping upstream in the same repo but I didn't find a clear path
> forward. So I just stuck with what I am currently doing since it works
> and from the documentation I have read is not "incorrect". If you could
> recommend specific documentation for this I can have a look.

I do find the ruby team's approach to be very nice here. Adding link to
a sample ruby package for reference[0]. The approach is to keep the
upstream files and tar ball deltas in separate branches (upstream and
pristine-tar). The tooling makes maintaining this pretty seamless.
Please take a look at these page for more information:
- https://wiki.debian.org/SimplePackagingTutorial
- https://wiki.abrahamraji.in/simple-packaging-tutorial.html

[0]: https://salsa.debian.org/ruby-team/atig

Best,
Abraham

[Nick Hastings]

Hi,

* Abraham Raji <wo...@abrahamr.in> [230620 17:37]:

>
> On 20/06/23 13:54, Nick Hastings wrote:
>
> > As far as I know the d/copyright file covers everything.
> >
> > > Or is there a licensing issue here?
> >
> > I don't think there is a licensing issue.
> >
> > It is the specifics of the d/copyright file I produced. Please see the
> > RFS bug for details.
>
> Will take a look. Thanks.
>
>
> > > I have created a zig-team namespace on salsa and I've invited you there.
> > > We can move the packaging work there as it will make it easier for
> > > potential contributors to find it.
> >
> > I joined it.
>
> Great let's get the zig package there.

Done.

> > > Also is there any particular reason you are only committing the debian
> > > directory?
> >
> > That is all that exists in the repo. Builds are done by downloading the
> > source with uscan with the info from the d/watch file. I did try to look
> > into keeping upstream in the same repo but I didn't find a clear path
> > forward. So I just stuck with what I am currently doing since it works
> > and from the documentation I have read is not "incorrect". If you could
> > recommend specific documentation for this I can have a look.
>
> I do find the ruby team's approach to be very nice here. Adding link
> to a sample ruby package for reference[0]. The approach is to keep
> the upstream files and tar ball deltas in separate branches (upstream
> and pristine-tar). The tooling makes maintaining this pretty seamless.

I see. I'll try to have a look.

> Please take a look at these page for more information: -
> https://wiki.debian.org/SimplePackagingTutorial -
> https://wiki.abrahamraji.in/simple-packaging-tutorial.html

I'm familiar with this level of packaging.

IIRC when I tried to look at a "proper" packaging work flow using git
there did not appear to be a "correct" way to do it. Multiple different
approaches only quite briefly documented. Perhaps without much
information about why particular things were done, and seemingly to
expecting knowledge of the other approaches.

Cheers,

Nick.

> [0]: https://salsa.debian.org/ruby-team/atig

[Abraham Raji]

On 20/06/23 14:50, Nick Hastings wrote:
> IIRC when I tried to look at a "proper" packaging work flow using git
> there did not appear to be a "correct" way to do it. Multiple different
> approaches only quite briefly documented. Perhaps without much
> information about why particular things were done, and seemingly to
> expecting knowledge of the other approaches.

In Debian there is no one correct way of doing things. It's just a
matter of taste and preferences. The git packaging conventions used by
the ruby team are tried and tested and in my opinion pretty elegant.

Anyways this seems like a good start.

Best,
Abraham

[Maytham Alsudany]

Hi Nick,

Are you still working on the zig package?
I'd be happy to help you fix any problems that remain.

Kind regards,
Maytham

[Nick Hastings]

Hi,

* Maytham Alsudany <maytha...@gmail.com> [240117 11:37]:

>
> Are you still working on the zig package?
> I'd be happy to help you fix any problems that remain.

I updated the package to Zig 0.10.1. The RFS bug still still open.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1012286

I'll likely not even try packaging 0.11 or newer until it can once again
bootstrap. However I don't expect that to happen anytime soon.

Nick.

[Maytham Alsudany]

Hi Nick,

On Sat, 2024-01-20 at 13:58 +0900, Nick Hastings wrote:
> I updated the package to Zig 0.10.1. The RFS bug still still open.

You should probably try to ping bage and ask them to have a look at your RFS
once more. Software written in Zig keeps popping up, for example, wayprompt.

> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1012286
>
> I'll likely not even try packaging 0.11 or newer until it can once again
> bootstrap. However I don't expect that to happen anytime soon.

Kind regards,
Maytham