building data.tar.gz for *.ipk files with less fuss

[Joshua DeWeese]

I thought this might be useful for others and worth bringing up here. It seems like nobody has figured out a decent way to build the data.tar.gz file inside of pkg files without alot of fuss (at least from my searching when trying to make pkgs for angstrom and poky). So I thought I'd take a whack at it. Attached is a script which builds tar files with the desired permissions/ownership and a sample control file showing how the output is controlled. It was made in about an hour, and I am already thinking of a better way to implement it in c or python where the tar file could be constructed directly using tar libraries rather in in /tmp as I have done here in shell script. Let me know what you think. If there is interest I will start a proper project.

[Alejandro del Castillo]

Hi Josh,

Have you looked at opkg-build?

http://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils/

Those are the tools used by OE/Yocto to package software up when opkg is the
package manger backend. It expects a dir with a tree with the files that will go
into the package. A tool like yours can help create the image, but in my
experience, people usually create custom scripts since the file source varies
from build system to build system.

my 2 cents...

--
Cheers,

Alejandro

[Joshua DeWeese]

Thanks for the input. I investigated opkg-build (actually I ipkg-build), but it seemed to miss key functionality for what I wanted (assign file ownership to users of the target system which don't exist on the build host). Also it seemed like many such tools used to make deb/ipkg/opkg files assume:
- building pkgs and software are done as separate projects, and/or by separate groups of people
- source is distributed as tar-balls and patching is done with things like patch or quilt

My goal is to be able to fork a project, patch via git (or some other such DVCS), and add the necessary bits to create a opkg/ipkg/deb file. From there I only simply need to merge or rebase (or even better push my changes upstream) to keep things up to date. I found a similar such tool to mine mentioned in a wiki, but could never actually find the tool itself anywhere, which prompted me to build this one.

I originally sought to build the entire pkg file using the Makefile already in my project, but found building a data dir (to be tared up) with complex permissions a little difficult. Next I thought to just build a script to do it called by make, but then I would have to maintain data.tar.gz's dependency list in my Makefile and in the shell script. The next logical thought was to make a more general script taking a control file that expressed the dependencies that both it and make could use. Now by adding the following lines to my Makefile:

-include $(shell find -name "*.dep")

angstrom/can_logger.ipk:\
 angstrom/debian-binary\
 angstrom/control.tar.gz\
 angstrom/data.tar.gz
    ar rc $@ $?

angstrom/control.tar.gz: angstrom/control
    tar czf $@ $<

angstrom/data.tar.gz: angstrom/data.cntrl
    ./mktar $@ $<
    @ echo -n "$@:" > $@.dep
    @ awk '!/^-type-/ && !/^($$|[[:space:]]*#)/ {printf " angstrom/" $$1}'\
       angstrom/data.cntrl >> $@.dep

I have a very easy way to rebuild the pkg whenever I change source code, the control file, or any additional config/script/data files I wish to include for distribution. Of course it's always possible I have create a solution to a problem that doesn't exist, but it seemed handy to me.

For what its worth, I have updated t script to tolerate comments and empty lines, and have attached the update and a better sample control file.

[Alejandro del Castillo]

On 06/24/2016 02:26 PM, Joshua DeWeese wrote:
> Thanks for the input. I investigated opkg-build (actually I ipkg-build), but it
> seemed to miss key functionality for what I wanted (assign file ownership to
> users of the target system which don't exist on the build host). Also it seemed
> like many such tools used to make deb/ipkg/opkg files assume:
> - building pkgs and software are done as separate projects, and/or by separate
> groups of people
> - source is distributed as tar-balls and patching is done with things like patch
> or quilt

You are right, tools like OpenEmbedded have code in charge of fetching the
source, building it, then stagging it for packaging (opkg-build). To get proper
file ownership, OpenEmbedded in particular uses pseudo, which is similar to
fakeroot.

yeah, that is straightforward indeed, for people not using OpenEmbedded a tool
like this one can be really handy.

> For what its worth, I have updated t script to tolerate comments and empty
> lines, and have attached the update and a better sample control file.

looks good to me, I will shoot you an email once the opkg homepage is ready as I
envision it will have a tools page. We can add this tool there.

I am hoping I can get to writing a homepage at some point this summer, will keep
you updated.

Thanks!

--
Cheers,

Alejandro