docker and building sqitch

[Dan Lynch]

Hello! 

I'm a HUGE sqitch fan, and I'm trying to build a custom docker image that uses sqitch along with other tools I'm working on for building sqitch projects (that I will be sharing soon!). Currently been spending a few days on this, and been banging my head for a while... ;)

Seems that CPAN modules are either changing, and I also kept having issues like this one https://github.com/audreyt/module-signature/issues/30, and then even the Dockerfile https://github.com/sqitchers/sqitch/blob/develop/dev/Dockerfile doesn't build for me... I love Perl and CPAN for this ;) 

so I just started to build from source... which has seemingly been working... I've gotten DBD-Pg to build (yay) and then even sqitch. 

MY ISSUE: However, when I try to run sqitch, it says it cannot locate Path/Class.pm

Can't locate Path/Class.pm in @INC (@INC contains: /usr/local/lib/perl5/5.10.0/x86_64-linux /usr/local/lib/perl5/5.10.0 /usr/local/lib/perl5/site_perl/5.10.0/x86_64-linux /usr/local/lib/perl5/site_perl/5.10.0 .) at /usr/local/lib/perl5/site_perl/5.10.0/App/Sqitch.pm line 11.

BEGIN failed--compilation aborted at /usr/local/lib/perl5/site_perl/5.10.0/App/Sqitch.pm line 11.

Compilation failed in require at /usr/local/bin/sqitch line 12.

BEGIN failed--compilation aborted at /usr/local/bin/sqitch line 12.

Even if I installed the Path:Class module... 

Here is the relevant portion of my Dockerfile, any pointers are appreciated :) if you want to try running the exact box I'm building: https://gist.github.com/pyramation/88c1b1d588e85418dacb55bdbed7f5a4

# add sqitch

ENV TZ UTC

RUN apk add --no-cache postgresql-dev

RUN mkdir -p /perli && \

cd /perli && \

wget https://www.cpan.org/src/5.0/perl-5.10.0.tar.gz && \

tar -xzvf perl-5.10.0.tar.gz

RUN cd /perli/perl-5.10.0 && sh Configure -de && make && make install

RUN mkdir -p /perli && \

cd /perli && \

wget https://www.cpan.org/modules/by-module/DBD/DBD-Pg-3.13.0.tar.gz && \

tar -xzvf DBD-Pg-3.13.0.tar.gz

RUN mkdir -p /perli && \

cd /perli && \

git clone https://github.com/perl5-dbi/dbi && \

cd dbi && perl Makefile.PL && make && make install

RUN apk add --no-cache postgresql-client

RUN mkdir -p /perli && \

cd /perli && \

git clone https://github.com/Test-More/test-more && \

cd test-more && perl Makefile.PL && make && make install

RUN cd /perli/DBD-Pg-3.13.0 && perl Makefile.PL && make && make install

# add sqitch

ENV TZ UTC

RUN curl -L https://cpanmin.us/ -o cpanm

RUN chmod +x cpanm

RUN cp cpanm /home/launchql/bin/cpanm

RUN PERL_MM_USE_DEFAULT=1 cpan App::cpanminus

RUN PERL_MM_USE_DEFAULT=1 cpan Menlo::CLI::Compat

RUN PERL_MM_USE_DEFAULT=1 cpan Path::Class

RUN PERL_MM_USE_DEFAULT=1 cpan Module:Build

RUN mkdir -p /perli && \

cd /perli && \

wget https://cpan.metacpan.org/authors/id/D/DW/DWHEELER/App-Sqitch-v1.1.0.tar.gz && \

tar -xzvf App-Sqitch-v1.1.0.tar.gz

RUN cd /perli/App-Sqitch-v1.1.0 && \

perl ./Build.PL

RUN cd /perli/App-Sqitch-v1.1.0 && \

./Build installdeps

RUN cd /perli/App-Sqitch-v1.1.0 && \

./Build

RUN cd /perli/App-Sqitch-v1.1.0 && \

./Build install

[Dan Lynch]

I managed to get it all to compile after building Path Class and PathTools from source, and I found a new easter egg when running sqitch init

bash-5.0# sqitch init testproj

"init" is not a valid command

"testproj" is not a valid command

UTF-16 surrogate 0xd800 at /usr/local/lib/perl5/5.10.0/Pod/Simple/BlackBox.pm line 67.

Usage

      sqitch [--etc-path | --help | --man | --version]

      sqitch <command> [--chdir <path>] [--no-pager] [--quiet] [--verbose]

             [<command-options>] [<args>]

[Dan Lynch]

managed to get it to work finally! pyramation/sqitch 10.21.0-alpine3.11 147MB pyramation/sqitch 12.18.2-alpine3.11 152MB pyramation/sqitch 14.5.0-alpine3.11 180MB It's hundreds of megabytes less IF YOU ONLY NEED POSTGRES and can use ALPINE: * I ported over the image found using slim-stretch to alpine, and removed all but postgres requirements * Also, if you don't need node, you can just replace that with the base alpine3.11 image FROM node:14.5.0-alpine3.11 AS sqitch-build # Install system dependencies. WORKDIR /work ARG VERSION RUN mkdir -p /usr/share/man/man1 /usr/share/man/man7 \ && apk add --no-cache --virtual .build-deps \ alpine-sdk \ perl-dev \ curl \ postgresql-dev \ perl-dbd-pg \ tzdata \ gnupg \ && apk add --no-cache perl \ && curl -LO https://www.cpan.org/authors/id/D/DW/DWHEELER/App-Sqitch-v$VERSION.tar.gz \ && mkdir src \ && tar -zxf App-Sqitch-v$VERSION.tar.gz --strip-components 1 -C src # Install cpan and build dependencies. ENV PERL5LIB /work/local/lib/perl5 RUN curl -sL --compressed https://git.io/cpm > cpm && chmod +x cpm \ && ./cpm install -L local --verbose --no-test ExtUtils::MakeMaker \ && ./cpm install -L local --verbose --no-test --with-recommends \ --with-configure --cpanfile src/dist/cpanfile ENV TZ UTC RUN cp /usr/share/zoneinfo/UTC /etc/localtime && \ echo UTC > /etc/timezone # Build, test, bundle, prune. WORKDIR /work/src RUN perl Build.PL --quiet --install_base /app --etcdir /etc/sqitch \ --config installman1dir= --config installsiteman1dir= --config installman3dir= --config installsiteman3dir= \ --with postgres \ && ln -s /usr/include/ibase.h \ && ./Build test && ./Build bundle \ && rm -rf /app/man \ && find /app -name '*.pod' | grep -v sqitch | xargs rm -rf RUN apk del .build-deps ################################################################################ # Copy to the final image without all the build stuff. FROM node:14.5.0-alpine3.11 AS sqitch # Install runtime system dependencies and remove unnecesary files. RUN mkdir -p /usr/share/man/man1 /usr/share/man/man7 \ && apk add --no-cache --virtual .build-deps2 \ # perl \ ca-certificates \ # libpq5 \ # postgresql-client \ && apk add --no-cache \ perl \ postgresql-client \ # ca-certificates \ # libpq5 \ && mkdir -p /etc/pki/tls && ln -s /etc/ssl/certs /etc/pki/tls/ \ && rm -rf /var/cache/apt/* /var/lib/apt/lists/* /usr/bin/mysql?* \ && rm -rf /plibs /man /usr/share/man /usr/share/doc /usr/share/postgresql \ /usr/share/nano /etc/nanorc \ && find / -name '*.pod' | grep -v sqitch | xargs rm -rf \ && find / -name '*.ph' -delete \ && find / -name '*.h' -delete # Copy the app and config from the build image. COPY --from=sqitch-build /app . COPY --from=sqitch-build /etc/sqitch /etc/sqitch/ RUN apk del .build-deps2 # Set up environment, entrypoint, and default command. ENV LESS=-R LC_ALL=C.UTF-8 LANG=C.UTF-8 SQITCH_EDITOR=vi SQITCH_PAGER=less ENTRYPOINT ["/bin/sh"]

-- You received this message because you are subscribed to the Google Groups "Sqitch Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqitch-users+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sqitch-users/1add1ad1-c900-47f4-9441-d5a66d08e0efo%40googlegroups.com. -- Dan Lynch (734) 657-4483

[David E. Wheeler]

On Jul 9, 2020, at 21:13, Dan Lynch <pyram...@gmail.com> wrote:

> managed to get it to work finally!
>
> pyramation/sqitch
> 10.21.0-alpine3.11 147MB
> pyramation/
> sqitch
> 12.18.2-alpine3.11 152MB
> pyramation/
> sqitch 14.5.0-alpine3.11 180MB
>
> It's hundreds of megabytes less IF YOU ONLY NEED POSTGRES and can use ALPINE:
>
> * I ported over the image found using slim-stretch to alpine, and removed all but postgres requirements
> * Also, if you don't need node, you can just replace that with the base alpine3.11 image

Nice! I alway intended to get back to an Alpine-based image, but never got ’round to it. Would you be interested in integrating your work into the project’s Docker distributions?

https://hub.docker.com/r/sqitch/sqitch
http://github.com/sqitchers/docker-sqitch/

This variant uses debian:slim (needed to include Firebase support at the time I made it), and I’ve thought for a while it’d be useful to script things to:

1. Build separate images for each engine
2. Build on both debian:slim and Alpine for the engines Alpine supports

Best,

David

[Dan Lynch]

Sure, I'd love to. Are you thinking a separate image for each engine?  If that's the case, I suppose a naming convention that would be something like sqitch/sqitch:1.1.0-postgres-stretch-slim sqitch/sqitch:1.1.0-postgres-alpine ... sqitch/sqitch:1.1.0-mysql-alpine

Dan Lynch
(734) 657-4483

[David E. Wheeler]

On Jul 13, 2020, at 19:25, Dan Lynch <pyram...@gmail.com> wrote:

> Sure, I'd love to. Are you thinking a separate image for each engine?

Yeah. I’m thinking :latest would be the same as today (Debian slim with all four OSS engines), plus separate images for each engine.

> If that's the case, I suppose a naming convention that would be something like
>
> sqitch/sqitch:1.1.0-postgres-stretch-slim
> sqitch/sqitch:1.1.0-postgres-alpine
> ...
> sqitch/sqitch:1.1.0-mysql-alpine

Hrm. Not sure whether the engine or the OS should go first. Wonder if there are any precedents?

D