Failing installation of pair/semver extensions

[Daniele Varrazzo]

Hello,

I'm trying to set up some smoke tests before releasing pgxn client
1.0. I haven't heard of any success/failure with the client in the
latest weeks, and this isn't enough for me as QA. So it looks I will
have to do myself.

The tests are currently in the smoketest branch of my repos: I want
them to be fully automatic, so they start downloading and compiling
postgres and setting up a cluster. See
<https://github.com/dvarrazzo/pgxnclient/tree/smoketest/dbtest>.
Currently there are the scripts:

- make_env.sh (will compile postgres and create a cluster)
- pg_ctl.sh start (will start the test cluster on port 15432)
- test_WHAT.sh (will install, test, uninstall the extension WHAT)
- pg_ctl.sh stop (guess what)

I'm having problems in installing both pair and semver extensions in
PG 9.1.1, not only with the client but manually as well: "make
install" returns:

...
/bin/sh /home/piro/dev/pgxnclient/dbtest/postgresql-9.1.1/root/lib/pgxs/src/makefiles/../../config/install-sh
-c -m 755 src/semver.so
'/home/piro/dev/pgxnclient/dbtest/postgresql-9.1.1/root/lib/'
/bin/sh /home/piro/dev/pgxnclient/dbtest/postgresql-9.1.1/root/lib/pgxs/src/makefiles/../../config/install-sh
-c -m 644 '/home/piro/dev/pgxnclient/dbtest/postgresql-9.1.1/root/share/doc/extension/'
/home/piro/dev/pgxnclient/dbtest/postgresql-9.1.1/root/lib/pgxs/src/makefiles/../../config/install-sh:
no input file specified.
make: *** [install] Error 1

It looks like there is a blank file in the list of files to install.
Other extensions work as expected: pgmp can be installed, and I've
sampled a couple other ones which installed successfully.

Maybe it's a stupid thing, in which case I apologise: I'm starting
being a little bit tired... Dave, do you have any idea why these
extensions fail being installed?

Thank you very much.

-- Daniele

[David E. Wheeler]

On Nov 10, 2011, at 6:33 PM, Daniele Varrazzo wrote:

> I'm trying to set up some smoke tests before releasing pgxn client
> 1.0. I haven't heard of any success/failure with the client in the
> latest weeks, and this isn't enough for me as QA. So it looks I will
> have to do myself.
>
> The tests are currently in the smoketest branch of my repos: I want
> them to be fully automatic, so they start downloading and compiling
> postgres and setting up a cluster. See
> <https://github.com/dvarrazzo/pgxnclient/tree/smoketest/dbtest>.
> Currently there are the scripts:
>
> - make_env.sh (will compile postgres and create a cluster)
> - pg_ctl.sh start (will start the test cluster on port 15432)
> - test_WHAT.sh (will install, test, uninstall the extension WHAT)
> - pg_ctl.sh stop (guess what)

Thanks for doing this. Really awesome.

> I'm having problems in installing both pair and semver extensions in
> PG 9.1.1, not only with the client but manually as well: "make
> install" returns:
>
> ...
> /bin/sh /home/piro/dev/pgxnclient/dbtest/postgresql-9.1.1/root/lib/pgxs/src/makefiles/../../config/install-sh
> -c -m 755 src/semver.so
> '/home/piro/dev/pgxnclient/dbtest/postgresql-9.1.1/root/lib/'

That looks fine.

> /bin/sh /home/piro/dev/pgxnclient/dbtest/postgresql-9.1.1/root/lib/pgxs/src/makefiles/../../config/install-sh
> -c -m 644 '/home/piro/dev/pgxnclient/dbtest/postgresql-9.1.1/root/share/doc/extension/'

Hrm. Weird.

> It looks like there is a blank file in the list of files to install.
> Other extensions work as expected: pgmp can be installed, and I've
> sampled a couple other ones which installed successfully.
>
> Maybe it's a stupid thing, in which case I apologise: I'm starting
> being a little bit tired... Dave, do you have any idea why these
> extensions fail being installed?

I don't. I just confirmed the same bug here…

And am getting the bug when I do it all manually (as opposed to using your client)…

Ah, here we go. My Makefile has this line:

DOCS = $(wildcard doc/*.txt)

However:

> ll doc
total 24
-rw-r--r--@ 1 david staff 8.2K May 12 2011 semver.mmd

I renamed it and forgot to update the Makefile. I've since fixed it in my repos, but the fix is not released.

So, my fault, and apologies for the wild goose chase. I'll get new versions out (working on a new version of pgTAP right now, so I'm in that space anyway).

Thanks,

David

PS: I go by “David”, not “Dave”. :-)

[David E. Wheeler]

On Nov 10, 2011, at 9:18 PM, David E. Wheeler wrote:

> So, my fault, and apologies for the wild goose chase. I'll get new versions out (working on a new version of pgTAP right now, so I'm in that space anyway).

I've now released the fixed versions. Let me know if you encounter any problems.

Thanks!

David

[Daniele Varrazzo]

On Fri, Nov 11, 2011 at 5:18 AM, David E. Wheeler <da...@kineticode.com> wrote:

> PS: I go by “David”, not “Dave”. :-)

Oops sorry... Told you I was tired :) (The Dave/David namespace is by
far the most saturated in my office)

Thank you for looking into the issue!

-- Daniele

[Daniele Varrazzo]

pair has still a problem: it defines MODULES, so the pgxs makefile
fails because it doesn't find a .so

-- Daniele

[Dickson dos Santos Guedes]

Just commenting the MODULES line works here.

[]s
Guedes

[David E. Wheeler]

On Nov 11, 2011, at 2:45 AM, Dickson dos Santos Guedes wrote:

>> pair has still a problem: it defines MODULES, so the pgxs makefile
>> fails because it doesn't find a .so
>
> Just commenting the MODULES line works here.

Huh. It works for me as-is, and I assumed it would everywhere else, too.

Er…I see, it fails on `make install`. My mistake for not trying that before. Will fix.

David

[David E. Wheeler]

On Nov 11, 2011, at 9:51 AM, David E. Wheeler wrote:

>> Just commenting the MODULES line works here.
>
> Huh. It works for me as-is, and I assumed it would everywhere else, too.
>
> Er…I see, it fails on `make install`. My mistake for not trying that before. Will fix.

Fixed and released v0.1.5. I also updated the How-To:

https://github.com/pgxn/pgxn-manager/commit/914136b9bc140a3fddf724fcd96ffa9e43cbfe38

Thanks!

David

[Daniele Varrazzo]

On Fri, Nov 11, 2011 at 6:04 PM, David E. Wheeler <da...@kineticode.com> wrote:
> On Nov 11, 2011, at 9:51 AM, David E. Wheeler wrote:
>
>>> Just commenting the MODULES line works here.
>>
>> Huh. It works for me as-is, and I assumed it would everywhere else, too.
>>
>> Er…I see, it fails on `make install`. My mistake for not trying that before. Will fix.
>
> Fixed and released v0.1.5.

Thank you. May it be that it's only with the latest PG versions that
the makefile has started spewing these extra command? I'd assume we
have tested and tested pairs in the previous months and nothing like
this has ever come out.

Another curious thing is that pairs fails the regression test (I'm not
asking to fix this, it's just something I've noticed and I wonder
why), with a diff such as:

*** /tmp/tmpzxBV0_/pair-0.1.5/test/expected/base.out 2011-11-11
18:37:27.610681183 +0000
--- /tmp/tmpzxBV0_/pair-0.1.5/results/base.out 2011-11-11
18:37:28.694678621 +0000
***************
***************
*** 11,20 ****
------------
(1,12)
(12.3,foo)
- (HEY,bar)
(foo,1)
(foo,bar)
(foo,woah)
(ick,foo)
(7 rows)

--- 11,20 ----
------------
(1,12)
(12.3,foo)
(foo,1)
(foo,bar)
(foo,woah)
+ (HEY,bar)
(ick,foo)
(7 rows)

Looks like "expected" was created with a database collating to C,
while the latter is a more human one. Don't know if such details can
really be controlled.

-- Daniele

[David E. Wheeler]

On Nov 11, 2011, at 11:01 AM, Daniele Varrazzo wrote:

>> Fixed and released v0.1.5.
>
> Thank you. May it be that it's only with the latest PG versions that
> the makefile has started spewing these extra command? I'd assume we
> have tested and tested pairs in the previous months and nothing like
> this has ever come out.

Yeah, but that was likely before I converted the Makefile to build the 9.1-style extension files. I did that only a few months ago, so I’m not surprised some things broke.

Oy. I should update the test to be collation-neutral. Thanks!

David

[rdunklau]

Hello.

Sorry if I'm late to the party, but I tested the latest pgxn client
(devel branch) on our extension using the following script, and it
worked like a charm.
Feel free to add it to your smoketests if you want.

#!/bin/bash
source env.sh
export EXTENSION=multicorn
export LEVEL=--testing
source _test_template.sh

Our extension installs a bunch of python modules in the global system
site_packages (via a "python setup.py install" in the Makefile), and I
wondered if you had any advice on how to manage the problem of
permissions.

Thank you very much.

--

Ronan Dunklau

[Daniele Varrazzo]

On Tue, Nov 15, 2011 at 8:33 AM, rdunklau <rdun...@gmail.com> wrote:
> Hello.
>
> Sorry if I'm late to the party, but I tested the latest pgxn client
> (devel branch) on our extension using the following script, and it
> worked like a charm.
> Feel free to add it to your smoketests if you want.
>
> #!/bin/bash
> source env.sh
> export EXTENSION=multicorn
> export LEVEL=--testing
> source _test_template.sh
>
> Our extension installs a bunch of python modules in the global system
> site_packages (via a "python setup.py install" in the Makefile), and I
> wondered if you had any advice on how to manage the problem of
> permissions.

Thank you. I think I had already tried installing your extension (I've
tested some of the latest in the network) and it worked fine (I've
also just find a bug in the client testing it again: the --pg_config
path should be made absolute before something else changes the pwd...)

Testing it in a repeatable way is a little bit of a problem, because
as you say install wants to put stuff in some system directory: used
normally "pgxn install" will ask for "sudo", but it wouldn't be
repeatable. I've tested yours by changing the install command to

python_code: setup.py
$(PYEXEC) ./setup.py install $(PYINSTALL_ARGS)

and passing PYINSTALL_ARGS="--install-lib=`pwd`/lib" to the
environment, in order to put the lib somewhere locally.

I've also seen you have several magic arguments to be used to compile
the python extension: I suggest you to use the output of python_config
instead to have the script more portable. take a look at this makefile
for example: <http://code.google.com/p/py-setproctitle/source/browse/Makefile>:
it is enough to pass a PYCONFIG executable to get all the other
parameters.

Cheers,

-- Daniele