"pkg: pkg is not installed" even after installing it

[Perry Hutchison]

I am in the process of setting up a FreeBSD 10 system, which will be
my first experience with pkg(8). In the interest of minimizing the
need to start over after things go badly, I'm trying to carefully
explore how pkg behaves, on an older system (FreeBSD 8) that does
not normally use it:

Mount the 10.2 memstick image read-only (on FreeBSD 8):
# mdconfig -a -t vnode -u 0 -f /path/to/FreeBSD-10.2-RELEASE-i386-memstick.img
md0
# ll -d /dev/md0*
crw-r----- 1 root operator 0, 175 Nov 30 01:46 /dev/md0
crw-r----- 1 root operator 0, 176 Nov 30 01:46 /dev/md0p1
crw-r----- 1 root operator 0, 178 Nov 30 01:46 /dev/md0p2
crw-r----- 1 root operator 0, 179 Nov 30 01:46 /dev/md0p3
# file -s /dev/md0*
/dev/md0: x86 boot sector; ...
/dev/md0p1: data
/dev/md0p2: Unix Fast File system ...
/dev/md0p3: data
# gpart show md0
=> 3 1346448 md0 GPT (657M)
3 32 1 freebsd-boot (16K)
35 1344368 2 freebsd-ufs (656M)
1344403 2048 3 freebsd-swap (1.0M)
# mount -r -t ufs /dev/md0p2 /mnt

Chroot into the image and run "pkg -N". Yes, this is running a newer
userland on an older kernel, so it might fail with an unimplemented
system call if the commands I run use kernel facilities not present
in the older kernel. However I don't actually get any error messages.
# chroot -u 105 -g 0 -G 105,0,5,20,25 /mnt which pkg
/usr/sbin/pkg
# chroot -u 105 -g 0 -G 105,0,5,20,25 /mnt ldd usr/sbin/pkg
usr/sbin/pkg:
libarchive.so.6 => /usr/lib/libarchive.so.6 (0x28071000)
libfetch.so.6 => /usr/lib/libfetch.so.6 (0x2810f000)
libucl.so.1 => /usr/lib/private/libucl.so.1 (0x28121000)
libsbuf.so.6 => /lib/libsbuf.so.6 (0x28133000)
libssl.so.7 => /usr/lib/libssl.so.7 (0x28136000)
libcrypto.so.7 => /lib/libcrypto.so.7 (0x28193000)
libm.so.5 => /lib/libm.so.5 (0x28323000)
libc.so.7 => /lib/libc.so.7 (0x2834a000)
libz.so.6 => /lib/libz.so.6 (0x284bf000)
libbz2.so.4 => /usr/lib/libbz2.so.4 (0x284d3000)
liblzma.so.5 => /usr/lib/liblzma.so.5 (0x284e3000)
libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x28506000)
# chroot -u 105 -g 0 -G 105,0,5,20,25 /mnt pkg -N
pkg: pkg is not installed
(as expected)

Mount a tmpfs on the (otherwise read-only) /usr/local, and copy
pkg.tgz there, so it will be visible in the chroot environment.
# set MD4UL=`mdconfig -a -t swap -s 30m`
# echo /dev/$MD4UL
/dev/md1
# newfs -b 4096 -f 512 -m 0 -n /dev/$MD4UL
Warning: changing optimization to space because minfree is less than 8%
/dev/md1: 30.0MB (61440 sectors) block size 4096, fragment size 512
using 4 cylinder groups of 7.50MB, 1921 blks, 3856 inodes.
super-block backups (for fsck -b #) at:
144, 15512, 30880, 46248
# tunefs -p /dev/$MD4UL
...
tunefs: maximum blocks per file in a cylinder group: (-e) 512
...
tunefs: optimization preference: (-o) space
...
# tunefs -e 1921 -o time /dev/$MD4UL
tunefs: maximum blocks per file in a cylinder group changes from 512 to 1921
tunefs: optimization preference changes from space to time
tunefs: should optimize for space with minfree < 8%
# tunefs -p /dev/$MD4UL
...
tunefs: maximum blocks per file in a cylinder group: (-e) 1921
...
tunefs: optimization preference: (-o) time
...
# mount /dev/$MD4UL /mnt/usr/local
# mkdir /mnt/usr/local/pkg
# chmod 777 /mnt/usr/local /mnt/usr/local/pkg
# cp -p /path/to/pkg.txz* /mnt/usr/local/pkg
# ll /mnt/usr/local/pkg
total 2445
-r--r--r-- 1 perryh perryh 2487404 Jul 29 18:13 pkg.txz
-r--r--r-- 1 perryh perryh 727 Jul 30 17:59 pkg.txz.sig
# chroot -u 0 -g 0 -G 105,0,5,20,25 /mnt pkg add -f /usr/local/pkg/pkg.tgz
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done

Note, no error messages. However:
# chroot -u 105 -g 0 -G 105,0,5,20,25 /mnt pkg -N
pkg: pkg is not installed

Why does it claim that pkg is not installed, when it just got through
installing it? Or, if the install failed part way through, why was there
no error message? And yes, the "pkg add" really did install something --
/mnt/usr/local/sbin did not exist before running "pkg add" -- and what it
installed is at least partly functional:
# ls -lR /mnt/usr/local
total 1
drwxrwxrwx 2 root wheel 512 Jan 9 18:11 pkg
drwxr-xr-x 2 perryh wheel 512 Jan 9 18:19 sbin
/mnt/usr/local/pkg:
total 2445
-r--r--r-- 1 perryh perryh 2487404 Jul 29 18:13 pkg.txz
-r--r--r-- 1 perryh perryh 727 Jul 30 17:59 pkg.txz.sig
/mnt/usr/local/sbin:
total 4472
-rwxr-xr-x 1 root wheel 4559004 Jul 29 18:13 pkg-static

# chroot -u 105 -g 0 -G 105,0,5,20,25 /mnt pkg-static help
Usage: pkg [-v] [-d] [-l] [-N] [-j <jail name or id>|-c <chroot path>|-r <rootdir>] [-C <configuration file>] [-R <repo config dir>] [-o var=value] [-4|-6] <command> [<args>]
Global options supported:
-d Increment debug level
-j Execute pkg(8) inside a jail(8)
-R Execute pkg(8) using relocating installation to <rootdir>
-c Execute pkg(8) inside a chroot(8)
-C Use the specified configuration file
-R Directory to search for individual repository configurations
-l List available commands and exit
-v Display pkg(8) version
-N Test if pkg(8) is activated and avoid auto-activation
-o Override configuration option from the command line
-4 Only use IPv4
-6 Only use IPv6

Commands supported:
add Compatibility interface to install a package
alias List the command line aliases
annotate Add, modify or delete tag-value style annotations on packages
audit Reports vulnerable packages
autoremove Removes orphan packages
backup Backs-up and restores the local package database
check Checks for missing dependencies and database consistency
clean Cleans old packages from the cache
config Display the value of the configuration options
convert Convert database from/to pkgng
create Creates software package distributions
delete Deletes packages from the database and the system
fetch Fetches packages from a remote repository
help Displays help information
info Displays information about installed packages
install Installs packages from remote package repositories and local archives
lock Locks package against modifications or deletion
plugins Manages plugins and displays information about plugins
query Queries information about installed packages
register Registers a package into the local database
remove Deletes packages from the database and the system
repo Creates a package repository catalogue
rquery Queries information in repository catalogues
search Performs a search of package repository catalogues
set Modifies information about packages in the local database
ssh Package server (to be used via ssh)
shell Opens a debug shell
shlib Displays which packages link against a specific shared library
stats Displays package database statistics
unlock Unlocks a package, allowing modification or deletion
update Updates package repository catalogues
updating Displays UPDATING information for a package
upgrade Performs upgrades of packaged software distributions
version Displays the versions of installed packages
which Displays which package installed a specific file

For more information on the different commands see 'pkg help <command>'.
_______________________________________________
freebsd-...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questi...@freebsd.org"

[Matthew Seaman]

On 10/01/2016 04:09, Perry Hutchison wrote:
> Why does it claim that pkg is not installed, when it just got through
> installing it? Or, if the install failed part way through, why was there
> no error message? And yes, the "pkg add" really did install something --
> /mnt/usr/local/sbin did not exist before running "pkg add" -- and what it
> installed is at least partly functional:
> # ls -lR /mnt/usr/local
> total 1
> drwxrwxrwx 2 root wheel 512 Jan 9 18:11 pkg
> drwxr-xr-x 2 perryh wheel 512 Jan 9 18:19 sbin
> /mnt/usr/local/pkg:
> total 2445
> -r--r--r-- 1 perryh perryh 2487404 Jul 29 18:13 pkg.txz
> -r--r--r-- 1 perryh perryh 727 Jul 30 17:59 pkg.txz.sig
> /mnt/usr/local/sbin:
> total 4472
> -rwxr-xr-x 1 root wheel 4559004 Jul 29 18:13 pkg-static

pkg(7) -- the shim that comes with the system -- is what produces the
'pkg is not installed' message. That's the result of checking that
${LOCALBASE}/sbin/pkg exists and is executable. ${LOCALBASE} defaults
to /usr/local.

It seems you have only /usr/local/sbin/pkg-static so your installation
of pkg(8) -- the actual pkg software -- looks to have fallen over
somewhere in the middle. pkg-static is pulled out of the pkg.txz
tarball and installed first, then that copy of pkg-static is used to do
a full install of the entire set of stuff comprising pkg(8) from the
pkg.txz tarball. That includes the dynamically linked pkg binary and
numerous man pages plus various other bits and pieces.

What it seems you are missing is are writable /var/db/pkg/ and
/var/cache/pkg directories -- this are where pkg(8) holds its database
of what packages have been installed, and where it holds copies of the
package tarballs, respectively.

It's definitely a bug that your procedure fails without any sort of
error message. There should be load complaints from pkg(8) that it
cannot initialize the database, but somehow those are not making it
through to your console. You should see some messages if you try running:

/usr/local/sbin/pkg-static add /usr/local/pkg/pkg.txz

and that should show exactly what pkg(8) is having trouble with.

Cheers,

Matthew

[Perry Hutchison]

Matthew Seaman <mat...@freebsd.org> wrote:
> On 10/01/2016 04:09, Perry Hutchison wrote:
> > Why does it claim that pkg is not installed, when it just got through
> > installing it? Or, if the install failed part way through, why was

> > there no error message? ...

>
> What it seems you are missing is are writable /var/db/pkg/ and
> /var/cache/pkg directories -- this are where pkg(8) holds its database
> of what packages have been installed, and where it holds copies of the
> package tarballs, respectively.
>
> It's definitely a bug that your procedure fails without any sort of
> error message. There should be load complaints from pkg(8) that it
> cannot initialize the database, but somehow those are not making it
> through to your console. You should see some messages if you try
> running:
>
> /usr/local/sbin/pkg-static add /usr/local/pkg/pkg.txz
>
> and that should show exactly what pkg(8) is having trouble with.

That doesn't produce any output, either. However, after mounting
tmpfs on /var/db/pkg and /var/cache, I got:

# chroot -u 0 -g 0 -G 105,0,5,20,25 /mnt pkg-static add /usr/local/pkg/pkg.txz
Installing pkg-1.5.4...
Extracting pkg-1.5.4: 100%
ldconfig: mkstemp(/var/run/ld-elf.so.hints.6wHhnd): Read-only file system
pkg-static: POST-INSTALL script failed
Message for pkg-1.5.4:
If you are upgrading from the old package format, first run:

# pkg2ng

# chroot -u 0 -g 0 -G 105,0,5,20,25 /mnt pkg -N
pkg: 1 packages installed

This time it installed a bunch of stuff in /usr/local and a
"local.sqlite" in /var/db/pkg -- but nothing in /var/cache,
so maybe that isn't needed when installing from a tarball
that is already local.

[Matthew Seaman]

On 11/01/2016 03:10, Perry Hutchison wrote:
> That doesn't produce any output, either. However, after mounting
> tmpfs on /var/db/pkg and /var/cache, I got:

Hmmm... that's certainly a bug. Could you open an issue at
https://github.com/freebsd/pkg/issues please, explaining exactly what
you did and what happened (or didn't as the case may be)? pkg(8)
shouldn't die quietly in those or any circumstances.

> # chroot -u 0 -g 0 -G 105,0,5,20,25 /mnt pkg-static add /usr/local/pkg/pkg.txz
> Installing pkg-1.5.4...
> Extracting pkg-1.5.4: 100%
> ldconfig: mkstemp(/var/run/ld-elf.so.hints.6wHhnd): Read-only file system
> pkg-static: POST-INSTALL script failed
> Message for pkg-1.5.4:
> If you are upgrading from the old package format, first run:
>
> # pkg2ng
>
> # chroot -u 0 -g 0 -G 105,0,5,20,25 /mnt pkg -N
> pkg: 1 packages installed
>
> This time it installed a bunch of stuff in /usr/local and a
> "local.sqlite" in /var/db/pkg -- but nothing in /var/cache,
> so maybe that isn't needed when installing from a tarball
> that is already local.

That looks to have worked. Yes, if you use pkg add to install from a
tarball directly on your system already, /var/cache/pkg isn't needed.
However, it will be necessary as soon as you do anything like 'pkg
update' or 'pkg install'.

Curious as to why you're using an older version of pkg there -- we're up
to 1.6.2 nowadays.

Cheers,

Matthew

[Perry Hutchison]

Matthew Seaman <mat...@freebsd.org> wrote:

> On 11/01/2016 03:10, Perry Hutchison wrote:
> > That doesn't produce any output, either. However, after mounting
> > tmpfs on /var/db/pkg and /var/cache, I got:
>
> Hmmm... that's certainly a bug. Could you open an issue at
> https://github.com/freebsd/pkg/issues please, explaining exactly
> what you did and what happened (or didn't as the case may be)?
> pkg(8) shouldn't die quietly in those or any circumstances.

I will try to get to it later this week.

> > # chroot -u 0 -g 0 -G 105,0,5,20,25 /mnt pkg-static add /usr/local/pkg/pkg.txz
> > Installing pkg-1.5.4...
> > Extracting pkg-1.5.4: 100%
> > ldconfig: mkstemp(/var/run/ld-elf.so.hints.6wHhnd): Read-only file system
> > pkg-static: POST-INSTALL script failed
> > Message for pkg-1.5.4:
> > If you are upgrading from the old package format, first run:
> >
> > # pkg2ng
> >
> > # chroot -u 0 -g 0 -G 105,0,5,20,25 /mnt pkg -N
> > pkg: 1 packages installed
> >
> > This time it installed a bunch of stuff in /usr/local and a

> > "local.sqlite" in /var/db/pkg ...

>
> That looks to have worked.

Any concern about the POST-INSTALL failure due to /var/run being
read-only? (That one is not as simple to fix as /var/cache and
/var/db/pkg, since the memstick's /var/run is not empty.)

> Curious as to why you're using an older version of pkg there --
> we're up to 1.6.2 nowadays.

It is the version I downloaded from
pkg.freebsd.org/freebsd:10:x86:32/release_2/Latest
on 12/21 (when I started on this exercise).

If I'm remembering correctly, .../release_2 appeared to be the area
which most nearly corresponded to what would have been current when
10.2 was released. I have gotten burned before by trying to install
package sets that are newer than the release on which they are being
installed -- sticking with what presumably worked at release time
seems safer, at least to get an initial, working configuration.

[JosC]

In een bericht van 11-1-2016 11:15:

> I will try to get to it later this week.

Perhaps a stupid remark but try periodic weekly first and check then
again...

Best regards,
Jos Chrispijn