pkg update broke

[William F. Dudley Jr.]

I am running "FreeBSD 14.3-RELEASE-p16 amd64".

I did "freebsd-update fetch" followed by "freebsd-update install", and
the power (and my UPS) failed, I think, during the "freebsd-update
install" step.

Now, when I run "pkg update", I get this:

pkg: Failed to fetch
https://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly/meta.conf: SSL peer
certificate or SSH remote key was not OK

And identical lines for meta.txz, data.pkg, data.tzst, etc.

I have a nearly identical machine, and that machine can do pkg update
successfully.

I have tried to figure out where pkg stores the SSL certificate or SSH
key that it's complaining about but have been unsuccessful. Running
"man pkg", "man pkg.conf", scanning output of "pkg -vv", running "pkg -f
bootstrap" all failed to help.

What is broken and how can I fix it?

Thanks,
Bill Dudley

[William F. Dudley Jr.]

Dan,

No, clock is correct. I saw that problem mentioned in a google search,
and checked it before I wrote to the list.

Thanks,
Bill Dudley

On 7/5/26 11:51 PM, Dan Mahoney wrote:

> Is your system clock off?
>
> -Dan
>

[Steve Rikli]

Longshot guess: is your system clock time very far off, perhaps?

If you're okay with re-bootstrap, maybe try
pkg-static -f bootstrap

... rather than the normal 'pkg' command.

Cheers,
sr.

[Dan Mahoney (ports)]

> On Jul 5, 2026, at 9:37 PM, William F. Dudley Jr. <wfdu...@gmail.com> wrote:
>

Okay, try the following:

openssl s_client -connect pkg.freebsd.org:443

and see what the verification prints, it should show the results, and I think by default it uses the system cert store. It should also use the same DNS you're using so should get the same pkg server.

-Dan

[William F. Dudley Jr.]

Steve,

The clock is correct.

pkg-static bootstrap -f

made no improvement. pkg update still fails as before.

Interestingly, the position of the "-f" is important:

pkg-static -f bootstrap
pkg-static: illegal option -- f
pkg-static: Invalid argument provided

Thanks,
Bill Dudley

[William F. Dudley Jr.]

That was "interesting". Rather than try to understand the output, I ran
the command on both machines, and they give identical results.

I think this excerpt is the punchline:

SSL handshake has read 2426 bytes and written 401 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 256 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)

Thanks,
Bill Dudley

[Steve Rikli]

Yes, sorry -- that's my badness on the flag|command mis-ordering.

Last thought for now: afaik the pkg commands use fetch(1) to do the
download work, the fetch(1) and fetch(3) man pages mention an env var:

SSL_NO_VERIFY_PEER

among others. Maybe try setting that and re-do your bootstrap attempt.

Wrt your system files, I dunno what would have gotten removed/corrupted
to cause this particular error. Since you have a still-working system
for reference, I'd compare things like their /var/db/pkg/ subdirs, see
if anything obvious is missing or scrambled.

If you find something obvious that can be restored from backups that's
great; otherwise I suspect you're still looking at a re-bootstrap if
we can work out how to force that.

sr.

[William F. Dudley Jr.]

Steve,

Interestingly, setting that environment variable allows "pkg update" to
work.

I'll try to compare /var/db/pkg/*

Thanks,
Bill Dudley

[Marco Moock]

Am 06.07.26 um 06:52 schrieb William F. Dudley Jr.:

> Interestingly, setting that environment variable allows "pkg update" to
> work.

Yes, but this check is security-relevant. If someone is between you and
the repo server, he can forge the certificate. This check detects that.

--
Gruß
Marco

Junk-Mail bitte an tras...@stinkedores.dorfdsl.de

[Erwan David]

On Mon, Jul 06, 2026 at 06:52:39AM CEST, "William F. Dudley Jr." <wfdu...@gmail.com> said:
> Steve,
>
> Interestingly, setting that environment variable allows "pkg update" to
> work.
>
> I'll try to compare /var/db/pkg/*

I would suspect your crash happened when upgrading the ca-root-nss package, leaving you with an ill-configured list of acceted CAs for certificates verification


--
Erwan David

[Michael Sierchio]

A hacktastic shortcut to disable SSL/TLS certificate validation?  Why does that seem like a good idea to anyone?

[Dag-Erling Smørgrav]

"William F. Dudley Jr." <wfdu...@gmail.com> writes:
> I am running "FreeBSD 14.3-RELEASE-p16 amd64".
>
> I did "freebsd-update fetch" followed by "freebsd-update install", and
> the power (and my UPS) failed, I think, during the "freebsd-update
> install" step.

Assuming a standard ZFS install, I would strongly advise that you roll
back to the snapshot that was created at the start of `freebsd-update
install` and re-run `freebsd-update install`.

> Now, when I run "pkg update", I get this:
>
> pkg: Failed to fetch
> https://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly/meta.conf: SSL peer
> certificate or SSH remote key was not OK

This can most likely be fixed by running `certctl rehash` as root.

DES
--
Dag-Erling Smørgrav - d...@FreeBSD.org

[William F. Dudley Jr.]

relax, it's just a debugging step.  I don't plan to do that forever.

Bill Dudley

[William F. Dudley Jr.]

ding ding ding ding -- we have a winner!

Thanks!  All fixed now after re-installing ca_root_nss

Bill Dudley

[Marco Moock]

Am 06.07.26 um 16:04 schrieb William F. Dudley Jr.:

> relax, it's just a debugging step.  I don't plan to do that forever.

Although it is enough to compromise a system, even if the probability is
low.

--
Gruß
Marco
Muell und Spam bitte an abfalle...@stinkedores.dorfdsl.de

[Dan Mahoney (ports)]

> On Jul 6, 2026, at 11:28 AM, Marco Moock <m...@dorfdsl.de> wrote:
>
> Am 06.07.26 um 16:04 schrieb William F. Dudley Jr.:
>> relax, it's just a debugging step. I don't plan to do that forever.
>
> Although it is enough to compromise a system, even if the probability is low.

Waitaminute,

How (better question: WHY) does ca_root_nss figure in to pkg?

The system has its own trust store, shipped as part of base and maintained with freebsd-update, rehashed on any update. Pkg/fetch/freebsd-update/etc uses it. The need for ca_root_nss is slowly diminishing, and most of my systems don't have it as an auto-dependency at all (I don't know which ports still haven't been massaged to just use the system store by default). I do wonder how many users who have it as a result of some long-ago dependency could just autoremove it and be fine.

Evidence: You can run pkg or freebsd-update or fetch https://google.com on a system with no installed packages. Pkg itself is an installed pkg (not in base) but...it also doesn't install ca_root_nss as a dependency.

The bug here is pkg also looks for a monolithic CAFile (which was installed by ca_root_nss, and was damaged in OP's case), and if reading your CAFile breaks (fails to load) for some reason, the invocation as called by libfetch doesn't properly fall back to trying the CAPath.

This seems counterintuitive in the use-case of pkg, where a damaged port can break a core util. I'd argue that pkg should have used only use the system stores. If a user needs to somehow trust a private CA/cert for a pkg repo, they can do it via the usual /usr/local/share/certs & certctl mechanisms.

Also, thanks to [an hour I won't get back of...] looking into openssl invocations, the openssl s_client command line worked but pkg failed because openssl s_client is more forgiving and didn't bail out of loading the CA paths when the CAFile load failed, when called with no -CAFile/-CAPath args. If I had told OP to specify those, we probably would have seen the failure, but there's likely no way to tell now that user's back up and running.

-Dan

[Dag-Erling Smørgrav]

"William F. Dudley Jr." <wfdu...@gmail.com> writes:
> ding ding ding ding -- we have a winner!
>
> Thanks! All fixed now after re-installing ca_root_nss

Only because that triggers `certctl rehash`, which I already told you
was the solution.