Having troubles getting the latest version to work correctly

[mic...@vanbaak.eu]

Hi all,

First of all, thanks for creating this project.

I really hope it will get the current state of public/private keyservers somewhere usable again.

This week I started testing the latest and greatest version (current master version of hockeypuck/server) and I seem to run into a couple of problems.

Maybe someone here can help me a little.

I got everything setup on a FreeBSD 12.0 vm, compiled hockeypuck with go 1.11.5 and using postgresql 9.6 as backend.

The hockeypuck process starts and I get output when I go to the stats page.

Adding a key with opengpg works as well:

gpg --keyserver 192.168.10.117 --send-keys 0x6FFC75A2679ED069

I see the key and subkeys in the postgresql database, so that looks ok.

Now when I try to search for this key, or get the key itself, I get errors:

Tested in a browser using the same url parameters as I use for SKS:

time="2019-02-21T11:53:11+01:00" level=error msg="HTTP 500: [{/root/ports/security/hockeypuck/work/src/github.com/hockeypuck/server/vendor/gopkg.in/hockeypuck/hkp.v1/handler.go:246: } {/root/ports/security/hockeypuck/work/src/github.com/hockeypuck/server/vendor/gopkg.in/hockeypuck/pghkp.v1/storage.go:337: } {json: cannot set embedded pointer to unexported struct: jsonhkp.publicKey}]"

time="2019-02-21T11:53:11+01:00" level=info msg= GET="/pks/lookup?op=get&search=0x6FFC75A2679ED069" duration=7.778631ms from="192.168.10.72:52241"

time="2019-02-21T12:03:21+01:00" level=error msg="HTTP 500: [{/root/ports/security/hockeypuck/work/src/github.com/hockeypuck/server/vendor/gopkg.in/hockeypuck/hkp.v1/handler.go:264: } {/root/ports/security/hockeypuck/work/src/github.com/hockeypuck/server/vendor/gopkg.in/hockeypuck/pghkp.v1/storage.go:337: } {json: cannot set embedded pointer to unexported struct: jsonhkp.publicKey}]"

time="2019-02-21T12:03:21+01:00" level=info msg= GET="/pks/lookup?search=0x6FFC75A2679ED069&fingerprint=on&op=index" duration=7.52244ms from="192.168.10.72:52319"

Any help would be appreciated.

Thanks

This is my config file:

[hockeypuck]

version="1.1.6"

contact="0x6FFC75A2679ED069"

hostname="keyserver.vanbaak.eu"

logfile="/tmp/hockeypuck.log"

loglevel="DEBUG"

indexTemplate="/var/db/hockeypuck/templates/index.html.template"

statsTemplate="/var/db/hockeypuck/templates/stats.html.template"

[hockeypuck.openpgp.db]

driver="postgres-jsonb"

dsn="dbname=sks-hp-01 user=YYY password=XXX host=pgsql-01 port=5432 sslmode=disable"

[hockeypuck.conflux.recon]

version="1.1.6"

filters=["yminsky.dedup", "yminsky.merge"]

[hockeypuck.conflux.recon.leveldb]

path="/var/db/hockeypuck/recon-sks-hp-01"

[Jason H.]

On Thursday, February 21, 2019 at 7:11:10 AM UTC-4, mic...@vanbaak.eu wrote:

 

First of all, thanks for creating this project.

I really hope it will get the current state of public/private keyservers somewhere usable again.

Me too.  In fact, I finally installed hockeypuck after your FreeBSD port of it was committed.  Thank you and Casey!

This week I started testing the latest and greatest version (current master version of hockeypuck/server) and I seem to run into a couple of problems.

Maybe someone here can help me a little.

I got everything setup on a FreeBSD 12.0 vm, compiled hockeypuck with go 1.11.5 and using postgresql 9.6 as backend.

There _shouldn't_ be a problem using the latest version of PostgreSQL:

  %pg_config

  VERSION = PostgreSQL 11.2

BTW, here's a patch for the FreeBSD port Makefile:

-USES=          go

+USES=          go pgsql:9.4+

 

(This defaulted to installing databases/postgresql11-server.)

The hockeypuck process starts and I get output when I go to the stats page.

Some additional notes.  I loaded a keydump "raw" - without the SQL constraints in hockeypuck/pghkp.v1/storage.go's crIndexesSQL.

  #Dump-started: 2019-02-25

  #Key-Count: 5438728

Without the indices/indexes, my reported key count on the stats page is 0, but it shows 5301660 keys added in the Daily Histogram...

However, I still saw these "failed to insert" errors at least twice:

  ERRO[19483] some keys failed to insert from "/tmp/sks-dump-0204.pgp": [{0 duplicates, 3238 errors}]

Do we know what data in the keydump(s) is causing this?

Loading "raw" also revealed the SKS dump had some primary key duplication:

select count (distinct(rfingerprint)) from keys;        = 5088442

select count (*) from keys;                             = 5088728

5088728 - 5088442 = 286 dupe keys by [r]fingerprint

in addition to the known packet duplication/corruption that needs to phased out with stronger filters...

 

Adding a key with opengpg works as well:

gpg --keyserver 192.168.10.117 --send-keys 0x6FFC75A2679ED069

I see the key and subkeys in the postgresql database, so that looks ok.

Now when I try to search for this key, or get the key itself, I get errors:

Tested in a browser using the same url parameters as I use for SKS:

time="2019-02-21T11:53:11+01:00" level=error msg="HTTP 500: [{/root/ports/security/hockeypuck/work/src/github.com/hockeypuck/server/vendor/gopkg.in/hockeypuck/hkp.v1/handler.go:246: } {/root/ports/security/hockeypuck/work/src/github.com/hockeypuck/server/vendor/gopkg.in/hockeypuck/pghkp.v1/storage.go:337: } {json: cannot set embedded pointer to unexported struct: jsonhkp.publicKey}]"

time="2019-02-21T11:53:11+01:00" level=info msg= GET="/pks/lookup?op=get&search=0x6FFC75A2679ED069" duration=7.778631ms from="192.168.10.72:52241"

time="2019-02-21T12:03:21+01:00" level=error msg="HTTP 500: [{/root/ports/security/hockeypuck/work/src/github.com/hockeypuck/server/vendor/gopkg.in/hockeypuck/hkp.v1/handler.go:264: } {/root/ports/security/hockeypuck/work/src/github.com/hockeypuck/server/vendor/gopkg.in/hockeypuck/pghkp.v1/storage.go:337: } {json: cannot set embedded pointer to unexported struct: jsonhkp.publicKey}]"

time="2019-02-21T12:03:21+01:00" level=info msg= GET="/pks/lookup?search=0x6FFC75A2679ED069&fingerprint=on&op=index" duration=7.52244ms from="192.168.10.72:52319"

Any help would be appreciated.

What was the solution to this "unexported struct" error?  I'm seeing it too.

Thanks.