Golang - get-proof-by-hash - 400 BadRequest (No leaf hash found)

[Dario]

Hi Guys,

i'm trying to check if a certificate has been incorporated into a CT log using the official certificate transparency Go library (https://github.com/google/certificate-transparency-go).

In particolar i'm using the logClient.GetProofByHash method (https://github.com/google/certificate-transparency-go/blob/master/client/logclient.go).

Below all the step executed (for this test i used the root CA cert exposed by 'google.com'):

1) Calculating the leaf hash:
     - leafEntry := ct.CreateX509MerkleTreeLeaf(ct.ASN1Cert{Data: rootCA.Raw}, sct.Timestamp) --> https://github.com/google/certificate-transparency-go/blob/master/serialization.go
     - leafHash, err := ct.LeafHashForLeaf(leafEntry) --> https://github.com/google/certificate-transparency-go/blob/master/serialization.go

2) Creating logClient instance (i got the publicKey from https://www.gstatic.com/ct/log_list/v2/all_logs_list.json and it is related to CT Log rean SCT logID included in root CA cert used for the test )
    - got SCT from google.com certificate
    - got logID from SCT
    - got CT Log public key from https://www.gstatic.com/ct/log_list/v2/all_logs_list.json using above logID
    - opts := jsonclient.Options{UserAgent: "ct-go-ctclient/1.0", PublicKey: publicKeyPem}      
    - logClient, err := ctClient.New(ctLog.URL, httpClient, opts)

3) Got Signed Tree Head (STH) from that CT Log
    - sth, err := logClient.GetSTH(ctx)

4) Running of GetProofByHash method:
    - sthSize := int64(sth.TreeSize)
    - rsp, err := logClient.GetProofByHash(ctx, leafHash[:], uint64(sthSize))
   

the API fails with :

HTTP Error - 400 BadRequest
Not Found
backend GetInclusionProofByHash request failed: rpc error: code = NotFound desc = No leaf found for hash: 484341b83622f400dee4914754ab0a31b9d5e215409bc53e195b250a07019bb5 in tree size 976351352

What am i doing wrong? Do i miss some configuration?

Thanks in advance,
Dario

[Al Cutter]

Hi Dario,

From your description it's a little unclear if you're trying to use SCTs from the google.com End Entity cert to prove inclusion of the root CA from the chain - if so, that's not going to work - you need the SCT for the cert you're trying to prove inclusion for. 

(Another point to note is that when the SCTs are embedded in an end-entity certificate you need to reconstruct the Precertificate and fetch an inclusion proof for that.)

Have you had a look at the slightly higher-level code for doing this verification in the repo

https://github.com/google/certificate-transparency-go/blob/master/ctutil/loginfo.go#L180

which is used by the sctcheck tool:

https://github.com/google/certificate-transparency-go/blob/master/ctutil/sctcheck/sctcheck.go#L219

$ go run github.com/google/certificate-transparency-go/ctutil/sctcheck --logtostderr -v 2 https://google.com
I0116 15:43:20.152217  191620 sctcheck.go:168] Retrieve certificate chain from TLS connection to "google.com:443"
I0116 15:43:20.163916  191620 sctcheck.go:181] Found chain of length 2
E0116 15:43:20.164542  191620 sctcheck.go:82] Found 0 external SCTs for "https://google.com", of which 0 were validated
I0116 15:43:20.164800  191620 sctcheck.go:226] Examine embedded SCT[0] with timestamp: 1575970968053 (2019-12-10 09:42:48.053 +0000 GMT) from logID: b21e05cc8ba2cd8a204e8766f92bb98a2520676bdafa70e7b249532def8b905e
I0116 15:43:20.164905  191620 sctcheck.go:239] Validate embedded SCT[0] against log "Google 'Argon2020' log"...
I0116 15:43:20.165412  191620 sctcheck.go:244] Validate embedded SCT[0] against log "Google 'Argon2020' log"... validated
I0116 15:43:20.165440  191620 sctcheck.go:248] Check embedded SCT[0] inclusion against log "Google 'Argon2020' log"...
I0116 15:43:20.165475  191620 client.go:179] GET https://ct.googleapis.com/logs/argon2020/ct/v1/get-sth?
I0116 15:43:20.886477  191620 client.go:179] GET https://ct.googleapis.com/logs/argon2020/ct/v1/get-proof-by-hash?hash=eW5Eno1UvgeibhdT%2F2MsU7V%2FW%2B%2FyNF%2BcMHKSSP%2BALOs%3D&tree_size=257528536
I0116 15:43:21.162764  191620 sctcheck.go:259] Check embedded SCT[0] inclusion against log "Google 'Argon2020' log"... included at 179700609
I0116 15:43:21.162837  191620 sctcheck.go:226] Examine embedded SCT[1] with timestamp: 1575970968081 (2019-12-10 09:42:48.081 +0000 GMT) from logID: 5ea773f9df56c0e7b536487dd049e0327a919a0c84a112128418759681714558
I0116 15:43:21.163025  191620 sctcheck.go:239] Validate embedded SCT[1] against log "Cloudflare 'Nimbus2020' Log"...
I0116 15:43:21.163576  191620 sctcheck.go:244] Validate embedded SCT[1] against log "Cloudflare 'Nimbus2020' Log"... validated
I0116 15:43:21.163605  191620 sctcheck.go:248] Check embedded SCT[1] inclusion against log "Cloudflare 'Nimbus2020' Log"...
I0116 15:43:21.163625  191620 client.go:179] GET https://ct.cloudflare.com/logs/nimbus2020/ct/v1/get-sth?
I0116 15:43:21.196178  191620 client.go:179] GET https://ct.cloudflare.com/logs/nimbus2020/ct/v1/get-proof-by-hash?hash=XZcZ17ecNy0eFJPwF178BPdI2Cnco72UBa%2FRjxgPU8I%3D&tree_size=207033881
I0116 15:43:21.382774  191620 sctcheck.go:259] Check embedded SCT[1] inclusion against log "Cloudflare 'Nimbus2020' Log"... included at 161053546
E0116 15:43:21.382821  191620 sctcheck.go:103] Found 2 embedded SCTs for "https://google.com", of which 2 were validated

Cheers,

Al.

--
You received this message because you are subscribed to the Google Groups "certificate-transparency" group.
To unsubscribe from this group and stop receiving emails from it, send an email to certificate-transp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/certificate-transparency/15a13b61-6b7d-401a-abc8-5b5e627003cb%40googlegroups.com.

[Dario]

Hi AI,

thanks for your reply.

Analizying the sctcheck.go file i found the issue: i was using a wrong method to build the Merkle tree leaf.

Wrong method --> https://github.com/google/certificate-transparency-go/blob/master/serialization.go#L83

Correct method --> https://github.com/google/certificate-transparency-go/blob/master/serialization.go#L197

After this change the get-proof-by-hash API works fine.

Thanks a lot! :)

Cheers,
Dario

To unsubscribe from this group and stop receiving emails from it, send an email to certificate-transparency+unsub...@googlegroups.com.

[Al Cutter]

Hi Dario,

awesome, I'm happy to hear it! :)

Cheers,

Al.

To unsubscribe from this group and stop receiving emails from it, send an email to certificate-transp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/certificate-transparency/15a13b61-6b7d-401a-abc8-5b5e627003cb%40googlegroups.com.

--

You received this message because you are subscribed to the Google Groups "certificate-transparency" group.

To unsubscribe from this group and stop receiving emails from it, send an email to certificate-transp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/certificate-transparency/106217c3-870b-4af6-b028-69fa7c72af4e%40googlegroups.com.