uploading .buildinfo files… (from Debian reproducible builds…)

[Holger Levsen]

Hi Linus,

we recently discussed uploading .buildinfo files to your log and you said it
would be simple, so… what's the exact command to upload? :-)

We currently create ~6000 .buildinfo files per day.


cheers,
Holger

[Linus Nordberg]

Holger Levsen <hol...@layer-acht.org> wrote
Wed, 9 Mar 2016 12:10:18 +0100:

Hi Holger,

Do once (per host you're going to submit from):

alias curl-tor='curl -A "" -x socks4a://127.0.0.1:9050/'
curl-tor -O https://www.ct.nordu.net/gaol.ct.nordu.net.pem
curl-tor -O https://www.ct.nordu.net/gaol.ct.nordu.net.pem.asc
gpg --verify gaol.ct.nordu.net.pem.asc

Do once per .buildinfo file:

printf "{\"blob\": \"$(cat file | base64)\"}" | \
curl-tor --data @- \
http://mvkhztpvqcxpdbn3.onion/open/gaol/v1/add-blob


NOTE0: If the size of your submissions (after base64 encoding) exceeds
~2MB they will fail.

NOTE1: All data may disappear at any time (but i'll try hard to avoid
that).

NOTE2: The format for submitted data might change, most likely adding a
requirement for a "sig" field with a signature over "blob"

NOTE3: you might want to put something in "blob" that makes it easy for
you to select your entries from the log

[Holger Levsen]

Hi Linus,

(added the Debian reproducible builds lists to cc:)

On Fri, Mar 18, 2016 at 11:36:56AM +0100, Linus Nordberg wrote:
> Do once (per host you're going to submit from):
>
> alias curl-tor='curl -A "" -x socks4a://127.0.0.1:9050/'

that has an result I understand…

> curl-tor -O https://www.ct.nordu.net/gaol.ct.nordu.net.pem
> curl-tor -O https://www.ct.nordu.net/gaol.ct.nordu.net.pem.asc
> gpg --verify gaol.ct.nordu.net.pem.asc

but this is rather incomplete or meaningless? ;-) Or I don't see the
point as that certificate aint used anywhere?

> Do once per .buildinfo file:
>
> printf "{\"blob\": \"$(cat file | base64)\"}" | \
> curl-tor --data @- \
> http://mvkhztpvqcxpdbn3.onion/open/gaol/v1/add-blob

ok, seems easy enough.

So I just did:

printf "{\"test-h01ger\": \"$(cat /etc/motd | base64)\"}" | curl -A "" \
-x socks4a://127.0.0.1:9050/ --data @- \
http://mvkhztpvqcxpdbn3.onion/open/gaol/v1/add-blob

Did the log receive that? If so, it's trivial to send them all to your
log…

> NOTE0: If the size of your submissions (after base64 encoding) exceeds
> ~2MB they will fail.

ok, that's fine. currently the biggest .buildinfo file we have
(gcc-5-cross-ports_7_amd64.buildinfo) is 120K which transforms into 162k
base64 encoded.

> NOTE1: All data may disappear at any time (but i'll try hard to avoid
> that).

ok, noted.

> NOTE2: The format for submitted data might change, most likely adding a
> requirement for a "sig" field with a signature over "blob"

ok, please just tell us.

> NOTE3: you might want to put something in "blob" that makes it easy for
> you to select your entries from the log

I guess the filename of the .buildinfo file will do. What if I reuse the
"blob" value?


--
cheers,
Holger

[Linus Nordberg]

Holger Levsen <hol...@layer-acht.org> wrote
Fri, 18 Mar 2016 10:00:48 -0400:

| > curl-tor -O https://www.ct.nordu.net/gaol.ct.nordu.net.pem> curl-tor -O https://www.ct.nordu.net/gaol.ct.nordu.net.pem.asc> gpg --verify gaol.ct.nordu.net.pem.asc
|
| but this is rather incomplete or meaningless? ;-) Or I don't see the
| point as that certificate aint used anywhere?

That's correct. Let's call it preparation for future verification of
SCT's and STH's. :D (In reality, I forgot adding info about that and
now I've decided to wait until someone asks for it.)

| > Do once per .buildinfo file:
| >
| > printf "{\"blob\": \"$(cat file | base64)\"}" | \
| > curl-tor --data @- \
| > http://mvkhztpvqcxpdbn3.onion/open/gaol/v1/add-blob
|
| ok, seems easy enough.
|
| So I just did:
|
| printf "{\"test-h01ger\": \"$(cat /etc/motd | base64)\"}" | curl -A "" \
| -x socks4a://127.0.0.1:9050/ --data @- \
| http://mvkhztpvqcxpdbn3.onion/open/gaol/v1/add-blob
|
| Did the log receive that? If so, it's trivial to send them all to your
| log…

Should've rejected it ("blob" is magic and required). What did curl tell
you? I bet it was 4xx rather than 200.

| > NOTE2: The format for submitted data might change, most likely adding a
| > requirement for a "sig" field with a signature over "blob"
|
| ok, please just tell us.

Will do.

| > NOTE3: you might want to put something in "blob" that makes it easy for
| > you to select your entries from the log
|
| I guess the filename of the .buildinfo file will do. What if I reuse the
| "blob" value?

Sorry for using sloppy language. You might want to put something
hopefully unique in the _value_ of the name/value pair with the name
"blob".

[Holger Levsen]

Hi,

On Fri, Mar 18, 2016 at 03:18:30PM +0100, Linus Nordberg wrote:
> That's correct. Let's call it preparation for future verification of
> SCT's and STH's. :D (In reality, I forgot adding info about that and
> now I've decided to wait until someone asks for it.)

ah, ok.

> Should've rejected it ("blob" is magic and required). What did curl tell
> you? I bet it was 4xx rather than 200.

curl spews lots of base encoded output at me, as well as "add-blob:
missing input: blob"

> Sorry for using sloppy language. You might want to put something
> hopefully unique in the _value_ of the name/value pair with the name
> "blob".

so I tried

printf "{\"blob=test-h01ger\": \"$(cat /etc/motd | base64)\"}" | curl -A

"" -x socks4a://127.0.0.1:9050/ --data @-
http://mvkhztpvqcxpdbn3.onion/open/gaol/v1/add-blob

and

printf "{\"blob/test-h01ger\": \"$(cat /etc/motd | base64)\"}" | curl -A

"" -x socks4a://127.0.0.1:9050/ --data @-
http://mvkhztpvqcxpdbn3.onion/open/gaol/v1/add-blob

but both didnt work and failed as before. What did work though was

printf "{\"blob\": \"$(cat /etc/motd | base64)\"}" | curl -A "" -x

socks4a://127.0.0.1:9050/ --data @-
http://mvkhztpvqcxpdbn3.onion/open/gaol/v1/add-blob

but that doesnt have the unique identifier…(?)


--
cheers,
Holger

[Linus Nordberg]

Holger Levsen <hol...@layer-acht.org> wrote
Fri, 18 Mar 2016 10:46:59 -0400:

| but that doesnt have the unique identifier…(?)

You could let "blob" contain another JSON object.