Signing audio and video to prevent deep fakes ?

[Raphael Jolivet]

Hi

I've just discovered this project and I think it's great: it's a timely response to a growing problem of information manipulation and deep fakes.

It seems to me an essential building block for a decentralized signature protocol for audio content:
- Calculate a fingerprint of the audio every X seconds
- Sign with the private key of an authority (journalist, institution, etc.)
- Add the public key + the signature as a watermark in the audio

The client could decode this information, verify the signature and display the authenticated source of the content. For example, using the WKD standard, which associates a public email with a PGP key.

What do you think ?

Regards,

Raphaël Jolivet

[Stefan Westerfeld]

Hi!

Am 02.09.23 um 12:56 schrieb Raphael Jolivet:

> I've just discovered this project and I think it's great: it's a timely
> response to a growing problem of information manipulation and deep fakes.
>
> It seems to me an essential building block for a decentralized signature
> protocol for audio content:
> - Calculate a fingerprint of the audio every X seconds
> - Sign with the private key of an authority (journalist, institution, etc.)
> - Add the public key + the signature as a watermark in the audio
>
> The client could decode this information, verify the signature and
> display the authenticated source of the content. For example, using the

> WKD standard <https://wiki.gnupg.org/WKD>, which associates a public

> email with a PGP key.
>
> What do you think ?

Well, that is an interesting idea. From the watermarking point of view,
I have two comments to make. One is that usually for watermarking one
requirement many users have is that the watermark cannot be altered or
removed easily. For this audiowmark uses a secret key for embedding and
decoding the watermark message bits. But as I understand your use case,
you want everyone to be able to read the message bits from the
watermark. So the "secret" key would have to be known for everyone. This
also means that journalist B can claim he created the content by
journalist A by altering the watermark. Your system would not avoid this
tampering.

But at least we can guarantee that a deep fake of for instance
journalist A will never be confirmed to be from journalist A by the
system, as the system would then say journalist B created the content.

The other - maybe more critical - issue is the number of bits you need
to store inside the watermark. Storing bits is expensive. So audiowmark
usually just stores 128 bit hashes, and the person that created the
watermark keeps a database of those hashes. See:

https://github.com/swesterfeld/audiowmark#recommendations-for-the-watermarking-payload

But this would require a central point where those hashes would be
stored. Maybe this would be ok for you. But if you want a fully
decentralized system, you would probably want to store something other
than a simple hash of all payload. Storing a complete public key and a
signature sounds like a lot more bits than 128. Maybe you could store
something foo.com/1234 saying that this is article 1234 published by
somebody from foo.com. Then the rest of the information (signature,...)
could be downloaded from foo.com via https. Still it would be better if
we had an upper limit to the number of bits, since
www.some-long-domain-from-whatever.com/2137891 could still be too many
bits to be practical.

Cu... Stefan
--
Stefan Westerfeld, http://space.twc.de/~stefan