Dupe checking through search or API

[bitb...@gmail.com]

Is there a way to check image files for duplicates before uploading?

For example, with Ponibooru you could search by MD5, and I used this in a script to check for dupes before checking out artist name and the like.

[K_A]

We use SHA-512, here. Try searching sha512_hash:<checksum>

[K_A]

But be aware that we perform image optimization here, so you may not get the results you want without running the same optimizations yourself. All of them except JPG optimization are lossless but will obviously affect the checksum.

[Lexi Anevay]

Maybe you should start storing the original hash as well, to help with this.

[Clover the Clever]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

We are now doing this (to fix our own issues with exact dedupe so for
more recent images we can provide this. We'll be exposing it in the API
soon. Images without an original hash stored (before approx 120000
id_number) will have a nil value for the original hash key.

Cheers,
Clover

> --
> You received this message because you are subscribed to the Google
> Groups "Derpibooru Discussion" group.
> To post to this group, send an email to
> derpiboor...@googlegroups.com.
> To unsubscribe from this group, send email to
> derpibooru-disc...@googlegroups.com.
> To view this discussion on the web, visit
> https://groups.google.com/d/msg/derpibooru-discuss/-/Ky1rvSmi4kkJ.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBAgAGBQJQexuKAAoJEG5qwsD99qALNkMP/074NT6NwPK6xusbFJ0Hk6uG
bh1WmBViFm8SMgZNyDVcrorcm5wQwtpi9SDSPe8mj9pH0W/lhjTbbxPH0qmOhi+d
uWDu4GXzuaf+Nn7X4/9zGfWt0CKKahc/WNwh0Ffv6uYBf9RCZ7wwOG8XFSh3hADo
oLcQGK3aaekCr0QEqLT4SdKBP0d5St8tX6GFHuXlqwHpnkO9VMezhHuQA4YXlsbY
gYYMo/DBOCEThb/AAlR64QL+mvSZ2S+nJzKTif1661VvT/+mjDl1wiOZzfPvuLSm
we/kWcUUSD5j5Hzyj40yX/ScZ492WlsUEEGgB1fZeTMU/+0Gt+8MxCqmbJMqEERh
2Id2vXpUMtOnh9jYZwIFEKzJTTr8dwDC5gSM7A7Yge6JF6RK0MisVHvWrWqLvfYZ
qhdNX4leFY8DZkEmMrKj6RzCYLB93MLlWb7tNLTRX3B/zkPhsNgiN/GAx3BXAl60
+LywSdCQy5sy/2vhenZEQS8FQ9dOT6jk3tyF1mOXe9k9Kn8xJl7pt5DYkc1/77NF
nPPh1AdpSizXdzolrEQbvKG19mRMyNf3DzuvYszL8HmxzKEgfMIyfNdaXO9g29wF
OBGjGUH3FZiA7ZbqIVv1dzeNrrJIVnMoj5WLKDzt7ZrdmLhUHC5vLDhZTlcXBa6J
ITJYpPH9MAC+3bwXl7EI
=Y0Yi
-----END PGP SIGNATURE-----