SUPERCOP NISTPQC integration status

424 views
Skip to first unread message

D. J. Bernstein

unread,
May 5, 2020, 4:37:25 PM5/5/20
to pqc-...@list.nist.gov
Changes in NISTPQC primitives and implementations often affect speed.
Here's a chart showing my current understanding of how the changes are
related to SUPERCOP's latest Haswell benchmarks. Please send corrections
to the list if I've missed anything.

As a general note, I'm _guessing_ that NIST is most interested in
whatever the submission teams designate as their newest primitives.
However, I'm not sure about this. The call for proposals said the
following regarding round 1:

Because of limited resources, and also to avoid moving evaluation
targets (i.e., modifying the submitted algorithms undergoing public
review), NIST will NOT accept modifications to the submitted
algorithms during this initial phase of evaluation.

The call also said that "small modifications to the submitted algorithms
will be permitted" before the beginning of the second round, but I
haven't found a statement making clear whether this also applies to the
third round.

The following submissions have unchanged primitives since the beginning
of round 2, and have submitted their latest implementations to SUPERCOP:

* Classic McEliece: mceliece{348864,460896,6688128,6960119,8192128}{,f}.

* Dilithium: dilithium{2,3,4}{,aes}.

* Falcon: falcon{512,1024}{dyn,tree}.

* Frodo: frodokem{640,976,1344}{aes,shake}. Email dated 25 Mar 2020
14:33:42 -0400 doesn't indicate any implementation changes.

* Kyber: kyber{,90s}{512,768,1024}.

* NTRU: ntru{hps{2048509,2048677,4096821},hrss701}.

* NTRU Prime: {ntrulpr,sntrup}{653,761,857}.

* NTS-KEM: Email dated 7 Mar 2020 01:54:15 -0000 announced merger
with Classic McEliece, so I won't track this separately.

* SABER: {light,,fire}saber2. The last code update was before the
email dated 8 Apr 2020 16:11:42 -0700 announcing speedups, but I
think it includes those speedups.

* SPHINCS+: sphincs{f,s}{128,192,256}{haraka,sha256,shake256}{robust,simple}.

The following submissions have unchanged primitives since the beginning
of round 2, but what's in SUPERCOP predates round 2, and perhaps this
affects speed:

* qTESLA: qtesla* predate round 2. Email dated 20 Aug 2019 13:51:48
+0200 restricted the submission to the "p" parameter sets.

* Rainbow: rainbow{1a,1b,1c,3b,3c,4a,5c,6a,6b} predate round 2.

The following submissions have proposed new primitives after the
beginning of round 2, and have submitted their latest implementations of
those primitives to SUPERCOP:

* GeMSS: {blue,,red}gemss{128,192,256}v2 are the new primitives.

* Picnic: picnic3l{1,3,5} are the new primitives.

* ThreeBears: threebears{624,936,1248}r2c{p,c}a{,x} are the new
primitives.

The following submissions have proposed new primitives after the
beginning of round 2, and have not submitted their latest
implementations to SUPERCOP:

* BIKE: bike* predate round 2. Email dated 1 May 2020 18:07:39 -0700
(and some previous email) announced new primitives.

* HQC: hqc{1281,1921,1922,2561,2562,2563} are the round-2 primitives.
Email dated 22 Apr 2020 00:55:15 +0200 announced new primitives.
Email dated 4 May 2020 12:37:59 +0200 announced new software.

* LAC: lac{128,192,256} predate round 2. Email dated 15 Apr 2020
23:16:13 +0800 announced new primitives.

* LEDA: ledakem{{1,3,5}{2,3,4},lt{1,3,5}{0,1}} are the round-2
primitives. Email dated 19 Mar 2020 19:26:25 +0100 announced new
primitives.

* LUOV: luov* predate round 2. Email dated 7 Sep 2019 21:15:00 +0200
announced new primitives.

* MQDSS: mqdss{48,64} predate round 2. Email dated 14 Apr 2020
20:49:33 +0200 announced new primitives.

* NewHope: newhope{512,1024}cca predate round 2 (and don't include
the cpa versions). Email dated 14 Apr 2020 16:58:36 +0000 announced
new primitives (although saying "the impact on performance is very
small").

* ROLLO: rollo{i,ii,iii}{128,192,256} are the round-2 primitives.
Email dated 22 Apr 2020 00:23:58 +0200 announced new primitives.

* Round5: r5n{{1,d}{1,3,5}kem0d,d{{1,3,5}kem5d,0kem2iot,1kem4longkey}}
are the round-2 versions. Email dated 15 Apr 2020 03:23:40 -0700
announced new primitives.

* RQC: rqc{128,192,256} are the round-2 versions. Email dated 21 Apr
2020 21:30:09 +0200 announced new primitives.

* SIKE: sikep503 predates round 2. Email dated 16 Apr 2020 22:20:02
-0400 announced new primitives.

---Dan
signature.asc

Moody, Dustin (Fed)

unread,
May 6, 2020, 12:06:04 PM5/6/20
to D. J. Bernstein, pqc-forum

Dan (and everyone),

      In evaluating the Round 2 candidates, we consider the implementations for the versions that were submitted at the start of Round 2, in addition to any updates provided by the submission team.  We factor both of them into our evaluation (provided an update does not incorporate major changes that substantially alter the design of the submission). 

 

Regarding modifications (or tweaks), for candidates selected to move forward for the 3rd round we will allow small changes similar to what we did at the start of the 2nd round.  

 

Dustin





From: pqc-...@list.nist.gov on behalf of D. J. Bernstein
Sent: Tuesday, May 5, 2020 4:36 PM
To: pqc-forum
Subject: [pqc-forum] SUPERCOP NISTPQC integration status

--
You received this message because you are subscribed to the Google Groups "pqc-forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pqc-forum+...@list.nist.gov.
To view this discussion on the web visit https://groups.google.com/a/list.nist.gov/d/msgid/pqc-forum/20200505203659.25334.qmail%40cr.yp.to.

D. J. Bernstein

unread,
May 11, 2020, 12:06:09 AM5/11/20
to pqc-...@list.nist.gov
supercop-20200510 benchmarks are online for Haswell (hiphop), now
including the new LEDA software. Revised chart below takes this update
into account. As before, please send any corrections to the list.

Overall SUPERCOP now has the latest Haswell software for 14 of the 26
submissions, including

* 10 of the 12 submissions that have unchanged primitives since the
beginning of round 2,

* 2 of the 3 submissions that have proposed new primitives after the
beginning of round 2 but not within the last month, and

* 2 of the 11 submissions that have proposed new primitives within
the last month.

Happy to take further code updates.

---Dan
* GeMSS: {blue,,red}gemss{128,192,256}v2 are the new primitives,
announced in email dated 15 Apr 2020 20:12:08 +0200.

* LEDA: ledakem{{1,3,5}{2,3,4}{64,sl},cpa{1,3,5}{2,3,4}} are the new
primitives, announced in email dated 19 Mar 2020 19:26:25 +0100.
(There's also crypto_encrypt/ledapkc*.)

* Picnic: picnic3l{1,3,5} are the new primitives, announced in email
dated 15 Apr 2020 18:13:17 +0000.

* ThreeBears: threebears{624,936,1248}r2c{p,c}a{,x} are the new
primitives, announced in email dated 25 Jul 2019 10:57:21 -0700.

The following submissions have proposed new primitives after the
beginning of round 2, and have not submitted their latest
implementations to SUPERCOP:

* BIKE: bike* predate round 2. Email dated 1 May 2020 18:07:39 -0700
(and some previous email) announced new primitives.

* HQC: hqc{1281,1921,1922,2561,2562,2563} are the round-2 primitives.
Email dated 22 Apr 2020 00:55:15 +0200 announced new primitives.
Email dated 4 May 2020 12:37:59 +0200 announced new software.

* LAC: lac{128,192,256} predate round 2. Email dated 15 Apr 2020
23:16:13 +0800 announced new primitives.

* LUOV: luov* predate round 2. Email dated 7 Sep 2019 21:15:00 +0200
announced new primitives.

* MQDSS: mqdss{48,64} predate round 2. Email dated 14 Apr 2020
20:49:33 +0200 announced new primitives.

* NewHope: newhope{512,1024}cca predate round 2 (and don't include
the cpa versions). Email dated 14 Apr 2020 16:58:36 +0000 announced
new primitives (although saying "the impact on performance is very
small").

* ROLLO: rollo{i,ii,iii}{128,192,256} are the round-2 primitives.
Email dated 22 Apr 2020 00:23:58 +0200 announced new primitives.

* Round5: r5n{{1,d}{1,3,5}kem0d,d{{1,3,5}kem5d,0kem2iot,1kem4longkey}}
are the round-2 primitives. Email dated 15 Apr 2020 03:23:40 -0700
announced new primitives. (There's also crypto_encrypt/r5*.)

* RQC: rqc{128,192,256} are the round-2 primitives. Email dated 21
signature.asc

D. J. Bernstein

unread,
May 23, 2020, 10:18:53 PM5/23/20
to pqc-...@list.nist.gov
supercop-20200522 benchmarks are online for Haswell (hiphop) with the
new HQC primitives (hqc{128,192,256} and hqcrmrs{128,192,256}). Revised
chart below takes this update into account.

* HQC: hqc{128,192,256} and hqcrmrs{128,192,256} are the new
primitives. The primitives were announced in email dated 22 Apr
2020 00:55:15 +0200, and the software was announced in email dated
4 May 2020. (hqc{1281,1921,1922,2561,2562,2563} are the original
round-2 primitives.)

* LEDA: ledakem{{1,3,5}{2,3,4}{64,sl},cpa{1,3,5}{2,3,4}} are the new
primitives, announced in email dated 19 Mar 2020 19:26:25 +0100.
(There's also crypto_encrypt/ledapkc*.)

* Picnic: picnic3l{1,3,5} are the new primitives, announced in email
dated 15 Apr 2020 18:13:17 +0000.

* ThreeBears: threebears{624,936,1248}r2c{p,c}a{,x} are the new
primitives, announced in email dated 25 Jul 2019 10:57:21 -0700.

The following submissions have proposed new primitives after the
beginning of round 2, and have not submitted their latest
implementations to SUPERCOP:

* BIKE: bike* predate round 2. Email dated 1 May 2020 18:07:39 -0700
(and some previous email) announced new primitives.

signature.asc

D. J. Bernstein

unread,
May 26, 2020, 1:14:51 PM5/26/20
to pqc-...@list.nist.gov
supercop-20200525 benchmarks are online for Haswell (hiphop), with new
implementations for Kyber and Dilithium. No changes in primitives.

---Dan
signature.asc

D. J. Bernstein

unread,
May 31, 2020, 3:44:09 AM5/31/20
to pqc-...@list.nist.gov
supercop-20200530 benchmarks are online for Haswell (hiphop), with the
latest implementations for SABER and HQC. No changes in primitives.

---Dan
signature.asc

D. J. Bernstein

unread,
May 31, 2020, 11:22:37 PM5/31/20
to pqc-...@list.nist.gov
supercop-20200531 benchmarks are online for Haswell (hiphop) with the
new BIKE primitives. Overall SUPERCOP now has the latest Haswell
software for 16 of the 26 submissions, including

* 10 of the 12 submissions that have unchanged primitives since the
beginning of round 2,

* 2 of the 3 submissions that have proposed new primitives after the
beginning of round 2 but before April, and

* 4 of the 11 submissions that have proposed new primitives more
recently.

Revised chart below. Please send corrections to the list.

---Dan


The following submissions have unchanged primitives since the beginning
of round 2, and have submitted their latest implementations to SUPERCOP:

* Classic McEliece: mceliece{348864,460896,6688128,6960119,8192128}{,f}.

* Dilithium: dilithium{2,3,4}{,aes}.

* Falcon: falcon{512,1024}{dyn,tree}.

* Frodo: frodokem{640,976,1344}{aes,shake}. Email dated 25 Mar 2020
14:33:42 -0400 doesn't indicate any implementation changes.

* Kyber: kyber{,90s}{512,768,1024}.

* NTRU: ntru{hps{2048509,2048677,4096821},hrss701}.

* NTRU Prime: {ntrulpr,sntrup}{653,761,857}.

* NTS-KEM: Email dated 7 Mar 2020 01:54:15 -0000 announced merger
with Classic McEliece, so I won't track this separately.

* SABER: {light,,fire}saber2.

* SPHINCS+: sphincs{f,s}{128,192,256}{haraka,sha256,shake256}{robust,simple}.

The following submissions have unchanged primitives since the beginning
of round 2, but what's in SUPERCOP predates round 2, and perhaps this
affects speed:

* qTESLA: qtesla* predate round 2. Email dated 20 Aug 2019 13:51:48
+0200 restricted the submission to the "p" parameter sets.

* Rainbow: rainbow{1a,1b,1c,3b,3c,4a,5c,6a,6b} predate round 2.

The following submissions have proposed new primitives after the
beginning of round 2, and have submitted their latest implementations of
those primitives to SUPERCOP:

* BIKE: bikel{1,3} are the new primitives, announced in email dated 1
May 2020 18:07:39 -0700 (and some previous email).

* GeMSS: {blue,,red}gemss{128,192,256}v2 are the new primitives,
announced in email dated 15 Apr 2020 20:12:08 +0200.

* HQC: hqc{128,192,256} and hqcrmrs{128,192,256} are the new
primitives. The primitives were announced in email dated 22 Apr
2020 00:55:15 +0200, and the software was announced in email dated
4 May 2020. (hqc{1281,1921,1922,2561,2562,2563} are the original
round-2 primitives.)

* LEDA: ledakem{{1,3,5}{2,3,4}{64,sl},cpa{1,3,5}{2,3,4}} are the new
primitives, announced in email dated 19 Mar 2020 19:26:25 +0100.
(There's also crypto_encrypt/ledapkc*.)

* Picnic: picnic3l{1,3,5} are the new primitives, announced in email
dated 15 Apr 2020 18:13:17 +0000.

* ThreeBears: threebears{624,936,1248}r2c{p,c}a{,x} are the new
primitives, announced in email dated 25 Jul 2019 10:57:21 -0700.

The following submissions have proposed new primitives after the
beginning of round 2, and have not submitted their latest
implementations to SUPERCOP:

signature.asc

D. J. Bernstein

unread,
Jun 3, 2020, 3:28:30 PM6/3/20
to pqc-...@list.nist.gov
supercop-20200602 benchmarks are online for Haswell (hiphop) with the
latest software for mceliece* and for saberx4. One saberx4 operation is
a batch of 4 saber2 operations, so the listed saberx4 median cycle
counts of 204692 keygen, 251484 enc, 272656 dec are handling each saber2
operation with 51173 keygen, 62871 enc, 68164 dec.

---Dan
signature.asc

D. J. Bernstein

unread,
Jun 4, 2020, 4:03:11 AM6/4/20
to pqc-...@list.nist.gov
supercop-20200603 benchmarks are online for Haswell (hiphop) with the
new ROLLO and RQC primitives. Revised chart below.

---Dan


The following submissions have unchanged primitives since the beginning
of round 2, and have submitted their latest implementations to SUPERCOP:

* Classic McEliece: mceliece{348864,460896,6688128,6960119,8192128}{,f}.

* Dilithium: dilithium{2,3,4}{,aes}.

* Falcon: falcon{512,1024}{dyn,tree}.

* Frodo: frodokem{640,976,1344}{aes,shake}. Email dated 25 Mar 2020
14:33:42 -0400 doesn't indicate any implementation changes.

* Kyber: kyber{,90s}{512,768,1024}.

* NTRU: ntru{hps{2048509,2048677,4096821},hrss701}.

* NTRU Prime: {ntrulpr,sntrup}{653,761,857}.

* NTS-KEM: Email dated 7 Mar 2020 01:54:15 -0000 announced merger
with Classic McEliece, so I won't track this separately.

* SABER: {light,,fire}saber2; also saberx4 = saber2 on 4 inputs.

* SPHINCS+: sphincs{f,s}{128,192,256}{haraka,sha256,shake256}{robust,simple}.

The following submissions have unchanged primitives since the beginning
of round 2, but what's in SUPERCOP predates round 2, and perhaps this
affects speed:

* qTESLA: qtesla* predate round 2. Email dated 20 Aug 2019 13:51:48
+0200 restricted the submission to the "p" parameter sets.

* Rainbow: rainbow{1a,1b,1c,3b,3c,4a,5c,6a,6b} predate round 2.

The following submissions have proposed new primitives after the
beginning of round 2, and have submitted their latest implementations of
those primitives to SUPERCOP:

* BIKE: bikel{1,3} are the new primitives, announced in email dated 1
May 2020 18:07:39 -0700 (and some previous email).

* GeMSS: {blue,,red}gemss{128,192,256}v2 are the new primitives,
announced in email dated 15 Apr 2020 20:12:08 +0200.

* HQC: hqc{128,192,256} and hqcrmrs{128,192,256} are the new
primitives. The primitives were announced in email dated 22 Apr
2020 00:55:15 +0200, and the software was announced in email dated
4 May 2020. (hqc{1281,1921,1922,2561,2562,2563} are the original
round-2 primitives.)

* LEDA: ledakem{{1,3,5}{2,3,4}{64,sl},cpa{1,3,5}{2,3,4}} are the new
primitives, announced in email dated 19 Mar 2020 19:26:25 +0100.
(There's also crypto_encrypt/ledapkc*.)

* Picnic: picnic3l{1,3,5} are the new primitives, announced in email
dated 15 Apr 2020 18:13:17 +0000.

* ROLLO: rollo{i,ii}{128,192,256} are (now) the new primitives,
announced in email dated 22 Apr 2020 00:23:58 +0200.

* RQC: rqc{128,192,256} are (now) the new primitives, announced in
email dated 21 Apr 2020 21:30:09 +0200.

* ThreeBears: threebears{624,936,1248}r2c{p,c}a{,x} are the new
primitives, announced in email dated 25 Jul 2019 10:57:21 -0700.

The following submissions have proposed new primitives after the
beginning of round 2, and have not submitted their latest
implementations to SUPERCOP:

* LAC: lac{128,192,256} predate round 2. Email dated 15 Apr 2020
23:16:13 +0800 announced new primitives.

* LUOV: luov* predate round 2. Email dated 7 Sep 2019 21:15:00 +0200
announced new primitives.

* MQDSS: mqdss{48,64} predate round 2. Email dated 14 Apr 2020
20:49:33 +0200 announced new primitives.

* NewHope: newhope{512,1024}cca predate round 2 (and don't include
the cpa versions). Email dated 14 Apr 2020 16:58:36 +0000 announced
new primitives (although saying "the impact on performance is very
small").

* Round5: r5n{{1,d}{1,3,5}kem0d,d{{1,3,5}kem5d,0kem2iot,1kem4longkey}}
are the round-2 primitives. Email dated 15 Apr 2020 03:23:40 -0700
announced new primitives. (There's also crypto_encrypt/r5*.)

signature.asc

D. J. Bernstein

unread,
Jun 19, 2020, 3:17:19 AM6/19/20
to pqc-...@list.nist.gov
supercop-20200618 benchmarks are online for Haswell (hiphop) with new
Rainbow primitives.

At this point 19 out of 26 submissions have sent their latest software
to SUPERCOP. Revised chart below. Please send corrections to the list.

---Dan


The following submissions have unchanged primitives since the beginning
of round 2, and have submitted their latest implementations to SUPERCOP:

* Classic McEliece: mceliece{348864,460896,6688128,6960119,8192128}{,f}.

* Dilithium: dilithium{2,3,4}{,aes}.

* Falcon: falcon{512,1024}{dyn,tree}.

* Frodo: frodokem{640,976,1344}{aes,shake}. Email dated 25 Mar 2020
14:33:42 -0400 doesn't indicate any implementation changes.

* Kyber: kyber{,90s}{512,768,1024}.

* NTRU: ntru{hps{2048509,2048677,4096821},hrss701}.

* NTRU Prime: {ntrulpr,sntrup}{653,761,857}.

* NTS-KEM: Email dated 7 Mar 2020 01:54:15 -0000 announced merger
with Classic McEliece, so I won't track this separately.

* SABER: {light,,fire}saber2; also saberx4 = saber2 on 4 inputs.

* SPHINCS+: sphincs{f,s}{128,192,256}{haraka,sha256,shake256}{robust,simple}.

The following submissions have unchanged primitives since the beginning
of round 2, but what's in SUPERCOP predates round 2, and perhaps this
affects speed:

* qTESLA: qtesla* predate round 2. Email dated 20 Aug 2019 13:51:48
+0200 restricted the submission to the "p" parameter sets.

The following submissions have proposed new primitives after the
beginning of round 2, and have submitted their latest implementations of
those primitives to SUPERCOP:

* BIKE: bikel{1,3} are the new primitives, announced in email dated 1
May 2020 18:07:39 -0700 (and some previous email).

* GeMSS: {blue,,red}gemss{128,192,256}v2 are the new primitives,
announced in email dated 15 Apr 2020 20:12:08 +0200.

* HQC: hqc{128,192,256} and hqcrmrs{128,192,256} are the new
primitives. The primitives were announced in email dated 22 Apr
2020 00:55:15 +0200, and the software was announced in email dated
4 May 2020. (hqc{1281,1921,1922,2561,2562,2563} are the original
round-2 primitives.)

* LEDA: ledakem{{1,3,5}{2,3,4}{64,sl},cpa{1,3,5}{2,3,4}} are the new
primitives, announced in email dated 19 Mar 2020 19:26:25 +0100.
(There's also crypto_encrypt/ledapkc*.)

* Picnic: picnic3l{1,3,5} are the new primitives, announced in email
dated 15 Apr 2020 18:13:17 +0000.

* Rainbow: rainbow*{classic,compres,cyclic}* are new primitives, as
per email dated 9 May 2020 18:27:57 -0700. (The pre-round-2
primitives were rainbow{1a,1b,1c,3b,3c,4a,5c,6a,6b}.)
signature.asc
Reply all
Reply to author
Forward
0 new messages