Question regarding Benchmark Results

[jctu...@gmail.com]

Hey David or Dayeol,

I'm running the RV8 benchmarks and saw something interesting. For AES, when I run it without Keystone it takes longer to complete versus with Keystone. This is running on hardware (VCU118). 

But I've also verified with QEMU. Is there something within Keystone that allows AES to complete faster than native Linux?

For example: 

AES Linux: cycles: 8194551853

AES Keystone: Init: 7674267264

Runtime: 5864480216

Here are some pictures running AES in QEMU showing the above values:

[David William Kohlbrenner]

Hi,

Yes, this is expected behavior for some programs.

What is happening is that when Keystone loads a program, it fully loads every page and doesn't do things like zfod (zero-fill-on-demand.)

There are a few different things that can then be happening under normal execution to make it appear slower.

e.g. the test program has a few zfod pages that cause faults when first accessed while running normally.

These faults don't happen during execution in Keystone, so it 'runs' considerably faster.

I'll note that this isn't free. Keystone's program startup is much longer than normal program startup, so we aren't actually faster under that metric.

-David

--
You received this message because you are subscribed to the Google Groups "Keystone Enclave Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keystone-enclave-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keystone-enclave-forum/a7927f35-c499-48c4-adb1-03b076baaf8bn%40googlegroups.com.

[jctu...@gmail.com]

Ah! Thanks David. That clears up the confusion and makes sense.

[jctu...@gmail.com]

Hey David,

So I'm seeing on another processor where if I run AES under Keystone, the runtime + initialization cycles are just a tad under when I run it on base Linux. This is a big performance gap that has AES running significantly better, but I don't think it's due to paging. When I reported this result in August, I actually went back and changed the timebase frequency in the DTS to a value we see in the Rocket Core DTS under the Sifive/Freedom repo: 1000000. Even though that's a wrong value for the processor I'm working on (the actual value based on the hardware for when the cycle register gets updated is 6250000 for comparison). This is a processor running at 100MHz FYI. 

After changing it to the lower value just to see, this processor was reporting what I believe would be the correct cycle count, even though the Linux date/time is completely off. 

Here's some comparisons:

AES with 625000 Timebase Frequency (correct timebase based on the hardware):

Base AES Runtime: 14920622241

Base Real Time: 2m29.226s
Keystone AES Runtime: 8720936538

Keystone Real Time: 2m25.808s

Keystone Initialization: 5524916405

Cycle Difference: 6199685703

AES with 1000000 Timebase Frequency (incorrect timebase):

Base AES Runtime: 18016480566

Base Real Time: 18m46.314s

Keystone AES Runtime: 14962720786

Keystone Real Time: 21m51.782s

Keystone Initialization: 5657233091

Cycle Difference: 3095858325

Ignoring the Real Time that's off, the run with the incorrect DTS looks more what I would expect to see running the base vs Keystone. There's a bit faster performance with Keystone, but adding the initialization to it then makes the performance slightly longer. This lines up with what you've described and what I've seen.  

Would you happen to have any idea what would cause the large cycle difference with Keystone in the run with a correct timebase frequency? Or would the performance gap truly be that much? 

On Thursday, August 13, 2020 at 1:33:50 PM UTC-4 dkoh...@berkeley.edu wrote:

[David Kohlbrenner]

I'm not aware of anything in the Eyrie runtime or Keystone SM that should be affected by changing the timebase.

My best guess is that you are seeing a different frequency of context switches between the two cases, since the timebase freq is different (and so the scheduling quantum is now a different number of cycles?).

I'd measure the number of context switches that occur between the two cases and see if there is a notable difference.

-David

To view this discussion on the web visit https://groups.google.com/d/msgid/keystone-enclave-forum/31c77f19-fcfc-4731-877f-2d3e5dcd4267n%40googlegroups.com.