Does GCE support hardware TCP offloading?

[Junxian Zhu]

If yes, how? Do we need extra configuration?

I am reading post re: andromeda, does andromeda already included feature of tcp offloading? Thank you!

[Marilu]

As pert this article, you have this feature in the Debian backports-image. you might need to turn it on using the ethtool command.

Marilu

[Anthony Voellm]

One place to look for how to create an optimal image is to review the differences between the standard Debian image and the Debian Backports image provided on GCE.  The Backports images should likely be called Google Optimized Debian because it is the image where we have done the most work to get the best Virtio[net,scsi,...] drivers along with adding a more recent OpenSSL that uses AES/NI,  interrupt settings, etc.

On Mon, Jul 14, 2014 at 9:08 AM, Marilu <ad...@supportdomain060.info> wrote:

As pert this article, you have this feature in the Debian backports-image. you might need to turn it on using the ethtool command.

Marilu

--
© 2014 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/5785ff9e-e27a-4145-851e-6eeb2a90eddb%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Anthony F. Voellm (aka Tony)

Google Voice:  (650) 516-7382

Blog: http://perfguy.blogspot.com/

[Junxian Zhu]

Many thanks to Tony and Marilu.

Will this feature be available for CentOS image?

[Anthony Voellm]

GCE is working with the community to make sure the released images + kernels have the right components.  So the short answer is yes.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/5a42c6c1-b5d6-4af0-98e8-23046eb236e0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Junxian Zhu]

Many thanks, it is really great news!

[Junxian Zhu]

Sorry I have one more question, besides to use ethtool to enable tcp offload, do we have extra steps to enable support for OpenSSL? Or it is enabled by default.

[Evan Anderson]

You need a version of OpenSSL which is compiled with support for AES-NI.  I believe AES-NI support was implemented in OpenSSL 1.0.1.  If you have an old (several months old) VM, you may find that the virtual hardware does not advertise support for AES-NI, as many operating systems get confused if the "physical" hardware changes while running.  Starting a new instance (or maybe restart?) will cause the VM to advertise the AES-NI support.

TCP Offload is similar; you need both the hardware to advertise support and the kernel to be recent enough to detect and use that advertised hardware.

--

© 2014 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/7c5463a2-ea83-4e43-bc54-2ace84a30daf%40googlegroups.com.

[Junxian Zhu]

Hello!

Sorry for the late response! It appears GCE doesn't support aesni engine.

I built my server using the latest image - backports-debian-7-wheezy-v20141021.

My OpenSSL version is 1.0.1e which I believe already included support for aesni engine, but if I run: openssl engine, I can get below:

(rsax) RSAX engine support

(rdrand) Intel RDRAND engine

(dynamic) Dynamic engine loading support

Aesni support is not included.

Can anyone advise?

[Venkatesh Srinivas]

On Sat, Nov 8, 2014 at 1:31 AM, Junxian Zhu <thorv...@gmail.com> wrote:

Hello!

Sorry for the late response! It appears GCE doesn't support aesni engine.

I built my server using the latest image - backports-debian-7-wheezy-v20141021.

My OpenSSL version is 1.0.1e which I believe already included support for aesni engine, but if I run: openssl engine, I can get below:

(rsax) RSAX engine support

(rdrand) Intel RDRAND engine

(dynamic) Dynamic engine loading support

Aesni support is not included.

Can anyone advise?

OpenSSL does not use AES-NI via its engines support; it is used if available via CPUID. (If you look at /proc/cpuinfo, you will see 'aes' in flags).

To see the performance difference (on GCM mode, for example):

$ openssl speed -elapsed -evp aes-128-gcm

type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes

aes-128-gcm     202378.18k   433277.33k   587599.27k   646111.91k   672675.16k

(to disable AES-NI use:)

$ OPENSSL_ia32cap="~0x200000200000000" openssl speed -elapsed -evp aes-128-gcm

type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes

aes-128-gcm      37831.31k    45423.23k   110289.32k   120525.48k   122549.59k

(This was on an n1-highcpu-2 instance in us-central-1b, on an Intel Sandy Bridge CPU; your numbers may vary with other instance types and in Ivy Bridge zones).

HTH,

-- vs;

[Junxian Zhu]

HI VS,

Thanks for getting back to me.

I do see AES in /proc/cpuinfo, are you saying AES-NI support is enabled by default, and there is no action required to activate that? Thanks.

[Junxian Zhu]

Can anyone answer my question please?

[Anthony Voellm]

Full TCP Offloading / TOE is not supported today and its very likely you don't need it.  When TOE was first designed it was trying to work around issues in hitting 10Gbit line rates.  Today CPU's are more modern and can achieve high throughput and low latency.  There are also other offloads that NICs provide and ways of reducing overheads.

GCE does have TCP checksum offloads, Send and Receive Offloads, etc.

At the Cloud Live Event we displayed the following graph all without TOE:

One way to see the NIC capabilities is to boot a Debian Backports image and run ethtool to get the features used.  I've done that for you below:

voellm@voellm1:~$ sudo ethtool -k eth0

Features for eth0:

rx-checksumming: on [fixed]

tx-checksumming: on

tx-checksum-ipv4: off [fixed]

tx-checksum-ip-generic: on

tx-checksum-ipv6: off [fixed]

tx-checksum-fcoe-crc: off [fixed]

tx-checksum-sctp: off [fixed]

scatter-gather: on

tx-scatter-gather: on

tx-scatter-gather-fraglist: on

tcp-segmentation-offload: on

tx-tcp-segmentation: on

tx-tcp-ecn-segmentation: off [fixed]

tx-tcp6-segmentation: off [fixed]

udp-fragmentation-offload: off [fixed]

generic-segmentation-offload: on

generic-receive-offload: on

large-receive-offload: off [fixed]

rx-vlan-offload: off [fixed]

tx-vlan-offload: off [fixed]

ntuple-filters: off [fixed]

receive-hashing: off [fixed]

highdma: on [fixed]

rx-vlan-filter: off [fixed]

vlan-challenged: off [fixed]

tx-lockless: off [fixed]

netns-local: off [fixed]

tx-gso-robust: off [fixed]

tx-fcoe-segmentation: off [fixed]

tx-gre-segmentation: off [fixed]

tx-ipip-segmentation: off [fixed]

tx-sit-segmentation: off [fixed]

tx-udp_tnl-segmentation: off [fixed]

tx-mpls-segmentation: off [fixed]

fcoe-mtu: off [fixed]

tx-nocache-copy: off

loopback: off [fixed]

rx-fcs: off [fixed]

rx-all: off [fixed]

tx-vlan-stag-hw-insert: off [fixed]

rx-vlan-stag-hw-parse: off [fixed]

rx-vlan-stag-filter: off [fixed]

l2-fwd-offload: off [fixed]

busy-poll: off [fixed]

Hope this addresses your concern.

--
© 2014 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/CAK9J1LJ8CN0B6Hk_XQMW_%3Dk%3DnDf58AMKpVTa1Csk1Qc4XHdGzw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Junxian Zhu]

Hey, that addressed my concern, many thanks for your reply! It is very helpful!

[Venkatesh Srinivas]

On Sat, Nov 8, 2014 at 11:31 PM, Junxian Zhu <thorv...@gmail.com> wrote:

HI VS,

Thanks for getting back to me.

I do see AES in /proc/cpuinfo, are you saying AES-NI support is enabled by default, and there is no action required to activate that? Thanks.

For OpenSSL, that is correct.

HTH,

 -- vs;