TCP Segmentation Offloading

TCP Segmentation Offloading

TCP Segmentation Offloading, as the name suggests, offload work of segmenting application data to Ethernet Card, reducing kernel work of data processing/ packetization at TCP/ IP layer. At TCP/ IP layer, application data is splitted into data segments and TCP/ IP headers are appended to the segments to form packets. This work of kernel along with performing checksum could be bottleneck on host's CPU utilization for applications which send large packets like 64KB or larger with high speed.

Fig 1. shows the task performed by networking TCP/ IP stack of of segmentation, adding header, performing checksums etc.

Fig 1. Packet Processing without TCP Segment Offload

Fig 2. shows the task performed by networking TCP/ IP stack has been delegated directly to Ethernet controller. TCP/ IP stack just passes the large data directly to NIC. NIC performs segmentation, appending header, checksums and passing frames to physical layer.

Fig 2. Packet Processing with TCP Segment Offload

Obvious question is why do we need Segmentation Offloading and where will it be useful?
Consider your host does lot of network file transfers. These files if smaller in size wont show lot of CPU/ IO overhead, but if these files are larger like 1MB, while passing down them to the NIC, kernel needs to do lot of data segmenting and processing, causing CPU load to be higher. Modern network card offer high speed processors. Almost every new system comes with a 1 Gigabit per second (Gbps) network card. Offloading work of TCP/ IP stack to network card could decrease CPU load by 10-30% and keep your machine happy.
NOTE: This does not mean that your transfer would go fast, but yes in case of servers doing large file transfers you could see CPU usage drops.

Commands to check/ enable/ disable TSO settings on Centos

#Check offload settings

[root@machine1 ~]# ethtool -k eth1
Offload parameters for eth1:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
udp fragmentation offload: off
generic segmentation offload: on
generic-receive-offload: on

#Switch off TSO
[root@machine1 ~]# ethtool -K eth1 tso off

#Switch on TSO
[root@machine1 ~]# ethtool -K eth1 tso on