Google announced open source offload friendly protocol PSP

Google today announced that the PSP (short for PSP Security Protocol) protocol is open source. The protocol is designed to handle data center-scale encryption hardware offloading and is currently deployed in Google's production.

Link: https://cloud.google.com/blog/products/identity-security/announcing-psp-security-protocol-is-now-open-source

To better protect user privacy, Google encrypted traffic between data centers more than a decade ago. In subsequent developments, nearly all data transmitted by Google has been encrypted. While this work provides valuable privacy and security benefits, software encryption comes at a significant cost: encrypting and decrypting RPCs requires about 0.7% of Google's processing power, and a corresponding amount of memory.

These costs prompted Google to use PSP (short for PSP Security Protocol) to offload encryption to the network interface card (NIC). Offload means that some of the packet processing (such as TCP segmentation, IP fragmentation, reassembly, checksum, TCP protocol processing, etc.) that would otherwise be done by the operating system is put into the NIC hardware, reducing system CPU consumption while increasing processing performance.

Given that TLS is not friendly enough, lacks support for UDP, and has some flaws with IPsec, Google took it upon themselves to develop their own Offload friendly protocol. PSP as their solution is described as a TLS-like, transport-independent protocol for per-connection security and Offload friendliness.

For PSP, Amin Vahdat of the Google Cloud team explains:

PSP is designed to meet the requirements of large-scale data center traffic. It does not mandate a specific key exchange protocol and provides few choices for packet formats and encryption algorithms. It achieves per-connection security by allowing each Layer 4 connection (e.g., TCP connection) to use encryption keys.

It supports stateless operation because the encryption state can be passed to the device through the packet descriptor as the packet is transmitted, and the packet can be received while using the Security Parameter Index (SPI) and the master key on the device. This allows us to maintain minimal state in the hardware and avoid hardware state explosion compared to typical state encryption techniques that maintain large device tables.

PSP uses User Datagram Protocol (UDP) encapsulation with a custom header and tail tag. A PSP packet begins with a raw IP header, followed by a UDP header on a pre-specified destination port, followed by a PSP header containing PSP information, followed by a raw TCP/UDP packet (including header and payload) with a PSP trailer containing an Integrity Checksum Value (ICV).

Layer 4 packets (header and payload) can be encrypted or validated based on a user-supplied offset called Crypt Offset. For example, this field can be used to keep part of the TCP header authenticated but unencrypted during transmission, while keeping the rest of the packet encrypted to support packet sampling and inspection in the network (if necessary).

Google patched PSP into their production Linux kernel, their Andromeda network virtualization stack and their Snap networking system. The PSP encryption offload reportedly saves about 0.5% of Google's overall processing power. Today, they are open-sourcing the PSP security protocol to encourage further adoption. They have released their architectural specification, reference software implementation and a set of test cases.