Fully Homomorphic Encryption (FHE) for Data Vectors

The Cheon-Kim-Kim-Song (CKKS) scheme was selected for its unique ability to handle real and complex numbers, making it ideal for applications requiring precise arithmetic operations, such as LLM training and proof-of-work computations. CKKS works by encoding real or complex numbers into polynomials, which are then encrypted. This approach allows the scheme to approximate arithmetic on encrypted real numbers. Encrypted data can undergo homomorphic operations, such as addition and multiplication, which correspond to operations on the coefficients of these polynomials.

Another FHE method that was considered was Fast Fully Homomorphic Encryption over the Torus (TFHE). While this method is sufficient for POS, it is not selected for handling data vectors due to its incompatibility with approximate arithmetic, which is a requirement for LLMs. The benefits of CKKS over TFHE are as follows:

• Arithmetic on Encrypted Data: CKKS supports efficient arithmetic operations on encrypted data, enabling complex calculations directly on ciphertexts. TFHE is optimized for exact computations on binary data, which is less flexible for applications needing approximate arithmetic.

• Handling Real and Complex Numbers: CKKS is uniquely suited for applications involving real and complex numbers, broadening its applicability to various fields. TFHE is primarily focused on Boolean gate operations and binary data, making it less suitable for tasks involving real or complex numbers.

• Noise Management: CKKS includes sophisticated mechanisms to manage and mitigate noise growth during arithmetic operations, ensuring that computations remain precise and reliable over multiple operations. TFHE manages noise effectively for gate operations, but faces challenges with the noise levels in arithmetic computations on large datasets.

• Better Suitability for Machine Learning: CKKS's ability to handle real numbers and perform efficient arithmetic operations makes it better suited for machine learning applications, including training and inference on encrypted data. TFHE is designed for binary gate operations, making it incompatible with the arithmetic needs of machine learning tasks, particularly for LLMs.