Fully Homomorphic Encryption (FHE) allows computing on encrypted data, enabling secure offloading of computation to untrusted servers. Though it provides ideal security, FHE is expensive when executed in software, 4 to 5 orders of magnitude slower than computing on unencrypted data. These overheads are a major barrier to FHE’s widespread adoption. We present F1, the first FHE accelerator that is programmable, i.e., capable of executing full FHE programs. F1 builds on an in-depth architectural analysis of the characteristics of FHE computations that reveals acceleration opportunities. F1 is a wide-vector processor with novel functional units deeply specialized to FHE primitives, such as modular arithmetic, number-theoretic transforms, and structured permutations. This organization provides so much compute throughput that data movement becomes the key bottleneck. Thus, F1 is primarily designed to minimize data movement. Hardware provides an explicitly managed memory hierarchy and mechanisms to decouple data movement from execution. A novel compiler leverages these mechanisms to maximize reuse and schedule off-chip and on-chip data movement. We evaluate F1 using cycle-accurate simulation and RTL synthesis. F1 is the first system to accelerate complete FHE programs, and outperforms state-of-the-art software implementations by gmean 5,400 × and by up to 17,000 ×. These speedups counter most of FHE’s overheads and enable new applications, like real-time private deep learning in the cloud.

https://dl.acm.org/doi/pdf/10.1145/3466752.3480070

F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption

Low-delay hierarchical coding structure (LD-HCS), as one of the most important components in the latest High Efficiency Video Coding (HEVC) standard, greatly improves coding performance. It groups consecutive P/B frames into different layers and encodes them with different quantization parameters (QPs) and reference mechanisms in such a way that temporal dependency among frames can be exploited. However, due to varying characteristics of video contents, temporal dependency among coding units differs significantly from each other in the same or different layers, while a fixed LD-HCS scheme cannot take full advantage of the dependency, leading to a substantial loss in coding performance. This paper addresses the temporally dependent rate distortion optimization (RDO) problem by attempting to exploit varying temporal dependency of different units. First, the temporal relationship of different frames under the LD-HCS is examined, and hierarchical temporal propagation chains are constructed to represent the temporal dependency among coding units in different frames. Then, a hierarchical temporally dependent RDO scheme is developed specifically for the LD-HCS based on a source distortion propagation model. Experimental results show that our proposed scheme can achieve 2.5% and 2.3% BD-rate gain in average compared with the HEVC codec under the same configuration of P and B frames, respectively, with a negligible increase in encoding time. Furthermore, coupled with QP adaption, our proposed method can achieve higher coding gains, e.g., with multi-QP optimization, about 5.4% and 5.0% BD-rate saving in average over the HEVC codec under the same setting of P and B frames, respectively.

Temporally Dependent Rate-Distortion Optimization for Low-Delay Hierarchical Video Coding

The high efficiency video coding (HEVC) partial encryption (PE) technique depends on encrypting the highly sensitive data on the video bit stream. The HEVC PE technique should keep the video format compliance, should be of the same bit rate, and ensure real-time constraints. The paper suggests an effective RC6 HEVC PE technique which encrypts sensible video data bits with low complexity overhead, fast encoding time for real-time applications, and fixed HEVC bitrate. These features result from using the low computational complexity RC6 block cipher for encrypting the selective video bins. The proposed RC6 HEVC PE encrypts the discrete cosine transform (DCT) coefficients sign bit, the DCT remaining absolute values suffixes that are binarized by Exp-Golomb (EGk) order zero, the motion vector difference (MVD) sign bits, and MVD absolute values suffixes that are binarized by EGk order one. Also, this paper introduces experimental results that compare between the proposed RC6 HEVC PE and HEVC PE algorithms that use the Advanced Encryption Standard in different operation modes. This paper presents more details about the security analysis of RC6 HEVC PE including encryption quality test, key space test, statistical analysis such as histogram and correlation coefficient analysis, and sensitivity analysis such as the key sensitivity analysis. The achieved test results ensured and confirmed the security, reliability, and robustness of RC6 HEVC PE technique.

HEVC Selective Encryption Using RC6 Block Cipher Technique

Power constraints constitute a critical design issue for the portable video codec system, in which the external dynamic random access memory (DRAM) accounts for more than half of the overall system power requirements. With the ultrahigh-definition video specifications, the power consumed by accessing reference frames in the external DRAM has become the bottleneck for the portable video encoding system design. To relieve the dynamic power stresses introduced by the DRAM, a lossless compression algorithm is devised to reduce the external traffic and the memory requirements of reference frames. First, pixel-granularity directional prediction is adopted to decrease the prediction residual energy by 54.1% over the previous horizontal prediction. Second, the dynamic $k$ th-order unary/Exp-Golomb rice coding is applied to accommodate the large-valued prediction residues. With the aforementioned techniques, an average data traffic reduction of 68.5% for the off-chip reference frames is obtained, which consequently reduces the dynamic power requirements of the DRAM by 42.3%. Based on the high data reduction ratio of the proposed compression algorithm, a partition group table-based storage space reduction scheme is provided to improve the utilization of row buffers in the DRAM. Consequently, an additional 14.5% of the DRAM dynamic power can be saved by reducing the number of row buffer activations. In total, a 56.8% decrease in the dynamic power requirements of the external reference frame access can be obtained using our strategies. With TSMC 65-nm CMOS logic technology, our algorithm was implemented in a parallel VLSI architecture based on a compressor and decompressor at the cost of 36.5k and 34.7k, respectively, in terms of gate count. The throughputs of the proposed compressor and decompressor are 1.54 and 0.78 Gpixels/s, which are suitable for quad full high definition (4K) @ 94 frames/s real-time encoding with the level-D reference data reuse scheme.

Lossless Frame Memory Compression Using Pixel-Grain Prediction and Dynamic Order Entropy Coding

Ultrahigh definition (UHD), such as 4K/8K, is becoming the mainstream of video resolution nowadays. High Efficiency Video Coding (HEVC) is the emerging video coding standard to process the encoding and decoding of UHD video. This paper first develops multiple techniques that allow the proposed hardware architecture for intra prediction of HEVC working in full pipeline. The proposed techniques include: 1) a novel buffer structure for reference samples; 2) a mode-dependent scanning order; and 3) an inverse method for reference sample extension. The size of the buffer is 3K b for luma component and 3K b for chroma components, providing sufficient accessing to the reference samples. Since the data dependency between two neighboring blocks is addressed by the mode-dependent scanning order, the proposed fully pipelined design can produce 4 pixels/clock cycle. As a result, the throughput of the proposed architecture is capable to support $3840 \times 2160$ videos at 30 frames/s.

A Fully Pipelined Hardware Architecture for Intra Prediction of HEVC

As the next generation standard of video coding, the High Efficiency Video Coding (HEVC) aims to provide significantly improved compression performance in comparison with all existing video coding standards. We propose a four-stage pipeline hardware architecture on a quarter-LCU basis of deblocking filter in HEVC. Coupled with the novel filter order, a memory interlacing technique is adopted to increase the throughput, which can access the data in the process of both vertical and horizontal filtering efficiently. As a result, our design can support 4K×2K (4096×2048) at 30 fps applications with merely 28 MHz working frequency.

A high-throughput VLSI architecture for deblocking filter in HEVC

The latest video coding standard high-efficiency video coding (HEVC) provides 50% improvement in coding efficiency compared to H.264/AVC to meet the rising demands for video streaming, better video quality, and higher resolution. The deblocking filter (DF) and sample adaptive offset (SAO) play an important role in the HEVC encoder, and the SAO is newly adopted in HEVC. Due to the high throughput requirement in the video encoder, design challenges such as data dependence, external memory traffic, and on-chip memory area become even more critical. To solve these problems, we first propose an interlacing memory organization on the basis of quarter-LCU to resolve the data dependence between vertical and horizontal filtering of DF. The on-chip SRAM area is also reduced to about 25% on the basis of quarter-LCU scheme without throughput loss. We also propose a simplified bitrate estimation method of rate-distortion cost calculation to reduce the computational complexity in the mode decision of SAO. Our proposed hardware architecture of combined DF and SAO is designed for the HEVC intraencoder, and the proposed simplified bitrate estimation method of SAO can be applied to both intra- and intercoding. As a result, our design can support ultrahigh definition 7680 × 4320 at 40 f/s applications at merely 182 MHz working frequency. Total logic gate count is 103.3 K in 65 nm CMOS process.

A Combined Deblocking Filter and SAO Hardware Architecture for HEVC

This brief describes a new method to implement the single-port SRAM-based transpose memory for large size discrete cosine transform (DCT)/indiscrete cosine transform (IDCT) which are used in the latest video coding standard, such as high efficiency video coding. Instead of shift-register array or multiport SRAM, only single-port SRAM is used in the proposed design. A new diagonal data mapping scheme is proposed to reduce the number of SRAM banks used to implement the transpose memory. This design can be flexibly extended to support DCT/IDCT of different transform sizes and different data throughput rates. To support larger size DCT/IDCT, only the depth of SRAM needs to be increased. To support different data throughput rate, multiple SRAM banks are well organized according to the required throughput. Row access and column access can be perfectly supported under single port SRAM. The equivalent gate count per bit (EGC) of proposed approach is less than two, which is much more efficient than the previous method. It is suitable for real-time processing of the video with the resolution up to 1080P HD or even higher.

Single-Port SRAM-Based Transpose Memory With Diagonal Data Mapping for Large Size 2-D DCT/IDCT

Frame recompression is an efficient way to reduce the huge bandwidth of external memory for video encoder, especially for P/B frame compression. A novel algorithm, which is called mixed lossy and lossless (MLL) reference frame recompression, is proposed in this paper. The bandwidth reduction comes from two sources in our scheme, which differs from its previous designs and achieves a much higher compression ratio. First, it comes from pixel truncation. We use truncated pixels (PR) for integer motion estimation (IME) and acquire truncated residuals for factional motion estimation (FME) and motion compensation (MC). Because the pixel access of IME is much larger than FME and MC, it saves about 37.5% bandwidth under 3-b truncation. Second, embedded compression of PR helps to further reduce data. The truncated pixels in the first stage greatly help to achieve a higher compression ratio than current designs. From our experiments, 3-b truncated PR can be compressed to 15.4% of the original data size, while most current embedded compressions can only achieve around 50%. For PR compression, two methods are proposed: in-block prediction and small-value optimized variable length coding. With these experiments, the total bandwidth can be reduced to 25.5%. Our proposed MLL is hardware/software friendly and also fast IME algorithm friendly frame recompression scheme. It is more suitable to work together with the data-reuse strategy than the previous schemes, and the video quality degradation is controllable and negligible.

In-Block Prediction-Based Mixed Lossy and Lossless Reference Frame Recompression for Next-Generation Video Encoding

The invention belongs to the technical field of high-definition digital video compression coding and decoding, and particularly relates to an encoder intra-frame prediction device and method applicable to HEVC standards. The encoder intra-frame prediction device comprises a control unit, two prediction engines, a rebuilt loop and a series of buffers. One prediction engine (A) is arranged at the first stage of a system stage assembly line and is responsible for selecting the optimal prediction mode and the best block dividing situation, the other prediction engine (B) and the rebuilt loop are combined to be arranged at the second stage of the system stage assembly line, results of the prediction engine (A) are used for conducting prediction and calculating residual errors, and then conversion processing, quantization processing, inverse quantization processing and inverse conversion processing are carried out on the residual errors. According to the encoder intra-frame prediction device and method, the data dependency in the prediction process can be effectively reduced, and the throughput rate can be improved. In addition, a common prediction unit and an appropriate scanning sequence reduce the cost of hardware, an internal high-streamlined structure enables the working frequency to be remarkably improved, and accordingly real-time coding on the high-definition video is achieved.

Shang Qing

Papers

A high-throughput VLSI architecture for deblocking filter in HEVC

A Combined Deblocking Filter and SAO Hardware Architecture for HEVC

Single-Port SRAM-Based Transpose Memory With Diagonal Data Mapping for Large Size 2-D DCT/IDCT

In-Block Prediction-Based Mixed Lossy and Lossless Reference Frame Recompression for Next-Generation Video Encoding

Encoder intra-frame prediction device and method applicable to HEVC standards