Latency (engineering)

About: Latency (engineering) is a research topic. Over the lifetime, 7278 publications have been published within this topic receiving 115409 citations. The topic is also known as: lag.

C3: cutting tail latency in cloud data stores via adaptive replica selection

Abstract: Achieving predictable performance is critical for many distributed applications, yet difficult to achieve due to many factors that skew the tail of the latency distribution even in well-provisioned systems. In this paper, we present the fundamental challenges involved in designing a replica selection scheme that is robust in the face of performance fluctuations across servers. We illustrate these challenges through performance evaluations of the Cassandra distributed database on Amazon EC2. We then present the design and implementation of an adaptive replica selection mechanism, C3, that is robust to performance variability in the environment. We demonstrate C3's effectiveness in reducing the latency tail and improving throughput through extensive evaluations on Amazon EC2 and through simulations. Our results show that C3 significantly improves the latencies along the mean, median, and tail (up to 3 times improvement at the 99.9th percentile) and provides higher system throughput.

ChargeCache: Reducing DRAM latency by exploiting row access locality

Abstract: DRAM latency continues to be a critical bottleneck for system performance. In this work, we develop a low-cost mechanism, called Charge Cache, that enables faster access to recently-accessed rows in DRAM, with no modifications to DRAM chips. Our mechanism is based on the key observation that a recently-accessed row has more charge and thus the following access to the same row can be performed faster. To exploit this observation, we propose to track the addresses of recently-accessed rows in a table in the memory controller. If a later DRAM request hits in that table, the memory controller uses lower timing parameters, leading to reduced DRAM latency. Row addresses are removed from the table after a specified duration to ensure rows that have leaked too much charge are not accessed with lower latency. We evaluate ChargeCache on a wide variety of workloads and show that it provides significant performance and energy benefits for both single-core and multi-core systems.

Effect of click rate on the latency of auditory brain stem responses in humans.

Abstract: Auditory brain stem responses are the far-field reflections of electrical activity originating in the auditory pathway in its course from the cochlea to cortex that can be recorded from scalp electrodes using computer averaging techniques. There are seven components in the initial 10 msec following a click signal which have been shown to have an orderly change in latency as a function of signal intensity. The results of this study show that click repetition rate can also significantly affect the response latency measure. Responses were measured in six normal hearing subjects at click rates of 10, 30, 50 and 100/sec and af four intensity levels (30, 40, 50, and 60 dB sensation level). The mean latency shift of component V was approximately 0.5 msec when the responses at 10 and 100/sec were compared. This is equivalent to a 15-20 dB decrease in signal intensity at the 10/sec click rate. An analysis of the time of occurrence of this shift using brief click trains at 100/sec showed the shift in latency to be complete by the fifth click. The latency shift was similar at the four signal levels tested. The latency shift was similar at the four signal levels tested. The latency shift of component V appeared to be a monaural and therefore a potentially peripheral process. The results are interpreted as an objective measure of adaptation in the human auditory system with implications for the measurement in disorders of hearing.

A Streaming On-Device End-To-End Model Surpassing Server-Side Conventional Model Quality and Latency

Abstract: Thus far, end-to-end (E2E) models have not been shown to outperform state-of-the-art conventional models with respect to both quality, i.e., word error rate (WER), and latency, i.e., the time the hypothesis is finalized after the user stops speaking. In this paper, we develop a first-pass Recurrent Neural Network Transducer (RNN-T) model and a second-pass Listen, Attend, Spell (LAS) rescorer that surpasses a conventional model in both quality and latency. On the quality side, we incorporate a large number of utterances across varied domains [1] to increase acoustic diversity and the vocabulary seen by the model. We also train with accented English speech to make the model more robust to different pronunciations. In addition, given the increased amount of training data, we explore a varied learning rate schedule. On the latency front, we explore using the end-of-sentence decision emitted by the RNN-T model to close the microphone, and also introduce various optimizations to improve the speed of LAS rescoring. Overall, we find that RNN-T+LAS offers a better WER and latency tradeoff compared to a conventional model. For example, for the same latency, RNN-T+LAS obtains a 8% relative improvement in WER, while being more than 400-times smaller in model size.

Understanding spatio-temporal uncertainty in medium access with ALOHA protocols

Abstract: The goal of this paper is understand how location-dependent propagation latency affects medium access control (MAC) by using ALOHA as a case study. MAC protocols in underwater acoustic networks suffer from latency that is five orders-of-magnitude larger than that in radio networks. Existing work on analyzing MAC throughput in RF networks, where the propagation latency is negligible, generally makes assumptions that render propagation latency irrelevant. As a result, only transmit time is considered as being uncertain in contention-based protocols. In this paper, we investigate the spatial dimension of uncertainty that is inherent to varying locations of transmitters, resulting in unequal propagation latency to a receiver. We show through simulation that the benefit of synchronization in slotted ALOHA is completely lost due to such latency. To handle spatial uncertainty, we propose a modification that adds guard bands to transmission slots. We then perform simulation and first-order analysis on this modified MAC to find its optimal operating parameters. Our simulation and analytic results suggest that shorter hops improve throughput.