Name-based route lookup is a key function for Named Data Networking (NDN). The NDN names are hierarchical and have variable and unbounded lengths, which are much longer than IPv4/6 address, making fast name lookup a challenging issue. In this paper, we propose an effective Name Component Encoding (NCE) solution with the following two techniques: (1) A code allocation mechanism is developed to achieve memory-efficient encoding for name components, (2) We apply an improved State Transition Arrays to accelerate the longest name prefix matching and design a fast and incremental update mechanism which satisfies the special requirements of NDN forwarding process, namely to insert, modify, and delete name prefixes frequently. Furthermore, we analyze the memory consumption and time complexity of NCE. Experimental results on a name set containing 3,000,000 names demonstrate that compared with the character trie NCE reduces overall 30% memory. Besides, NCE performs a few millions lookups per second (on an Intel 2.8 GHz CPU), a speedup of over 7 times compared with the character trie. Our evaluation results also show that NCE can scale up to accommodate the potential future growth of the name sets.

/pdf/scalable-name-lookup-in-ndn-using-effective-name-component-12e201quww.pdf

Scalable Name Lookup in NDN Using Effective Name Component Encoding

In this paper we design, implement and evaluate NameFilter, a two-stage Bloom filter-based scheme for Named Data Networking name lookup, in which the first stage determines the length of a name prefix, and the second stage looks up the prefix in a narrowed group of Bloom filters based on the results from the first stage. Moreover, we optimize the hash value calculation of name strings, as well as the data structure to store multiple Bloom filters, which significantly reduces the memory access times compared with that of non-optimized Bloom filters. We conduct extensive experiments on a commodity server to test NameFilter's throughput, memory occupation, name update as well as scalability. Evaluation results on a name prefix table with 10M entries show that our proposed scheme achieves lookup throughput of 37 million searches per second at low memory cost of only 234.27 MB, which means 12 times speedup and 77% memory savings compared to the traditional character trie structure. The results also demonstrate that NameFilter can achieve 3M per second incremental updates and exhibit good scalability to large-scale prefix tables.

NameFilter: Achieving fast name lookup with low memory cost via applying two-stage Bloom filters

URL filtering systems provide a simple and effective way to prevent people from browsing undesirable or malicious websites. These systems require a well-designed URL lookup method as the core operation. A high-performance URL lookup engine is proposed in this paper for URL filtering systems. It combines a URL compression algorithm with a multiple string matching based (Wu-Manber-like) matching algorithm. Using this method, the proposed URL lookup engine can achieve high URL lookup performance and efficient memory utilization for storing the ever-increasing URL blacklist with the ability of prefix matching. Experiments with actual URL blacklists and requested URL sets show that our engine can save about 80% memory usage for storing URL blacklists, and reduce 58%-162% URL lookup time compared with the state-of-the-art URL lookup methods.

A High-Performance URL Lookup Engine for URL Filtering Systems

Name lookup is an essential function, but a performance bottleneck in both today's and future network architectures. Variable-length and unbounded names rather than fixed-length addresses, as well as much larger and more dynamic forwarding tables call for a careful re-engineering of lookup structures for fast, memory-efficient, and scalable packet forwarding. We propose a novel data structure, called NameTrie, to store and index forwarding table entries efficiently and to support fast name lookups and updates. Its novelty lies in the optimized design and implementation of a character-trie structure. The nodes of NameTrie are stored compactly, improving cache efficiency and speeding up packet processing. Its edges are implemented using a hash table, facilitating fast name lookups and updates. A new scheme is used to encode control information without consuming additional memory. Running on conventional commodity hardware and using large-scale real-world name datasets, our implementation of NameTrie in software achieves 2.82~3.56, 3.48~3.72, and 2.73~3.25 million name insertions, lookups, and removals per second, respectively, for various datasets while requiring a small memory footprint. We have conducted a comprehensive performance evaluation against the state-of-the-art of named data networking (NDN) as a typical use-case. It is shown to require at least 35% less memory and runs at least 3x faster for name table lookups and updates than two well-known trie-based schemes in NDN.

A Fast and Memory-Efficient Trie Structure for Name-Based Packet Forwarding

Name lookup is an essential function but a performance bottleneck in both today’s and future network architectures. Trie is an excellent candidate data structure and has been widely used for looking up and updating names. However, the granularity of trie—at bit, byte (character), or component level—can dramatically affect the network performance in terms of memory usage and packet-processing speed, which has not yet been studied adequately. To fill this gap, we first show that the choice of trie’s granularity for name lookups and updates (i.e., insertions and removals) is not a trivial problem due to the complex performance tradeoffs involved. We also introduce a new tool, called NameGen, which uses a Markov-based name learning model and generates pseudo-real datasets with different tunable name characteristics. We compare different trie granularities based on a collection of datasets and performance metrics, highlight the strengths and weaknesses of each granularity, and draw a conclusion on the choice of granularity. Surprisingly, our experimental evaluation finds that there are only two key rules to choose the proper trie’s granularity for any kind of dataset: 1) bit-level trie is the choice when the memory requirement is a real concern and 2) character- and component-level tries are preferred for faster lookups and updates when dealing with names composed of short and long components, respectively.

On the Granularity of Trie-Based Data Structures for Name Lookups and Updates

Hashing large collection of URLs is an inevitable problem in many Web research activities. Through a large scale experiment, three hash functions are compared in this paper. Two metrics were developed for the comparison, which are related to web structure analysis and Web crawling, respectively. The finding is that the well-known function for hashing sequence of symbols, ELFhash, is not very good in this regard, and the other two functions are better and thus recommended.

/pdf/two-effective-functions-on-hashing-url-2hbmdv2d68.pdf

LI Xiao-Ming

Papers

Two Effective Functions on Hashing URL