Data storage operations, including content-indexing, containerized deduplication, and policy-driven storage, are performed within a cloud environment. The systems support a variety of clients and cloud storage sites that may connect to the system in a cloud environment that requires data transfer over wide area networks, such as the Internet, which may have appreciable latency and/or packet loss, using various network protocols, including HTTP and FTP. Methods are disclosed for content indexing data stored within a cloud environment to facilitate later searching, including collaborative searching. Methods are also disclosed for performing containerized deduplication to reduce the strain on a system namespace, effectuate cost savings, etc. Methods are disclosed for identifying suitable storage locations, including suitable cloud storage sites, for data files subject to a storage policy. Further, systems and methods for providing a cloud gateway and a scalable data object store within a cloud environment are disclosed, along with other features.

Data object store and server for a cloud storage environment, including data deduplication and data management across multiple cloud storage sites

A method and system for processing network metadata is described. Network metadata may be processed by dynamically instantiated executable software modules which make policy-based decisions about the character of the network metadata and about presentation of the network metadata to consumers of the information carried by the network metadata. The network metadata may be type classified and each subclass within a type may be mapped to a definition by a unique fingerprint value. The fingerprint value may be used for matching the network metadata subclasses against relevant policies and transformation rules. For template-based network metadata such as NetFlow v9, an embodiment of the invention can constantly monitor network traffic for unknown templates, capture template definitions, and informs administrators about templates for which custom policies and conversion rules do not exist. Conversion modules can efficiently convert selected types and/or subclasses of network metadata into alternative metadata formats.

Streaming method and system for processing network metadata

Systems and methods of mobile traffic categorization and policy implementation for network use optimization are disclosed. In one aspect, embodiments of the present disclosure include a method, which may be implemented on a system, of detecting an activity state of an application on a mobile device to which traffic is originated from or directed to, detecting a time criticality of data contained in the traffic to be sent between the mobile device and the host server, determining a timing with which to allow the traffic to pass through, based on the activity state or the time criticality and/or controlling radio use on the mobile device based on the timing with which the traffic is allowed to pass through. The method may further include selecting a network configuration in the wireless network for use in passing traffic to and from the mobile device.

Mobile traffic categorization and policy for network use optimization while preserving user experience

Systems and methods for mobile network traffic coordination across multiple applications are disclosed. In one aspect, embodiments of the present disclosure include a distributed proxy and cache system, including, a local proxy on a mobile device for intercepting a data request made via a mobile device, and a proxy server coupled to the mobile device and a content server to which the data request is directed. One embodiment includes, delaying transfer of a first data transfer request initiated by a first application until another data transfer request initiated by a second application is detected on the mobile device and transferring, the first data transfer request of the first application and the other data transfer request of the second application a single transfer operation over the network.

Mobile network traffic coordination across multiple applications

A logging system includes an event receiver and a storage manager. The receiver receives log data, processes it, and outputs a column-based data “chunk.” The manager receives and stores chunks. The receiver includes buffers that store events and a metadata structure that stores metadata about the contents of the buffers. Each buffer is associated with a particular event field and includes values from that field from one or more events. The metadata includes, for each “field of interest,” a minimum value and a maximum value that reflect the range of values of that field over all of the events in the buffers. A chunk is generated for each buffer and includes the metadata structure and a compressed version of the buffer contents. The metadata structure acts as a search index when querying event data. The logging system can be used in conjunction with a security information/event management (SIEM) system.

Storing log data efficiently while supporting querying

A logging system includes an event receiver and a storage manager. The receiver receives log data, processes it, and outputs a data 'chunk.' The manager receives data chunks and stores them so that they can be queried. The receiver includes buffers that store events and a metadata structure that stores metadata about the contents of the buffers. The metadata includes a unique identifier associated with the receiver, the number of events in the buffers, and, for each 'field of interest,' a minimum value and a maximum value that reflect the range of values of that field over all of the events in the buffers. A chunk includes the metadata structure and a compressed version of the contents of the buffers. The metadata structure acts as a search index when querying event data. The logging system can be used in conjunction with a security information/event management (SIEM) system.

Storing log data efficiently while supporting querying to assist in computer network security

A data storage system includes a storage engine to partition data across multiple dimensions. The storage engine determines chunks according to the partitioning, and performs column-based storage of the chunks.

Multidimension column-based partitioning and storage

In some examples, a set of events is received. A row-based chunk includes the set of events and metadata about the set of events is generated, and a column-based chunk that includes metadata about the set of events and, for each event in the set of events, a value of a first field of the multiple fields. The metadata about the set of events includes at least one of a minimum value or a maximum value of the first field over the events in the set of events.

Wei Huang

Papers

Storing log data efficiently while supporting querying

Storing log data efficiently while supporting querying to assist in computer network security

Multidimension column-based partitioning and storage

Generating row-based and column-based chunks

Effiziente speicherung von protokolldaten während der unterstützung einer abfrage zur erhöhung der sicherheit eines computernetzwerks