Showing papers by "Sony Broadcast & Professional Research Laboratories published in 2019"

Emerging MPEG Standards for Point Cloud Compression

Abstract: Due to the increased popularity of augmented and virtual reality experiences, the interest in capturing the real world in multiple dimensions and in presenting it to users in an immersible fashion has never been higher. Distributing such representations enables users to freely navigate in multi-sensory 3D media experiences. Unfortunately, such representations require a large amount of data, not feasible for transmission on today’s networks. Efficient compression technologies well adopted in the content chain are in high demand and are key components to democratize augmented and virtual reality applications. Moving Picture Experts Group, as one of the main standardization groups dealing with multimedia, identified the trend and started recently the process of building an open standard for compactly representing 3D point clouds, which are the 3D equivalent of the very well-known 2D pixels. This paper introduces the main developments and technical aspects of this ongoing standardization effort.

Open-Unmix - A Reference Implementation for Music Source Separation

Abstract: Music source separation is the task of decomposing music into its constitutive components, e.g., yielding separated stems for the vocals, bass, and drums. Such a separation has many applications ranging from rearranging/repurposing the stems (remixing, repanning, upmixing) to full extraction (karaoke, sample creation, audio restoration). Music separation has a long history of scientific activity as it is known to be a very challenging problem. In recent years, deep learning-based systems-for the first time-yielded high-quality separations that also lead to increased commercial interest. However, until now, no open-source implementation that achieves state-of-the-art results is available. Open-Unmix closes this gap by providing a reference implementation based on deep neural networks. It serves two main purposes. Firstly, to accelerate academic research as Open-Unmix provides implementations for the most popular deep learning frameworks, giving researchers a flexible way to reproduce results. Secondly, we provide a pre-trained model for end users and even artists to try and use source separation. Furthermore, we designed Open-Unmix to be one core component in an open ecosystem on music separation, where we already provide open datasets, software utilities, and open evaluation to foster reproducible research as the basis of future development.

PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things

Abstract: We propose PanopticFusion, a novel online volumetric semantic mapping system at the level of stuff and things. In contrast to previous semantic mapping systems, PanopticFusion is able to densely predict class labels of a background region (stuff) and individually segment arbitrary foreground objects (things). In addition, our system has the capability to reconstruct a large-scale scene and extract a labeled mesh thanks to its use of a spatially hashed volumetric map representation. Our system first predicts pixel-wise panoptic labels (class labels for stuff regions and instance IDs for thing regions) for incoming RGB frames by fusing 2D semantic and instance segmentation outputs. The predicted panoptic labels are integrated into the volumetric map together with depth measurements while keeping the consistency of the instance IDs, which could vary frame to frame, by referring to the 3D map at that moment. In addition, we construct a fully connected conditional random field (CRF) model with respect to panoptic labels for map regularization. For online CRF inference, we propose a novel unary potential approximation and a map division strategy. We evaluated the performance of our system on the ScanNet (v2) dataset. PanopticFusion outperformed or compared with state-of-the-art offline 3D DNN methods in both semantic and instance segmentation benchmarks. Also, we demonstrate a promising augmented reality application using a 3D panoptic map generated by the proposed system.

Recursive Speech Separation for Unknown Number of Speakers.

Abstract: In this paper we propose a method of single-channel speaker-independent multi-speaker speech separation for an unknown number of speakers. As opposed to previous works, in which the number of speakers is assumed to be known in advance and speech separation models are specific for the number of speakers, our proposed method can be applied to cases with different numbers of speakers using a single model by recursively separating a speaker. To make the separation model recursively applicable, we propose one-and-rest permutation invariant training (OR-PIT). Evaluation on WSJ0-2mix and WSJ0-3mix datasets show that our proposed method achieves state-of-the-art results for two- and three-speaker mixtures with a single model. Moreover, the same model can separate four-speaker mixture, which was never seen during the training. We further propose the detection of the number of speakers in a mixture during recursive separation and show that this approach can more accurately estimate the number of speakers than detection in advance by using a deep neural network based classifier.

Wideband Beam-Switchable 28 GHz Quasi-Yagi Array for Mobile Devices

Abstract: The goal of this paper is to propose a new antenna array architecture that aims to solve the most known limitations of phased antenna arrays, resulting a good candidate for next 5G mobile handsets. The architecture consists of five quasi-Yagi antennas printed on the short edge of a Roger RO3003 substrate, pointing in different directions, and a switch to feed each antenna and steer the beam. Simulations prove that the antenna array can cover the angle of over 180° with high gain over the frequency range from 26 to 40 GHz. Alternative designs to make the structure more compact further demonstrate the validity of the concept. The optimized corner array of four elements is fabricated and passive and active measurements are performed with the Microwave Vision Group (MVG) Starlab 50 GHz. The results of the passive measurements are in accordance with the simulations and show that the proposed quasi-Yagi antenna array has large coverage over the whole bandwidth and peak gain of 8 dBi at 28 and 38 GHz. The active measurements of the array connected to the front end module (FEM) and integrated in the phone case further confirm the radiation properties of the switchable antenna array at 28 GHz in a quasi-real scenario.

Measurements of electrically tunable refractive index of MoS2 monolayer and its usage in optical modulators

Abstract: Two-dimensional materials hold a great promise for developing extremely fast, compact and inexpensive optoelectronic devices. A molybdenum disulphide (MoS2) monolayer is an important example which shows strong, stable and gate tunable optical response even at room temperature near excitonic transitions. However, optical properties of a MoS2monolayer are not documented well. Here, we investigate the electric field effect on optical properties of a MoS2 monolayer and extract the dependence of MoS2 optical constants on gating voltage. The field effect is utilised to achieve ~10% visible light modulation for a hybrid electro-optical waveguide modulator based on MoS2. A suggested hybrid nanostructure consists of a CMOS compatible Si3N4 dielectric waveguide sandwiched between a thin gold film and a MoS2 monolayer which enables a selective enhancement of polarised electro-absorption in a narrow window of angles of incidence and a narrow wavelength range near MoS2 exciton binding energies. The possibility to modulate visible light with 2D materials and the robust nature of light modulation by MoS2 could be useful for creation of reliable ultra-compact electro-optical hybrid visible-light modulators.

Random access in next generation wireless systems

Abstract: A wireless transmit/receive unit (WTRU) may initiate a random access. The WTRU may determine whether to select a first random access channel (RACH) procedure or a second RACH procedure for the random access. The first RACH procedure may be a legacy RACH procedure. The second RACH procedure may be an enhanced RACH (eRACH) procedure. The WTRU may determine whether to select the first RACH procedure or the second RACH procedure based at least on a type of uplink data to be transmitted. When the second RACH procedure is selected, the WTRU may determine at least one physical random access channel (PRACH) resource associated with the second RACH procedure. The WTRU may determine a preamble sequence associated with the second RACH procedure. The WTRU may determine a data resource for the uplink data. The WTRU may send a RACH transmission that includes the preamble sequence and the uplink data.

Transformer ASR with Contextual Block Processing

Abstract: The Transformer self-attention network has recently shown promising performance as an alternative to recurrent neural networks (RNNs) in end-to-end (E2E) automatic speech recognition (ASR) systems. However, the Transformer has a drawback in that the entire input sequence is required to compute self-attention. In this paper, we propose a new block processing method for the Transformer encoder by introducing a context-aware inheritance mechanism. An additional context embedding vector handed over from the previously processed block helps to encode not only local acoustic information but also global linguistic, channel, and speaker attributes. We introduce a novel mask technique to implement the context inheritance to train the model efficiently. Evaluations of the Wall Street Journal (WSJ), Librispeech, VoxForge Italian, and AISHELL-1 Mandarin speech recognition datasets show that our proposed contextual block processing method outperforms naive block processing consistently. Furthermore, the attention weight tendency of each layer is analyzed to clarify how the added contextual inheritance mechanism models the global information.

Nested Named Entity Recognition via Second-best Sequence Learning and Decoding

Abstract: When an entity name contains other names within it, the identification of all combinations of names can become difficult and expensive. We propose a new method to recognize not only outermost named entities but also inner nested ones. We design an objective function for training a neural model that treats the tag sequence for nested entities as the second best path within the span of their parent entity. In addition, we provide the decoding method for inference that extracts entities iteratively from outermost ones to inner ones in an outside-to-inside way. Our method has no additional hyperparameters to the conditional random field based model widely used for flat named entity recognition tasks. Experiments demonstrate that our method performs better than or at least as well as existing methods capable of handling nested entities, achieving the F1-scores of 85.82%, 84.34%, and 77.36% on ACE-2004, ACE-2005, and GENIA datasets, respectively.

PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things

Abstract: We propose PanopticFusion, a novel online volumetric semantic mapping system at the level of stuff and things. In contrast to previous semantic mapping systems, PanopticFusion is able to densely predict class labels of a background region (stuff) and individually segment arbitrary foreground objects (things). In addition, our system has the capability to reconstruct a large-scale scene and extract a labeled mesh thanks to its use of a spatially hashed volumetric map representation. Our system first predicts pixel-wise panoptic labels (class labels for stuff regions and instance IDs for thing regions) for incoming RGB frames by fusing 2D semantic and instance segmentation outputs. The predicted panoptic labels are integrated into the volumetric map together with depth measurements while keeping the consistency of the instance IDs, which could vary frame to frame, by referring to the 3D map at that moment. In addition, we construct a fully connected conditional random field (CRF) model with respect to panoptic labels for map regularization. For online CRF inference, we propose a novel unary potential approximation and a map division strategy. We evaluated the performance of our system on the ScanNet (v2) dataset. PanopticFusion outperformed or compared with state-of-the-art offline 3D DNN methods in both semantic and instance segmentation benchmarks. Also, we demonstrate a promising augmented reality application using a 3D panoptic map generated by the proposed system.

Recursive speech separation for unknown number of speakers

Abstract: In this paper we propose a method of single-channel speaker-independent multi-speaker speech separation for an unknown number of speakers. As opposed to previous works, in which the number of speakers is assumed to be known in advance and speech separation models are specific for the number of speakers, our proposed method can be applied to cases with different numbers of speakers using a single model by recursively separating a speaker. To make the separation model recursively applicable, we propose one-and-rest permutation invariant training (OR-PIT). Evaluation on WSJ0-2mix and WSJ0-3mix datasets show that our proposed method achieves state-of-the-art results for two- and three-speaker mixtures with a single model. Moreover, the same model can separate four-speaker mixture, which was never seen during the training. We further propose the detection of the number of speakers in a mixture during recursive separation and show that this approach can more accurately estimate the number of speakers than detection in advance by using a deep neural network based classifier.

Comparator, AD converter, solid-state imaging device, electronic apparatus, and method of controlling comparator

Abstract: The present disclosure relates to a comparator, an AD converter, a solid-state imaging device, an electronic apparatus, and a comparator control method that can reduce power consumption while increasing the determination speed of the comparator. The comparator includes a comparison unit, a positive feedback circuit, and a current limiting unit. The comparison unit compares the voltage of an input signal and the voltage of a reference signal, and outputs a comparison result signal. The positive feedback circuit increases the transition speed at the time when the comparison result signal is inverted. The current limiting unit limits the current flowing in the comparison unit after the inversion of the comparison result signal. The present disclosure can be applied to comparators, for example.

Information processing device, information processing method, and program

Abstract: There is provided an information processing device, including a processing unit configured to associate, based on a captured image obtained by imaging a remote controller including an operation device and an operation signal transmitted from the remote controller in response to an operation on the operation device, the operated operation device with the operation signal.

Image processing device and method, surgical system, and surgical member

Abstract: The present invention acquires a microscopic image (201) obtained by photographing the surgical member (121) inserted to the subject with a surgical microscope (112), estimates the relative posture of the surgical member in the subject on the basis of the acquired microscopic image (201), and outputs posture information (222, 223, 224) associated with the posture that has been estimated. The present invention can be applied to ophthalmic surgery, in which an intraocular endoscope or surgical tool is observed with a surgical microscope, for example.

More Results on Shortest Linear Programs

Abstract: At the FSE conference of ToSC 2018, Kranz et al. presented their results on shortest linear programs for the linear layers of several well known block ciphers in literature. Shortest linear programs are essentially the minimum number of 2-input xor gates required to completely describe a linear system of equations. In the above paper the authors showed that the commonly used metrics like d-xor/s-xor count that are used to judge the “lightweightedness” do not represent the minimum number of xor gates required to describe a given MDS matrix. In fact they used heuristic based algorithms of Boyar/Peralta and Paar to find implementations of MDS matrices with even fewer xor gates than was previously known. They proved that the AES mixcolumn matrix can be implemented with as little as 97 xor gates. In this paper we show that the values reported in the above paper are not optimal. By suitably including random bits in the instances of the above algorithms we can achieve implementations of almost all matrices with lesser number of gates than were reported in the above paper. As a result we report an implementation of the AES mixcolumn matrix that uses only 95 xor gates.

Electronic device in wireless communication system and wireless communication method

Abstract: The present disclosure relates to an electronic device in a wireless communication system and a wireless communication method. The electronic device comprises: one or more processing circuits, wherein the processing circuits are configured to execute the following operations: acquiring an on/off state of a small cell base station in a pre-determined geographical area; and on the basis of the acquired on/off state of the small cell base station, generating reconfiguration information about a positioning reference signal (PRS) used for the small cell base station in the pre-determined geographical area, so as to position user equipment in the pre-determined geographical area. By using the electronic device and the wireless communication method in the present disclosure, an existing PRS signal can be reconfigured, so as to improve the positioning accuracy, and achieve the purpose of accelerating the positioning process.

Phase-Responsive Fourier Nanotransducers for Probing 2D Materials and Functional Interfaces

Abstract: We report a methodology for the fabrication of neutralized chitosan-based nanofiber matrices decorated with bare Au nanoparticles, which demonstrate stable characteristics even after prolonged contact with a biological environment. The methodology consists of electrospinning of a mixture of bare (ligand-free) laser-synthesized Au nanoparticles (AuNPs) and solutions of chitosan/polyethylene oxide (ratio 1/3) containing chitosan of a relatively high molecular weight (200 kDa) and concentration of 3% (w/v). Our studies reveal a continuous morphology of hybrid nanofibers with the mean fiber diameter of 189 nm ± 86 nm, which demonstrate a high thermal stability. Finally, we describe a protocol for the neutralization of nanofibers, which enabled us to achieve their structural stability in phosphate-buffered saline (PBS) for more than six months, as confirmed by microscopy and FTIR measurements. The formed hybrid nanofibers exhibit unique physicochemical properties essential for the development of future tissue engineering platforms.

Mixed Precision DNNs: All you need is a good parametrization

Abstract: Efficient deep neural network (DNN) inference on mobile or embedded devices typically involves quantization of the network parameters and activations. In particular, mixed precision networks achieve better performance than networks with homogeneous bitwidth for the same size constraint. Since choosing the optimal bitwidths is not straight forward, training methods, which can learn them, are desirable. Differentiable quantization with straight-through gradients allows to learn the quantizer's parameters using gradient methods. We show that a suited parametrization of the quantizer is the key to achieve a stable training and a good final performance. Specifically, we propose to parametrize the quantizer with the step size and dynamic range. The bitwidth can then be inferred from them. Other parametrizations, which explicitly use the bitwidth, consistently perform worse. We confirm our findings with experiments on CIFAR-10 and ImageNet and we obtain mixed precision DNNs with learned quantization parameters, achieving state-of-the-art performance.

Wireless telecommunications apparatus and methods

Abstract: A method of operating a terminal device in a wireless telecommunications network comprising the terminal device and a base station. The wireless telecommunications network supports a random access procedure comprising an exchange of random access procedure messages between the terminal device and the base station. At least one of the random access procedure messages is a variable size uplink message. The method comprises: receiving from the base station an instruction to perform the random access procedure, wherein the instruction to perform the random access procedure comprises an indication of a message size to be used by the terminal device for the variable size uplink message during the random access procedure; and performing the random access procedure by exchanging random access procedure messages with the base station, including transmitting the uplink message of variable size with a size corresponding to the indicated message size received from the base station.

Terminal device, base station device, communication method, and program

Abstract: [Solution] A terminal device includes: a communication unit configured to perform wireless communication; and a control unit configured to perform control such that control information regarding a supported communication scheme is transmitted to an external device through the wireless communication. In a case in which a first communication scheme and a second communication scheme different from the first communication scheme are supported, the control unit associates, with the control information, a parameter indicating whether or not dual connectivity based on the first communication scheme and the second communication scheme is supported.

Wireless communication method and wireless communication device

Abstract: Disclosed are a wireless communication method and a wireless communication device Provided is a resource management device, comprising one or a plurality of processors, said processor or processors being configured to: obtain interference information from another resource management device, said interference information comprising mutual interference between a plurality of managed systems managed by the resource management device, and interference of a plurality of managed systems by one or a plurality of non-managed systems not managed by the resource management device; determine a communication resource for each managed system on the basis of the obtained interference information; and notify a corresponding managed system of the determined communication resource

Wireless communication method and wireless communication device

Abstract: A wireless communication method and a wireless communication device. An electronic device for a first communication device in a wireless communication system includes: storage device configured to store an analog codebook for the first communication device, the analog codebook including a plurality of sets of first configuration parameters for a set of phase shifters of the first communication device; and a processing circuit configured to: perform channel estimation on a first channel from a second communication device to the first communication device respectively based on the plurality of sets of first configuration parameters and signal transmission from the second communication device, select a set of first configuration parameters corresponding to ones of channel estimation results that satisfy a first predetermined condition to generate a reduced analog sub-codebook, configure signal transmission from the first communication device to the second communication device based on the analog sub-codebook.

Skin-Based Identification From Multispectral Image Data Using CNNs

Abstract: User identification from hand images only is still a challenging task. In this paper, we propose a new biometric identification system based solely on a skin patch from a multispectral image. The system is utilizing a novel modified 3D CNN architecture which is taking advantage of multispectral data. We demonstrate the application of our system for the example of human identification from multispectral images of hands. To the best of our knowledge, this paper is the first to describe a pose-invariant and robust to overlapping real-time human identification system using hands. Additionally, we provide a framework to optimize the required spectral bands for the given spatial resolution limitations.

Circuit, base station, method, and recording medium

Abstract: Circuitry for an aerial vehicle that includes processing circuitry that acquires altitude information indicating a measurement result of altitude of the aerial vehicle, receives a reference signal transmitted from a base station, receives altitude zone setting information transmitted from the base station, the altitude zone setting information being information for setting an altitude zone for classifying a state of the altitude zone of the aerial vehicle, identifies the altitude zone of the aerial vehicle based on the altitude information and the altitude zone setting information, and controls measurement report processing of reporting a measurement report message including reference signal information indicating a measurement result of the reference signal and the altitude information to the base station based on the altitude zone of the aerial vehicle, such that a frequency of transmitting the measurement report message is changed according to the identified altitude zone.

Transmission device, transmission method, and communication system

Abstract: A transmission device of the present disclosure includes: a driver unit that transmits a data signal with use of a first voltage state, a second voltage state, and a third voltage state interposed between the first voltage state and the second voltage state, and is configured to make a voltage in the third voltage state changeable; and a controller that changes the voltage in the third voltage state to cause the driver unit to perform emphasis.

Programmable data pattern for repeated writes to memory

Abstract: A programmable data pattern for repeated writes to memory can enable efficient writing of a data pattern to multiple memory locations without transmitting the data pattern for each write. In one embodiment, a memory device includes input/output (I/O) circuitry to receive a command, a register to store a value to indicate a source of a data pattern to write in response to receipt of the command, and access circuitry to, in response to receipt of the command, write the data pattern to memory based on the source indicated by the value in the register.

Global and Local Mode-domain Adaptive Algorithms for Spatial Active Noise Control Using Higher-order Sources

Abstract: The aim of spatial active noise control (ANC) is to attenuate noise over a certain space. Although a large-scale system is required to achieve spatial ANC, mode-domain signal processing makes it possible to reduce the computational cost and improve the performance. A higher-order source (HOS) has an advantage in sound field control due to its controllable directivity patterns. An array of HOS can suppress an undesired exterior sound propagation while occupying a smaller physical space than a conventional omnidirectional loudspeaker array. In this paper, we propose two types of adaptive algorithm for spatial ANC using HOSs, which provide a trade-off between efficiency and error robustness against loudspeaker placements. Numerical simulations in a reverberant environment show the efficacy of the proposed algorithms compared with the conventional multipoint adaptive spatial ANC algorithm.