Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

The reference human genome sequence set the stage for studies of genetic variation and its association with human disease, but epigenomic studies lack a similar reference. To address this need, the NIH Roadmap Epigenomics Consortium generated the largest collection so far of human epigenomes for primary cells and tissues. Here we describe the integrative analysis of 111 reference human epigenomes generated as part of the programme, profiled for histone modification patterns, DNA accessibility, DNA methylation and RNA expression. We establish global maps of regulatory elements, define regulatory modules of coordinated activity, and their likely activators and repressors. We show that disease- and trait-associated genetic variants are enriched in tissue-specific epigenomic marks, revealing biologically relevant cell types for diverse human traits, and providing a resource for interpreting the molecular basis of human disease. Our results demonstrate the central role of epigenomic information for understanding gene regulation, cellular differentiation and human disease.

Integrative analysis of 111 reference human epigenomes

Macrophage Development Rewritten Macrophages provide protection against a wide variety of infections and critically shape the inflammatory environment in many tissues. These cells come in many flavors, as determined by differences in gene expression, cell surface phenotype and specific function. Schulz et al. (p. 86, published online 22 March) investigated whether adult macrophages all share a common developmental origin. Immune cells, including most macrophages, are widely thought to arise from hematopoietic stem cells (HSCs), which require the transcription factor Myb for their development. Analysis of Myb-deficient mice revealed that a population of yolk-sac–derived, tissue-resident macrophages was able to develop and persist in adult mice in the absence of HSCs. Importantly, yolk sac–derived macrophages also contributed substantially to the tissue macrophage pool even when HSCs were present. In mice, a population of tissue-resident macrophages arises independently of bone marrow–derived stem cells. Macrophages and dendritic cells (DCs) are key components of cellular immunity and are thought to originate and renew from hematopoietic stem cells (HSCs). However, some macrophages develop in the embryo before the appearance of definitive HSCs. We thus reinvestigated macrophage development. We found that the transcription factor Myb was required for development of HSCs and all CD11bhigh monocytes and macrophages, but was dispensable for yolk sac (YS) macrophages and for the development of YS-derived F4/80bright macrophages in several tissues, such as liver Kupffer cells, epidermal Langerhans cells, and microglia—cell populations that all can persist in adult mice independently of HSCs. These results define a lineage of tissue macrophages that derive from the YS and are genetically distinct from HSC progeny.

A lineage of myeloid cells independent of Myb and hematopoietic stem cells

Single-cell expression profiles of melanoma Tumors harbor multiple cell types that are thought to play a role in the development of resistance to drug treatments. Tirosh et al. used single-cell sequencing to investigate the distribution of these differing genetic profiles within melanomas. Many cells harbored heterogeneous genetic programs that reflected two different states of genetic expression, one of which was linked to resistance development. Following drug treatment, the resistance-linked expression state was found at a much higher level. Furthermore, the environment of the melanoma cells affected their gene expression programs. Science, this issue p. 189 Melanoma cells show transcriptional heterogeneity. To explore the distinct genotypic and phenotypic states of melanoma tumors, we applied single-cell RNA sequencing (RNA-seq) to 4645 single cells isolated from 19 patients, profiling malignant, immune, stromal, and endothelial cells. Malignant cells within the same tumor displayed transcriptional heterogeneity associated with the cell cycle, spatial context, and a drug-resistance program. In particular, all tumors harbored malignant cells from two distinct transcriptional cell states, such that tumors characterized by high levels of the MITF transcription factor also contained cells with low MITF and elevated levels of the AXL kinase. Single-cell analyses suggested distinct tumor microenvironmental patterns, including cell-to-cell interactions. Analysis of tumor-infiltrating T cells revealed exhaustion programs, their connection to T cell activation and clonal expansion, and their variability across patients. Overall, we begin to unravel the cellular ecosystem of tumors and how single-cell genomics offers insights with implications for both targeted and immune therapies.

Dissecting the multicellular ecosystem of metastatic melanoma by single-cell RNA-seq

A grating coupler with aligned Si3N4 and Si grating teeth is proposed and demonstrated. The measured coupling efficiency is -1.3 dB (competitive with the best Si-only grating couplers), and the 1-dB bandwidth is a record 80 nm.

Ultra-Efficient and Broadband Dual-Level Si3N4-on-SOI Grating Coupler

A 3-D ring oscillator integrated with through silicon vias (TSVs) is designed and fabricated for testing multilayer-stacked integrated circuits (ICs) with TSV. The proposed 3-D ring oscillator consists of 13 stages and 65-nm Complementary Metal-Oxide-Semiconductor Transistor (CMOS) dies with two current-starved inverters and via-last TSVs designed for the five middle layers of the 3-D ring oscillator. The two cascaded inverters are connected to the upside layer through a TSV and to the downside layer through a microbump. One chip with two inverters but without TSV is stacked in the top layer of the 3-D ring oscillator to realize the ring oscillator loop, and one logic chip with one inverter and via-middle TSVs are in the bottom of the ring oscillator. The characteristics of via-last and via-middle TSVs in the 3-D ring oscillator are analyzed based on the equivalent circuits. The oscillation frequency responses of the designed 3-D ring oscillator are measured finally to verify the design concept, and to assess the performance of the 3-D ring oscillator. The measured results demonstrate that the proposed 3-D ring oscillator is an attractive candidate for testing the stacked 3-D IC, and the effect of TSVs dominates the delay of the 3-D ring oscillator.

Built-in-Self-Test-Stacked 3-D Ring Oscillator Based on Through Silicon Vias

The structure of Si nanocrystals (nc-Si) embedded in SiO2 is promising for achieving optical gain and waveguiding with the advantage of full compatibility with the matured Si technology. In this paper, we report an approach to optical-constant profiling for such a planar waveguide structure formed by Si ion implantation into a SiO2 thin film based on spectroscopic ellipsometry (SE). With the nc-Si optical constants calculated from the Forouhi–Bloomer model and the nc-Si depth profile obtained from secondary ion mass spectroscopy (SIMS) measurements, the optical properties at a given depth are simulated with the Maxwell–Garnett effective medium approximation (EMA). Then an SE fitting is carried out, and the optical constants of nc-Si are extracted from the best fitting. Finally, the depth profile of optical constants of the structure is obtained from the EMA calculation. The result also suggests that the structure has a very low optical loss in the visible to infrared spectral range.

An approach to optical-property profiling of a planar-waveguide structure of Si nanocrystals embedded in SiO2

This paper describes the University of Sydney & JD’s joint submission of the IWSLT 2021 low resource speech translation task. We participated in the Swahili->English direction and got the best scareBLEU (25.3) score among all the participants. Our constrained system is based on a pipeline framework, i.e. ASR and NMT. We trained our models with the officially provided ASR and MT datasets. The ASR system is based on the open-sourced tool Kaldi and this work mainly explores how to make the most of the NMT models. To reduce the punctuation errors generated by the ASR model, we employ our previous work SlotRefine to train a punctuation correction model. To achieve better translation performance, we explored the most recent effective strategies, including back translation, knowledge distillation, multi-feature reranking, and transductive finetuning. For model structure, we tried auto-regressive and non-autoregressive models, respectively. In addition, we proposed two novel pre-train approaches, i.e. de-noising training and bidirectional training to fully exploit the data. Extensive experiments show that adding the above techniques consistently improves the BLEU scores, and the final submission system outperforms the baseline (Transformer ensemble model trained with the original parallel data) by approximately 10.8 BLEU score, achieving the SOTA performance.

/pdf/the-usyd-jd-speech-translation-system-for-iwslt2021-47ex1b7upv.pdf

The USYD-JD Speech Translation System for IWSLT2021

Pre-training (PT) and back-translation (BT) are two simple and powerful
methods to utilize monolingual data for improving the model performance of
neural machine translation (NMT). This paper takes the first step to
investigate the complementarity between PT and BT. We introduce two probing
tasks for PT and BT respectively and find that PT mainly contributes to the
encoder module while BT brings more benefits to the decoder. Experimental
results show that PT and BT are nicely complementary to each other,
establishing state-of-the-art performances on the WMT16 English-Romanian and
English-Russian benchmarks. Through extensive analyses on sentence originality
and word frequency, we also demonstrate that combining Tagged BT with PT is
more helpful to their complementarity, leading to better translation quality.
Source code is freely available at https://github.com/SunbowLiu/PTvsBT.

Liang Ding

Papers

Ultra-Efficient and Broadband Dual-Level Si3N4-on-SOI Grating Coupler

Built-in-Self-Test-Stacked 3-D Ring Oscillator Based on Through Silicon Vias

An approach to optical-property profiling of a planar-waveguide structure of Si nanocrystals embedded in SiO2

The USYD-JD Speech Translation System for IWSLT2021

On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation