Showing papers by "Kyunghyun Cho published in 2023"
TL;DR: This article proposed imitation learning from language feedback (ILF), which is a new approach that utilizes more informative language feedback to improve the performance of language models and improve human-level summarization performance.
Abstract: Pretrained language models often generate outputs that are not in line with human preferences, such as harmful text or factually incorrect summaries. Recent work approaches the above issues by learning from a simple form of human feedback: comparisons between pairs of model-generated outputs. However, comparison feedback only conveys limited information about human preferences. In this paper, we introduce Imitation learning from Language Feedback (ILF), a new approach that utilizes more informative language feedback. ILF consists of three steps that are applied iteratively: first, conditioning the language model on the input, an initial LM output, and feedback to generate refinements. Second, selecting the refinement incorporating the most feedback. Third, finetuning the language model to maximize the likelihood of the chosen refinement given the input. We show theoretically that ILF can be viewed as Bayesian Inference, similar to Reinforcement Learning from human feedback. We evaluate ILF's effectiveness on a carefully-controlled toy task and a realistic summarization task. Our experiments demonstrate that large language models accurately incorporate feedback and that finetuning with ILF scales well with the dataset size, even outperforming finetuning on human summaries. Learning from both language and comparison feedback outperforms learning from each alone, achieving human-level summarization performance.
17 citations
TL;DR: In this article , the authors compared four policosanols, including one from Cuba (Raydel® policosanol) and three from China (Xi’an Natural sugar cane, Xi‘an Realin sugar cane and Shaanxi rice bran), using the sodium cholate dialysis method.
Abstract: Policosanols from various sources, such as sugar cane, rice bran, and insects, have been marketed to prevent dyslipidemia, diabetes, and hypertension by increasing the blood high-density lipoproteins cholesterol (HDL-C) levels. On the other hand, there has been no study on how each policosanol influences the quality of HDL particles and their functionality. Reconstituted high-density lipoproteins (rHDLs) with apolipoprotein (apo) A-I and each policosanol were synthesized using the sodium cholate dialysis method to compare the policosanols in lipoprotein metabolism. Each rHDL was compared regarding the particle size and shape, antioxidant activity, and anti-inflammatory activity in vitro and in zebrafish embryos. This study compared four policosanols including one policosanol from Cuba (Raydel® policosanol) and three policosanols from China (Xi’an Natural sugar cane, Xi’an Realin sugar cane, and Shaanxi rice bran). The synthesis of rHDLs with various policosanols (PCO) from Cuba or China using a molar ratio of 95:5:1:1 with palmitoyloleoyl phosphatidylcholine (POPC): free cholesterol (FC): apoA-I:PCO (wt:wt) showed that rHDL containing Cuban policosanol (rHDL-1) showed the largest particle size and the most distinct particle shape. The rHDL-1 showed a 23% larger particle diameter and increased apoA-I molecular weight with a 1.9 nm blue shift of the maximum wavelength fluorescence than rHDL alone (rHDL-0). Other rHDLs containing Chinese policosanols (rHDL-2, rHDL-3, and rHDL-4) showed similar particle sizes with an rHDL-0 and 1.1–1.3 nm blue shift of wavelength maximum fluorescence (WMF). Among all rHDLs, the rHDL-1 showed the strongest antioxidant ability to inhibit cupric ion-mediated LDL oxidation. The rHDL-1-treated LDL showed the most distinct band intensity and particle morphology compared with the other rHDLs. The rHDL-1 also exerted the highest anti-glycation activity to inhibit the fructose-mediated glycation of human HDL2 with the protection of apoA-I from proteolytic degradation. At the same time, other rHDLs showed a loss of anti-glycation activity with severe degradation. A microinjection of each rHDL alone showed that rHDL-1 had the highest survivability of approximately 85 ± 3%, with the fastest developmental speed and morphology. In contrast, rHDL-3 showed the lowest survivability, around 71 ± 5%, with the slowest developmental speed. A microinjection of carboxymethyllysine (CML), a pro-inflammatory advanced glycated end product, into zebrafish embryos resulted in severe embryo death of approximately 30 ± 3% and developmental defects with the slowest developmental speed. On the other hand, the phosphate buffered saline (PBS)-injected embryo showed 83 ± 3% survivability. A co-injection of CML and each rHDL into adult zebrafish showed that rHDL-1 (Cuban policosanol) induced the highest survivability, around 85 ± 3%, while rHDL-0 showed 67 ± 7% survivability. In addition, rHDL-2, rHDL-3, and rHDL-4 showed 67 ± 5%, 62 ± 37, and 71 ± 6% survivability, respectively, with a slower developmental speed and morphology. In conclusion, Cuban policosanol showed the strongest ability to form rHDLs with the most distinct morphology and the largest size. The rHDL-containing Cuban policosanol (rHDL-1) showed the strongest antioxidant ability against LDL oxidation, anti-glycation activity to protect apoA-I from degradation, and the highest anti-inflammatory activity to protect embryo death under the presence of CML.
3 citations
TL;DR: The authors argue that self-consistency is an important criteria for valid multi-step reasoning and propose two types of selfconsistencies: hypothetical consistency (the ability for a model to predict what its output would be in a hypothetical other context) and compositional consistency, which is defined as consistency of a model's outputs for a compositional task even when an intermediate step is replaced with the model's output for that step).
Abstract: Large language models (LLMs) have achieved widespread success on a variety of in-context few-shot tasks, but this success is typically evaluated via correctness rather than consistency. We argue that self-consistency is an important criteria for valid multi-step reasoning and propose two types of self-consistency that are particularly important for multi-step logic -- hypothetical consistency (the ability for a model to predict what its output would be in a hypothetical other context) and compositional consistency (consistency of a model's outputs for a compositional task even when an intermediate step is replaced with the model's output for that step). We demonstrate that four sizes of the GPT-3 model exhibit poor consistency rates across both types of consistency on four different tasks (Wikipedia, DailyDialog, arithmetic, and GeoQuery).
2 citations
TL;DR: In this article , the authors compared the changes in HDL quantity and quality among middle-aged women depending on exercise intensity, frequency, and duration; participants were divided into a sedentary group (group 1), a middle-intensity group(group 2), and a high-intensity groups (group 3).
Abstract: Regular exercise, especially aerobic exercise, is beneficial for increasing serum high-density lipoprotein-cholesterol (HDL-C) levels in the general population. In addition to the HDL-C quantity, exercise enhances HDL functionality, antioxidants, and cholesterol efflux. On the other hand, the optimal intensity and frequency of exercise to increase HDL quantity and enhance HDL quality in middle-aged women need to be determined. The current study was designed to compare the changes in HDL quantity and quality among middle-aged women depending on exercise intensity, frequency, and duration; participants were divided into a sedentary group (group 1), a middle-intensity group (group 2), and a high-intensity group (group 3). There were no differences in anthropometric parameters among the groups, including blood pressure, muscle mass, and handgrip strength. Although there was no difference in serum total cholesterol (TC) among the groups, the serum HDL-C and apolipoprotein (apo)A-I levels remarkably increased to 17% and 12%, respectively, in group 3. Serum low-density lipoprotein-cholesterol (LDL-C), glucose, triglyceride, and the apo-B/apoA-I ratio were remarkably decreased in the exercise groups depending on the exercise intensity; group 3 showed 13%, 10%, and 45% lower LDL-C, glucose, and triglyceride (TG), respectively, than group 1. The hepatic and muscle damage parameter, aspartate aminotransferase (AST), was significantly decreased in the exercise groups, but high-sensitivity C-reactive protein (CRP), alanine aminotransferase (ALT), and γ-glutamyl transferase (γ-GTP) were similar in the three groups. In LDL, the particle size was increased 1.5-fold (p < 0.001), and the oxidation extent was decreased by 40% with a 23% lower TG content in group 3 than in group 1. In the exercise groups (groups 2 and 3), LDL showed the slowest electromobility with a distinct band intensity compared to the sedentary group (group 1). In HDL2, the particle size was 2.1-fold increased (p < 0.001) in the exercise group (group 3) with a 1.5-fold increase in TC content compared to that in group 1, as well as significantly enhanced antioxidant abilities, paraoxonase (PON) activity, and ferric ion reduction ability (FRA). In HDL3, the particle size was increased 1.2-fold with a 45% reduction in TG in group 3 compared to group 1. With increasing exercise intensity, apoA-I expression was increased in HDL2 and HDL3, and PON activity and FRA were enhanced (p < 0.001). In conclusion, regular exercise in middle-aged women is associated with the elevation of serum HDL-C and apoA-I with the enhancement of HDL quality and functionality and an increase in the TC content, particle size, and antioxidant abilities. With the reduction in TG and oxidized products in LDL and HDL, lipoproteins could have more anti-atherogenic properties through regular exercise in an intensity-dependent manner.
2 citations
TL;DR: In this article , a putative interaction between CIGB-258 and apoA-I was investigated to provide mechanistic insight into the protection of HDL in a paralyzed hyperlipidemic zebrafish and its embryo.
Abstract: Inflammation and atherosclerosis are intimately associated via the production of dysfunctional high-density lipoproteins (HDL) and modification of apolipoprotein (apo) A-I. A putative interaction between CIGB-258 and apoA-I was investigated to provide mechanistic insight into the protection of HDL. The protective activity of CIGB-258 was tested in the CML-mediated glycation of apoA-I. The in vivo anti-inflammatory efficacy was compared in paralyzed hyperlipidemic zebrafish and its embryo in the presence of CML. Treatment of CML induced greater glycation extent of HDL/apoA-I and proteolytic degradation of apoA-I. In the presence of CML, however, co-treatment of CIGB-258 inhibited the glycation of apoA-I and protected the degradation of apoA-I, exerting enhanced ferric ion reduction ability. Microinjection of CML (500 ng) into zebrafish embryos resulted in acute death with the lowest survivability with severe developmental defects with interleukin (IL)-6 production. Conversely, a co-injection of CIGB-258 or Tocilizumab produced the highest survivability with a normal development speed and morphology. In hyperlipidemic zebrafish, intraperitoneal injection of CML (500 μg) caused the complete loss of swimming ability and severe acute death with only 13% survivability 3 h post-injection. A co-injection of the CIGB-258 resulted in a 2.2-fold faster recovery of swimming ability than CML alone, with higher survivability of approximately 57%. These results suggest that CIGB-258 protected hyperlipidemic zebrafish from the acute neurotoxicity of CML. Histological analysis showed that the CIGB-258 group had 37% lower infiltration of neutrophils in hepatic tissue and 70% lower fatty liver changes than those of the CML-alone group. The CIGB-258 group exhibited the smallest IL-6 expression in the liver and the lowest blood triglyceride level. CIGB-258 displayed potent anti-inflammatory activity in hyperlipidemic zebrafish by inhibiting apoA-I glycation, promoting rapid recovery from the paralysis of CML toxicity and suppression of IL-6, and lowering fatty liver changes.
1 citations
TL;DR: In this paper , the efficacy and safety of 20 mg of Cuban policosanol in blood pressure and lipid/lipoprotein parameters of healthy Japanese subjects via a placebo-controlled, randomized, and double-blinded human trial was evaluated.
Abstract: This study evaluated the efficacy and safety of 20 mg of Cuban policosanol in blood pressure (BP) and lipid/lipoprotein parameters of healthy Japanese subjects via a placebo-controlled, randomized, and double-blinded human trial. After 12 weeks of consumption, the policosanol group showed significantly lower BP, glycated hemoglobin (HbA1c), and blood urea nitrogen (BUN) levels. The policosanol group also showed lower aspartate aminotransferase (AST), alanine aminotransferase (ALT), and γ-glutamyl transferase (γ-GTP) levels at week 12 than those at week 0: A decrease of up to 9% (p < 0.05), 17% (p < 0.05), and 15% (p < 0.05) was observed, respectively. The policosanol group showed significantly higher HDL-C level and HDL-C/TC (%), approximately 9.5% (p < 0.001) and 7.2% (p = 0.003), respectively, than the placebo group and a difference in the point of time and group interaction (p < 0.001). In lipoprotein analysis, the policosanol group showed a decrease in oxidation and glycation extent in VLDL and LDL with an improvement of particle shape and morphology after 12 weeks. HDL from the policosanol group showed in vitro stronger antioxidant and in vivo anti-inflammatory abilities. In conclusion, 12 weeks of Cuban policosanolconsumption in Japanese subjects showed significant improvement in blood pressure, lipid profiles, hepatic functions, and HbA1c with enhancement of HDL functionalities.
1 citations
TL;DR: In this article , an unsupervised algorithm is proposed to find good initialization for input data, given that a downstream task is d-way classification, and the success of learning is directly related to how diverse downstream tasks are in the vicinity of the initial parameters.
Abstract: Despite the recent success of stochastic gradient descent in deep learning, it is often difficult to train a deep neural network with an inappropriate choice of its initial parameters. Even if training is successful, it has been known that the initial parameter configuration may negatively impact generalization. In this paper, we propose an unsupervised algorithm to find good initialization for input data, given that a downstream task is d-way classification. We first notice that each parameter configuration in the parameter space corresponds to one particular downstream task of d-way classification. We then conjecture that the success of learning is directly related to how diverse downstream tasks are in the vicinity of the initial parameters. We thus design an algorithm that encourages small perturbation to the initial parameter configuration leads to a diverse set of d-way classification tasks. In other words, the proposed algorithm ensures a solution to any downstream task to be near the initial parameter configuration. We empirically evaluate the proposed algorithm on various tasks derived from MNIST with a fully connected network. In these experiments, we observe that our algorithm improves average test accuracy across most of these tasks, and that such improvement is greater when the number of labelled examples is small.
10 Jul 2023
TL;DR: AsmDepictor as discussed by the authors is a Transformer-based framework that generates a function symbol name from a set of assembly codes, which consists of three major components: binary code refinement, model training, and inference.
Abstract: Reverse engineering of a stripped binary has a wide range of applications, yet it is challenging mainly due to the lack of contextually useful information within. Once debugging symbols (e.g., variable names, types, function names) are discarded, recovering such information is not technically viable with traditional approaches like static or dynamic binary analysis. We focus on a function symbol name recovery, which allows a reverse engineer to gain a quick overview of an unseen binary. The key insight is that a well-developed program labels a meaningful function name that describes its underlying semantics well. In this paper, we present AsmDepictor, the Transformer-based framework that generates a function symbol name from a set of assembly codes (i.e., machine instructions), which consists of three major components: binary code refinement, model training, and inference. To this end, we conduct systematic experiments on the effectiveness of code refinement that can enhance an overall performance. We introduce the per-layer positional embedding and Unique-softmax for AsmDepictor so that both can aid to capture a better relationship between tokens. Lastly, we devise a novel evaluation metric tailored for a short description length, the Jaccard* score. Our empirical evaluation shows that the performance of AsmDepictor by far surpasses that of the state-of-the-art models up to around 400%. The best AsmDepictor model achieves an F1 of 71.5 and Jaccard* of 75.4.
TL;DR: Ozonated sunflower oil (OSO) is a well-known functional oil with antioxidant, antimicrobial, anti-allergic, and skin-moisturizing properties as mentioned in this paper .
Abstract: Ozonated sunflower oil (OSO) is a well-known functional oil with antioxidant, antimicrobial, anti-allergic, and skin-moisturizing properties. However, studies on the effects of OSO on high-cholesterol diet (HCD)-induced metabolic disorders have been scarce. In the current study, we aimed to determine the anti-inflammatory effects of OSO on lipid metabolism in adult hypercholesterolemic zebrafish and its embryos. Microinjection of OSO (final 2%, 10 nL) into zebrafish embryos under the presence of carboxymethyllysine (CML, 500 ng) protected acute embryo death up to 61% survival, while sunflower oil (final 2%) showed much less protection at around 42% survival. The microinjection of OSO was more effective than SO to inhibit reactive oxygen species (ROS) production and apoptosis in the CML induced embryo toxicity. Intraperitoneal injection of OSO under the presence of CML protected acute death from CML-induced neurotoxicity with improved hepatic inflammation, less detection of ROS and interleukin (IL)-6, and lowering blood total cholesterol (TC) and triglyceride (TG), while the SO-injected group did not protect the CML-toxicity. Long-term supplementation of OSO (final 20%, wt/wt) with HCD for 6 months resulted in higher survivability than the HCD alone group or HCD + SO group (final 20%, wt/wt) with significant lowering of plasma TC and TG levels. The HCD + OSO group showed the least hepatic inflammation, fatty liver change, ROS, and IL-6 production. In conclusion, short-term treatment of OSO by injection exhibited potent anti-inflammatory activity against acute neurotoxicity of CML in zebrafish and their embryo. Long-term supplementation of OSO in the diet also revealed the highest survivability and blood lipid-lowering effect through potent antioxidant and anti-inflammatory activity.
19 Apr 2023
TL;DR: In this article , the authors compare three state-of-the-art semi-supervised methods encompassing both unpaired text and audio as well as several of their combinations in a controlled setting using joint training.
Abstract: Unpaired text and audio injection have emerged as dominant methods for improving ASR performance in the absence of a large labeled corpus. However, little guidance exists on deploying these methods to improve production ASR systems that are trained on very large supervised corpora and with realistic requirements like a constrained model size and CPU budget, streaming capability, and a rich lattice for rescoring and for downstream NLU tasks. In this work, we compare three state-of-the-art semi-supervised methods encompassing both unpaired text and audio as well as several of their combinations in a controlled setting using joint training. We find that in our setting these methods offer many improvements beyond raw WER, including substantial gains in tail-word WER, decoder computation during inference, and lattice density.
TL;DR: In this article , the authors evaluated the efficacy of BWA and BRB-F supplements for treating liver damage and dyslipidemia in a zebrafish with 1% ethanol with supplementation of 10% fermented black rice bran and 10% BWA.
Abstract: Alcohol abuse, a global health problem, is closely associated with many pathological processes, such as dyslipidemia and cardiovascular disease. In particular, excessive alcohol consumption promotes dyslipidemia and liver damage, such as hepatic steatosis, fibrosis, and cirrhosis. Beeswax alcohol (BWA) is a natural product used for its antioxidant properties that has not been evaluated for its efficacy in alcohol-induced liver injury. In the present study, zebrafish were exposed to 1% ethanol with supplementation of 10% fermented black rice bran (BRB-F), 10% BWA, or 10% mixtures of BWA+BRB-F (MIX). The BRB-F, BWA, and MIX supplementation increased the survival rate dramatically without affecting the body weight changes. In histology of hepatic tissue, alcoholic foamy degeneration was ameliorated by the BWA or MIX supplements. Moreover, dihydroethidium (DHE) and immunohistochemistry staining suggested that the MIX supplement decreased the hepatic ROS production and interleukin-6 expression significantly owing to the enhanced antioxidant properties, such as paraoxonase. Furthermore, the MIX supplement improved alcohol-induced dyslipidemia and oxidative stress. The BWA and MIX groups showed lower blood total cholesterol (TC) and triglyceride (TG) levels with higher high-density lipoprotein-cholesterol (HDL-C) than the alcohol-alone group. The MIX group showed the highest HDL-C/TC ratio and HDL-C/TG ratio with the lowest low-density lipoprotein (LDL)-C/HDL-C ratio. In conclusion, BWA and BRB-F showed efficacy to treat alcohol-related metabolic disorders, but the MIX supplement was more effective in ameliorating the liver damage and dyslipidemia, which agrees with an enhanced antioxidant and anti-inflammatory activity exhibited by BWA/BRB-F in a synergistic manner.
21 Mar 2023
TL;DR: In this article , the authors derive the ionization field directly from the initial density field without using the ionizing sources' location, and hence emulating the radiative transfer process, which represents a step towards generating large-scale ionization maps with a minimal cost and hence enabling rapid parameter inference at the field level.
Abstract: Semi-numerical simulations are the leading candidates for evolving reionization on cosmological scales. These semi-numerical models are efficient in generating large-scale maps of the 21cm signal, but they are too slow to enable inference at the field level. We present different strategies to train a U-Net to accelerate these simulations. We derive the ionization field directly from the initial density field without using the ionizing sources' location, and hence emulating the radiative transfer process. We find that the U-Net achieves higher accuracy in reconstructing the ionization field if the input includes either white noise or a noisy version of the ionization map beside the density field during training. Our model reconstructs the power spectrum over all scales perfectly well. This work represents a step towards generating large-scale ionization maps with a minimal cost and hence enabling rapid parameter inference at the field level.
TL;DR: This article proposed a general framework for the use of natural language (NL) feedback to formalize system-level design decisions in a human-in-the-loop process to produce better models.
Abstract: Natural language (NL) feedback contains rich information about the user experience. Existing studies focus on an instance-level approach, where feedback is used to refine specific examples, disregarding its system-wide application. This paper proposes a general framework for unlocking the system-level use of NL feedback. We show how to use feedback to formalize system-level design decisions in a human-in-the-loop-process -- in order to produce better models. In particular this is done through: (i) metric design for tasks; and (ii) language model prompt design for refining model responses. We conduct two case studies of this approach for improving search query generation and dialog response generation, demonstrating the effectiveness of the use of system-level feedback. We show the combination of system-level feedback and instance-level feedback brings further gains, and that human written instance-level feedback results in more grounded refinements than GPT-3.5 written ones, underlying the importance of human feedback for building systems.
TL;DR: In this article , a human clinical trial with Japanese participants, the policosanol group (n = 26, male 13/female 13) showed a remarkable decrease in alanine aminotransferase (ALT) from baseline up to 21% (p = 0.041) and 8.7%(p =0.017), respectively.
Abstract: Policosanol consumption has been associated with treating blood pressure and dyslipidemia by increasing the level of high-density lipoproteins-cholesterol (HDL-C) and HDL functionality. Although policosanol supplementation also ameliorated liver function in animal models, it has not been reported in a human clinical study, particularly with a 20 mg doage of policosanol. In the current study, twelve-week consumption of Cuban policosanol (Raydel®) significantly enhanced the hepatic functions, showing remarkable decreases in hepatic enzymes, blood urea nitrogen, and glycated hemoglobin. From the human trial with Japanese participants, the policosanol group (n = 26, male 13/female 13) showed a remarkable decrease in alanine aminotransferase (ALT) and aspartate aminotransferase (AST) from baseline up to 21% (p = 0.041) and 8.7% (p = 0.017), respectively. In contrast, the placebo group (n = 26, male 13/female 13) showed almost no change or slight elevation. The policosanol group showed a 16% decrease in γ-glutamyl transferase (γ-GTP) at week 12 from the baseline (p = 0.015), while the placebo group showed a 1.2% increase. The policosanol group exhibited significantly lower serum alkaline phosphatase (ALP) levels at week 8 (p = 0.012), week 12 (p = 0.012), and after 4-weeks (p = 0.006) compared to those of the placebo group. After 12 weeks of policosanol consumption, the ferric ion reduction ability and paraoxonase of serum were elevated by 37% (p < 0.001) and 29% (p = 0.004) higher than week 0, while placebo consumption showed no notable changes. Interestingly, glycated hemoglobin (HbA1c) in serum was lowered significantly in the policosanol group 4 weeks after consumption, which was approximately 2.1% (p = 0.004) lower than the placebo group. In addition, blood urea nitrogen (BUN) and uric acid levels were significantly lower in the policosanol group after 4 weeks: 14% lower (p = 0.002) and 4% lower (p = 0.048) than those of the placebo group, respectively. Repeated measures of ANOVA showed that the policosanol group had remarkable decreases in AST (p = 0.041), ALT (p = 0.008), γ-GTP (p = 0.016), ALP (p = 0.003), HbA1c (p = 0.010), BUN (p = 0.030), and SBP (p = 0.011) from the changes in the placebo group in point of time and group interaction. In conclusion, 12 weeks of 20 mg consumption of policosanol significantly enhanced hepatic protection by lowering the serum AST, ALT, ALP, and γ-GTP via a decrease in glycated hemoglobin, uric acid, and BUN with an elevation of serum antioxidant abilities. These results suggest that improvements in blood pressure by consumption of 20 mg of policosanol (Raydel®) were accompanied by protection of liver function and enhanced kidney function.