The development of new CRISPR-Cas genome editing tools continues to drive major advances in the life sciences. Four classes of CRISPR-Cas-derived genome editing agents-nucleases, base editors, transposases/recombinases and prime editors-are currently available for modifying genomes in experimental systems. Some of these agents have also moved rapidly into the clinic. Each tool comes with its own capabilities and limitations, and major efforts have broadened their editing capabilities, expanded their targeting scope and improved editing specificity. We analyze key considerations when choosing genome editing agents and identify opportunities for future improvements and applications in basic research and therapeutics.

Genome editing with CRISPR–Cas nucleases, base editors, transposases and prime editors

The advent of clustered regularly interspaced short palindromic repeat (CRISPR) genome editing, coupled with advances in computing and imaging capabilities, has initiated a new era in which genetic diseases and individual disease susceptibilities are both predictable and actionable. Likewise, genes responsible for plant traits can be identified and altered quickly, transforming the pace of agricultural research and plant breeding. In this Review, we discuss the current state of CRISPR-mediated genetic manipulation in human cells, animals, and plants along with relevant successes and challenges and present a roadmap for the future of this technology. Description A decade of CRISPR In the decade since the publication of CRISPR-Cas9 as a genome-editing technology, the CRISPR toolbox and its applications have profoundly changed basic and applied biological research. Wang and Doudna now review the origins and utility of CRISPR-based genome editing, the successes and current limitations of the technology, and where innovation and engineering are needed. The authors describe important advances in the development of CRISPR genome-editing technology and make predictions about where the field is headed. They also highlight specific examples in medicine and agriculture that show how CRISPR is already affecting society, with exciting opportunities for the future. —DJ A review discusses the current state of CRISPR-mediated genetic manipulation in human cells, animals, and plants and considers its future potential. BACKGROUND The fields of molecular biology, genetics, and genomics are at a critical juncture—a moment in history when a convergence of knowledge and methods has made it both technically possible and incredibly useful to edit specific base pairs or segments of DNA in cells and living organisms. The advent of clustered regularly interspaced short palindromic repeat (CRISPR) genome editing, coupled with advances in computing and imaging capabilities, has initiated a new era in which we can not only diagnose human diseases and even predict individual susceptibility based on personal genetics but also act on that information. Likewise, we can both identify and rapidly alter genes responsible for plant traits, transforming the pace of agricultural research and plant breeding. The applications of this technology convergence are profound and far reaching—and they are happening now. In the decade since the publication of CRISPR-Cas9 as a genome editing technology, the CRISPR toolbox and its applications have profoundly changed biological research, impacting not only patients with genetic diseases but also agricultural practices and products. As a specific example from the field of genomic medicine, it has become feasible to obtain a complete sequence of the human genome in less than 24 hours—a staggering advance considering the first such sequence took 5 years to generate. Notably, designing and putting to use a potent CRISPR genome editor to obtain clinically actionable information from that genome—previously a near-intractable challenge—now takes only a matter of days. For additional background and related topics, we refer readers to in-depth reviews of the microbiology and structural biology of CRISPR systems and to articles about the considerable ethical and societal challenges of this technology. ADVANCES The past decade has witnessed the discovery, engineering, and deployment of RNA-programmed genome editors across many applications. By leveraging CRISPR-Cas9’s most fundamental activity to create a targeted genetic disruption in a gene or gene regulatory element, scientists have built successful platforms for the rapid creation of knockout mice and other animal models, genetic screening, and multiplexed editing. Beyond traditional CRISPR-Cas9–induced knockouts, base editing—a technology utilizing engineered Cas9’s fused to enzymes that alter the chemical nature of DNA bases—has also provided a highly useful strategy to generate site-specific and precise point mutations. Over the past decade, scientists have utilized CRISPR technology as a readily adaptable tool to probe biological function, dissect genetic interactions, and inform strategies to combat human diseases and engineer crops. This Review covers the origins and successes of CRISPR-based genome editing and discusses the most pressing challenges, which include improving editing accuracy and precision, implementing strategies for precise programmable genetic sequence insertions, improving targeted delivery of CRISPR editors, and increasing access and affordability. We examine current efforts addressing these challenges, including emerging gene insertion technologies and new delivery modalities, and describe where further innovation and engineering are needed. CRISPR genome editors are already being deployed in medicine and agriculture, and this Review highlights key examples, including a CRISPR-based therapy treating sickle cell disease, a more nutritious CRISPR-edited tomato, and a high-yield, disease-resistant CRISPR-edited wheat, to illustrate CRISPR’s current and potential future impacts in society. OUTLOOK In the decade ahead, genome editing research and applications will continue to expand and will intersect with advances in technologies, such as machine learning, live cell imaging, and sequencing. A combination of discovery and engineering will diversify and refine the CRISPR toolbox to combat current challenges and enable more wide-ranging applications in both fundamental and applied research. Just as during the advent of CRISPR genome editing, a combination of scientific curiosity and the desire to benefit society will drive the next decade of innovation in CRISPR technology. CRISPR: past, present, and future. The past decade of CRISPR technology has focused on building the platforms for generating gene knockouts, creating knockout mice and other animal models, genetic screening, and multiplexed editing. CRISPR’s applications in medicine and agriculture are already beginning and will serve as the focus for the next decade as society’s demands drive further innovation in CRISPR technology.

CRISPR technology: A decade of genome editing is only the beginning

In combination with transgenic mouse lines expressing Cre or Flp recombinases in defined cell types, recombinase-dependent adeno-associated viruses (AAVs) have become the tool of choice for localized cell-type-targeted gene expression. Unfortunately, applications of this technique when expressing highly sensitive transgenes are impeded by off-target, or "leak" expression, from recombinase-dependent AAVs. We investigated this phenomenon and find that leak expression is mediated by both infrequent transcription from the inverted transgene in recombinant-dependent AAV designs and recombination events during bacterial AAV plasmid production. Recombination in bacteria is mediated by homology across the antiparallel recombinase-specific recognition sites present in recombinase-dependent designs. To address both of these issues we designed an AAV vector that uses mutant "cross-over insensitive" recognition sites combined with an "ATG-out" design. We show that these CIAO (cross-over insensitive ATG-out) vectors virtually eliminate leak expression. CIAO vectors provide reliable and targeted transgene expression and are extremely useful for recombinase-dependent expression of highly sensitive transgenes.

https://www.pnas.org/content/pnas/116/52/27001.full.pdf

Sources of off-target expression from recombinase-dependent AAV vectors and mitigation with cross-over insensitive ATG-out vectors.

Protein engineering is used to generate novel protein folds and assemblages, to impart new properties and functions onto existing proteins, and to enhance our understanding of principles that govern protein structure. While such approaches can be employed to reprogram protein-protein interactions, modifying protein-DNA interactions is more difficult. This may be related to the structural features of protein-DNA interfaces, which display more charged groups, directional hydrogen bonds, ordered solvent molecules and counterions than comparable protein interfaces. Nevertheless, progress has been made in the redesign of protein-DNA specificity, much of it driven by the development of engineered enzymes for genome modification. Here, we summarize the creation of novel DNA specificities for zinc finger proteins, meganucleases, TAL effectors, recombinases and restriction endonucleases. The ease of re-engineering each system is related both to the modularity of the protein and the extent to which the proteins have evolved to be capable of readily modifying their recognition specificities in response to natural selection. The development of engineered DNA binding proteins that display an ideal combination of activity, specificity, deliverability, and outcomes is not a fully solved problem, however each of the current platforms offers unique advantages, offset by behaviors and properties requiring further study and development.

Engineering altered protein–DNA recognition specificity

We study the problem of efficient generative inference for Transformer models, in one of its most challenging settings: large deep models, with tight latency targets and long sequence lengths. Better understanding of the engineering tradeoffs for inference for large Transformer-based models is important as use cases of these models are growing rapidly throughout application areas. We develop a simple analytical model for inference efficiency to select the best multi-dimensional partitioning techniques optimized for TPU v4 slices based on the application requirements. We combine these with a suite of low-level optimizations to achieve a new Pareto frontier on the latency and model FLOPS utilization (MFU) tradeoffs on 500B+ parameter models that outperforms the FasterTransformer suite of benchmarks. We further show that with appropriate partitioning, the lower memory requirements of multiquery attention (i.e. multiple query heads share single key/value head) enables scaling up to 32x larger context lengths. Finally, we achieve a low-batch-size latency of 29ms per token during generation (using int8 weight quantization) and a 76% MFU during large-batch-size processing of input tokens, while supporting a long 2048-token context length on the PaLM 540B parameter model.

Efficiently Scaling Transformer Inference

The development of site-specific recombinases (SSRs) as genome editing agents is limited by the difficulty of altering their native DNA specificities. Here we describe Rec-seq, a method for revealing the DNA specificity determinants and potential off-target substrates of SSRs in a comprehensive and unbiased manner. We applied Rec-seq to characterize the DNA specificity determinants of several natural and evolved SSRs including Cre, evolved variants of Cre, and other SSR family members. Rec-seq profiling of these enzymes and mutants thereof revealed previously uncharacterized SSR interactions, including specificity determinants not evident from SSR:DNA structures. Finally, we used Rec-seq specificity profiles to predict off-target substrates of Tre and Brec1 recombinases, including endogenous human genomic sequences, and confirmed their ability to recombine these off-target sequences in human cells. These findings establish Rec-seq as a high-resolution method for rapidly characterizing the DNA specificity of recombinases with single-nucleotide resolution, and for informing their further development.

/pdf/high-resolution-specificity-profiling-and-off-target-v2h4bczzsu.pdf

High-resolution specificity profiling and off-target prediction for site-specific DNA recombinases

Quantization has become a popular technique to compress neural networks and reduce compute cost, but most prior work focuses on studying quantization without changing the network size. Many real-world applications of neural networks have compute cost and memory budgets, which can be traded off with model quality by changing the number of parameters. In this work, we use ResNet as a case study to systematically investigate the effects of quantization on inference compute cost-quality tradeoff curves. Our results suggest that for each bfloat16 ResNet model, there are quantized models with lower cost and higher ac-curacy; in other words, the bfloat16 compute cost-quality tradeoff curve is Pareto-dominated by the 4-bit and 8-bit curves, with models primarily quantized to 4-bit yielding the best Pareto curve. Furthermore, we achieve state-of-the-art results on ImageNet for 4-bit ResNet-50 with quantization-aware training, obtaining a top-1 eval accuracy of 77.09%. We demonstrate the regularizing effect of quantization by measuring the generalization gap. The quantization method we used is optimized for practicality: It requires little tuning and is designed with hardware capabilities in mind. Our work motivates further research into optimal numeric formats for quantization, as well as the development of machine learning accelerators supporting these formats. As part of this work, we contribute a quantization library written in JAX, which is open-sourced at https://github.com/google-research/google-research/tree/master/aqt.

/pdf/pareto-optimal-quantized-resnet-is-mostly-4-bit-1m6qcqam6r.pdf

Pareto-Optimal Quantized ResNet Is Mostly 4-bit

Quantization has become a popular technique to compress neural networks and reduce compute cost, but most prior work focuses on studying quantization without changing the network size. Many real-world applications of neural networks have compute cost and memory budgets, which can be traded off with model quality by changing the number of parameters. In this work, we use ResNet as a case study to systematically investigate the effects of quantization on inference compute cost-quality tradeoff curves. Our results suggest that for each bfloat16 ResNet model, there are quantized models with lower cost and higher accuracy; in other words, the bfloat16 compute cost-quality tradeoff curve is Pareto-dominated by the 4-bit and 8-bit curves, with models primarily quantized to 4-bit yielding the best Pareto curve. Furthermore, we achieve state-of-the-art results on ImageNet for 4-bit ResNet-50 with quantization-aware training, obtaining a top-1 eval accuracy of 77.09%. We demonstrate the regularizing effect of quantization by measuring the generalization gap. The quantization method we used is optimized for practicality: It requires little tuning and is designed with hardware capabilities in mind. Our work motivates further research into optimal numeric formats for quantization, as well as the development of machine learning accelerators supporting these formats. As part of this work, we contribute a quantization library written in JAX, which is open-sourced at this https URL.

Chas Leichner

Papers

High-resolution specificity profiling and off-target prediction for site-specific DNA recombinases

Pareto-Optimal Quantized ResNet Is Mostly 4-bit

Pareto-Optimal Quantized ResNet Is Mostly 4-bit