Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

PROBLEM TO BE SOLVED: To solve the problem, wherein it is impossible for an electronic content information provider to provide commercially secure and effective method, for a configurable general-purpose electronic commercial transaction/distribution control system. SOLUTION: In this system, having at least one protected processing environment for safely controlling at least one portion of decoding of digital information, a secure content distribution method comprises a process for encapsulating digital information in one or more digital containers; a process for encrypting at least a portion of digital information; a process for associating at least partially secure control information for managing interactions with encrypted digital information and/or digital container; a process for delivering one or more digital containers to a digital information user; and a process for using a protected processing environment, for safely controlling at least a portion of the decoding of the digital information. COPYRIGHT: (C)2006,JPO&NCIPI

https://apps.dtic.mil/dtic/tr/fulltext/u2/a423264.pdf

Systems and Methods for Secure Transaction Management and Electronic Rights Protection

This document describes an updated version of the "Security
Architecture for IP", which is designed to provide security services
for traffic at the IP layer. This document obsoletes RFC 2401
(November 1998). [STANDARDS-TRACK]

/pdf/security-architecture-for-the-internet-protocol-1x2l107p1d.pdf

Security Architecture for the Internet Protocol

As sensor networks edge closer towards wide-spread deployment, security issues become a central concern. So far, much research has focused on making sensor networks feasible and useful, and has not concentrated on security.We present a suite of security building blocks optimized for resource-constrained environments and wireless communication. SPINS has two secure building blocks: SNEP and mTESLA SNEP provides the following important baseline security primitives: Data confidentiality, two-party data authentication, and data freshness. A particularly hard problem is to provide efficient broadcast authentication, which is an important mechanism for sensor networks. mTESLA is a new protocol which provides authenticated broadcast for severely resource-constrained environments. We implemented the above protocols, and show that they are practical even on minimal hardware: the performance of the protocol suite easily matches the data rate of our network. Additionally, we demonstrate that the suite can be used for building higher level protocols.

http://www.csl.mtu.edu/cs5461/www/Reading/Perrig02.pdf

SPINS: security protocols for sensor networks

National Institute of Standards and Technology における超伝導研究及び生活

Classic cryptographic protocols based on user-chosen keys allow an attacker to mount password-guessing attacks. A combination of asymmetric (public-key) and symmetric (secret-key) cryptography that allow two parties sharing a common password to exchange confidential and authenticated information over an insecure network is introduced. In particular, a protocol relying on the counter-intuitive motion of using a secret key to encrypt a public key is presented. Such protocols are secure against active attacks, and have the property that the password is protected against offline dictionary attacks. >

/pdf/encrypted-key-exchange-password-based-protocols-secure-4zhvfjd0go.pdf

Encrypted key exchange: password-based protocols secure against dictionary attacks

The current Internet infrastructure has very few built-in protection mechanisms, and is therefore vulnerable to attacks and failures. In particular, recent events have illustrated the Internet's vulnerability to both denial of service (DoS) attacks and flash crowds in which one or more links in the network (or servers at the edge of the network) become severely congested. In both DoS attacks and flash crowds the congestion is due neither to a single flow, nor to a general increase in traffic, but to a well-defined subset of the traffic --- an aggregate. This paper proposes mechanisms for detecting and controlling such high bandwidth aggregates. Our design involves both a local mechanism for detecting and controlling an aggregate at a single router, and a cooperative pushback mechanism in which a router can ask upstream routers to control an aggregate. While certainly not a panacea, these mechanisms could provide some needed relief from flash crowds and flooding-style DoS attacks. The presentation in this paper is a first step towards a more rigorous evaluation of these mechanisms.

/pdf/controlling-high-bandwidth-aggregates-in-the-network-10zeldncmn.pdf

Controlling high bandwidth aggregates in the network

From the Book:

But after a time, as Frodo did not show any sign of writing a book on the spot, the
hobbits returned to their questions about doings in the Shire.

Lord of the Rings
—J.R.R. TOLKIEN



The first printing of the First Edition appeared at the Las Vegas Interop in May, 1994. At that same show appeared the first of many commercial firewall products. In many ways, the field has matured since then: You can buy a decent firewall off the shelf from many vendors.

The problem of deploying that firewall in a secure and useful manner remains. We have studied many Internet access arrangements in which the only secure component was the firewall itself—it was easily bypassed by attackers going after the “protected” inside machines. Before the trivestiture of AT&T/Lucent/NCR, there were over 300,000 hosts behind at least six firewalls, plus special access arrangements with some 200 business partners.

Our first edition did not discuss the massive sniffing attacks discovered in the spring of 1994. Sniffers had been running on important Internet Service Provider (ISP) machines for months—machines that had access to a major percentage of the ISP’s packet flow. By some estimates, these sniffers captured over a million host name/user name/password sets from passing telnet, ftp, and rlogin sessions. There were also reports of increased hacker activity on military sites. It’s obvious what must have happened: If you are a hacker with a million passwords in your pocket, you are going to look for the most interesting targets, and .mil certainly qualifies.

Since the First Edition, we have been slowlylosing the Internet arms race. The hackers have developed and deployed tools for attacks we had been anticipating for years. IP spoofing Shimomura, 1996 and TCP hijacking are now quite common, according to the Computer Emergency Response Team (CERT). ISPs report that attacks on the Internet’s infrastructure are increasing.

There was one attack we chose not to include in the First Edition: the SYN-flooding denial-of- service attack that seemed to be unstoppable. Of course, the Bad Guys learned about the attack anyway, making us regret that we had deleted that paragraph in the first place. We still believe that it is better to disseminate this information, informing saints and sinners at the same time. The saints need all the help they can get, and the sinners have their own channels of communication.Crystal Ball or Bowling Ball?The first edition made a number of predictions, explicitly or implicitly. Was our foresight accurate?

Our biggest failure was neglecting to foresee how successful the Internet would become. We barely mentioned the Web and declined a suggestion to use some weird syntax when listing software resources. The syntax, of course, was the URL...

Concomitant with the growth of the Web, the patterns of Internet connectivity vastly increased. We assumed that a company would have only a few external connections—few enough that they’d be easy to keep track of, and to firewall. Today’s spaghetti topology was a surprise.

We didn’t realize that PCs would become Internet clients as soon as they did. We did, however, warn that as personal machines became more capable, they’d become more vulnerable. Experience has proved us very correct on that point.

We did anticipate high-speed home connections, though we spoke of ISDN, rather than cable modems or DSL. (We had high-speed connectivity even then, though it was slow by today’s standards.) We also warned of issues posed by home LANs, and we warned about the problems caused by roaming laptops.

We were overly optimistic about the deployment of IPv6 (which was called IPng back then, as the choice hadn’t been finalized). It still hasn’t been deployed, and its future is still somewhat uncertain.

We were correct, though, about the most fundamental point we made: Buggy host software is a major security issue. In fact, we called it the “fundamental theorem of firewalls”:

Most hosts cannot meet our requirements: they run too many programs that are too large. Therefore, the only solution is to isolate them behind a firewall if you wish to run any programs at all.

If anything, we were too conservative.Our ApproachThis book is nearly a complete rewrite of the first edition. The approach is different, and so are many of the technical details. Most people don’t build their own firewalls anymore. There are far more Internet users, and the economic stakes are higher. The Internet is a factor in warfare.

The field of study is also much larger—there is too much to cover in a single book. One reviewer suggested that Chapters 2 and 3 could be a six-volume set. (They were originally one mammoth chapter.) Our goal, as always, is to teach an approach to security. We took far too long to write this edition, but one of the reasons why the first edition survived as long as it did was that we concentrated on the concepts, rather than details specific to a particular product at a particular time. The right frame of mind goes a long way toward understanding security issues and making reasonable security decisions. We’ve tried to include anecdotes, stories, and comments to make our points.

Some complain that our approach is too academic, or too UNIX-centric, that we are too idealistic, and don’t describe many of the most common computing tools. We are trying to teach attitudes here more than specific bits and bytes. Most people have hideously poor computing habits and network hygiene. We try to use a safer world ourselves, and are trying to convey how we think it should be.

The chapter outline follows, but we want to emphasize the following:

It is OK to skip the hard parts.

If we dive into detail that is not useful to you, feel free to move on.

The introduction covers the overall philosophy of security, with a variety of time-tested maxims. As in the first edition, Chapter 2 discusses most of the important protocols, from a security point of view. We moved material about higher-layer protocols to Chapter 3. The Web merits a chapter of its own.

The next part discusses the threats we are dealing with: the kinds of attacks in Chapter 5, and some of the tools and techniques used to attack hosts and networks in Chapter 6. Part III covers some of the tools and techniques we can use to make our networking world safer. We cover authentication tools in Chapter 7, and safer network servicing software in Chapter 8.

Part IV covers firewalls and virtual private networks (VPNs). Chapter 9 introduces various types of firewalls and filtering techniques, and Chapter 10 summarizes some reasonable policies for filtering some of the more essential services discussed in Chapter 2. If you don’t find advice about filtering a service you like, we probably think it is too dangerous (refer to Chapter 2).

Chapter 11 covers a lot of the deep details of firewalls, including their configuration, administration, and design. It is certainly not a complete discussion of the subject, but should give readers a good start. VPN tunnels, including holes through firewalls, are covered in some detail in Chapter 12. There is more detail in Chapter 18.

In Part V, we apply these tools and lessons to organizations. Chapter 13 examines the problems and practices on modern intranets. See Chapter 15 for information about deploying a hacking-resistant host, which is useful in any part of an intranet. Though we don’t especially like intrusion detection systems (IDSs) very much, they do play a role in security, and are discussed in Chapter 15.

The last part offers a couple of stories and some further details. The Berferd chapter is largely unchanged, and we have added “The Taking of Clark,” a real-life story about a minor break-in that taught useful lessons.

Chapter 18 discusses secure communications over insecure networks, in quite some detail. For even further detail, Appendix A has a short introduction to cryptography.

The conclusion offers some predictions by the authors, with justifications. If the predictions are wrong, perhaps the justifications will be instructive. (We don’t have a great track record as prophets.) Appendix B provides a number of resources for keeping up in this rapidly changing field.Errata and UpdatesEveryone and every thing seems to have a Web site these days; this book is no exception. Our “official” Web site is . We’ll post an errata list there; we’ll also keep an up-to-date list of other useful Web resources. If you find any errors—we hope there aren’t many—please let us know via e-mail at .AcknowledgmentsFor many kindnesses, we’d like to thank Joe Bigler, Steve “Hollywood” Branigan, Hal Burch, Brian Clapper, David Crocker, Tom Dow, Phil Edwards and the Internet Public Library, Anja Feldmann, Karen Gettman, Brian Kernighan, David Korman, Tom Limoncelli, Norma Loquendi, Cat Okita, Robert Oliver, Vern Paxson, Marcus Ranum, Eric Rescorla, Guido van Rooij, Luann Rouff (a most excellent copy editor), Abba Rubin, Peter Salus, Glenn Sieb, Karl Siil (we’ll always have Boston), Irina Strizhevskaya, Rob Thomas, Win Treese, Dan Wallach, Avishai Wool, Karen Yannetta, and Michal Zalewski, among many others.


BILL CHESWICK


STEVE BELLOVIN


AVI RUBIN






020163466XP01302003

Firewalls and Internet Security: Repelling the Wily Hacker

The TCP/IP protocol suite, which is very widely used today, was developed under the sponsorship of the Department of Defense. Despite that, there are a number of serious security flaws inherent in the protocols, regardless of the correctness of any implementations. We describe a variety of attacks based on these flaws, including sequence number spoofing, routing attacks, source address spoofing, and authentication attacks. We also present defenses against these attacks, and conclude with a discussion of broad-spectrum defenses such as encryption.

/pdf/security-problems-in-the-tcp-ip-protocol-suite-32o4kyp6h7.pdf

Security problems in the TCP/IP protocol suite

The encrypted key exchange (EKE) protocol is augmented so that hosts do not store cleartext passwords. Consequently, adversaries who obtain the one-way encrypted password file may (i) successfully mimic (spoof) the host to the user, and (ii) mount dictionary attacks against the encrypted passwords, but cannot mimic the user to the host. Moreover, the important security properties of EKE are preserved—an active network attacker obtains insufficient information to mount dictionary attacks. Two ways to accomplish this are shown, one using digital signatures and one that relies on a family of commutative one-way functions.

/pdf/augmented-encrypted-key-exchange-a-password-based-protocol-umol33k4rd.pdf

Steven M. Bellovin

Papers

Encrypted key exchange: password-based protocols secure against dictionary attacks

Controlling high bandwidth aggregates in the network

Firewalls and Internet Security: Repelling the Wily Hacker

Security problems in the TCP/IP protocol suite

Augmented encrypted key exchange: a password-based protocol secure against dictionary attacks and password file compromise