The reinforcement learning paradigm is a popular way to address problems that have only limited environmental feedback, rather than correctly labeled examples, as is common in other machine learning contexts. While significant progress has been made to improve learning in a single task, the idea of transfer learning has only recently been applied to reinforcement learning tasks. The core idea of transfer is that experience gained in learning to perform one task can help improve learning performance in a related, but different, task. In this article we present a framework that classifies transfer learning methods in terms of their capabilities and goals, and then use it to survey the existing literature, as well as to suggest future directions for transfer learning work.

/pdf/transfer-learning-for-reinforcement-learning-domains-a-4d00ui6nyo.pdf

Transfer Learning for Reinforcement Learning Domains: A Survey

Service interface description languages such as WSDL, and related standards, are evolving rapidly to provide a foundation for interoperation between Web services. At the same time, Semantic Web service technologies, such as the Ontology Web Language for Services (OWL-S), are developing the means by which services can be given richer semantic specifications. Richer semantics can enable fuller, more flexible automation of service provision and use, and support the construction of more powerful tools and methodologies. Both sets of technologies can benefit from complementary uses and cross-fertilization of ideas. This paper shows how to use OWL-S in conjunction with Web service standards, and explains and illustrates the value added by the semantics expressed in OWL-S.

https://eprints.soton.ac.uk/263000/1/OWL-S-SWSWPC2004-final.pdf

Bringing semantics to web services: the OWL-S approach

https://www.cs.umd.edu/~nau/papers/sirin2004htn.pdf

HTN planning for Web Service composition using SHOP2

A translation apparatus is provided which comprises: an inputting section for inputting a source document in a natural language; a layout analyzing section for analyzing layout information including cascade information, itemization information, numbered itemization information, labeled itemization information and separator line information in the source document inputted by the inputting section and specifying a translation range on the basis of the layout information; a translation processing section for translating a source document text in the specified translation range into a second language; and an outputting section for outputting a translated text provided by the translation processing section.

Automated Planning: Theory and Practice.

https://www.princeton.edu/~yael/Publications/BotvinickNivBarto2008.pdf

Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective

The SHOP2 planning system received one of the awards for distinguished performance in the 2002 International Planning Competition. This paper describes the features of SHOP2 which enabled it to excel in the competition, especially those aspects of SHOP2 that deal with temporal and metric planning domains.

/pdf/shop2-an-htn-planning-system-20n5kqaph4.pdf

SHOP2: an HTN planning system

We design the simple hierarchical ordered planner (SHOP) and its successor, SHOP2, with two goals in mind: to investigate research issues in automated planning and to provide some simple, practical planning tools. SHOP and SHOP2 are based on a planning formalism called hierarchical task network planning. SHOP and SHOP2 use a search-control strategy called ordered task decomposition, which breaks tasks into subtasks and generates the plan's actions in the same order that the plan executor executes them. So, throughout the planning process, the planner can tell what the state of the world at each step of the plan.

/pdf/applications-of-shop-and-shop2-3bx9pic6ve.pdf

Applications of SHOP and SHOP2

: In this paper, we describe a way to improve the performance of hand-tailorable planners by compiling each domain description into a separate domain-specific planner. We discuss why and when this approach can be useful, and we present experimental results showing that our approach produces significant increases in the speed of planning.

/pdf/a-general-approach-to-synthesize-problem-specific-planners-2cbrd25kwg.pdf

A General Approach to Synthesize Problem-Specific Planners

Documentation for JSHOP2

A great challenge in using any planning system to solve real-world problems is the difficulty of acquiring the domain knowledge that the system will need. We present away to address part of this problem, in the context of Hierarchical Task Network (HTN) planning, by having the planning system incrementally learn conditions for HTN methods under expert supervision. We present a general formal framework for learning HTN methods, and a supervised learning algorithm, named CaMeL, based on this formalism. We present theoretical results about CaMeL's soundness, completeness, and convergence properties. We also report experimental results about its speed of convergence under different conditions. The experimental results suggest that CaMeL has the potential to be useful in real-world applications.

/pdf/camel-learning-method-preconditions-for-htn-planning-430jke4g9z.pdf

Okhtay Ilghami

Papers

SHOP2: an HTN planning system

Applications of SHOP and SHOP2

A General Approach to Synthesize Problem-Specific Planners

Documentation for JSHOP2

CaMeL: learning method preconditions for HTN planning