Experiences of an In-Service Wizard-of-Oz Data Collection for the Deployment of a Call-Routing Application

TLDR: This paper describes the experiences of collecting a corpus of 42,000 dialogues for a call-routing application using a Wizard-of-Oz approach, and provides a detailed exposition of the data collection as such and the application used, and compares the approach to methods previously used.

Abstract: This paper describes our experiences of collecting a corpus of 42,000 dialogues for a call-routing application using a Wizard-of-Oz approach. Contrary to common practice in the industry, we did not use the kind of automated application that elicits some speech from the customers and then sends all of them to the same destination, such as the existing touch-tone menu, without paying attention to what they have said. Contrary to the traditional Wizard-of-Oz paradigm, our data-collection application was fully integrated within an existing service, replacing the existing touch-tone navigation system with a simulated call-routing system. Thus, the subjects were real customers calling about real tasks, and the wizards were service agents from our customer care. We provide a detailed exposition of the data collection as such and the application used, and compare our approach to methods previously used.