Deep neural nets (DNNs) have transformed speech recognition technology, offering a new level of accuracy. Neural networks showed great promise in the mid-1990s, but results proved disappointing and the approach was all but dead until new DNN methods began to power breakthroughs in performance and training speed. This tutorial explains how DNNs work and why they are so much better than previous methods. We review the latest techniques from several research centers and explain how much accuracy improves with each method. The session includes live demos and practice exercises. Participants are encouraged to bring a smartphone or laptop for part of the training. Attend this session to gain an understanding of DNN fundamentals, how DNNs fit into speech recognition systems, the role of open source tools, and where the next advances are expected to appear.

Mobile devices provide powerful computation and multiple sensors that enable users to perform tasks combining the modalities of voice, text, gesture, touch, type, and more. A multimodal user interface requires the integration of several recognition technologies together with sophisticated user interfaces for data input and output. The workshop discusses both the technology and usability aspects of interactive multimodal user interfaces. Learn which technologies are used in various types of user-device interactions as well as the available multimodal architectures and their design challenges. We discuss integration and deployment issues and give examples of real-world systems including automotive, wearables, and smartphones.

Spoken natural language interfaces are becoming ubiquitous. They can be found in IVRs, on smartphone apps, and on set-top box and gaming devices. This session introduces attendees to the underlying technologies used for such interfaces and addresses design issues associated with spoken natural dialogue. Emphasis is placed on conversational techniques required to build successful natural spoken interfaces including building robust NLU grammars, designing semantic schemas, scripting for responses, and dealing with ambiguous responses. People interested in natural language for customer service applications should be particularly interested in this session, but the techniques addressed apply equally to the emerging fields of smartphone intelligent assistants and other device-oriented applications.

Multimodal interfaces, combining speech, graphics, and sensor input, are becoming increasingly important for interaction with the rapidly expanding variety of nontraditional platforms, including mobile, wearable, robots, and devices in the Internet of Things. User interfaces on these platforms will need to be much more varied than traditional user interfaces. We demonstrate how to develop multimodal clients using standards such as WebRTC, WebAudio, and Web Sockets and the Open Web Platform, including open technologies such as HTML5, JavaScript and CSS. We also discuss integration with cloud resources for technologies such as speech recognition and natural language understanding. Attendees should have access to a browser that supports the Open Web Platform standards, for example, the current versions of Chrome, Firefox, or Opera. Basic knowledge of HTML5 and JavaScript would be very helpful.