Spoken multimodal human-computer dialogue in mobile environments /
副标题：无
作 者：edited by W. Minker, Dirk B鑥hler, and Laila Dybkj祌.
分类号：
ISBN：9781402030734
微信扫一扫,移动浏览光盘
简介
Summary:
Publisher Summary 1
The ongoing migration of computing and information access from stationary environments to mobile computing devices for eventual use in mobile environments, such as Personal Digital Assistants (PDAs), tablet PCs, next generation mobile phones, and in-car driver assistance systems, poses critical challenges for natural human-computer interaction. Spoken dialogue is a key factor in ensuring natural and user-friendly interaction with such devices which are meant not only for computer specialists, but also for everyday users. Speech supports hands-free and eyes-free operation, and becomes a key alternative interaction mode in mobile environments, e.g. in cars where driver distraction by manually operated devices may be a significant problem. On the other hand, the use of mobile devices in public places, may make the possibility of using alternative modalities possibly in combination with speech, such as graphics output and gesture input, preferable due to e.g. privacy issues. Researchers' interest is progressively turning to the integration of speech with other modalities such as gesture input and graphics output, partly to accommodate more efficient interaction and partly to accommodate different user preferences. This book:combines overview chapters of key areas in spoken multimodal dialogue (systems and components, architectures, and evaluation) with chapters focussed on particular applications or problems in the field. focusses on the influence of the environment when building and evaluating an application. Audience: Computer scientists, engineers, and others who work in the area of spoken multimodal dialogue systems in academia and in the industry.Graduate students and Ph.D. students specialising in spoken multimodal dialogue systems in general, or focusing on issues in these systems in mobile environments in particular
目录
Table Of Contents:
Preface xi
Contributing Authors xiii
Introduction xxi
Part I Issues in Multimodal Spoken Dialogue Systems and Components
1 Multimodal Dialogue Systems 3(10)
Alexander I. Rudnicky
1. Introduction 3(1)
2. Varieties of Multimodal Dialogue 4(2)
3. Detecting Intentional User Inputs 6(1)
4. Modes and Modalities 7(1)
5. History and Context 7(1)
6. Domain Reasoning 8(1)
7. Output Planning 9(1)
8. Dialogue Management 9(1)
9. Conclusion 10(1)
References 11(2)
2 Speech Recognition Technology in Multimodal/Ubiquitous Computing Environments 13(24)
Sadaoki Furui
1. Ubiquitous/Wearable Computing Environment 13(1)
2. State-of-the-Art Speech Recognition Technology 14(2)
3. Ubiquitous Speech Recognition 16(2)
4. Robust Speech Recognition 18(3)
5. Conversational Systems for Information Access 21(3)
6. Systems for Transcribing, Understanding and Summarising ubiquitous Speech Documents 24(8)
7. Conclusion 32(1)
References 33(4)
3 A Robust Multimodal Speech Recognition Method using Optical Flow Analysis 37(18)
Satoshi Tamura, Koji Iwano, Sadaoki Furui
1. Introduction 38(1)
2. Optical Flow Analysis 39(1)
3. A Multimodal Speech Recognition System 40(3)
4. Experiments for Noise-Added Data 43(5)
5. Experiments for Real-World Data 48(1)
6. Conclusion and Future Work 49(3)
References 52(3)
4 Feature Functions for Tree-Based Dialogue Course Management 55(22)
Klaus Macherey, Hermann Ney
1. Introduction 55(1)
2. Basic Dialogue Framework 56(3)
3. Feature Functions 59(4)
4. Computing Dialogue Costs 63(1)
5. Selection of Dialogue State/Action Pairs 64(1)
6. XML-based Data Structures 65(3)
7. Usability in Mobile Environments 68(1)
8. Results 69(5)
9. Summary and Outlook 74(1)
References 74(3)
5 A Reasoning Component for Information-Seeking and Planning Dialogues 77(16)
Dirk B眉hler, Wolfgang Minker
1. Introduction 77(3)
2. State-of-the-Art in Problem Solving Dialogues 80(1)
3. Reasoning Architecture 81(4)
4. Application to Calendar Planning 85(3)
5. Conclusion 88(2)
References 90(3)
6 A Model for Multimodal Dialogue System Output Applied to an Animated Talking Head 93(24)
Jonas Beskow, Jens Edlund, Magnus Nordstrand
1. Introduction 93(4)
2. Specification 97(6)
3. Interpretation 103(2)
4. Realisation in an Animated Talking Head 105(4)
5. Discussion and Future Work 109(2)
References 111(6)
Part II System Architecture and Example Implementations
7 Overview of System Architecture 117(16)
Andreas Kellner
1. Introduction 117(1)
2. Towards Personal Multimodal Conversational User Interface 118(4)
3. System Architectures for Multimodal Dialogue Systems 122(4)
4. Standardisation of Application Representation 126(3)
5. Conclusion 129(1)
References 130(3)
8 XISL: A Modality-Independent MMI Description Language 133(16)
Kouichi Katsurada, Hirobumi Yamada, Yusaku Nakamura, Satoshi Kobayashi, Tsuneo Nitta
1. Introduction 133(1)
2. XISL Execution System 134(2)
3. Extensible Interaction Scenario Language 136(4)
4. Three Types of Front-Ends and XISL Descriptions 140(6)
5. XISL and Other Languages 146(1)
6. Discussion 147(1)
References 148(1)
9 A Path to Multimodal Data Services for Telecommunications 149(20)
Georg Niklfeld, Michael Pucher, Robert Finan, Wolfgang Eckhart
1. Introduction 149(1)
2. Application Considerations, Technologies and Mobile Terminals 150(4)
3. Projects and Commercial Developments 154(2)
4. Three Multimodal Demonstrators 156(5)
5. Roadmap for Successful Versatile Interfaces in Telecommunications 161(2)
6. Conclusion 163(1)
References 164(5)
10 Multimodal Spoken Dialogue with Wireless Devices 169(16)
Roberto Pieraccini, Bob Carpenter, Eric Woudenberg, Sasha Caskey, Stephen Springer, Jonathan Bloom, Michael Phillips
1. Introduction 169(2)
2. Why Multimodal Wireless? 171(1)
3. Walking Direction Application 172(1)
4. Speech Technology for Multimodal Wireless 173(1)
5. User Interface Issues 174(5)
6. Multimodal Architecture Issues 179(3)
7. Conclusion 182(2)
References 184(1)
11 The SmartKom Mobile Car Prototype System for Flexible Human-Machine Communication 185(18)
Dirk B眉hler, Wolfgang Minker
1. Introduction 185(1)
2. Related Work 186(3)
3. SmartKom - Intuitive Human-Machine Interaction 189(2)
4. Scenarios for Mobile Use 191(2)
5. Demonstrator Architecture 193(1)
6. Dialogue Design 194(3)
7. Outlook - Towards Flexible Modality Control 197(2)
8. Conclusion 199(1)
References 200(3)
12 LARRI: A Language-Based Maintenance and Repair Assistant 203(18)
Dan Bohus, Alexander I. Rudnicky
1. Introduction 203(1)
2. LARRI - System Description 204(4)
3. LARRI - Hardware and Software Architecture 208(5)
4. Experiments and Results 213(2)
5. Conclusion 215(2)
References 217(4)
Part III Evaluation and Usability
13 Overview of Evaluation and Usability 221(26)
Laila Dybkj忙r, Niels Ole Bernsen, Wolfgang Minker
1. Introduction 221(2)
2. State-of-the-Art 223(4)
3. Empirical Generalisations 227(7)
4. Frameworks 234(2)
5. Multimodal SDSs Usability, Generalisations and Theory 236(2)
6. Discussion and Outlook 238(3)
References 241(6)
14 Evaluating Dialogue Strategies in Multimodal Dialogue Systems 247(22)
Steve Whittaker, Marilyn Walker
1. Introduction 247(4)
2. Wizard-of-Oz Experiment 251(11)
3. Overhearer Experiment 262(4)
4. Discussion 266(1)
References 267(2)
15 Enhancing the Usability of Multimodal Virtual Co-drivers 269(18)
Niels Ole Bernsen, Laila Dybkj忙r
1. Introduction 269(2)
2. The VICO System 271(1)
3. VICO Haptics - How and When to Make VICO Listen? 272(2)
4. VICO Graphics - When might the Driver Look? 274(4)
5. Who is Driving this Time? 278(2)
6. Modelling the Driver 280(4)
7. Conclusion and Future Work 284(1)
References 285(2)
16 Design, Implementation and Evaluation of the SENECA Spoken Language Dialogue System 287(24)
Wolfgang Minker, Udo Haiber, Paul Heisterkamp, Sven Scheible
1. Introduction 288(2)
2. The SENECA SLDS 290(11)
3. Evaluation of the SENECA SLDS Demonstrator 301(7)
4. Conclusion 308(1)
References 309(2)
17 Segmenting Route Descriptions for Mobile Devices 311(18)
Sabine Geldof, Robert Dale
1. Introduction 311(4)
2. Structured Information Delivery 315(1)
3. Techniques 315(7)
4. Evaluation 322(4)
5. Conclusion 326(1)
References 327(2)
18 Effects of Prolonged Use on the Usability of a Multimodal Form-Filling Interface 329(20)
Janienke Sturm, Bert Cranen, Jacques Terken, Isle Bakx
1. Introduction 329(3)
2. The Matis System 332(3)
3. Methods 335(2)
4. Results and Discussion 337(8)
5. Conclusion 345(1)
References 346(3)
19 User Multitasking with Mobile Multimodal Systems 349(30)
Anthony Jameson, Kerstin Kl枚ckner
1. The Challenge of Multitasking 350(4)
2. Example System 354(1)
3. Analyses of Single Tasks 354(5)
4. Analyses of Task Combinations 359(5)
5. Studies with Users 364(7)
6. The Central Issues Revisited 371(4)
References 375(4)
20 Speech Convergence with Animated Personas 379(20)
Sharon Oviatt, Courtney Darves, Rachel Coulston, Matt Wesson
1. Introduction to Conversational Interfaces 379(3)
2. Research Goals 382(1)
3. Method 383(4)
4. Results 387(4)
5. Discussion 391(2)
6. Conclusion 393(1)
References 394(5)
Index 399
Preface xi
Contributing Authors xiii
Introduction xxi
Part I Issues in Multimodal Spoken Dialogue Systems and Components
1 Multimodal Dialogue Systems 3(10)
Alexander I. Rudnicky
1. Introduction 3(1)
2. Varieties of Multimodal Dialogue 4(2)
3. Detecting Intentional User Inputs 6(1)
4. Modes and Modalities 7(1)
5. History and Context 7(1)
6. Domain Reasoning 8(1)
7. Output Planning 9(1)
8. Dialogue Management 9(1)
9. Conclusion 10(1)
References 11(2)
2 Speech Recognition Technology in Multimodal/Ubiquitous Computing Environments 13(24)
Sadaoki Furui
1. Ubiquitous/Wearable Computing Environment 13(1)
2. State-of-the-Art Speech Recognition Technology 14(2)
3. Ubiquitous Speech Recognition 16(2)
4. Robust Speech Recognition 18(3)
5. Conversational Systems for Information Access 21(3)
6. Systems for Transcribing, Understanding and Summarising ubiquitous Speech Documents 24(8)
7. Conclusion 32(1)
References 33(4)
3 A Robust Multimodal Speech Recognition Method using Optical Flow Analysis 37(18)
Satoshi Tamura, Koji Iwano, Sadaoki Furui
1. Introduction 38(1)
2. Optical Flow Analysis 39(1)
3. A Multimodal Speech Recognition System 40(3)
4. Experiments for Noise-Added Data 43(5)
5. Experiments for Real-World Data 48(1)
6. Conclusion and Future Work 49(3)
References 52(3)
4 Feature Functions for Tree-Based Dialogue Course Management 55(22)
Klaus Macherey, Hermann Ney
1. Introduction 55(1)
2. Basic Dialogue Framework 56(3)
3. Feature Functions 59(4)
4. Computing Dialogue Costs 63(1)
5. Selection of Dialogue State/Action Pairs 64(1)
6. XML-based Data Structures 65(3)
7. Usability in Mobile Environments 68(1)
8. Results 69(5)
9. Summary and Outlook 74(1)
References 74(3)
5 A Reasoning Component for Information-Seeking and Planning Dialogues 77(16)
Dirk B眉hler, Wolfgang Minker
1. Introduction 77(3)
2. State-of-the-Art in Problem Solving Dialogues 80(1)
3. Reasoning Architecture 81(4)
4. Application to Calendar Planning 85(3)
5. Conclusion 88(2)
References 90(3)
6 A Model for Multimodal Dialogue System Output Applied to an Animated Talking Head 93(24)
Jonas Beskow, Jens Edlund, Magnus Nordstrand
1. Introduction 93(4)
2. Specification 97(6)
3. Interpretation 103(2)
4. Realisation in an Animated Talking Head 105(4)
5. Discussion and Future Work 109(2)
References 111(6)
Part II System Architecture and Example Implementations
7 Overview of System Architecture 117(16)
Andreas Kellner
1. Introduction 117(1)
2. Towards Personal Multimodal Conversational User Interface 118(4)
3. System Architectures for Multimodal Dialogue Systems 122(4)
4. Standardisation of Application Representation 126(3)
5. Conclusion 129(1)
References 130(3)
8 XISL: A Modality-Independent MMI Description Language 133(16)
Kouichi Katsurada, Hirobumi Yamada, Yusaku Nakamura, Satoshi Kobayashi, Tsuneo Nitta
1. Introduction 133(1)
2. XISL Execution System 134(2)
3. Extensible Interaction Scenario Language 136(4)
4. Three Types of Front-Ends and XISL Descriptions 140(6)
5. XISL and Other Languages 146(1)
6. Discussion 147(1)
References 148(1)
9 A Path to Multimodal Data Services for Telecommunications 149(20)
Georg Niklfeld, Michael Pucher, Robert Finan, Wolfgang Eckhart
1. Introduction 149(1)
2. Application Considerations, Technologies and Mobile Terminals 150(4)
3. Projects and Commercial Developments 154(2)
4. Three Multimodal Demonstrators 156(5)
5. Roadmap for Successful Versatile Interfaces in Telecommunications 161(2)
6. Conclusion 163(1)
References 164(5)
10 Multimodal Spoken Dialogue with Wireless Devices 169(16)
Roberto Pieraccini, Bob Carpenter, Eric Woudenberg, Sasha Caskey, Stephen Springer, Jonathan Bloom, Michael Phillips
1. Introduction 169(2)
2. Why Multimodal Wireless? 171(1)
3. Walking Direction Application 172(1)
4. Speech Technology for Multimodal Wireless 173(1)
5. User Interface Issues 174(5)
6. Multimodal Architecture Issues 179(3)
7. Conclusion 182(2)
References 184(1)
11 The SmartKom Mobile Car Prototype System for Flexible Human-Machine Communication 185(18)
Dirk B眉hler, Wolfgang Minker
1. Introduction 185(1)
2. Related Work 186(3)
3. SmartKom - Intuitive Human-Machine Interaction 189(2)
4. Scenarios for Mobile Use 191(2)
5. Demonstrator Architecture 193(1)
6. Dialogue Design 194(3)
7. Outlook - Towards Flexible Modality Control 197(2)
8. Conclusion 199(1)
References 200(3)
12 LARRI: A Language-Based Maintenance and Repair Assistant 203(18)
Dan Bohus, Alexander I. Rudnicky
1. Introduction 203(1)
2. LARRI - System Description 204(4)
3. LARRI - Hardware and Software Architecture 208(5)
4. Experiments and Results 213(2)
5. Conclusion 215(2)
References 217(4)
Part III Evaluation and Usability
13 Overview of Evaluation and Usability 221(26)
Laila Dybkj忙r, Niels Ole Bernsen, Wolfgang Minker
1. Introduction 221(2)
2. State-of-the-Art 223(4)
3. Empirical Generalisations 227(7)
4. Frameworks 234(2)
5. Multimodal SDSs Usability, Generalisations and Theory 236(2)
6. Discussion and Outlook 238(3)
References 241(6)
14 Evaluating Dialogue Strategies in Multimodal Dialogue Systems 247(22)
Steve Whittaker, Marilyn Walker
1. Introduction 247(4)
2. Wizard-of-Oz Experiment 251(11)
3. Overhearer Experiment 262(4)
4. Discussion 266(1)
References 267(2)
15 Enhancing the Usability of Multimodal Virtual Co-drivers 269(18)
Niels Ole Bernsen, Laila Dybkj忙r
1. Introduction 269(2)
2. The VICO System 271(1)
3. VICO Haptics - How and When to Make VICO Listen? 272(2)
4. VICO Graphics - When might the Driver Look? 274(4)
5. Who is Driving this Time? 278(2)
6. Modelling the Driver 280(4)
7. Conclusion and Future Work 284(1)
References 285(2)
16 Design, Implementation and Evaluation of the SENECA Spoken Language Dialogue System 287(24)
Wolfgang Minker, Udo Haiber, Paul Heisterkamp, Sven Scheible
1. Introduction 288(2)
2. The SENECA SLDS 290(11)
3. Evaluation of the SENECA SLDS Demonstrator 301(7)
4. Conclusion 308(1)
References 309(2)
17 Segmenting Route Descriptions for Mobile Devices 311(18)
Sabine Geldof, Robert Dale
1. Introduction 311(4)
2. Structured Information Delivery 315(1)
3. Techniques 315(7)
4. Evaluation 322(4)
5. Conclusion 326(1)
References 327(2)
18 Effects of Prolonged Use on the Usability of a Multimodal Form-Filling Interface 329(20)
Janienke Sturm, Bert Cranen, Jacques Terken, Isle Bakx
1. Introduction 329(3)
2. The Matis System 332(3)
3. Methods 335(2)
4. Results and Discussion 337(8)
5. Conclusion 345(1)
References 346(3)
19 User Multitasking with Mobile Multimodal Systems 349(30)
Anthony Jameson, Kerstin Kl枚ckner
1. The Challenge of Multitasking 350(4)
2. Example System 354(1)
3. Analyses of Single Tasks 354(5)
4. Analyses of Task Combinations 359(5)
5. Studies with Users 364(7)
6. The Central Issues Revisited 371(4)
References 375(4)
20 Speech Convergence with Animated Personas 379(20)
Sharon Oviatt, Courtney Darves, Rachel Coulston, Matt Wesson
1. Introduction to Conversational Interfaces 379(3)
2. Research Goals 382(1)
3. Method 383(4)
4. Results 387(4)
5. Discussion 391(2)
6. Conclusion 393(1)
References 394(5)
Index 399
Spoken multimodal human-computer dialogue in mobile environments /
- 名称
- 类型
- 大小
光盘服务联系方式: 020-38250260 客服QQ：4006604884
云图客服：
用户发送的提问，这种方式就需要有位在线客服来回答用户的问题，这种 就属于对话式的,问题是这种提问是否需要用户登录才能提问
Video Player
×
Audio Player
×
pdf Player
×
