Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog
J. Thomason et al., “Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog,” in Late-breaking Track at the SIGDIAL Special Session on Physically Situated Dialogue (RoboDIAL-18), 2018.
Abstract
In this work, we present methods for parsing natural language to underlying meanings, and using robotic sensors to create multi-modal models of perceptual concepts. We combine these steps towards language understanding into a holistic agent for jointly improving parsing and perception on a robotic platform through human-robot dialog. We train and evaluate this agent on Amazon Mechanical Turk, then demonstrate it on a robotic platform initialized from that conversational data. Our experiments show that improving both parsing and perception components from conversations improves communication quality and human ratings of the agent.
BibTeX Entry
@inproceedings{thomason2018robodial, title = {Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog}, author = {Thomason, Jesse and Padmakumar, Aishwarya and Sinapov, Jivko and Walker, Nick and Jiang, Yuqian and Yedidsion, Harel and Hart, Justin and Stone, Peter and Mooney, Raymond J.}, booktitle = {Late-breaking Track at the SIGDIAL Special Session on Physically Situated Dialogue (RoboDIAL-18)}, month = jul, year = {2018}, wwwtype = {lbr}, wwwhidden = {true}, wwwvideo = {https://www.superlectures.com/sigdial2018/jointly-improving-parsing-and-perception-for-natural-language-commands-through-human-robot-dialog}, wwwpdf = {https://jessethomason.com/publication_supplements/SIGDIAL18_late_breaking.pdf} }