Blog

Making robots that learn is still a hard problem, so it’s no surprise that most robot learning research is focused on algorithmic challenges. As we’re considering how to build robot learners though, we must keep in mind what people will actually think about them. After all, if a learning robot makes its owners think that it’s broken, we may have to redesign the learner entirely.

In our work for HRI, we present a large scale online video study that probes people’s thoughts on a robot that decides to learn on its own. The paper gives a full breakdown of the results, and I’ll be presenting it at HRI in Cambridge next month. As one of the first studies tackling perceptions of intrinsically motivated robots, we think it’ll be of interest both to people working on robot learning methods and to those looking to understand how intelligent robots are perceived by users.

I attended RoboCup for the second time. I’ve moved up to UW, but Maya (my advisor) and her postdoc, Maru, won the inaugural RoboCup Fellows award, so I tagged along to help evaluate whether we wanted to form a team for 2020. Treking over to Sydney just to watch seemed like a bit of waste, so I embedded with UT Austin Villa@Home for the competition.

Pre-competition

Australia is far away and the Sydney airport is awful. Villa landed together, so we rented a van and made the harrowing wrong-side drive to Darling Harbour.

For the past couple of years, the team has stayed together in an Airbnb. This works out to be a little cheaper, and definitely saves on logistical overhead. Stocking the apartment with a weeks worth of local groceries is both an absolute joy and a chance for the team to preview objects that are likely to appear in the competition. Having a largish apartment also means that it’s theoretically possible to set up a smallish robot like the HSR and get some work done before the setup days even begin.

Still, I’m mixed on the wisdom of keeping so many people under the same roof as they go through the RoboCup wringer. Also there are never enough restrooms.

As we were settling in, we learned that the team’s robot was caught in a customs snafu. It was in the country, but it wouldn’t be released without our paying a comically large tariff. Initially it seemed like it would be a matter of hours and some phone calls to sort out the misunderstanding. Then it seemed like we’d just have to get it during the setup days. The timezone difference between Australia and the US, where the logistics provider’s office was, didn’t help.

Naturally, I had strong opinions about the team’s poster, so I used the free time to make some tweaks and got it reprinted. I think it turned out quite nicely. We rehearsed a bit and waited for the competition to kick off.

Competition

The morning of the first setup day arrived, and we still didn’t have our robot. Fortunately, Toyota generously loaned us their backup HSR. A considerable amount of effort went into setting up the robot to behave like Austin. Helping to set up the robot was probably the most useful I would be during the competition. I’d handled much of the robo-sysadmin while at UT.

I wasn’t able to contribute much to on-site tuning. I had provided some input from afar during the team’s preparation, mostly via pull request reviews and discussion with team members, but even having read the code and knowing the framework fairly well, it was obvious to me that I shouldn’t make code changes. I see this situation as a bit of a RoboCup anti-pattern. Odds are high that someone will need to tune code they haven’t written during the competition. The original author may not be on-site, and even if they are, they might be busy handling something else. There need to be built in affordances for making tuning easy, or team level procedures to ensure that there’s enough people familiar with any given part of the system.

Set up usually extends long into the night during the competition, but some (yes, multiple) battery fires in the soccer leagues caused the venue to clamp down hard and kick all participants out by 11PM. As a team with lots of preparation catchup to do, this hurt us, but I did enjoy actually getting to go get dinner and sleep. Our apartment was in Ultimo just a couple minutes walk from the venue and from China town.

By the middle of the second set up day it was clear that we’d have no choice but to compete with the Toyota loaner. We passed inspection in the evening, then we raced the clock to tune each task in the arena.

Maru shared some details and photos for each of the three competition days (Stage I, Stage II, Finals), so I won’t go into much detail beyond this point.

Ultimately, the only task points Villa would score were for taking out a garbage bag. Though the robot had some success in parts of other challenges, scoring was largely binary to encourage teams to build complete systems instead of cherry picking easy points. Most teams came away from most tasks with no points. The winners differentiated themselves by outstanding performances in a small number of tasks.

The competition is peppered with team leader meetings in which the league officials and participants gather for announcements and Q&A on the day’s activities. At the final meeting of the competition, with some prompting from Alex Moriarity, I volunteered to serve on the Technical Committee for RoboCup 2020. I’ve kept a close eye on rule developments since I started participating in @Home, and next year I’ll help with the drafting process.

Overall, the competition was smoothly-run. The WiFi actually worked, a relief after our terrible experience going without networking in Montreal. Our robot arrived just after we had been eliminated, giving us enough time to roll it over to Toyota for some repairs, then pack it back in for the return trip. Maya and Maru got the chance to see every part of the competition process, from the grueling set up process, to the team leader meetings and the Trustee meetings. We spent the rest of the competition watching other teams and enjoying the raucous child-sized soccer league games.

Symposium

The symposium was a mix of keynotes, contributed talks and poster sessions. The organizers picked great speakers, and it was interesting to get to hear from Australian roboticists (Gamini Dissanayake’s bridge robots!). Even coming up in Peter Stone’s group, I found it hard to piece together the story of the early days of RoboCup, so I greatly appreciated Manuela’s talk on RoboCup’s history. @Home is hardly new, but it gets only an oblique mention. In the future, it would be nice to hear one of the original @Home organizers discuss the genesis of the league.

I gave a talk on some work we’ve been doing on improving command understanding for the general-purpose service robot @Home task. I think GPSR has great potential as a benchmark domain for embodied language understanding, but the barrier to entry is still pretty steep; very few language researchers have access to a team that can mount a serious entry, and teams that do well in the task rarely focus much effort on making their command understanding robust. Hopefully teams can use our work to bootstrap better systems for next year.

Miscellany

Due to unavoidable physiological and scheduling constraints, the week of RoboCup was supposed to be my marathon training peak week. Not only did I not do the 20 mile long run that I was supposed to, I barely scrounged up 12 miles total for the 9 days I was in Sydney. The runs I did get in were before the set up days even began, but I’m so glad I did them. The Darling Harbor/ Pyrmont area is beautiful, and I wouldn’t have gotten to see it much otherwise. Running with Villa folks was also part of the fun. I look forward to organizing runs at conferences in the future to spread the joy. Maybe we’ll have the first RoboCup@Home social run in 2020.

We spent some time thinking about how to build some recent advances in neural natural language understanding into a system that could be useful for a general-purpose service robot. The approach strike a nice balance between flexibility, performance, and ease of implementation. I’ll present the paper at this year’s RoboCup Symposium in Australia.

Yuqian and I also put down some of thoughts on what the desirable features of a GPSR planning system are, a nice complement to our other work to appear at ICAPS.

Yuqian and I will have a paper at ICAPS on “Open World Planning for Service Robots,” the first conference publication to come from UT Austin’s RoboCup@Home team! We describe how we make plans for user commands that involve references to objects that the robot hasn’t seen recently.

Jesse’s work on improving a robot’s semantic parser through interaction was accepted at ICRA 2019. It’s the most complete description of how to build a continually improving robot language understanding system to date.

We put down some of our thoughts after more than a year of thinking about RoboCup@Home. LAAIR gives the outline of how we’ve put together a general purpose reasoning system and more conventional robotics components to address the challenge of building a general-purpose service robot. The second argues that there are synergies in domestic and office service robotics research efforts, and highlights what the @Home competition has to offer for researchers interested in service robots.