Under the Hood: Merging Google Assistant and Waze for Smarter Drives

Last updated: 10-12-2020

Read original article here

Under the Hood: Merging Google Assistant and Waze for Smarter Drives

What’s interesting about voice is the way it unlocks things that are hard to do by hand, and that’s obviously especially hard to do when you’re behind the wheel. In driving, the “hotword” (the word that activates a personal assistant) is critical for a truly hands-free experience.

Before the team and I started working on this feature, if you turned on voice commands in Waze (by saying “OK Waze”), you wouldn’t be able to use Siri or Google Assistant to complete tasks outside of Waze. We want to enable safe driving, but we also want to help you be productive behind the wheel. Now, with Google Assistant, you can do more while you drive — like make phone calls and report hazards — without taking your eyes off the road or your hands off the wheel.

The integration is really about how Waze and Assistant talk to each other. It’s very simple, actually. When you say, “Hey Google, report a crash,” all Assistant needs to know is if you have Waze open when you say this. If you do, it makes a highly educated guess you want to do this in Waze. It knows how to adjust its response and the action it takes based on this. So a big part of the integration is to enable this communication, to pass this information, so the Google Assistant will know this.

For the most part, we worked remotely between the Waze and Google teams, collaborating in different time zones like Tel Aviv, New York, and Mountain View. In the beginning, the final product didn’t seem so far away. But then it took almost a year to build the product, test it in Google, test it with Wazers, and then fully launch it.

We don’t get the chance to collaborate like this often, so it was pretty challenging to bring the Google system into Waze for the first time. I’ll never forget the first week our three teams met. We sat in a room together for one week and built prototypes to trigger Google Assistant from Waze. “Hey Google, report a crash” was the first action we checked.

For voice reporting to work, we had to “teach” Assistant to recognize and understand Waze’s different reporting capabilities, like crashes, cars stopped on shoulders, potholes, and much more. Assistant needs to know what your intent is when you ask to report something, and understand what kind of report it is. Once the intent is clear, Assistant can pass this information back to Waze to make the report and confirm the action by saying “OK, I reported a crash.” The ability to report things on the road is what makes Waze, Waze. And it’s really like magic to do it with Google Assistant.

Typically, Google Assistant takes up a lot of space on your phone screen. But when it’s working with Waze, it’s voice-forward without the visual elements, so it’s more friendly for driving. Since it automatically knows when you’re in the Waze app, you can say commands like “report a crash,” and Google Assistant will take care of it faster than you could do it by hand. Personally, I can’t go back to submitting reports by hand.

We started with the main actions that are important for Wazers, like “Hey Google, report a crash.” You can also ask to navigate home and it will navigate to the home address you have saved in Waze.

But the ones that really make a difference are things that would be difficult to do yourself while driving. The tolls feature, for example, is one you can easily toggle on and off by asking Google to avoid or allow tolls, which is helpful while driving but might be harder to find in the UI by hand. You can also ask to turn voice directions on or off, avoid highways, or see alternate routes. If you say “Hey Google, navigate to the closest ice cream shop,” it will navigate with Waze. It’s a pretty magical moment.

We tested the feature with a few Community members before anyone else and they provided the initial feedback that helped us make improvements. One of the biggest suggestions from our Community was to make it available in more countries and languages. That’s much easier with this feature because Google Assistant already knows how to enter queries in other languages, so we don’t have to do the work from scratch. After months of work, we got to do the exciting part: bringing the final product to Wazers in a few countries like the U.S. and soon, even more.


Read the rest of this article here