I'm a developer trying to learn to interact apps with Google Assistant.
I noticed that as a developer we can use app actions (action.xml or shortcut.xml) to define how we want the google assistant to communicate with the app. Besides, there are conversational actions that can also do a similar job.
I wonder which one is preferred by Google and what are the differences in between. Are the apps developed by Google using conversational actions or app actions? and finally how can I tell if an app is using either of them or both?
Broadly speaking, app actions are a way to launch an Android app (possibly into a specific Android Intent and with specific information) from the Google Assistant running on Android, while conversational actions are a way to interact with a webhook-based app through the Google Assistant, typically over multiple turns.
While the two are similar, in that they both work through the Assistant, they are rather different.
App Actions
Only work on Android devices, not everywhere that the Assistant runs.
Are generally used to launch an already installed app, possibly providing specific information to a deep link in that app.
Can also be used (in some cases, with features that are coming soon) to provide widgets from the app into the Assistant.
Once the app is launched, you are (usually) no longer interacting with the Assistant - you're in the app, and have the UI from that app, which is generally not voice driven
Conversational Actions
Work across all platforms where the Assistant runs - from Smart Speakers to Android devices
Do not require an app to be "installed" - you can invoke it by name just like you open a web page
Primarily uses voice interaction for all of the work - there does not need to be a visual component.
Code runs "in the cloud", not on the device, which acts more like a web browser.
Google doesn't "prefer" either, and they develop both types. (For example, anything that works with a smart speaker is a conversational action, while apps like Google Maps include app action support). It depends on your use case and what you already have available:
If you have an existing Android app, then app actions may be a reasonable approach.
If you are starting from scratch, then you may want to look at conversational actions.
Related
I am currently developing an updated version of a voice app that it was previously developed with Google Actions console as a conversational app. However, when started to create a new project for this updated app, a banner says that Conversational apps will be sunsetted on June 13th.
Reading through the documentation, it is not clear to me how to develop now that this options will not be available in the future. Among the options, App Actions and Dialogflow CX might be the route for development. However, my app to be developed reqquires integration with smart devices such as Google nest Hub and Google nest mini for interaction.
It seems that App Actions might not be the solution because it adds voice capabilities to an existing Android App, which I am not sure that will work with Smart devices directly.
On the other hand, dialogflow CX looks to be focused on chatbox, i.e. based on text. Again, I am not sure if using Dialogflow CX will be able to provide an appp that uses voice interactions in smart devices.
In addition to these options, I also read about Dialogflow ES, Cloud-to-Cloud for smart for Google Home, Content Actions, and Media Actions. It does not look like either of these solutions might be a replacement to Google Actions. For example, I think smart home is not the option because I am not looking to interact with home devices besides google nest hub.
I hope anybody can help and help me understand development path or lead me to information that I might be missing. Thank you
It sounds like you are looking to create a new version of a conversational app that was previously developed using the Google Actions console, but are now facing the sunset of the conversational apps feature on June 13th.
One option that you may want to consider is Dialogflow CX, which is a platform for building and managing natural language conversational experiences. Dialogflow CX is built on top of Dialogflow ES, and is designed for more complex and large-scale conversational apps.
It allows you to create a conversation flow using a visual editor, and supports integration with various platforms including Google Assistant, Alexa, and WhatsApp.
Regarding the integration with smart devices like Google Nest Hub and Google Nest Mini, you can use the Google Assistant Actions API to build custom actions that can be invoked by users through the Google Assistant on these devices. This API allows you to define a conversation flow and handle user inputs, and it can be integrated with Dialogflow CX to handle natural language understanding and generation.
App Actions, on the other hand, are a way to surface your app's functionality through the Google Assistant on Android devices, and it's not a replacement for Google Actions.
Cloud-to-Cloud for smart home, Content Actions, and Media Actions are more specific solutions for different use cases, for example, providing actions for smart home devices, providing actions for media content and etc.
In summary, Dialogflow CX along with the Google Assistant Actions API seems to be the most relevant solution for your use case. It allows you to build natural language conversational experiences that can be integrated with smart devices like Google Nest Hub and Google Nest Mini.
It's worth checking the official documentation and tutorials for more information about Dialogflow CX, the Google Assistant Actions API, and how to integrate them. Also, you may want to reach out to Google support team for more specific guidance for your case.
Google recently announced that Google Conversational action are getting deprecated, and will be removed completely by 13th June 2023.
What are the alternatives to the conversational action available now?
I understand that Google recommends app action over conversational action. As I understand, App action are used to interact with android action, this does not fulfil my requirements.
I want to control my smart home device using voice commands. The required device type is treadmill and required trait is speedControl. I have checked the Google Smart Home action documentation, I could not find any device type and trait that would satisfy my requirements, hence using smart home action is not the option.
What are the alternative options to support the device control for device types and traits not supported by smart home action ?
I would recommend filing a ticket request for a new device type/trait to the Smart Home support team. That will help inform them what device types/traits to prioritize as they continue to build out device compatiblities for the new Matter protocol. Ideally one of these paths would be implemented for you by the time conversational actions are deprecated next June.
Alternatively, you could use custom intents to jump to your treadmill controller mobile app, which can then route the controls to your device. This fulfillment path would not work directly through the Google Home app on Nest devices, but would still enable you to have voice control for your device.
The official documentation describes two kind of applications that can be implemented for Android Automotive, Media applications and Messaging applications. Also when creating a new project in Android Studio, the IDE prompts to select a template from those options.
My question is specific to third-party apps. Is Android Automotive limited to build one of these flavors? That is, interact with the user only through the native media UI of the vehicle or the messaging/notifications mechanism. Or as a third-party application developer I can build any other kind of applications (like generic phone/tablet android apps) with custom activities/screens? In case that any kind of applications are allowed, are they limited to run when the vehicle is parked?
Android Automotive is Android so you could build whatever android app you like and run it on Automotive. The catch is that Google is, as per now, only allowing Media/Messaging apps into Play Store and has quite specific design guidelines that developers need to follow.
When your app follows Googles Driver Distraction Guidelines then it can run when a vehicle is moving.
I am developing Google Actions for Google Assistant. Is there an emulator of Google Assistant for Linux available? I believe this would be easier during development, instead of having to use my phone all the time.
I know there is https://console.actions.google.com, but I am not sure if this has all capabilities built in. Should Google Actions (such as cards, suggestion clips, carousel, ...) also work here?
You should be able to do almost all of your development using the actions simulator: https://developers.google.com/actions/tools/simulator
There are a few exceptions dealing with cross device scenarios and intent invocation for Android.
The simulator is better than developing on a phone since it provides detailed logging and request/response JSON payloads.
I am new to google home. I had an experience in Amazon Alexa custom skill development. In Alexa, I had deployed my codebase in lambda function and also test my custom skill using actual Alexa device register with my email id.
So, Now I need to develop similar skill in google home device. Till now, I didn't get any good tutorials.
Is it possible to create & test Google Home app like Alexa skill?
The steps you go through to develop a Google Home action or app are very similar to creating an Alexa skill. There are a couple of differences, but logically they are the same.
If you use a NLP system such as Dialogflow (which is strongly suggested), you build the suggested phrases that the system responds to and the Intents they correspond to. You would specify your webhook as part of building these phrases. If you don't wish to use an NLP, you can specify the initial Intent phrases using the Actions SDK and specifying the configuration as part of a .json file. Other actions you'd do in the Alexa console are similar to what you'd do in the Assistant console.
You can deploy your Action on any public server that accepts HTTPS connections. This can include AWS Lambda with an AWS API Gateway trigger, or a Firebase Function, or a web server you more directly control that has a valid SSL certificate. This webhook would get a JSON body and needs to send back a valid JSON response. Google has libraries for node.js to help with this.
Google has a relatively full-featured simulator which you can use to test your Action. Once it is available in the simulator, it is also immediately available on every device attached to that account. You can permit other accounts to the project as well and, once they have activated it through the simulator, it is available on all their devices as well.
A full set of documentation is available at https://developers.google.com/actions/. It includes links to sample code, and you can find more step-by-step codelabs at https://codelabs.developers.google.com/?cat=Assistant
If you're familiar with how to develop skills for Alexa you might want to check out the jovo-framework. It makes it pretty easy to create skills that work for both Amazon Alexa and Google Home.
Here is a good starter template and walk-through that will get you going. https://github.com/rmtuckerphx/ask-cli-jovo-starter