Can we get Google send something when the person stays silent in a conversation? A "no.response" intent before closing the microphone. I'm thinking of an "are you still there" question scenario or timed question/response games.
The point is not to close the session and give a chance to continue.
This would happen only once (or configurable times) so the microphone would not stay open.
When using the client library assistant.ask method, remember to specify values for the no-inputs parameters. You can use that to prompt the user to respond when the microphone doesn't hear anything.
Related
I'm working on Google Actions Console.
I want to have my google agent to verbally warn that time is up (instead of setting a timer, for instance).
I have now two main scenes:
user says "I am ready", the agent responds "OK. Ready, set, go!";
(user says nothing and) the agent says "please stop now".
I would like the prompt in 2 to proactively run 5 minutes after the end of the prompt in 1, without the user having to say anything.
Is it possible to create a timer/delay fo 5 min before the transition from 1 to 2 or to have the prompt in 2 delayed of 5 min during scene 2? How can I create this delay? Is there any workaround otherwise?
NB: I'm not a developer so be patient :D
This is difficult to do without code, but not impossible.
First - in general, Actions on Google is poorly suited for this. It is much better for conversational systems rather than timed events.
You have two options for how to do this:
As part of an Interactive Canvas game.
Using a Media response.
As part of an Interactive Canvas game
This scenario has you controlling the timer using JavaScript code that is part of an Interactive Canvas page that you have loaded on a Smart Display or Smart Phone device. As part of the "Ready Set Go" response, you send data back to indicate that your local code should start the timer.
You'll capture this data as part of the onUpdate() callback and in your callback function set the timer. This timer is done using JavaScripts setTimeout() function. In the function that setTimeout() triggers when it is done, you can call the sendTextQuery() function to continue the conversation.
Using a Media response
This will work on devices that can play long-form audio, but do not have a screen (so they can't use the Interactive Canvas).
In this scenario, when you send the "Ready Set Go" response, you also include a Media prompt which plays a 5-minute long audio.
When the audio finishes playing, it will send a MEDIA_STATUS_FINISHED System Intent which you can handle and then reply to continue the conversation.
Which should you use?
Well... maybe both. Media works better on Smart Speakers, while the Interactive Canvas works better on Smart Displays and Smart Phones (assuming your Action is a Game).
Media handling with media response and exit handling with exit intents conflict somehow with users expectations. Links to the docs supplied below. Expected/actual behaviour also described.
Is there any chance to get that handled (at least in the near future) by defining custom utterances for media handling? As far as I know there is no possibility to define custom utterances/intents for "play" / "pause" / "stop" / "start over".
I set up a AoG to play streams through the media response.
When I want to completely end the conversation with an exit intent from within media response it doesn't really stop the conversation, but just pauses the player with no voice output whereas on any visual device it shows a play button instead of the pause button. On second "stop" utterance (or whatever calls the exit intent) the action finally finishes and plays the desired exit audio/shows desired text making clear that the action really ended.
Whereas this is somehow the expected behaviour this is still pretty annoying according user expectations. When a user says "STOP" s/he probably wants to end the conversation and not pause a stream, or am I wrong here? User case studies in our company showed that at least.
Solution would be to be able to add custom voice output when stopping media playback.
This is a known bug in the Media response.
Ending a conversation with a MediaResponse will open up player features like pause, forward, backwards and raising / lowering the volume. This is however currently broken on a Google Home device as the audio will never play. We had it working for several months until it broke and we had to switch to keeping the conversation open in order to make the audio play.
We have noticed that there are several undocumented commands which does strange things... For instance, when playing an audio with the conversation open and saying "Lämna" which is the Swedish equivalent to "Leave", the conversation will end and the audio will keep playing. The command "Stopp" will stop the audio but keep the conversation open. "Avsluta" = "End" will stop the audio and close the conversation.
None of these actions will callback to our backend and things seems to change on a weekly basis.
I submitted a background audio app for certification and has failed with two reasons in which I could not figure out why.
Reason 1:
This app failed to correctly respond to at least one of the play,
pause, or play/pause events.
I understand that the MediaControl events for Play, Pause, Stop and PlayPause need to be catered, and have done so (and tested on both tablets and local devices that they are working) in the code. However, due to the reason that stopping a media stream and restarting it requires a longer-than-expected time, I used MediaElement.Pause() for both "Pause" and "Stop".
I read another post who had similar problem at the certification phase. Somebody recommended to use MediaElement.PlaybackRate = 0; instead. However, this is not ideal for long pauses as the stream will not move on.
What I wish to know is am I doing this the right way? For all my MediaControl events I have made sure that the MediaControl.IsPlaying property is correctly set as well.
Also, another reason it failed was this:
App failed the Perf test in the Windows ACK. See the following links
for more information: Test cases ran:
http://msdn.microsoft.com/en-us/library/windows/apps/hh920274.aspx
I have ran my app against the ACK and it all passed. The only thing I can think of is that the app does not enter suspend mode when the hardware (or on-screen) media control pause button is pressed. I have placed a debugger in the App_Suspending event but it never hits there.
As the description is too vague I am not sure if this is the problem. But if it's the case, can I know how do I force the app to enter suspended mode? I tried looking in the Window.Current class and Application.Current class, but to no avail.
Thanks!
For your first issue be sure that your media element is ready to play using :
while (CurrentTrack.CurrentState == MediaElementState.Opening || CurrentTrack.CurrentState == MediaElementState.Buffering)
{
await Task.Delay(100);
}
CurrentTrack.Play();
Also you have to stop your media element when the view is unload.
Regards.
After nearly 10 attempts in releasing the app, I finally got to the root of the problem, thanks to some guessing work by the folks at Microsoft too.
My app will automatically start the MediaElement streaming after the app is started. The background-capable audio will prevent the app from passing WACK because it will never enter suspended mode!
So, in order to get pass the store's WACK I had to remove the auto-starting feature, and now the app is in the store! (Phew).
I am trying to determine feasibility of certain features required in a (potential) project. I am not (yet) looking for a how-to, just a can-do. I apologize for any vagueness and ignorance: the former due to an NDA that makes Apple's NDA look like GPL, and the latter due to the fact that I have no iPhone or MAC experience.
I do have a solid understanding of objective-c and interface builder (going back to NextStep) and some PDA development experience, so I'll probably understand the answers, even if my questions are naive. I have done a bit of browsing, so I know some buzzwords.
I can't go into detail about the actual project, but I have come up with a lame analogy.
a large number of users in the same room are asked to complete task(s) in the app (say a puzzle)
they are under supervision, but the monitor cannot watch them all closely
they are not allowed to leave the application until done.
they cannot send/receive phone calls or messages during the task.
the monitor receives notification of various steps during the task
the monitor is notified when the task is completed, or the app is exited
the app sends a heartbeat, so the monitor also knows if signal is lost
Jailbreak is not an option.
The app should also work on iPod Touch.
So the things I need to do that seem dicey to me are
can I turn off (or require the user to turn off) phone and sms but still be able send http to my server
can I prevent an app from being switched out (even if, say an alarm app triggers or the phone rings).
failing prevention, can I at least detect any of these events and notify my server.
failing notification, can I record the event for the monitor to check later.
The user will be aware (and in fact welcome) these restrictions. It's a trust issue - the user must not seek help or use a helper app to solve the puzzle, and wants the monitor (and other contestants) to know that he did not. It's feasible for the user to click on an "OK, I understand and approve" screen at the beginning, but not for each communication to the server.
The app would only communicate with a central server (run by my company) - the monitor would not be able to buy the server software, and the url's of the server would not be user (or monitor) modifiable.
Hey! This looks like an app for taking exams. Not what I am doing, but that would be cool too!
EDIT --
I changed the title and am adding a few more parts to the question, based in part on mmc's answer. The App may run in an offline mode that would have to do the following:
So using the exam analogy, the user off line experience would be something like this
Launch App
App download test questions, registers start time, etc.
Turn off phone (if app can't do it by itself)
Disable any app that might interrupt my App (can app do this?)
Resume and Take Test
Indicate test done (or finish last problem).
Turn on phone (if app can't) and restart App (if needed).
App uploads test results and log of any interruptions.
So the question becomes
Am I sure that I at least get to log any interruption I can't prevent
can I know the cause of the interruption (phone answered, alarm launch, user initiated)?
can the user be prevented from modifying the log
can I know what other Apps are running when I start? (to guard against a daemon that occasionally displays a hint or something.
I would still like to run with real time uploads, so a few other ideas come to mind.
If I can reliably detect and record that the phone or another app was used, that might be almost as good as preventing it.
Can the user prevent the phone from ringing even if it's on (eg. call forward + ringer off)
Can my app know if the ringer is off
Same question for sms messaging
If I can't block it, can the user just ignore (silent) call or message and not leave my App. Would my app know that?
Sounds like you might be better off doing away with the phone, and making it exclusive to the iPod Touch.
You've nailed your trouble spots.
There is no way to disable phone functionality, and at the same time maintain network functionality of any type (3G, WiFi, or bluetooth) If you disable the phone operations with Airplane mode, all of them are disabled.
There is no way to prevent the Home button from returning you to Springboard
You can notify a server of a premature app interruption (there is an applicationWillTerminate: method on your app delegate) but is not reliable. If the operation takes too long, your app will be forcibly terminated.
You could write to the local file system that a premature interruption happened, and this would be far more reliable, as this operation would be much faster.
This question seems to be the essence of several others on this forum. I believe that it is possible for the active iPhone application to continue running, and specifically, to continue receiving timer call-backs, after it has entered the inactive state (either through the idle timer kicking in the screen lock, or through the user pressing the hardware lock button).
The documentation specifically says that while an application is inactive, it is executing, but not dispatching incoming events (I'm not giving a link because I'm jumpy about NDA - should I relax about that? Is this whole post a breach? sigh).
Also, two answers by user "Ambr Str" directly state that it is possible to continue receiving timer call-backs, and he supplies a snip of code to achieve it (I can't link to this because I'm a new user, sorry - search for the question: "What happens to an iPhone app when iPhone goes into stand-by mode?" to find his answer).
I've tried to create my call-backs as he suggests, but once my application becomes inactive, the call-backs cease firing.
I've just noticed that while the iPhone is plugged in, if the application becomes inactive (due to idle time out or me pressing the sleep button), call-backs do continue to occur - perhaps I should get my users to carry a battery pack with them ;-)
There is a good answer to this question on Apple's forums. Search for "Timer" and "Eskimo" (the helpful chap who provides the answers).
In brief, shortly after the application becomes inactive, the phone really does go to sleep. The only way to prevent this is to be playing some audio (or for some audio to be playing in a background application). While audio is playing the phone will not sleep, and your application continues to be run.
It is suggested that playing stay awake audio is a hack, and that you shouldn't do it if at all possible. I think in my application (which performs audio playback interspersed with silent periods), the approach is valid, if not ideal!
I have an app. which includes an embedded webserver. I'm planning to offer an option of disabling auto-sleep IFF the server is turned on, AND the device is on power. So you might consider checking the batteryState property of UIDevice. So if batteryState != UIDeviceBatteryStateUnplugged, go ahead and disable idle timer. (note: the docs say that UIDeviceBatteryStateUnknown will be returned when in simulator.)
You'll also want to listen for UIDeviceBatteryStateUnplugged notifications and set batteryMonitoringEnabled appropriately.