I am using Ionic Native speech recognition as outlined in the docs: https://ionicframework.com/docs/native/speech-recognition:
startListening() {
this.speechRecognition.startListening()
.subscribe(
(matches: Array<string>) => {
console.log(matches)
},
(onerror) => {
console.log('error:', onerror)
}
)
}
However, this only listens for a certain amount of time. If nothing has been said, or the speech was unable to hear it, I receive the message "Tap to speak. Try again". However, for this the user needs to press a button to be able to speak again. Is there therefore a way to catch when this happens such that I can trigger this.startListening() again?
I find in the doc, the speech recognition only 5 times and after that it stop.
https://github.com/pbakondy/cordova-plugin-speechrecognition.
You can start listening with options :
let options = {
String language,
Number matches, // increase this number if don't want to stop listening
String prompt, // Android only
Boolean showPopup, // Android only
Boolean showPartial
}
this.speechRecognition.startListening(options)
.subscribe(
(matches: Array<string>) => console.log(matches),
(onerror) => console.log('error:', onerror)
)
Related
I'm building a PWA using Web Bluetooth API. Connection is OK, I set an eventlistener for getting data and when my Arduino with BLE module HM10 send data, I get them.
But my PWA has more than one page. So on first one I have a "Connection" button, and after connection I "listen".
When I go to a second page, I use navigator.bluetooth.getDevices() to get previsouly connected device.
But I get an empty list.
I though my getDevices() function was bugged but I discoverd a strange thing:
my PWA run a browser tab on Chrome. I connect to the device. Then, on another tab I open this url with a Google example (https://googlechrome.github.io/samples/web-bluetooth/get-devices.html) and call Get Bluetooth Devices to get list of connected device, I see my device in this example page.
Without closing this second tab, I use my PWA: all page are able to get the device (using navigator.bluetooth.getDevices()), set the even and get data from the Arduino. Great!
If I close this tab, connection is lost...
So I think my "connection" is not perfect and when I close my PWA first page, I loose it... But as example avaible on internet are about "one page", I'm a bit lost (like the device connection in fact...).
Here is the code I use:
function reconnect_ble()
{
console.log('Search previously connected devices...');
navigator.bluetooth.getDevices()
.then(devices => {
console.log('> Got ' + devices.length + ' Bluetooth devices.');
if (devices.length == 0)
{
alert("No module");
return false;
}
var flag_module = false;
for (const device of devices)
{
var nom = device.name;
// Check if it's our modle
if (nom == pwa_hm10)
{
flag_module = connectToBluetoothDevice(device);
}
}
if (flag_module == false)
{
alert("Nothing for us");
return false;
}
else
{
return true;
}
})
.catch(error => {
console.log('Erreur:' + error);
});
}
//===================================================================================
// First Connection
function connect_ble()
{
return (deviceCache ? Promise.resolve(deviceCache) :
requestBluetoothDevice()).
then(device => connectToBluetoothDevice(device)).
catch(error => display_info(error));
}
//-------------------------------------------------------------------------------------
function connectToBluetoothDevice(device)
{
const abortController = new AbortController();
// Evenement sur ce device
device.addEventListener('advertisementreceived', (event) =>
{
console.log('connectToBluetoothDevice- Advertisements from "' + device.name + '"...');
// Stop watching advertisements to conserve battery life.
abortController.abort();
// Connexion au serveur GATT
device.gatt.connect()
.then(() =>
{
console.log('connectToBluetoothDevice - Server GATT from "' + device.name + '"...');
// Set our event
connectDeviceAndCacheCharacteristic(device).
then(characteristic => startNotifications(characteristic)).
catch(error => display_info(error));
})
.catch(error =>
{
// Erreur pas de connexion
console.log(error);
});
}, { once: true });
console.log('connectToBluetoothDevice - Watching advertisements from "' + device.name + '"...');
device.watchAdvertisements({ signal: abortController.signal })
.catch(error =>
{
console.log(error);
});
}
//----------------------------------------------------------------------------------
// Form to choose device
function requestBluetoothDevice()
{
return navigator.bluetooth.requestDevice(
{
acceptAllDevices: true,
optionalServices: [0xFFE0]
}).
then(device => {
display_info('Selected: ' + device.name);
deviceCache = device;
deviceCache.addEventListener('gattserverdisconnected',handleDisconnection);
return deviceCache;
});
}
//-----------------------------------------------------------------------------------
function connectDeviceAndCacheCharacteristic(device)
{
if (device.gatt.connected && characteristicCache)
{
return Promise.resolve(characteristicCache);
}
return device.gatt.connect().
then(server => {
return server.getPrimaryService(0xFFE0);
}).
then(service => {
return service.getCharacteristic(0xFFE1);
}).
then(characteristic => {
display_info('characteristic founded');
characteristicCache = characteristic;
return characteristicCache;
});
}
//------------------------------------------------------------------------------------
function startNotifications(characteristic)
{
return characteristic.startNotifications().
then(() => {
characteristic.addEventListener('characteristicvaluechanged',handleCharacteristicValueChanged);
});
}
//--------------------------------------------------------------------------------------
function handleCharacteristicValueChanged(event)
{
// Decoding received data
var decodage = new TextDecoder().decode(event.target.value);
traitement_data_ble(decodage);
}
Edit
If you connect to the bluetooth on Page 1, and go to Page 2 on same tab, connection is lost.
If you connect to the bluetooth on Page 1, set an event listenner to get data then open Page 2 on another tab, Page 2 will "see" the bluetooth module, but even if Page 2 set an event listener, this is even from Page 1 that will receive the data.
If you connect to the bluetooth on Page 1, don't set event listenner, open Page 2 in other tab and set event listenner on Page 2, Page 2 will receive the data.
So case 3 seems to be the only way. Maybe using iFrame for the connection page??
Accessing the same device from multiple pages is not supported and the fact that it seems to partially work is more of a bug than a feature. A device can't effectively distinguish between multiple pages on the same host trying to communicate with it so you need to have a single piece of code which manages the connection to the device.
The correct way to do this today is to create a single-page application so that the connection held by that page can remain own even if the user navigates to different "pages" of your application. If you need to create separate windows these can communicate with the device by sending commands back through the single page that owns the connection using postMessage or Broadcast Channel.
The ideal solution, which is not currently supported, is to use a Shared Worker for this. Shared Workers are a type of web worker which is owned by all the currently open pages of your app at once. It can thus hold shared state (such as the Bluetooth connection) in behalf of your application and won't exit unless every window of your app is closed.
Task at hand is to add support for fullscreen mode to an WebGL application written in Dart.
canvas.requestFullscreen() works for simple test cases, but fails on the full app.
Please point out the way to tell what is preventing the browser from switching to fullscreen.
The code is:
void trapFullscreenError() {
document.onFullscreenError.listen((e) {
log("fullscreenerror: $e");
});
}
void toggleFullscreen(CanvasElement c) {
log(
"fullscreenSupport=${document.fullscreenEnabled} fullscreenElement=${document.fullscreenElement}"
);
if (document.fullscreenElement != null) {
log("exiting fullscreen");
document.exitFullscreen();
} else {
log("requesting fullscreen");
c.requestFullscreen();
}
}
In Chrome that code results in:
fullscreenSupport=true fullscreenElement=null
requesting fullscreen
fullscreenerror: Instance of 'Event'
Dartium debugger shows these fields:
Event [id=4]
_selector null
bubbles true
cancelable false
clipboardData null
currentTarget #document [id=5]
defaultPrevented false
eventPhase 3
hashCode 234642739
path NodeList[6] [id=6]
target canvas#main_canvas [id=7]
timeStamp 1398779450832
type "webkitfullscreenerror"
For security reasons requestFullscreen can only be called in an event handler of a keyboard or click event.
see also Javascript request fullscreen is unreliable
I have a one-scene Action that calls my webhook 'randomSpeech' (mentioned below) upon invocation, which plays an MP3. I added a "skip" intent to skip to the next MP3. When I say "skip", the Action should transition (loop) back into the webhook 'randomSpeech', and since there is a counter, x, the Action should begin playing the 2nd MP3 in the switch statement.
However, I have to say the word "skip" twice in order for it to work.
The 1st time I say "skip", the system intent, MEDIA_STATUS_FINISHED automatically calls the 'mediaStatus' handler and the text 'Media has finished.' is added to the conversation. Even though I've configured the "skip" intent to call the handler, 'randomSpeech', it doesn't appear to happen as no new Media is added to the conversation. It's almost like 'randomSpeech', is completely ignored!
The 2nd time I say "skip", the second MP3 finally begins playing.
My main question is, how can I make it so the user only has to say "skip" one time?
let x = 1;
app.handle('randomSpeech', (conv) => {
switch(x) {
case(1):
conv.add(new Media({
mediaObjects: [
{
name: 'NEVER GIVE UP',
description: 'some athlete',
url: 'http://zetapad.com/speeches/nevergiveup.mp3',
image: {
large: {
url: 'https://www.keepinspiring.me/wp-content/uploads/2020/02/motivation-gets-you-started-jim-ryun-quote-min.jpg'
}
}
}
],
mediaType: 'AUDIO',
optionalMediaControls: ['PAUSED', 'STOPPED'],
startOffset: '5s'
}));
x++;
break;
case(2):
conv.add(new Media({
mediaObjects: [
{
name: 'SPEECHLESS',
description: 'Denzel Washington (feat Will Smith)',
url: 'http://zetapad.com/speeches/denzel.mp3',
image: {
large: {
url: 'https://www.keepinspiring.me/wp-content/uploads/2020/02/motivational-quotes-2-min.jpg'
}
}
}
],
mediaType: 'AUDIO',
optionalMediaControls: ['PAUSED', 'STOPPED']
}));
break;
}
});
app.handle('media_status', (conv) => {
const mediaStatus = conv.intent.params.MEDIA_STATUS.resolved;
switch(mediaStatus) {
case 'FINISHED':
conv.add('Media has finished.');
break;
case 'FAILED':
conv.add('Media has failed.');
break;
case 'PAUSED' || 'STOPPED':
if (conv.request.context) {
// Persist the media progress value
const progress = conv.request.context.media.progress;
}
conv.add(new Media({
mediaType: 'MEDIA_STATUS_ACK'
}));
break;
default:
conv.add('Unknown media status received.');
}
});
Images from the only scene, "Motivation":
Scene
On enter
Intent handling
Further notes:
MEDIA_STATUS_PAUSED / MEDIA_STATUS_FINISHED / MEDIA_STATUS_STOPPED all only call the 'media_status' wehbook
The issue at the heart of your question is that "skip" is a built-in Media Player command (although this is not clearly documented), so when the user says "skip", the player treats this as the audio being completed, so it sends the MEDIA_STATUS_FINISHED Intent, just as it the user listened to it all the way through.
The good news is - you actually want to handle both these cases the same way! So if the user skips to the next audio, or finishes the first and it should advance to the next audio - you want to play the next audio.
In your code, "playing the next audio" is all done as part of your switch statement. So you should probably put that into a regular JavaScript function by itself. You can then call that function from the different handlers that you have setup.
It might look something like this (without some of the code details):
function nextAudio( conv ){
// Code goes here to figure out the next audio to play and send it back
}
app.handle('randomSpeech', (conv) => {
nextAudio( conv );
}
app.handle('media_status', (conv) => {
const mediaStatus = conv.intent.params.MEDIA_STATUS.resolved;
switch(mediaStatus) {
case 'FINISHED':
nextAudio( conv );
break;
// Other media status cases can go here
}
});
I am using phonegap-barcodescanner-plugin to read qr while a service is reading instant payments on background.
I am detecting a new instant payment in the service and launching an event subscribed in the page where the barcodeScanner is launched
event.subscribe('instant-payment', (val) => {
console.log("Hi--------------", val)
this.navCtrl.pop()
});
the event is correctly fired and the log is ok but I'm trying to do a navCtrl.pop() and the Activity is never closed, I thought was going to work same way like cancelling scanner
scan() {
this.barcodeScanner.scan(this.options).then(barcodeData => {
if (barcodeData.cancelled == true) {
this.navCtrl.pop()
} else {
}
}).catch(err => {
console.log('Error', err);
});
}
Is there anyway to force close barcodeScanner and go back to the last "IonicPage".
Thanks for any help.
I'm working with cordova's BLE (bluetooth low energy)
After I subscribe to notifications of BLE (which returns Observable), I want to send some message to the ble device, what is the best way to perform this, basically I need to run a function once after the subscription is made so that once device responds back to me, the code in the subscription is run.
ble.startNotification(deviceId, uuid1, uuid2).subscribe(bufferData=> {
//do something with bufferData
})
now after this, I want to run something like a callback,
.then(()=> {
//send message to device (only once), after the message is sent, the device will respond back and the `do something with bufferData` code will be run
})
I could easily do a setTimeout and send a message to the device after few seconds, and of course it works, but I want to do it cleanly, after I'm sure the subscription happened (subscription to the Observable of course)
You can wrap existing method using create operator and add custom code that will be executed on every new subscription.
See the example:
// emulate cordova with "dummy" bluetooth interface
const BLE = {
startNotification: () => Rx.Observable.interval(1000)
}
const wrappedBLE = (...params) =>
Rx.Observable.create(observer => {
// constructor fn will be executed on every new subscribtion
const disposable = BLE.startNotification(...params).subscribe(observer);
// place code to send notification here, instead of console log
console.log('New subscriber for BLE with params: ', params);
return disposable;
});
wrappedBLE("param1", "param2", "param3")
.subscribe(e => console.log("received notification: ", e));
<script src="https://unpkg.com/rxjs#5.4.3/bundles/Rx.min.js"></script>