Problems with WebAudio - web-audio-api

I'm creating a research experiment that uses WebAudio API to record audio files spoken by the user.
I came up with a solution for this using recorder.js and everything was working fine... until I tried it yesterday.
I am now getting this error in Chrome:
"The AudioContext was not allowed to start. It must be resumed (or
created) after a user gesture on the page."
And it refers to this link: Web Audio API policy.
This appears to be a consequence of Chrome's new policy outlined at the link above.
So I attempted to solve the problem by using resume() like this:
var gumStream; //stream from getUserMedia()
var rec; //Recorder.js object
var input; //MediaStreamAudioSourceNode we'll be recording
// shim for AudioContext when it's not avb.
var AudioContext = window.AudioContext || window.webkitAudioContext;
var audioContext = new AudioContext; //new audio context to help us record
function startUserMedia() {
var constraints = { audio: true, video:false };
audioContext.resume().then(() => { // This is the new part
console.log('context resumed successfully');
});
navigator.mediaDevices.getUserMedia(constraints).then(function(stream) {
console.log("getUserMedia() success, stream created, initializing Recorder.js");
gumStream = stream;
input = audioContext.createMediaStreamSource(stream);
rec = new Recorder(input, {numChannels:1});
audio_recording_allowed = true;
}).catch(function(err) {
console.log("Error");
});
}
Now in the console I'm getting:
Error
context resumed successfully
And the stream is not initializing.
This happens in both Firefox and Chrome.
What do I need to do?

I just had this exact same problem! And technically, you helped me to find this answer. My error message wasn't as complete as yours for some reason and the link to those policy changes had the answer :)
Instead of resuming, it's best practise to create the audio context after the user interacted with the document (when I say best practise, if you have a look at padenot's first comment of 28 Sept 2018 on this thread, he mentions why in the first bullet point).
So instead of this:
var audioContext = new AudioContext; //new audio context to help us record
function startUserMedia() {
audioContext.resume().then(() => { // This is the new part
console.log('context resumed successfully');
});
}
Just set the audio context like this:
var audioContext;
function startUserMedia() {
if(!audioContext){
audioContext = new AudioContext;
}
}
This should work, as long as startUserMedia() is executed after some kind of user gesture.

Related

postMessage() event listener not working

I am working with a 3rd party who supplies a URL to be put into an iFrame to display some hosted video playback.
this is cross-domain
they use JWPlayer as their player of choice
I requested a way to 'know' when the video playback is complete. From reading, looks like the postMessage() callback is what many use.. and is what the 3rd vendor suggested, and mentioned they would implement.
I was given a TEST url that has this 'call back' function in it... and to see if I can could use it.
I can not seem to get any alert from the callback/listener functions?
As this is the first time I am implementing this, Im not sure if the error stems from my end or theirs?
I'm thinking it may be the path form the postMessage() function?
After firebugging the code.. I eventually fund their JS/callback set up here:
jwPI.on('complete', function(event){
playbackTime= playbackTime + (jwPI.getPosition() - positionA);
positionA=jwPI.getPosition();
parent.postMessage('EndVideo','*');
});
My side of things has the simple event listener added like so:
window.addEventListener("message", function(evt) {
//do whatever
alert("VIDEO CALLBACK FIRED");
});
My questions are:
1.) Why is this not working? a target/scope issue?
2.) Do I need to have the 3rd party vendor update the path in their postMessage() callback? where does '.parent' actually point to? (if this is an embedded iFrame?) and there are DIV's..etc..etc..etc housing the nested iFrame content?
my listener function is in the main parent file that loads this iFrame?
3.) Can I just leave it as 'as-is' and somehow change the path/target in my listener?
Solution posted:
here is a both a jQuery and JS solution
** note the jQuery approach need to use originalEvent in the scope
//jQuery approach
$(window).on("message onmessage", function(evt) {
//message
var targetData = evt.originalEvent.data;
//origin
var targetOrigin = evt.originalEvent.origin;
//check origin for security and to make Scott proud
if(targetOrigin !== 'https://example.com'){
//no same origin, exploit attempt in process possibly
}
//do whatever
});
//Javascript approach
window.addEventListener("message", function(evt) {
//message
var targetData = evt.data;
//source
var targetSource = evt.source; //iframe source message stems from - doesnt work
//origin
var targetOrigin = evt.origin;
if(targetOrigin !== 'https://example.com'){
//no same origin, exploit attempt in process possibly
}
//do whatever
});
(Posted an answer on behalf of the question author).
Here is a both a jQuery and JS solution. Note the jQuery approach need to use originalEvent in the scope.
//jQuery approach
$(window).on("message onmessage", function(evt) {
//message
var targetData = evt.originalEvent.data;
//origin
var targetOrigin = evt.originalEvent.origin;
//check origin for security and to make Scott proud
if(targetOrigin !== 'https://example.com'){
//no same origin, exploit attempt in process possibly
}
//do whatever
});
//Javascript approach
window.addEventListener("message", function(evt) {
//message
var targetData = evt.data;
//source
var targetSource = evt.source; //iframe source message stems from - doesnt work
//origin
var targetOrigin = evt.origin;
if(targetOrigin !== 'https://example.com'){
//no same origin, exploit attempt in process possibly
}
//do whatever
});

How to detect Google places AutoComplete load issues?

I'm using the API successfully but encountered an error this morning with "OOPS! Something went wrong" sitting in the textbox and the user cannot type into it. I found the issue to be key related and fixed, however, this brought to light that some issue may arise and the user cannot complete because of this blocking. I'd like to be able to detect in javascript if there is some issue with the google.maps.places.Autocomplete object and not bind it to the textbox.
For anyone else wanting to do this.
Thanks to the folks for the idea over at:
Capturing javascript console.log?
// error filter to capture the google error
(function () {
var oldError = console.error;
console.error = function (message) {
if (message.toLowerCase().includes("google maps api error")) {
document.getElementById('<%=hdnGoogleSelected.ClientID %>').value = "DISABLE";
triggerUpdatePanel();
//alert(message);
}
oldError.apply(console, arguments);
};
})();
Mine is in an update panel so I triggered the update which sets the onfocus back to this.select(); for the textbox which effectively disables the autocomplete attempts.
tbAddress1.Attributes["onfocus"] = "javascript:this.select();";
Another option:
Google will return an error after about 5 seconds from loading.
"gm-err-autocomplete" class indicates any error with the autocomplete component.
You can periodically check for the error class google returns. I do it for 10 seconds after loading:
function checkForGoogleApiErrors() {
var secCounter = 0;
var googleErrorCheckinterval = setInterval(function () {
if (document.getElementById("AddressAutocomplete").classList.contains("gm-err-autocomplete")) {
console.log("error detected");
clearInterval(googleErrorCheckinterval);
}
secCounter++;
if (secCounter === 10){
clearInterval(googleErrorCheckinterval);
}
}, 1000);
}

Xamarin.Forms Taking picture with Plugin.Media not working

I'm using the Plugin.Media from #JamesMontemagno version 2.4.0-beta (which fixes picture orientation), it's working on Adroind 4.1.2 (Jelly Bean) and Marshmallow, but NOT on my Galaxy S5 Neo with Android version 5.1.1.
Basically when I take a picture it never returns back on the page from where I started the process; always returns back to the initial home page.
On devices where it works, when I take a picture, I see that first of all the application fires OnSleep, then after taking the picture fires OnResume.
On my device where is NOT working it fires OnSleep and after taking the picture doesn't fire OnResume, it fires the initialization page and then OnStart.
For this reason it doesn't open the page where I was when taking the picture.
What should I do to make sure it fires OnResume returning to the correct page and not OnStart which returns on initial fome page ?
In addition, when I take a picture it takes almost 30 seconds to get back to the code after awaiting TakePhotoAsync process, and it's too slow!
Following my code:
MyTapGestureRecognizerEditPicture.Tapped += async (sender, e) =>
{
//Display action sheet
String MyActionResult = await DisplayActionSheet(AppLocalization.UserInterface.EditImage,
AppLocalization.UserInterface.Cancel,
AppLocalization.UserInterface.Delete,
AppLocalization.UserInterface.TakePhoto,
AppLocalization.UserInterface.PickPhoto);
//Execute action result
if (MyActionResult == AppLocalization.UserInterface.TakePhoto)
{
//-----------------------------------------------------------------------------------------------------------------------------------------------
//Take photo
await CrossMedia.Current.Initialize();
if (!CrossMedia.Current.IsCameraAvailable || !CrossMedia.Current.IsTakePhotoSupported)
{
await DisplayAlert(AppLocalization.UserInterface.Alert, AppLocalization.UserInterface.NoCameraAvailable, AppLocalization.UserInterface.Ok);
}
else
{
var MyPhotoFile = await CrossMedia.Current.TakePhotoAsync(new Plugin.Media.Abstractions.StoreCameraMediaOptions
{
Directory = "MyApp",
Name = "MyAppProfile.jpg",
SaveToAlbum = true,
PhotoSize = Plugin.Media.Abstractions.PhotoSize.Small
});
if (MyPhotoFile != null)
{
//Render image
MyProfilePicture.Source = ImageSource.FromFile(MyPhotoFile.Path);
//Save image on database
MemoryStream MyMemoryStream = new MemoryStream();
MyPhotoFile.GetStream().CopyTo(MyMemoryStream);
byte[] MyArrBytePicture = MyMemoryStream.ToArray();
await SaveProfilePicture(MyArrBytePicture);
MyPhotoFile.Dispose();
MyMemoryStream.Dispose();
}
}
}
if (MyActionResult == AppLocalization.UserInterface.PickPhoto)
{
//-----------------------------------------------------------------------------------------------------------------------------------------------
//Pick photo
await CrossMedia.Current.Initialize();
if (!CrossMedia.Current.IsPickPhotoSupported)
{
await DisplayAlert(AppLocalization.UserInterface.Alert, AppLocalization.UserInterface.PermissionNotGranted, AppLocalization.UserInterface.Ok);
}
else
{
var MyPhotoFile = await CrossMedia.Current.PickPhotoAsync();
if (MyPhotoFile != null)
{
//Render image
MyProfilePicture.Source = ImageSource.FromFile(MyPhotoFile.Path);
//Save image on database
MemoryStream MyMemoryStream = new MemoryStream();
MyPhotoFile.GetStream().CopyTo(MyMemoryStream);
byte[] MyArrBytePicture = MyMemoryStream.ToArray();
await SaveProfilePicture(MyArrBytePicture);
MyPhotoFile.Dispose();
MyMemoryStream.Dispose();
}
}
}
};
Please help!! We need to deploy this app but we cannot do it with this problem.
Thank you in advance!
It is perfectly normal to have the Android OS terminate and restart an Activity. As you are seeing, your app's Activity it will be automatically restarted when the camera app exits and the OS returns control to your app. The odds are it just needed more memory in order to take that photo with the Neo's 16MP camera, you can watch the logcat output to confirm that.
Restarted – It is possible for an activity that is anywhere from paused to stopped in the lifecycle to be removed from memory by Android. If the user navigates back to the activity it must be restarted, restored to its previously saved state, and then displayed to the user.
What to do:
So on the Xamarin.Forms OnStart lifecycle method you need to restore your application to a valid running state (initializing variables, preforming any bindings, etc...).
Plug code:
The Android platform code for the TakePhotoAsync method looks fine to me, but remember that the memory for that image that is passed back via the Task will be doubled as it is marshaled from the ART VM back the Mono VM. Calling GC.Collect() as soon as possible after the return will help (but your Activity is restarting anyway...)
public async Task<MediaFile> TakePhotoAsync(StoreCameraMediaOptions options)
{
~~~
var media = await TakeMediaAsync("image/*", MediaStore.ActionImageCapture, options);
In turn calls:
this.context.StartActivity(CreateMediaIntent(id, type, action, options));
Not much less you can really do within the Android OS to popup the Camera.
In addition, when I take a picture it takes almost 30 seconds to get back to the code after awaiting TakePhotoAsync process, and it's too slow!
Is that on your Neo? Or all devices?
I would call that very suspect (ie. a bug) as even flushing all the Java memory after the native Camera Intent/Activity and the restart time for your app's Activity should not take 30 seconds on a oct-core 1.6 GHz Cortex... but I do not have your device, app and code in front of me....

Why explodes this window

i have a form loaded with the user data (from Json Store). When i click the first time, the form show good. But, the second time crash :
Firebug says:
Also crash, when i show other form after this form.
Any ideas ?
Edit for more info:
The code to load form is:
cargarFormularioEdicion: function(idPaciente){
var store = Ext.create('cp.store.form.Paciente',{});
store.load({params:{idUsuario: idPaciente}});
var form = Ext.create('cp.view.form.EditPaciente',{
action: 'bin/paciente/modificar.php'
});
// Ver la forma de pasar este listener al controller
form.on('afterrender',function(form,idPaciente){
form.getForm().loadRecord(store.first());
form.getForm().findField('idUsuario').setValue(idPaciente);
});
var win = Ext.create('cp.view.ui.DecoratorForm',{
aTitle: 'Editar paciente',
aForm: form
});
win.show();
}
Hypothetical solution:
Load the store with async = false.
var store = Ext.create('cp.store.form.Paciente',{});
Ext.apply(Ext.data.Connection.prototype, {
async: false
});
store.load({params:{idUsuario: idPaciente}});
Your code does not guarantee that the store is loaded at the time form.getForm().loadRecord(store.first()); is called. Since it is probably not, somewhere in the processing of loadRecord expects tries to access an undefined variable (as your error message shows) and that crashes the javascript execution. That leaves the form component with a broken internal state, and so everything is ugly from there.
You've kind of found that already by loading your store synchronously, but that's really something you should avoid. It wastes processing time by blocking a thread, and probably freeze your application for a few seconds.
The correct way of handling asynchronous events is to pass them a function that they will call back when there done. Hence the name callback function (and the joke "Hollywood principle"). This is efficient and safe.
Since the callback function can be passed in multiple ways, you need to refer to the doc. Taking your code as example, look at the doc for Ext.data.Store#load. Here's how to fix your code elegantly:
cargarFormularioEdicion: function(idPaciente){
var store = Ext.create('cp.store.form.Paciente');
store.load({
params:{idUsuario: idPaciente}
// here's how to pass a callback to Store#load
// notice you get access to some useful parameters as well!
,callback: function(records, operation, success) {
if (!success) {
// the execution will continue from here when the store is loaded
var form = Ext.create('cp.view.form.EditPaciente',{
action: 'bin/paciente/modificar.php'
});
// Ver la forma de pasar este listener al controller
form.on('afterrender',function(form,idPaciente){
form.getForm().loadRecord(store.first());
form.getForm().findField('idUsuario').setValue(idPaciente);
});
var win = Ext.create('cp.view.ui.DecoratorForm',{
aTitle: 'Editar paciente',
aForm: form
});
win.show();
} else {
// you need to handle that case
}
}
});
}
Hypothetical solution: Load the store with async = false.
var store = Ext.create('cp.store.form.Paciente',{});
Ext.apply(Ext.data.Connection.prototype, {
async: false
});
store.load({params:{idUsuario: idPaciente}});

What's the best way to cancel an upload in progress using filepicker.io?

I'm using filepicker.makeDropPane to play around with some simple uploads. One thing that I cannot find in the docs at filepicker.io's web documentation is a method for canceling an upload that is in progress.
Intuitively I feel that the onStart function should be passed an additional parameter. An object that represents the upload that has a cancel() function which when called would immediately cancel the file uploads. Something like this does not seem to be available.
we don't actually have this functionality yet, but it makes sense. I'll take a look at adding something along these lines and let you know
Here's an ugly hack I figured out to protect against unwanted returned files as well some other fancy stuff around progress and multiple files etc. You definitely want to use a static js file include so it doesn't get changed on you and break this.
Use this in your progress handler:
var event = event || window.event;
// For firefox, go and grab the XHR progress event object.
if (typeof event == 'undefined' || event == null) {
if (typeof arguments.callee.caller.arguments[0] == 'object') {
event = arguments.callee.caller.arguments[0];
}
else {
event = arguments.callee.caller.arguments.callee.caller.arguments.callee.caller.arguments[0];
}
}
var sUploadId;
if (event.type.toLowerCase() == 'progress') {
// if we haven't already tagged this XHR object (one per file), tag it now
// if we receive further progress from this file, it will already have been tagged
if (!('id' in event.currentTarget)) {
sUploadId = goUploadProgress.files.length;
event.currentTarget.id = sUploadId;
goUploadProgress.files.push({id: sUploadId, total:event.total});
}
else {
sUploadId = event.currentTarget.id;
}
// get the loaded bytes for this file
goUploadProgress.files[sUploadId].loaded = event.loaded;
}
else {
// ignore readystatechange events
return;
}
I modified this to make more sense for this context, so I may have made a logical mistake, but this is the gist. You can get the event in Firefox by looking up the call stack, and you can append an id to each file's XHR upload object, which will exist until it finishes.
I repeat, this is a hack! Don't trust that it won't break.