I'm trying to build a web application for iPhone and Android that deals with audio input.
Is this possible?
Apparently ... yes, or it should be able to when it's finished at least. It will supposedly become possible using the device API which is due to be part of HTML5 when it's finished and released (HTML5 isn't finalised yet however, and information is subject to potential for change).
W3C Device API Requirements (camera section)
Sony Erricson community blog posting, with examples (pre-final API)
While it isn't explicitly mentioned in the W3C spec, audio recording as part of (web)camera interactions is, so it's definitely hopeful. There seems to be a shortage of good information at this stage though. I'd expect to see more as HTML5 comes closer to being finalised.
As of now, HTML5 Can't record Audio. but in future, it will be able to, by using the Device's native features.
HTML 5 can not record audio (at least currently). HTML basically is a markup language and therefore only declares how a browser should display certain content. Although HTML 5 introduces new features that make some interaction possible, you can't record audio straight into.. HTML (even saying that sounds wrong). When the HTML5 spec is finished, it might become reality, until then, no way.
Web applications that record audio normally require a plugin like Flash or Silverlight, because those can access system resources like audio hardware. Both are a no-go on iOS, although Flash is theoretically possible on Android, I don't know if it supports audio input.
I would suggest you write a native app (for iOS and Android) that can access the audio hardware and connects to your web application in the background, so that the recording takes place natively and the recorded audio will be transmitted to your servers (think of Shazam, for example).
Here are the basic developer guides on recording audio in:
Android
OS X, iOS
A new MediaStream Recording API is being worked on. It is currently availble only in the Firefox Nightly build for demo purposes
Here's the draft with the latest updates directly form W3C site:
https://dvcs.w3.org/hg/dap/raw-file/default/media-stream-capture/MediaRecorder.html
Also the following article covers up other attempts on recording audio and video directly in the browser:
http://hdfvr.com/html5-video-recording
Related
I ran into this problem in Safari where it appears that WebRTC is not fully supported. So when I call
navigator.webkitGetuserMedia()
I get an undefined error.
So my question to the community is what is the best way to write a Meteor app that captures Video on a mobile device and saves it on the said device.
If you have done this, I would appreciate it very much if you could share with me and the community how you went about this.
Specific Answer
The modern API is: navigator.mediaDevices.getUserMedia(constraints). See the docs here.
In the past, I've been unsuccessful with getUserMedia on iOS, but according to this post it can be done on iOS 11.
As for saving it, you can write to the browser's file system, but that API is only supported in Chrome. If you want to write to the camera roll, you'd need native code in the mix.
General Advice
I've spent several years of my life dealing with recording, uploading, and processing video using meteor. If you are doing anything more than trivial web recording, these observations may save you some time:
Chrome (on everything but iOS) has the best API for web recording. If you can require chrome for recording, that's ideal. Firefox is a close second, only because it doesn't support the file system API.
If you need to record and upload long videos on iOS, build a native app. Don't consider any kind of hybrid - that's a serious trap. The number of corner cases and things you need to check is pretty astounding, and the only way to get over those hurdles is with native code.
I want to build a facebook app featuring a personalized video which imports content assets from the user's facebook profile and their extended social graph and integrates these assets within the timeline. I am thinking of using Flash however a key stipulation is that the app works on mobile - and so I would need to use HTML5. My question is: Can I use Flash to build the application and then compile the app as HTML5 - or is there an alternative solution in the form of a HTML5 video toolkit with a programming layer that would allow me to build a web app / access the Facebook API?
I have done this a few times over the years and yes flash was the easiest however there are a few options which you have available to you that I know of which will be purely HTML5 based, personally I'd stay away from flash here as it will end up just getting int he way:
1- The cleanest method is to use a video compositing tool on the server side which can be programmed to accept variables. Personally I have only ever done this using ffmpeg however there a couple of alternatives which are out there.
The basic process would be to grab the media from FB then to composite them at certain point on top/below/around a base video which is sitting on the server using a shell script which you then pass the media assets to as variables. There are so many options as to how you might want this to be done, probably best id to have a look at some of these examples:
http://broadcasterproject.wordpress.com/2010/05/18/how-to-layerremix-videos-with-free-command-line-tools/
http://graphcomp.com/ffmpeg/
ffmpeg watermark without vhook?
note that last time I did this I used vhooks and custom filters, vhooks are now deprecated
This method will mean a reasonably heavy server load if your app is popular but it's probably the most robust across devices etc.
2- Use Popcorn.js, and let the processing be done on the client side. you could hard code it using css/js/html but popcorn is pretty stable although I havent seen how it runs on devices but in theory it should work (all standardized technologies). Basically the process would be to use javascript to fire the display of images overlayed on the video base file at preset cue points. Popcorn has all of the methods and means for you to do this already.
Hope this helps a bit. Good luck, sounds fun.
we realised some interactive video apps and one recent project was quite like your question describes.
We used adobe flash to track the motion - and published the project via create.js. You could have an image sequence from within create.js or put a video in a layer behind. This video would then control the player head time of the create.js motion tracked sequence via jquery.
worked fine - here a link to a testsetup with an image sequence.
Video Integration would be the next step.
http://www.jungeroemer.net/projekte/testpersvid/elftest01.html
(German text, sorry but it's nothing important to read there.
Just click the images and go for it)
you can download the sources from the link, if you need i can also upload the flash file to show you the motion tracking.
Firstly, as this is essentially a multi-disciplinary question, please feel free to submit answers which only answer one part of the post - I will upvote answers which satisfy any of the questions below.
Hi, we are putting together an installation in which:
Audio comes in through a microphone to an FM transmitter
Transmitted to receiver
Receiver is plugged into computer's Line In
Audio is recorded/streamed as an .m3u stream, or recorded, and then uploaded(?)
Stream is accessible by scanning QR code on iPhone/Android.
I have a few questions around the subject that I was wondering if people could help with.
This is kind of a minor first question, if you are able to answer the second question, but is the best way to deal with incoming audio to record it 'before' uploading it, resulting in a delay, or is it to stream it?
I'm not sure of the best way to host the upload, as I suppose that it would be either an .m3u stream or some kind of progressive .mp3 that would need to be uploaded somewhere that supported this. Any advice? Is this kind of thing possible with free webhosting solutions? It would preferably be above 128kbps, too.
Initially, I presumed that the easiest way to have an audio stream linked to a URL that allowed for QR code linking was to build a very simple webpage with a media player on it, so that the end user wouldn't have to download any external application, and it could be done through the Android/iPhone browsers. Is this the case? Is the easiest way of doing this with something like <video> or <audio>, or is the HTML5 support for the mobile browsers not yet good enough? What are the alternatives? Something like jPlayer looks like it would work?
Thank you for your responses, I can give any necessary technical information you may find relevant, but these are quite broad questions. We can run the stream from either OSX 10.7 or Windows 7, as we have both, so recommendations for suitable software for either platform are welcomed.
Thanks again!
EDIT: Information from jPlayer.org -
HTML5 Audio Streams
HTML5 browsers and their support for audio streams. (Note that, jPlayer's Flash fall-back for non-HTML5 browsers works with MP3 streams.)
Audio streams work on:
Firefox (OSX, Win): OGA
Safari (OSX): MP3
Mobile Safari (iOS4 iPad/iPhone/iPod): MP3
Opera (OSX, Win): OGA
Chrome (OSX, Win): MP3, OGA
IE9 (Win): MP3
No mention of iOS5 or Android, but I used my android phone to check out the jplayer 1.2 stream demo they have up on their site, and it works. Seems like this could be the answer, but I need to know a little more about hosting, and perhaps iOS5 compatibility, though I'm sure it must work going forward. Anyone?
I recommend going with a standard SHOUTcast/Icecast setup.
First, you need an encoder. I recommend Edcast if you're doing this under Windows. The encoder is responsible for listening to your sound card, and encoding/compressing the raw PCM data into MP3, or whatever format(s) you need.
Next, you need a server. Again, SHOUTcast or Icecast will work great for you. No, you typically cannot host this on any free web hosting provider. Fortunately, hosting for SHOUTcast streams is cheap, and readily available. You can also host the server yourself, if you have the necessary bandwidth. It doesn't take much CPU/RAM.
Finally, you need a player. Your jPlayer will work just fine. For iOS, I usually just link to the playlist file and let it play. On Android, you can do a Flash player. HTML5 compatibility is there though, and jPlayer would be the way to do it.
Also note that there is no such thing as a ".m3u stream". A .m3u file is nothing but a playlist file, that contains the URL to the streaming audio. That's all.
For the QR code, just link to a web page URL that contains your player. That's all you have to do.
If you have more specific questions as you go, you should post them as separate questions here.
There are a ton of SO posts on audio, HTML5, and mobile Safari, such as these:
Reusing HTML5 Audio Object in Mobile Safari
Autoplay an Audio File on Mobile Safari
Preloading HTML5 Audio in Mobile Safari
Will HTML5 support the access of offline cached audio?
However, they all are outdated.
We prefer solutions to support iOS 3+, but we will take anything that works -- even if it's restricted to iOS 5.
Anyone have the definitive answer as things stand today, or testers on iOS 5 have any insights?
Can audio files be cached in mobile Safari? If so, what are the limitations?
Is there a way to minimize lag or delay between pressing a button and playing a sound?
Thanks!
I recently wrote an HTML5 audio player. I had similar trouble with iOS4 and iOS5. First, the play has to be triggered by user input, meaning specifically that it has to be in the same call stack as a click event.
I tested this a lot, and iOS seemed to refuse to cache the audio at all. It fetched the audio with every play. I think this should be considered a bug, but perhaps they are trying to preserve local storage space (audio files can get rather large).
If your audio files are not too large, you might want to consider appending them together into a single file and then using pause / jump to position / play to switch between sounds. I haven't tried it, but it should work. I didn't use the technique because my app was a music player, and music files are a bit too large for that technique to be valuable.
I am working on a website that works across all mobile browsers specifically Android, iOS and BB. I have used the experimental jQuery mobile for the consistent look and feel. One of the pages require audio to be recorded and played back for this I did some research and found that HTML5 has support for this but still part of the spec so it may not available on these browsers yet, so I was wondering if there is any workaround for this. I atleast know of one such application that captures audio and its Google Voice for iPhone and I read somewhere that they use HTML5 so does that mean safari has support for HTML5 audio capture? Please suggest.
For audio playback, you can use this test page on all of your supported platforms and browsers. It should confirm which HTML5 audio playback constructs will work:
http://www.jplayer.org/HTML5.Audio.Support/
It sounds like there are issues with even getting playback to work on Android, see here for info:
http://textopiablog.wordpress.com/2010/06/25/browser-support-for-html5-audio/
I haven't seen anything about current browser support for audio recording through HTML5. You could try Java or Flash, but Java requires the user to have it installed (and accept suitable permissions), and Flash requires you to set up a server for the data to be sent to (e.g. Red5)...and remember that Flash does not work on iOS.