We are working on a prototyp application using unity3d. Your goal is to create a fluid and fun to use cross platform app.
The problem we facing right now is streaming (h.264 - mp4) video content over the web. This will be a major feature of our app.
I have already tried MovieTextures and the www class but it seems the files must be in ogg format which we can not provide. On the other hand handheld.playfullscreenmovie seems to be an android and ios only feature which uses the build in video player. This would be great if it would be supported on other platforms (e.g. Win8-Phone) as well.
Is there another cross platform option to stream (h.264 - mp4) video content over the web and display in full screen or as gui object? Or are there any plans to support something like this in the near future? Or is there a stable plugin for such a task?
Thanks
As of Unity 5 Handheld.PlayFullScreenMovie supports Windows Phone and Windows Store as per http://docs.unity3d.com/ScriptReference/Handheld.PlayFullScreenMovie.html
On Windows Phone 8, Handheld.PlayFullScreenMovie internally uses Microsoft Media Foundation for movie playback. On this platform, calling Handheld.PlayFullScreenMovie with full or minimal control mode is not supported.
On Windows Store Apps and Windows Phone 8.1, Handheld.PlayFullScreenMovie internally uses XAML MediaElement control.
On Windows Phone and Windows Store Apps, there generally isn't movie resolution or bitrate limit, however, higher resolution or bitrate movies will consume more memory for decoding. Weaker devices will also start skipping frames much sooner at extremely high resolutions. For example, Nokia Lumia 620 can only play videos smoothly up to 1920x1080. For these platforms, you can find list of supported formats here: Supported audio and video formats on Windows Store
mp4 is not a streamable container. If you read the ISO specification, you will see that MP4 can not be streamed. This is because the MOOV atom can not be written until all frames are know and accounted for. This 100% incompatible for live video. There are supersets of MP4 used in DASH that make this possible. Essentially, they create a little mp4 (called a fragment) file every couple seconds. Alternatively you can use a container designed for streaming such as FLV or TS.
You will probably need to step outside the unity sdk a bit to enable this.
Related
I'm a bit lost looking through all the various Agora.io modules (and not sure what it means that only some of them have Unity-specific downloads).
I want to make a Unity app where two remote phones exchange data as follows:
Streaming voice in both directions
Streaming video in one direction (recorded from device camera)
Streaming a small amount of continuously-changing custom data in the other direction (specifically, a position + orientation in a virtual world; probably encoded as 7 floats)
The custom data needs to have low latency but does not need reliability (it's fine if some updates get lost; app only cares about the most recent update). Updates basically every frame.
Ideally I want to support both Android and iOS.
I started looking at Agora video (successfully built a test project) and it seems like it will cover the voice and video, but I'm struggling to find a good way to send the custom data (position + orientation). It's probably theoretically possible to encode it as a custom video feed but that sounds complex and inefficient. Is there some out-of-band signalling mechanism I could use to send some extra data alongside/instead of a video?
Agora real-time messaging sounds like it would probably work for this, but I can't seem to find any info about integrating it with Unity (either on Agora's web site or in a general web search). Can I roll this in somehow?
Agora interactive gaming could maybe also be relevant? The overview doesn't seem real clear about how it's different from regular Agora video. I suspect it's overkill but that might be fine if there isn't a large performance cost.
Could anyone point me in the right direction?
I would also consider alternatives to Agora if there's a better plugin for implementing this feature set in Unity.
Agora's Video SDK for Unity supports exporting projects to Android, iOS, MacOS, and Windows (non-UWP).
Regarding your data streaming needs, Agora's RTM SDK is in the process of being ported to work within Unity. At the moment the best way to send data using the Agora SDK is to use CreateDataStream to leverage Agora's ability to open a data stream that is sent along with the frames. Data stream messages are limited to 1kb per frame and 30kb/s so I would be cautious about running it on every frame if you are using a frame-rate above 30fps.
I'm using MovieTexture now, but when a video file is added to unity Project, it will automatically be imported and converted to Ogg Theora format. and the quality is really bad.
I have tried changing the quality setting and even on the highest setting the video is still pretty bad quality, I have tried it in multiple file formats like .mov, .avi. .mpeg4 etc. I have even tried converting it to .ogv to try and get around unity converting it itself, and still the quality is poor. The platform is PC, and in the build the quality is the same as in the editor.
so the question is ,how to play high quality video in unity no matter using MovieTexture or anything else like some plugins?
Unity player on Windows only supports OGG, which is why Unity is transcoding your videos.
I have use the Renderheads AVPRo Quicktime plugin on Windows to play very high quality videos in kiosk setups. (They also have one for Windows Media format, but I used Quicktime).
Link: Renderheads AVPro (Quicktime)
I am not affiliated with them in any way, just a very happy customer, and here is the review I posted on the Unity Asset store:
Great work on your plugin! I've used so many plugins that don't work well over multiple platforms, or require switching between platforms, or manual steps, or manual licensing, or DLL hell, etc. I have to say you nailed it.
I develop on a Mac (and your plugin runs in the Unity Editor), then deploying on Windows. It all worked well straight forward and as documented. Even the events to detect when a video has loaded and is ready to play just what I needed (as we are loading a large video file).
Additionally, the error messages are very precise and pin-point a problem (missing file, bad format, etc) which means less time debugging.
I'm writing an Chrome Packaged App that needs to be able to play a lot of local video files. I can use the tag to play files encoded in h.264 and mp3, but not much else. I'll require playback of at least DivX videos and AC3 audio. Is there any way to do this using the HTML5 platform or otherwise using some kind of plugin?
There are alternatives, but in my opinion the final solution is not going to be very good.
1 - You can try to use a plug-in, for example:
VLC Plug-in - sorry, I have not enough reputation to post more than 2 links :(
Divx Web Player - sorry, I have not enough reputation to post more than 2 links :(
But then you need to rely on the user installing the plug-in. For VLC, the plug-in is not compatible with the latest versions of Mac OS X.
2 - Encode to H.264 or VP8 from a server with an ffmpeg or using a cloud video provider.
3 - Encode from the client side using JavaScript! There is a port of the ffmpeg on javascript (http://bgrins.github.io/videoconverter.js/). I didn't try this method with large files.
4 - Encode from the client side using a Native Client component (https://developers.google.com/native-client/dev/). But seems a daunting task to me.
If you are going to go with the first option, assure that your audience is going to install/configure your player and that their OS are supported.
VLC ported to NaCL would be a great first step.
According to a poster on https://forum.videolan.org/viewtopic.php?f=5&t=107178, libVLC has been ported to NaCL, but I am not familiar with VLC internals so I could not say how far this gets you in terms of being able to decode different streams.
I have a stock trading membership website, on which I put up stock videos on stock trading days. Currently, I have them in .swf and I am quiet happy with what I offer. Lately, my members are asking for availability on iPhone / iPad too, which means, I have to convert or upload new videos in one of the universally accepted formats, i.e., .mp4. I have tested few settings with the video conversion, but, up to, 640 x 480 resolution worked well on iPhone / iPad but for web viewing, it was disaster. I usually offer videos in 980 x 620 size. I changed the size to 800 x 600 and it was Okay for web, but iPhone / iPad din't do well. I am using JWPlayer for the test.
I want to ask, what format of the video should I use, which will work for all platforms. I want to keep the video size down, and as well 1 file for all platforms. Is there anything that I can do to achieve this?
Will be very thankful if someone helps me out on this.
Thanks!
I don't think this is really a code related issue but since I'm here:
Definitely use mp4 it works on all your desired platforms.
Definately stick with JWPlayer, I use it to stream to web and iPhone.
To keep the "file size down" and if you are encoding offline use handbrake http://handbrake.fr/
Using that software you will see some pre-sets for iDevices including iPhone and iPod mp4 / m4v file output. A new version of the software has just been released so it may include a pre-set for iPad too.
Personally I have a pre-set that is 720x400 which runs on iPod touch 2ndGen and iPhone very well.
Unfortunately one file does not suit all your target platforms. However, with JWPlayer you are able set it up with bitrate switching such that the best possible version of the different video files is delivered to the target platform:
http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v4/27/bitrate-switching
The JWPlayer will also deliver to Flash and HTML5 enabled browsers.
I'm trying to build a web application for iPhone and Android that deals with audio input.
Is this possible?
Apparently ... yes, or it should be able to when it's finished at least. It will supposedly become possible using the device API which is due to be part of HTML5 when it's finished and released (HTML5 isn't finalised yet however, and information is subject to potential for change).
W3C Device API Requirements (camera section)
Sony Erricson community blog posting, with examples (pre-final API)
While it isn't explicitly mentioned in the W3C spec, audio recording as part of (web)camera interactions is, so it's definitely hopeful. There seems to be a shortage of good information at this stage though. I'd expect to see more as HTML5 comes closer to being finalised.
As of now, HTML5 Can't record Audio. but in future, it will be able to, by using the Device's native features.
HTML 5 can not record audio (at least currently). HTML basically is a markup language and therefore only declares how a browser should display certain content. Although HTML 5 introduces new features that make some interaction possible, you can't record audio straight into.. HTML (even saying that sounds wrong). When the HTML5 spec is finished, it might become reality, until then, no way.
Web applications that record audio normally require a plugin like Flash or Silverlight, because those can access system resources like audio hardware. Both are a no-go on iOS, although Flash is theoretically possible on Android, I don't know if it supports audio input.
I would suggest you write a native app (for iOS and Android) that can access the audio hardware and connects to your web application in the background, so that the recording takes place natively and the recorded audio will be transmitted to your servers (think of Shazam, for example).
Here are the basic developer guides on recording audio in:
Android
OS X, iOS
A new MediaStream Recording API is being worked on. It is currently availble only in the Firefox Nightly build for demo purposes
Here's the draft with the latest updates directly form W3C site:
https://dvcs.w3.org/hg/dap/raw-file/default/media-stream-capture/MediaRecorder.html
Also the following article covers up other attempts on recording audio and video directly in the browser:
http://hdfvr.com/html5-video-recording