I am building a Unity application in which i need to use a h.264 encoded stream. The data will be read over the network and shown on the screen (feed).
I Have worked with ffmpeg, but well context close is causing a problem so i was thinking of making my own h264.dll decoder.
I cannot find anything to where to start, everyone has SoC solutions to hardware decoders there are encoders available, any code/base or reference available on where to start with ?
i'v used ffmpeg its good but well its becoming big to ship since all those dll's are a bit bulky, so i want to do like one dll which will be the decoder and then whatever dependencies are required can be shipped.
If your license permits GPL, then you can compile ffmpeg (libav*) yourself eliminating all the unnecessary dependencies. (./configure --help will show you what options are available). Or you can license a decoder from a company like corecodec. Creating your own decoder is a no go. The spec is about 1000 pages long, and assumes you are familiar with advanced maths.
Related
I am reading about fuzzing. I have some basic questions regarding fuzzing. I searched but couldn't find any good explanation.
Why image files are popular and common for fuzzing? What is the benefit of using image files?
Why png files are popular and common for fuzzing?
Why Libpng is popular and common for fuzzing?
Is it best to fuzz png images with libpng for beginners? Why?
If someone can answer, it will be very helpful for me.
Thank you in advance.
You fuzz not image files, but software that parses these. Typically
developers don't write code to parse images, but use third party
libraries like libpng. As a developer you don't need to fuzz third
party libraries, only the code of your project. As a security
engineer you can fuzz them.
It is easy to setup fuzzing of such an opensource library - you can build it
statically instrumented, create a small application that calls into
it and fuzz it with an easy to setup fuzzer like afl. This and the
fact that such libraries are widely used, thus errors in these can
have big impact on a lot of applications, make them a good target
for fuzzing.
But image files are not the only files that are widely used and have
popular libraries to handle them. Most fuzzers are unaware of input
structure of the tested binary. They mostly use input mutation
techniques at bit/byte level - changing values of some bit/byte of
the input, feeding it to the tested application and watching it's
behaviour. When the input is highly structured, a fuzzer fails to
test deep into the code. For example to test a browser feeding
html-files to it, requires a fuzzer to create inputs that have
correct lexical and syntactical structure. Typically the code for
lexical/syntax handling is autogenerated based on a language
grammar. By changing bits/bytes in html you most likely get bad
keywords, which would be rejected by such an autogenerated code,
thus testing mostly this code and not getting deeper. Image files
are typically not highly structured and easier to fuzz deeply, thus
can be fuzzed with better coverage.
It is also faster to fuzz a small input than a bigger one - less
bits to change. It's easy to create a small image file just by
taking a small image as a seed, than for example an html-file.
I don't know if png files are more popular for fuzzing than other binary media files, but they structure can include multiple headers/chunks of different types which results in more different handling paths in the code and thus makes it more likely to have errors.
As I said it's opensource, widely used, easy to set up and can be fuzzed fast - it's much faster to run a small application, than for example a browser.
I'm not sure there can be a 'best' criteria, but it's easy and therefore good for beginners.
I want to make a sort of operating system, but with some very specific features.
The only thing i need it to do is show a video, place some text over it, accept user input and some basic file manipulation.
Something that looks like an OS'es loading screen would do.
i am, however, a complete noob to this part of programming and i have no idea where to look. Don't worry i do not need a complete explaination for every single step, but it would be nice to know in what direction i should go search.
Anyone got an idea where to start searching?
Thanks in advance ;)
Although you narrowed down your specification it's still complicated enough. Playing video involves file I/O, a file system, storage device access, buffering mechanism, memory allocation, memory management primitives, GPU access. Accepting user input requires keyboard and mouse handling which requires a working USB layer. Not to mention that you needed to make the video decoding library work with your own system, unless you want to implement that from scratch too. That might require you to have an ABI or a POSIX emulation layer. You might need to port at least one or two graphics libraries like SDL.
That's why "OS loading screen"s don't play videos :)
You might get away with using BIOS only for accessing all devices. But BIOS and VBE are usually slow and may not work well with your video playing scenario. BIOS is slowly getting pushed out of PC ecosystem in favor of UEFI too.
If you don't need a custom OS, you can trim a Linux kernel down to the parts you need to have. Even that's not a trivial thing to do though.
I am using the LINK to take frame from video ..but it says to create a commercial project that decodes H264 video, we have to make sure that we get a license from MPEG LA.
Why should we get license?some tutorials say that we have to give complete source code to
LGPL , if we integrate statically FFMPEG.what is mean by statically? what is the other option?can we submit our App to Appstore(for storing video streams in server) without getting license from LGPL?i am getting confused? any help please?
There are multiple issues going on; the first is your legal right to use the FFMpeg code specifically. This code is governed by the license chosen by the FFMpeg developers, and they have portions with GPL and portions with LGPL embedded. If you compile with the GPL-licensed portions, then the entirety becomes GPL, and you must either distribute the source code for FFMpeg OR offer to make it available for three years. If you link your code against FFMpeg, then you must also provide your source code OR offer to make it available for three years.
If you disable the GPL portions, then you do not need to provide source code for your project, and you only need to provide FFMpeg source code if you modify the LGPL-portions of FFMpeg.
BUT, this is only to abide by the copyright on FFMpeg. You also must comply with the intellectual property of the algorithms inside FFMpeg. This means in the United States and most other countries you must license the underlying algorithms from the organizations that patented the algorithms. E.g., if you use FFMpeg to decode MPEG2 Layer 3 data, then you must license MP3 support from Fraunhofer or one of their licensees. They may or may not place restrictions on the source code that is legal to use their intellectual property.
There are probably dozens of patents on technology in FFMpeg across dozens of patent offices from countries around the world. Which is why a lot of companies buy their source (and licenses!) from a single source, to avoid the complication of licensing agreements in every country where you will have a user and a license holder has registered an intellectual property through the local patent office.
MPEG LA is a company that holds patent pools for several video related technologies.
FFMPEG is an actual implementation of a lot of video related technologies, which a.o. is licensed under the LGPL.
If you want to use FFMPEG to work with H264, you have to deal with both the FFMPEG licenses and the MPEG LA-held patent. The FFMPEG license will require you to redistribute FFMPEG source, whereas using H264 in any form will require you to pay for a license from MPEG LA.
FFMpeg is an open-source library, more information about this here http://www.ffmpeg.org/
If you want to include libraries into your iPhone projects static linking is the only allowed way of doing this. If you do not know the differences between static and dynamic linking, ask Google.
LGPL is just a type of software license, not a legal entity. AFAIK a library published under GPL can only be used if the end product is also licensed under GPL and is open-source. LGPL code can be used in closed-source projects, too if one includes the licence text in the software.
I am not a lawyer, but if you do not encode H.264 you do not have to pay any license fees at all to the MPEG LA.
Yes, linking ffmpeg into your projects is "app store compatible".
EDIT
As Theo pointed out in the comments, point 2 is wrong. This would render FFmpeg incompatible with the iPhone as no dynamic linking is allowed.
I am not a lawyer, but my interpretation of this is:
One of the intents of the LGPL is to allow the distribution of pre compiled binaries, but in such a way that substitutability is possible: LGPL preserves the right of the end user to swap a LGPL library for an equivalent library of their own choosing, by rebuilding the library from source or whatever.
This means the only way to be in compliance with LPGL licensed distributions is to release all your source code, OR, bundle the library as a stand alone dynamic library.
As iPhone software is required to use libraries statically linked, it doesn't seem possible to use LGPL libraries in AppStore Apps unless you also make your entire app source available - or enough of it such that users could re-link it with their own implementation of said library.
it is like an command line any to any video/image converter that can be extended to support as many media codecs as there are , for an instance you upload an video in some format to your site , you can convert it to flv or mp4 or any format u wish to support and host it just like you tube doing ...
http://www.ffmpeg.org/ffmpeg-doc.html check this for all possible functionality
I have developed a Windows application that captures video from an external device using DirectShow. The image resolution is 640x480 and the videos saved without compression have very huge sizes (approx. 27MB per second).
My goal is to reduce this size as much as possible, so I am looking for an encoder which will allow me to compress the video in real-time. It could be H.264, MPEG-2 or anything else. It must allow me to save the video to disk and it would be best if I also could stream it in real-time over network (Wi-Fi, so the size should be around 1MB per second, or less). The significant quality loss would be unacceptable.
I have found out that getting an appropriate DirectShow filter for this task is very difficult. It can be assumed that the client machine is reasonably modern (fast 2-core CPU) and can utilize CUDA/OpenCL. There are a few apps that allow to encode video using CUDA and offer good performance, however I have not found an appropriate DirectShow filter or an API which could be used to develop one. The NVIDIA nvcuvenc.dll seems to have private API so I am unable to use it directly. Any CPU-based encoders I have found are too slow for my requirements, but maybe I have missed some.
Could anybody recommend me a solution, i.e. an encoder (paid or free, that can be used in an closed-source app) that can achieve a good performance, regardless whether it is using CPU/CUDA/OpenCL or DirectCompute? Or maybe I should use some external hardware video encoder?
Best regards,
madbadger
Since you're using Directshow, by far the easiest thing to do would be to use WMV9 inside an ASF container. This is easier because it's available on almost all Windows machines (very few install time dependencies), decently fast (you should have no issues using it on a reasonably modern machine) and the quality is reasonable. But considering your limit is 8 mbit/sec (1 MB/sec), quality isn't an issue for you. 2 mbit/sec, VGA-resolution WMV9 should look quite good.
It's not nearly as good as a decent implementation of H264, but from an implementation standpoint, you're going to save yourself a lot of time by going this route.
See this:
http://msdn.microsoft.com/en-us/library/dd375008%28v=VS.85%29.aspx
Which filters have you tried?
If you're only dealing with 640x480, then any reasonable-quality commercial software-based encoder should be fine as long as you choose a realistic bitrate. Hardware acceleration using Cuda or OpenCL shouldn't be required. H264 takes a bit more horse-power and would require more CPU cores, but Mpeg2 or any of the h263-era codecs (xvid, wmv9, divx, etc) should have no problems even on a modest CPU. Streaming it over the network at the same time takes a little more effort, but should still be possible.
It's not DirectShow-based, but VLC Media Player can do all this. It's based on the FFMpeg open-source project. Some versions of it are LGPL-licensed, so the library could incorporated into your project without many restrictions.
If you just want a set of DirectShow filters that will handle all this for you, I've had good results with MainConcept's products before. They're at the expensive end of the spectrum though.
You dont specify what filters you've tried, or what 'significant' quality loss means, so about the best I suspect we can do is suggest some encoders to try to see if they meet your requirements.
Two good ones are the Theora and WebM video encoder filters (you can get them from a single installer at xiph.org). They're both high quality encoders which can be tweaked to balance performance vs quality. WebM can use multiple processors when encoding, which might help in your situation. Both are also used w/ HTML5 video, so that might be an extra plus for you.
Forget about WMV encoding for realtime streaming. WMV works well for realtime low quality streams, but it doesn't do high quality encoding in realtime.
I suggest that you take a look at MainConcept's SDK. They do a series of DirectShow filters for encoding H.264. I've implemented realtime streaming and muxing of streams encoded in H.264 using MainConcept's codec and DirectShow filters, and it's great.
Hope this helps
I am using Windows Media Encoder for real-time encoding, and it works well even in resolution 720x576. One such example of it's usage is in VideoPhill Recorder.
It is written in pure .NET with DirectShow.NET for capturing and windowsMedia.NET for encoding.
Using those two I am able to achieve real-time encoding with 24/7 stability.
And both libraries are free to use on Windows, so you won't have to pay any licenses except for OS.
ffdshow tryouts leverage ffmpeg's x264 stuff, which is said to be pretty fast (I think so anyway). Also libjpeg-turbo might help, or choosing some other codec made for high throughput like camstudio's or what not.
update: ffmpeg can take directshow input now: http://ffmpeg.zeranoe.com/forum/viewtopic.php?f=3&t=27
Have you seen this yet?
http://www.streamcoders.com/products/rtpavsource.html
http://www.streamcoders.com/products/rtpavrender.html
If you can stay at or below 1280x1024, Micorsofts MPEG-2 encoder (included in Vista and up) is quite good.
I haven't gotten it to work for 1080p content at all though. I suspect the encoder just aborts on that. Shame.
Here is one option : http://www.codeproject.com/Articles/421869/H-264-CUDA-Encoder-DirectShow-Filter-in-Csharp
It uses about 10% of my cpu (p4 3ghz) to encode a SD video to h264 in graph edit.
See the CaptureDS C# sample that comes with AVBlocks. It shows how to build a video recorder with AVBlocks and DirectShow. DirectShow is used for video capture and AVBlocks is used for video encoding:
I don't just mean publish, but pretty much everything between when the pure coding is finished and the first version is released. For example, how do games make it so that their save files are hidden/unhackable, how do they include their resources within the game as opposed to having a resource file containing all of the sprites, etc., how do they make it so that there are special file extensions like .rect and .screen_mode, and so on and so forth.
So does anyone know any good books, articles, websites, etc. that explain the process between completing the pure code for a game and the release of it?
I don't think developers make much of an effort to ensure saves are hidden or unhackable. PC games usually just save out to a folder, one file per save, and any obfuscation is likely the result of using a binary file format (which requires some level of effort to reverse-engineer) or plaintext values that aren't very meaningful out of context, but not deliberate attempts to circumvent hacking. There are probably a ton of PC games that have shipped with very easily hackable text or XML save files, but I've never been a save hacker so I don't have any specific examples. On consoles the save files are going to a memory card or the console's hard drive, which makes them inherently inconvenient to access, but beyond that I don't think console developers make much of an effort to encrypt or otherwise obfuscate save data. That energy would more likely be directed towards securing the game against cheating if it's on online game or just making other systems work better.
Special file extensions come from just using your own extensions and/or defining your own file formats. You can use any extension for any file, so there are tons of "special" file formats that are just text files with a different extension, I've done this plenty of times myself. In other cases, if they have defined their own binary file format, that means they also have their own file parsers to process those files at runtime.
I don't know what platforms you have in mind, but for PC and console games, resources are not embedded in the executable. You will generally see a separate executable and then various archives and configuration files. Depending on the game, it may be a single resource pack, or perhaps a handful of packs for related resources like graphics, sound, level data, etc. As a general observation console games are more aggressively archived (to minimize file operations on slow optical media, and perhaps to overcome limitations of the native file systems on more primitive platforms). Some PC games have very loose assets, with even script files hanging out in the open.
If you develop for Windows or XBox 360, Microsoft might offer some help here. Check out their Game Development tools for Visual Studio C++ Express Edition.
If you are looking for books the Game Development Essentials series should answer your questions.
For circumventing saved file modifications, you can implement a simple encryption algorithm and use it to encrypt saved files, and then decrypt them when loading. File extensions are simply a matter of choice.
To use special file extensions in your game, just do the following:
Create some files in a format of your choice that have that extension, and then
write some code that knows how to read that format, and point it at those files.
File extensions are conventions, nothing more; there's nothing magic about them.
ETA: As for embedding resources, there are a few different ways to approach that problem. One common technique is to keep all your resources bundled together in a small number of files - maybe only one (Guild Wars takes that approach).
At the other extreme, you can leave your resources spread across many files in a directory tree, maybe in a custom format that requires special tools to modify, and maybe not. Civilization 4 does things this way, as do all the Turbine games I'm familiar with. This is a matter of taste, and not very important either way.
I think a better solution is two break your images in tiles of some known size and then join them back to back in some random order in a new file. This random order is only known to you and hence only you know how to jumble the tiles to get the original image back.
The approach would be to maintain a single dimensional array and maintains the position of tiles in it. Know use the crop functions of MIDP to extract each tile and render each tile back to the console.
If you need, I can post the code for you.
I would suggest to check the presentation from the developers of World of Goo (great game):
http://2dboy.com/public/eyawtkagibwata.pdf.