MPEG-TS Encoding - encoding

I have a file that i need to convert to MPEG-TS so that it fits the below specification:
Elementary stream bitrate [kbit/s]
video: 2575
audio: 2 x 192
subtitle: -
PAT/PMT: -
Stuffing: -
Component TS bitrate [kbit/s]
video: 2652
audio: 395
subtitle: 45
PAT/PMT: 45
Stuffing: 62
Total: 3200 CBR
Additional required components:
PAT
PMT
Null packets
Components that might pop up:
NIT, SDT, EIT, etc.
vcodec="h264"
acodec="mpga"
bitrate="2500"
arate="192"
samplerate=48000
ext="mpg"
mux="ts"
vlc="/usr/bin/vlc"
fmt="mpg"
dst="/home/adam/test/"
for a in *$fmt; do
$vlc -I dummy -vvv "/home/adam/test/" --sout "#transcode{vcodec=$vcodec,venc=x264{profile=main,level=3.0,hrd=cbr,bframes=2},vb=$bitrate,acodec=$acodec,ab=$arate,samplerate=$samplerate,channels=2}:standard{mux=$mux,dst=\"$dst$a.$ext\",ac$
done
After encoding with the above script everything seems to be ok (for both video and audio bitrate codec is constant) apart from two things: Bitrate of the container should also remain CBR but this is not the case. Also, stuffing component (0x1 ffff) - null packet is missing. Is it possible for you to correct the script to make null packet as well as bitrate of the container constant (3,2 Mbps CBR) ?
The second option is encoding with ffmpeg:
ffmpeg -i video_input.mpg -i audio_input.mp2 -acodec copy -tune zerolatency -x264opts bitrate=2600:vbv-maxrate=2600:vbv-bufsize=166:nal_hrd=cbr -vpre libx264-iptv -vcodec libx264 -f mpegts -muxrate 3200K -y output.ts
but how to unset/disable/remove SDT table?

Given that you are aware of the concept of NULL packets, you might have been working with commercial grade software or hardware in this area.
There is a difference between CBR (of video) and system rate (or multiplexer rate). When video is encoded as CBR, say at 3.2 Mbps, it is quite ok that it fluctuate by a few hundred kbps around that margin. So peak bitrate, could be say 3.3 Mbps. This is quite ok. Adding another 100 kbps of Audio, the total maximum bitrate can be 3.4. Usually, one would set the System rate above 3.6 Mbps or more in that case; where balance are NULL packets.
The system rate 3.5 Mbps CANNOT fluctuate at all. If it does, the PCR based synchronization wont work and basically, things wont work in live environment. So basically, you can think of 3.5 Mbps (about 240 packets in 100 milisecond) can be thought of as a BUS. every seat must be filled up to ensure that transportation is continuous. Usually, it is done such a way that few seats will remain empty.
The sad part is, neither VLC nor any other tool in open source will do it for you.
The hack we used to use was, that we used to send such VLC produced stream over IP (where sending TS stream without NULL packets is ok - and receive that via the output from a Muxer with ASI or such interface, which would have added muxer.
Alternatively you can use Manzanita muxer to convert your Non-null TS stream to proper TS stream.
EDIT:
Based on the comment - if all you need to do is to disable SDT - there are two things needs to be done.
Remove all packets from the PID that corresponds to SDT table. If you are not fully demuxing and re-muxing - a quick way to do that could be to re-stamp the 13 bit PID number by a NULL packet PID number.
Remove the reference of SDT PID value in PAT table. This essentially means that you produce a 3 to 4 packets which is corrected PAT; and replace all PAT packet sequences in the stream by these corrected packets.

Related

MFCreateFMPEG4MediaSink does not generate MSE-compatible MP4

I'm attempting to stream a H.264 video feed to a web browser. Media Foundation is used for encoding a fragmented MPEG4 stream (MFCreateFMPEG4MediaSink with MFTranscodeContainerType_FMPEG4, MF_LOW_LATENCY and MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS enabled). The stream is then connected to a web server through IMFByteStream.
Streaming of the H.264 video works fine when it's being consumed by a <video src=".."/> tag. However, the resulting latency is ~2sec, which is too much for the application in question. My suspicion is that client-side buffering causes most of the latency. Therefore, I'm experimenting with Media Source Extensions (MSE) for programmatic control over the in-browser streaming. Chrome does, however, fail with the following error when consuming the same MPEG4 stream through MSE:
Failure parsing MP4: TFHD base-data-offset not allowed by MSE. See
https://www.w3.org/TR/mse-byte-stream-format-isobmff/#movie-fragment-relative-addressing
mp4dump of a moof/mdat fragment in the MPEG4 stream. This clearly shows that the TFHD contains an "illegal" base data offset parameter:
[moof] size=8+200
[mfhd] size=12+4
sequence number = 3
[traf] size=8+176
[tfhd] size=12+16, flags=1
track ID = 1
base data offset = 36690
[trun] size=12+136, version=1, flags=f01
sample count = 8
data offset = 0
[mdat] size=8+1624
I'm using Chrome 65.0.3325.181 (Official Build) (32-bit), running on Win10 version 1709 (16299.309).
Is there any way of generating a MSE-compatible H.264/MPEG4 video stream using Media Foundation?
Status Update:
Based on roman-r advise, I managed to fix the problem myself by intercepting the generated MPEG4 stream and perform the following modifications:
Modify Track Fragment Header Box (tfhd):
remove base_data_offset parameter (reduces stream size by 8bytes)
set default-base-is-moof flag
Add missing Track Fragment Decode Time (tfdt) (increases stream size by 20bytes)
set baseMediaDecodeTime parameter
Modify Track fragment Run box (trun):
adjust data_offset parameter
The field descriptions are documented in https://www.iso.org/standard/68960.html (free download).
Switching to MSE-based video streaming reduced the latency from ~2.0 to 0.7 sec. The latency was furthermore reduced to 0-1 frames by calling IMFSinkWriter::NotifyEndOfSegment after each IMFSinkWriter::WriteSample call.
There's a sample implementation available on https://github.com/forderud/AppWebStream
I was getting the same error (Failure parsing MP4: TFHD base-data-offset not allowed by MSE) when trying to play a fmp4 via MSE. The fmp4 had been created from a mp4 using the following ffmpeg comand:
ffmpeg -i myvideo.mp4 -g 52 -vcodec copy -f mp4 -movflags frag_keyframe+empty_moov myfmp4video.mp4
Based on this question I was able to find out that to have the fmp4 working in Chrome I had to add the "default_base_moof" flag. So, after creating the fmp4 with the following command:
ffmpeg -i myvideo.mp4 -g 52 -vcodec copy -f mp4 -movflags frag_keyframe+empty_moov+default_base_moof myfmp4video.mp4
I was able to play successfully the video using Media Source Extensions.
This Mozilla article helped to find out that missing flag:
https://developer.mozilla.org/en-US/docs/Web/API/Media_Source_Extensions_API/Transcoding_assets_for_MSE
The mentioned 0.7 sec latency (in your Status Update) is caused by the Media Foundation's MFTranscodeContainerType_FMPEG4 containterizer which gathers and outputs each roughly 1/3 seconds (from unknown reason) of frames in one MP4 moof/mdat box pair. This means that you need to wait 19 frames before getting any output from MFTranscodeContainerType_FMPEG4 at 60 FPS.
To output single MP4 moof/mdat per each frame, simply lie that MF_MT_FRAME_RATE is 1 FPS (or anything higher than 1/3 sec). To play the video at the correct speed, use Media Source Extensions' <video>.playbackRate or rather update timescale (i.e. multiply by real FPS) of mvhd and mdhd boxes in your MP4 stream interceptor to get the correctly timed MP4 stream.
Doing that, the latency can be squeezed to under 20 ms. This is barely recognizable when you see the output side by side on localhost in chains such as Unity (research) -> NvEnc -> MFTranscodeContainerType_FMPEG4 -> WebSocket -> Chrome Media Source Extensions display.
Note that MFTranscodeContainerType_FMPEG4 still introduces 1 frame delay (1st frame in, no output, 2nd frame in, 1st frame out, ...), hence the 20 ms latency at 60 FPS. The only solution to that seems to be writing own FMPEG4 containerizer. But that is order of magnitude more complex than intercepting of Media Foundation's MP4 streams.
The problem was solved by following roman-r's advise, and modifying the generated MPEG4 stream. See answer above.
Another way to do this is again using the same code #Fredrik mentioned but I write my own IMFByteStream and and I check the chunks written to the IMFByteStream.
FFMpeg writes the atoms almost once at a time. So you can check the atom name and do the mods. It is the same thing. I wish there was an MSE compliant windows sinker.
Is there one that can generate .ts files for HLS?

Are there tools to repair OPUS file stream?

Opus audio codec format [and OGG as its container] started to gain traction, which is great. However it also means that everyday problems surface. Mine today is a corrupted file, and the question is whether there are tools to repair the stream.
The actual stream is possibly created by a broken encoder, namely:
Encoded with Gagravarr.org Java Vorbis Tools v0.8 20160217
and opusinfo correctly complains:
**WARNING: Invalid header page in stream 1, contains multiple packets**
New logical stream (#1, serial: 000052c9): type opus
**WARNING: Implausibly low preskip in Opus stream (1)**
Encoded with Gagravarr.org Java Vorbis Tools v0.8 20160217
Opus stream 1:
Pre-skip: 0
Playback gain: 0 dB
Channels: 1
Original sample rate: 16000Hz
Packet duration: 60.0ms (max), 60.0ms (avg), 60.0ms (min)
Page duration: 60.0ms (max), 60.0ms (avg), 60.0ms (min)
Total data length: 12231171 bytes (overhead: 13.9%)
Playback length: 60m:49.920s
Average bitrate: 26.81 kb/s, w/o overhead: 23.08 kb/s
Logical stream 1 ended
which is fine. But while the data is visibly there, opusdec decides to force correctness:
Decoding to 48000 Hz (1 channel)
Extra packets on initial header page. Invalid stream.
and absolutely reject to process the file, which isn't very helpful.
My question is: how the user is supposed to salvage/repair/fix a broken, but salvageable file? Is there a (FOSS) code for that? Is there some hidden functionality in xiph stuff?
Please report the warnings and error to encoder's upstream. The point of opusinfo and opusdec complaining to get people to fix their software and stop making broken files. In addition to the extra packets and unset pre-skip value, 60 ms packet durations probably aren't what you want.
There's no fixup program in xiph's opus-tools package. You could try remuxing it with something like ffmpeg in.opus -acodec copy out.opus which might correct the incorrect paging. There's no good way to correct the missing pre-skip value.

What's difference result when I setup -frag and -dash with MP4Box?

I have read MP4Box Doc about Mpeg-Dash, but I don't clearly understand about "MP4Box -dash 10000 -frag 2000 largeFile.mp4" and MP4Box -dash 10000 -frag 1000 largeFile.mp4. When I open the *.mpd File I found the duration of SegmentList is 10023(about 10 sec). If the -frag 2000 or 1000 is no used?
I'm designing a HTML5 Video Player(like this sample), and I using MP4Box tool to create DASH Video.
But I don't clearly understand what's the difference when I convert my video with -frag 2000 and 1000. For example: I don't the mean about my video with 10 second segments and 1 second fragments. maybe My Video Player do not need to set this option?
GPAC contributor here. It is difficult to help you without a full example. I stronlgy recommend to describe bugs on our bug tracker (https://github.com/gpac/gpac/issues).
When I open the *.mpd File I found the duration of SegmentList is 10023(about 10 sec). If the -frag 2000 or 1000 is no used?
Three points:
1) You probably get 10023 ms (instead of 10000 ms) because you may use an old version of MP4Box. Please consider using the latest version.
2) Fragments are an MP4 feature and is not seen at the MPEG-DASH level. Segments is also an MP4 feature (basically a segment contains fragments) that is seen by MPEG-DASH. Therefore you can't see it in the MP4 but it may have consequences on your playback.
3) The blog article you mention (http://gpac.io/2011/02/02/mp4box-fragmentation-segmentation-splitting-and-interleaving/ contains all the information you need. If you think we can improve it, please leave a message there. Thanks!

Recording RaspberryPi camera to rosbag

I'm trying to configure a RaspberryPi2 to record video data from the camera module to a rosbag. To get the camera working with ROS, I used code I found here: https://github.com/fpasteau/raspicam_node.
This works fine, but I have a problem capturing the data to a rosbag. When capturing in raw mode at a high frame rate, it captures smoothly for a few seconds, then freezes for a few seconds, then captures smoothly for a few seconds, then freezes, ...
For instance, I tried capturing a file with 640x480#30FPS and this is what rosbag info yields:
duration: 2:51s (171s)
size: 2.9 GB
messages: 5049
compression: none [2504/2504 chunks]
types: rosgraph_msgs/Log [acffd30cd6b6de30f120938c17c593fb]
sensor_msgs/CameraInfo [c9a58c1b0b154e0e6da7578cb991d214]
sensor_msgs/Image [060021388200f6f0f447d0fcd9c64743]
topics: /camera/camera_info 2505 msgs : sensor_msgs/CameraInfo
/camera/image 2504 msgs : sensor_msgs/Image
/rosout 22 msgs : rosgraph_msgs/Log (2 connections)
/rosout_agg 18 msgs : rosgraph_msgs/Log
So if we have 171 seconds of video, at 90FPS, that should give 15390 messages, we only got 2504, which is about 14FPS. The file itself is 2.9GB in size. This means it had an average writing speed of ~17.5MB/s. Eventually I found a command to test the write speed of the SD card (dd if=/dev/zero of=~/test.tmp bs=500K count=1024), which says my writing speed is about ~19MB/s on average.
So my questions are:
If the SD writing speed is causing the problem, how come the RaspberryPi can't utilise the full 90MB/s?
Can I tune the RaspberryPi to write quicker to the SD card?
I thought about getting a BananaPi, which comes with SATA, so I could connect a SATA drive and shouldn't run into any write speed issues. Before making that investment, does anyone have experience with BananaPis? I saw a test here: http://314256.blogspot.co.uk/2014/11/banana-pi-sata-disk-throughput-test.html, which looks like the BananaPi should be able to handle it.
Any other ideas how to make it work on the RaspberryPi?
It looks like the raspicam_node publishes images with bgra8 encoding (raspicam_raw_node.cpp#L266), so we need to store 4*640*480*30 Bytes/second = 36.86 MB/s.
However ~18 MB/s seems to be pretty much the limit on a Raspberry 2 (microSD card performance comparison).
Instead of trying to save all the raw data, have rosbag store the sensor_msgs/CompressedImage from the /camera/image/compressed topic. You can tune the <base_topic>/compressed/jpeg_quality parameter (see compressed_image_transport's dynamic reconfigure parameters), but with the default of 80 you should get around 30:1 compression ratio, i.e. 1.23 MB/s.
The Raspberry should be able to handle this easily. Given the image quality of the tiny Raspberry camera, you will probably not even perceive any difference in quality.

x264 IDR access unit with a SPS and a PPS

I am trying to encode video in h.264 that when split with Apples HTTP Live Streaming tools media file segmenter will pass the media file validator I am getting two errors on the split MPEG-TS file
WARNING: Media segment contains a video track but does not contain any IDR access unit with a SPS and a PPS.
WARNING: 7 samples (17.073 %) do not have timestamps in track 257 (avc1).
After hours of research I think the "IDR" warning relates to not having keyframes in the right place on the segmented MPEG-TS file so in my ffmpeg command I set -keyint_min 1 to ensure keyframes where at every frame, but this didn't work.
Although it would be great to get an answer, if anyone can shed any light on what a "IDR access unit with a SPS and a PPS" is or what the timestamps warning means I would be very grateful, thanks.
Fix can be found on this thread https://devforums.apple.com/thread/45830?tstart=15