I've been using the FFmpeg libraries to read and write media files using the C API.
So far, reading seems to be pretty straightforward. I am able to read frames which I can then process, convert to RGB, process, and then convert back to YUV420 to be encoded.
The encoded files play back with VLC media player fine, and Windows Media Player if I have a codec pack installed. However, they do behave strangely: the stock Windows 10 player won't play them, same for Adobe Premiere. Also thumbnailers don't work on it.
Basically it seems like nothing other than VLC or FFmpeg itself can play/process the file. I have seen this with both MP4 and MKV, so it is not a format-specific issue.
The problems go away once you remux the file with FFmpeg, for example "ffmpeg -i input.mkv -c copy output.mkv". Everything can play the file correctly.
Also, the "remuxing.c" sample from the official samples works as well, with the same library version and compilers that I'm using (Visual Studio 2017, FFmpeg compiled with MinGW). It will fix the file and make it playable in all software.
I'm not sure what could be causing this. I also don't understand what the remuxing "fixed". It must be a container issue as the frames aren't touched by remuxing.
I have analysed the output MKVs with FFprobe -show_packets. It seems to have budged the packet timestamps a little constant factor, and the output stream now has
is_avc=true and nal_length_size=4 instead of is_avc=false and nal_length_size=0, but apart from that the files are identical.
Now here's the output of FFprobe with the 3 last test packets, stream info and format info for both streams. As you can see, they are identical except for a couple of field. But something in here must have been "fixed" during remuxing to make it work.
[PACKET]
codec_type=video
stream_index=0
pts=59050
pts_time=59.050000
dts=58890
dts_time=58.890000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=427
pos=277358
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=58970
pts_time=58.970000
dts=58970
dts_time=58.970000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=205
pos=277792
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=59130
pts_time=59.130000
dts=59050
dts_time=59.050000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=268
pos=278004
flags=__
[/PACKET]
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=Main
codec_type=video
codec_time_base=1/2000
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
width=720
height=576
coded_width=720
coded_height=576
has_b_frames=2
sample_aspect_ratio=N/A
display_aspect_ratio=N/A
pix_fmt=yuv420p
level=50
color_range=unknown
color_space=unknown
color_transfer=unknown
color_primaries=unknown
chroma_location=left
field_order=progressive
timecode=N/A
refs=1
is_avc=false
nal_length_size=0
id=N/A
r_frame_rate=299/12
avg_frame_rate=1000/1
time_base=1/1000
start_pts=0
start_time=0.000000
duration_ts=N/A
duration=N/A
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=737
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
TAG:DURATION=00:00:59.211000000
[/STREAM]
[FORMAT]
filename=testEncLeft.mkv
nb_streams=1
nb_programs=0
format_name=matroska,webm
format_long_name=Matroska / WebM
start_time=0.000000
duration=59.211000
size=278349
bit_rate=37607
probe_score=100
TAG:COMMENT=Slickline Player Export
TAG:ENCODER=Lavf57.83.100
[/FORMAT]
And the info after remuxing, which works:
[PACKET]
codec_type=video
stream_index=0
pts=59050
pts_time=59.050000
dts=58890
dts_time=58.890000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=427
pos=277418
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=58970
pts_time=58.970000
dts=58970
dts_time=58.970000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=205
pos=277852
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=59130
pts_time=59.130000
dts=59050
dts_time=59.050000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=268
pos=278064
flags=__
[/PACKET]
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=Main
codec_type=video
codec_time_base=1/2000
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
width=720
height=576
coded_width=720
coded_height=576
has_b_frames=2
sample_aspect_ratio=N/A
display_aspect_ratio=N/A
pix_fmt=yuv420p
level=50
color_range=unknown
color_space=unknown
color_transfer=unknown
color_primaries=unknown
chroma_location=left
field_order=progressive
timecode=N/A
refs=1
is_avc=true
nal_length_size=4
id=N/A
r_frame_rate=299/12
avg_frame_rate=1000/1
time_base=1/1000
start_pts=0
start_time=0.000000
duration_ts=N/A
duration=N/A
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=737
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
TAG:DURATION=00:00:59.212000000
[/STREAM]
[FORMAT]
filename=fixedLeft.mkv
nb_streams=1
nb_programs=0
format_name=matroska,webm
format_long_name=Matroska / WebM
start_time=0.000000
duration=59.212000
size=278409
bit_rate=37615
probe_score=100
TAG:COMMENT=Slickline Player Export
TAG:ENCODER=Lavf58.12.100
[/FORMAT]
Here is how I'm setting up the output context, for reference: it's pretty standard, following the sample code.
int ret;
avformat_alloc_output_context2(&outputFormatCtx, nullptr, nullptr, outFilePath.c_str());
av_dict_set(&outputFormatCtx->metadata, "comment", "FFmpeg Export", 0);
if (!outputFormatCtx)
{
LOG_AND_THROW("Could not allocate output context");
}
outputVideoStream = avformat_new_stream(outputFormatCtx, nullptr);
outputVideoStream->time_base = AVRational{ 1, AV_TIME_BASE }; // Stream timebase will be used by codec
if (!outputVideoStream)
{
LOG_AND_THROW("Failed allocating output stream");
}
// defaults to "libx264"
AVCodec *outCodec = avcodec_find_encoder_by_name(selectedCodecName.c_str());
if (!outCodec)
{
LOG_AND_THROW("Failed finding output codec");
}
AVDictionary *opts = nullptr;
if (selectedCodecName == "libx264")
{
opts = getX264CodecOptions();
}
encoderCtx = avcodec_alloc_context3(outCodec);
if (!encoderCtx)
{
LOG_AND_THROW("Failed to allocate the encoder context");
}
encoderCtx->width = width;
encoderCtx->height = height;
encoderCtx->pix_fmt = AV_PIX_FMT_YUV420P;
// time base for the frames we will provide to the encoder
encoderCtx->time_base = AVRational{ 1, AV_TIME_BASE };
// convert framerate from double to rational
encoderCtx->framerate = AVRational{ (int)(frameRate * AV_TIME_BASE), AV_TIME_BASE};
// Match encoderCtx time base for the stream
outputVideoStream->time_base = encoderCtx->time_base;
ret = avcodec_open2(encoderCtx, outCodec, &opts);
if (ret < 0)
{
LOG_AND_THROW_PARAM("Cannot open video encoder for stream: %d", ret);
}
// Fill in some params for MP4 stream, details about encoder
ret = avcodec_parameters_from_context(outputVideoStream->codecpar, encoderCtx);
if (ret < 0)
{
LOG_AND_THROW_PARAM("Failed to copy encoder parameters to output stream: %d", ret);
}
if (outputFormatCtx->oformat->flags & AVFMT_GLOBALHEADER)
{
encoderCtx->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
}
av_dump_format(outputFormatCtx, 0, filePath.c_str(), 1);
// End of encoder settings, setting up MP4
if (!(outputFormatCtx->oformat->flags & AVFMT_NOFILE))
{
ret = avio_open(&outputFormatCtx->pb, outFilePath.c_str(), AVIO_FLAG_WRITE);
if (ret < 0)
{
LOG_AND_THROW_PARAMSTR("Could not open output file '%s'", outFilePath.c_str());
}
}
ret = avformat_write_header(outputFormatCtx, nullptr);
if (ret < 0)
{
LOG_AND_THROW_PARAM("Error occurred when opening output file for writing: %d", ret);
}
Can anyone help me figure out why the container is not playing properly?
Thanks in advance.
-James
Related
I'm trying to run a gstreamer-1.0 python script (see below, works fine on an ubuntu laptop) on a Raspberry Pi. However, it seems to be unable to decode the stream:
0:00:11.237415476 9605 0xafb0cc60 ERROR vaapidecode ../../../gst/vaapi/gstvaapidecode.c:1025:gst_vaapidecode_ensure_allowed_caps: failed to retrieve VA display
0:00:11.239490439 9605 0xafb0cc60 WARN decodebin gstdecodebin2.c:2087:connect_pad:<decodebin0> Link failed on pad vaapidecode0:sink
0:00:11.244097356 9605 0xafb0cc60 WARN uridecodebin gsturidecodebin.c:939:unknown_type_cb:<decoder> warning: No decoder available for type 'video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)426, height=(int)240, framerate=(fraction)30/1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, level=(string)2.1, profile=(string)main'.
I searched for information about the error (the results didn't enlighten me) and the warnings but couldn't really find much advice other than to install gstreamer1.0-libav which had already been installed. Consequently the decoder should be available.
What might be wrong here and how do I fix it?
This is the script:
#!/usr/bin/env python
# GST_DEBUG=3,python:5,gnl*:5 python 01_parsepipeline.py http://www.ustream.tv/channel/17074538 worst novideo.png
from __future__ import print_function
import sys
import gi
from gi.repository import GObject as gobject, Gst as gst
from livestreamer import Livestreamer, StreamError, PluginError, NoPluginError
import cv2
import numpy
def exit(msg):
print(msg, file=sys.stderr)
sys.exit()
class Player(object):
def __init__(self):
self.fd = None
self.mainloop = gobject.MainLoop()
# This creates a playbin pipeline and using the appsrc source
# we can feed it our stream data
self.pipeline = gst.parse_launch('uridecodebin uri=appsrc:// name=decoder \
decoder. ! videorate ! video/x-raw,framerate=1/1 ! tee name=t \
t. ! queue ! videoconvert ! video/x-raw,format=RGB ! appsink name=appsink \
decoder. ! queue ! audioconvert ! fakesink')
if self.pipeline is None:
exit("couldn't build pipeline")
decoder = self.pipeline.get_by_name('decoder')
if decoder is None:
exit("couldn't get decoder")
decoder.connect("source-setup", self.on_source_setup)
vsink = self.pipeline.get_by_name('appsink')
if vsink is None:
exit("couldn't get sink")
vsink.set_property("emit-signals", True)
vsink.set_property("max-buffers", 1)
vsink.connect("new-sample", self.on_new_sample)
# Creates a bus and set callbacks to receive errors
self.bus = self.pipeline.get_bus()
self.bus.add_signal_watch()
self.bus.connect("message::eos", self.on_eos)
self.bus.connect("message::error", self.on_error)
def on_new_sample(self, sink):
sample = sink.emit("pull-sample")
buf = sample.get_buffer()
caps = sample.get_caps()
height = caps.get_structure(0).get_value('height')
width = caps.get_structure(0).get_value('width')
(result, mapinfo) = buf.map(gst.MapFlags.READ)
if result == True:
arr = numpy.ndarray(
(height,
width,
3),
buffer=buf.extract_dup(0, buf.get_size()),
dtype=numpy.uint8)
resized_refimage = cv2.resize(refArray, (width, height))
diff = cv2.norm(arr, resized_refimage, cv2.NORM_L2)
buf.unmap(mapinfo)
s = "diff = " + str(diff)
print(s)
return gst.FlowReturn.OK
def exit(self, msg):
self.stop()
exit(msg)
def stop(self):
# Stop playback and exit mainloop
self.pipeline.set_state(gst.State.NULL)
self.mainloop.quit()
# Close the stream
if self.fd:
self.fd.close()
def play(self, stream):
# Attempt to open the stream
try:
self.fd = stream.open()
except StreamError as err:
self.exit("Failed to open stream: {0}".format(err))
# Start playback
self.pipeline.set_state(gst.State.PLAYING)
self.mainloop.run()
def on_source_setup(self, element, source):
# When this callback is called the appsrc expects
# us to feed it more data
print("source setup")
source.connect("need-data", self.on_source_need_data)
print("done")
def on_pad_added(self, element, pad):
string = pad.query_caps(None).to_string()
print(string)
if string.startswith('video/'):
#type = pad.get_caps()[0].get_name()
#print(type)
#if type.startswith("video"):
pad.link(self.vconverter.get_static_pad("sink"))
def on_source_need_data(self, source, length):
# Attempt to read data from the stream
try:
data = self.fd.read(length)
except IOError as err:
self.exit("Failed to read data from stream: {0}".format(err))
# If data is empty it's the end of stream
if not data:
source.emit("end-of-stream")
return
# Convert the Python bytes into a GStreamer Buffer
# and then push it to the appsrc
buf = gst.Buffer.new_wrapped(data)
source.emit("push-buffer", buf)
#print("sent " + str(length) + " bytes")
def on_eos(self, bus, msg):
# Stop playback on end of stream
self.stop()
def on_error(self, bus, msg):
# Print error message and exit on error
error = msg.parse_error()[1]
self.exit(error)
def main():
if len(sys.argv) < 4:
exit("Usage: {0} <url> <quality> <reference png image path>".format(sys.argv[0]))
# Initialize and check GStreamer version
gi.require_version("Gst", "1.0")
gobject.threads_init()
gst.init(None)
# Collect arguments
url = sys.argv[1]
quality = sys.argv[2]
refImage = sys.argv[3]
global refArray
image = cv2.imread(refImage)
refArray = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# refArray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
refArray = cv2.blur(refArray, (3,3))
# Create the Livestreamer session
livestreamer = Livestreamer()
# Enable logging
livestreamer.set_loglevel("debug")
livestreamer.set_logoutput(sys.stdout)
# Attempt to fetch streams
try:
streams = livestreamer.streams(url)
except NoPluginError:
exit("Livestreamer is unable to handle the URL '{0}'".format(url))
except PluginError as err:
exit("Plugin error: {0}".format(err))
if not streams:
exit("No streams found on URL '{0}'".format(url))
# Look for specified stream
if quality not in streams:
exit("Unable to find '{0}' stream on URL '{1}'".format(quality, url))
# We found the stream
stream = streams[quality]
# Create the player and start playback
player = Player()
# Blocks until playback is done
player.play(stream)
if __name__ == "__main__":
main()
The pipeline you've set up appears to be trying to invoke a vaapi decoder to hardware-decode h264 - vaapi isn't available on the raspberry pi, since the closed source X server doesn't implement it. You may be able to use omxh264dec from the gstreamer1.0-omx package instead.
If not, you could do software h264 decoding, but that will be slower (maybe unacceptably slow on a raspberry pi).
I'm using logstash(2.3.2) to read gz file by using gzip_lines codec.
The log file example (sample.log) is
127.0.0.2 - - [11/Dec/2013:00:01:45 -0800] "GET /xampp/status.php HTTP/1.1" 200 3891 "http://cadenza/xampp/navi.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0"
The command I used to append to a gz file is:
cat sample.log | gzip -c >> s.gz
The logstash.conf is
input {
file {
path => "./logstash-2.3.2/bin/s.gz"
codec => gzip_lines { charset => "ISO-8859-1"}
}
}
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
#match => { "message" => "message: %{GREEDYDATA}" }
}
#date {
# match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
#}
}
output {
stdout { codec => rubydebug }
}
I have installed gzip_line plugin with bin/logstash-plugin install logstash-codec-gzip_lines
start logstash with ./logstash -f logstash.conf
When I feed s.gz with
cat sample.log | gzip -c >> s.gz
I expect that the console prints the data. but there is nothing print out.
I have tried it on mac and ubuntu, and get same result.
Is anything wrong with my code?
I checked the code for gzip_lines and it seemed obvious to me that this plugin is not working. At least for version 2.3.2. May be it is outdated. Because it does not implement the methods specified here:
https://www.elastic.co/guide/en/logstash/2.3/_how_to_write_a_logstash_codec_plugin.html
So current internal working is like that:
file input plugin reads file line by line and send it to codec.
gzip_lines codec tryies to create a new GzipReader object with GzipReader.new(io)
It then go through the reader line by line to create events.
Because you specify a gzip file, file input plugin tries to read gzip file as a regular file and sends lines to codec. Codec tries to create a GzipReader with that string and it fails.
You can modify it to work like that:
Create a file that contains list of gzip files:
-- list.txt
/path/to/gzip/s.gz
Give it to file input plugin:
file {
path => "/path/to/list/list.txt"
codec => gzip_lines { charset => "ISO-8859-1"}
}
Changes are:
Open vendor/bundle/jruby/1.9/gems/logstash-codec-gzip_lines-2.0.4/lib/logstash/codecs/gzip_lines.r file. Add register method:
public
def register
#converter = LogStash::Util::Charset.new(#charset)
#converter.logger = #logger
end
And in method decode change:
#decoder = Zlib::GzipReader.new(data)
as
#decoder = Zlib::GzipReader.open(data)
The disadvantage of this approach is it wont tail your gzip file but the list file. So you will need to create a new gzip file and append it to list.
I had a variant of this problem where I needed to decode bytes in a files to an intermediate string to prepare for a process input that only accepts strings.
The fact that encoding / decoding issues were ignored in Pyhton 2 is actually really bad IMHO. you may end up with various corrupt data problems especially if you needed to re-encode the string back into data.
using ISO-8859-1 works for both gz and text files alike. while utf-8 only worked for text files. I haven't tried it for png's yet.
Here's an example of what worked for me
data = os.read(src, bytes_needed)
chunk += codecs.decode(data,'ISO-8859-1')
# do the needful with the chunk....
I have been writing a lcd kernel driver for a LCD module. All was going well, I can write to the display, create a /dev/lcd node that I can write into and it will display the results on the screen. I thought using the llseek fops callback to position the cursor on the lcd would be good, this way I could use rewind fseek etc. However it is not working as I expected, below is a summary of what I am seeing:
The relevant lines of code from the driver side are:
loff_t lcd_llseek(struct file *filp, loff_t off, int whence)
{
switch (whence) {
case 0: // SEEK_SET
if (off > 4*LINE_LENGTH || off < 0) {
printk(KERN_ERR "unsupported SEEK_SET offset %llx\n", off);
return -EINVAL;
}
lcd_gotoxy(&lcd, off, 0, WHENCE_ABS);
break;
case 1: // SEEK_CUR
if (off > 4*LINE_LENGTH || off < -4*LINE_LENGTH) {
printk(KERN_ERR "unsupported SEEK_CUR offset %llx\n", off);
return -EINVAL;
}
lcd_gotoxy(&lcd, off, 0, WHENCE_REL);
break;
case 2: // SEEK_END (not supported, hence fall though)
default:
// how did we get here !
printk(KERN_ERR "unsupported seek operation\n");
return -EINVAL;
}
filp->f_pos = lcd.pos;
printk(KERN_INFO "lcd_llseek complete\n");
return lcd.pos;
}
int lcd_open(struct inode *inode, struct file *filp)
{
if (!atomic_dec_and_test(&lcd_available)) {
atomic_inc(&lcd_available);
return -EBUSY; // already open
}
return 0;
}
static struct file_operations fops = {
.owner = THIS_MODULE,
.write = lcd_write,
.llseek = lcd_llseek,
.open = lcd_open,
.release = lcd_release,
};
int lcd_init(void)
{
...
// allocate a new dev number (this can be dynamic or
// static if passed in as a module param)
if (major) {
devno = MKDEV(major, 0);
ret = register_chrdev_region(devno, 1, MODULE_NAME);
} else {
ret = alloc_chrdev_region(&devno, 0, 1, MODULE_NAME);
major = MAJOR(devno);
}
if (ret < 0) {
printk(KERN_ERR "alloc_chrdev_region failed\n");
goto fail;
}
// create a dummy class for the lcd
cl = class_create(THIS_MODULE, "lcd");
if (IS_ERR(cl)) {
printk(KERN_ERR "class_simple_create for class lcd failed\n");
goto fail1;
}
// create cdev interface
cdev_init(&cdev, &fops);
cdev.owner = THIS_MODULE;
ret = cdev_add(&cdev, devno, 1);
if (ret) {
printk(KERN_ERR "cdev_add failed\n");
goto fail2;
}
// create /sys/lcd/fplcd/dev so udev will add our device to /dev/fplcd
device = device_create(cl, NULL, devno, NULL, "lcd");
if (IS_ERR(device)) {
printk(KERN_ERR "device_create for fplcd failed\n");
goto fail3;
}
...
}
To test the lseek call I have the following unit test:
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#define log(msg, ...) fprintf(stdout, __FILE__ ":%s():[%d]:" msg, __func__, __LINE__, __VA_ARGS__)
int lcd;
void test(void)
{
int k;
// a lot of hello's
log("hello world test\n",1);
if (lseek(lcd, 0, SEEK_CUR) == -1) {
log("failed to seek\n", 1);
}
}
int main(int argc, char **argv)
{
lcd = open("/dev/lcd", O_WRONLY);
if (lcd == -1) {
perror("unable to open lcd");
exit(EXIT_FAILURE);
}
test();
close(lcd);
return 0;
}
The files are cross compiled like so:
~/Workspace/ts4x00/lcd-module$ cat Makefile
obj-m += fls_lcd.o
all:
make -C $(KPATH) M=$(PWD) modules
$(CROSS_COMPILE)gcc -g -fPIC $(CFLAGS) lcd_unit_test.c -o lcd_unit_test
clean:
make -C $(KPATH) M=$(PWD) clean
rm -rf lcd_unit_test
~/Workspace/ts4x00/lcd-module$ make CFLAGS+="-march=armv4 -ffunction-sections -fdata-sections"
make -C ~/Workspace/ts4x00/linux-2.6.29 M=~/Workspace/ts4x00/lcd-module modules
make[1]: Entering directory `~/Workspace/ts4x00/linux-2.6.29'
CC [M] ~/Workspace/ts4x00/lcd-module/fls_lcd.o
~/Workspace/ts4x00/lcd-module/fls_lcd.c:443: warning: 'lcd_entry_mode' defined but not used
Building modules, stage 2.
MODPOST 1 modules
CC ~/Workspace/ts4x00/lcd-module/fls_lcd.mod.o
LD [M] ~/Workspace/ts4x00/lcd-module/fls_lcd.ko
make[1]: Leaving directory `~/Workspace/ts4x00/linux-2.6.29'
~/Workspace/ts4x00/arm-2008q3/bin/arm-none-linux-gnueabi-gcc -g -fPIC -march=armv4 -ffunction-sections -fdata-sections lcd_unit_test.c -o lcd_unit_test
This is the output of running the driver with the unit test is:
root#ts4700:~/devel# insmod ./fls_lcd.ko
root#ts4700:~/devel# ./lcd_unit_test
lcd_unit_test.c:test():[61]:hello world test
lcd_unit_test.c:test():[63]:failed to seek
root#ts4700:~/devel# dmesg
FLS LCD driver started
unsupported SEEK_SET offset bf0a573c
I cannot figure out why the parameters are being mucked up so badly on the kernel side, I tried to SEEK_CUR to position 0 and in the driver I get a SEEK_SET (no matter what I put in the unit test) and a crazy big number for off?
Does anyone know what is going on please ?
btw I am compiling for kernel 2.6.29 on a arm dev kit
OK sorry guys after trying to debug this all last night it comes down to compiling against the wrong kernel (I had KPATH left to a different config of the kernel than was on the sdcard)
sorry for wasting everyones time, but hopefully if someone is seeing what looks like a crazy stack in their kernel driver this might set them straight.
oh and thanks for all the help :)
Trying to create a web-front end for a Python3 backed application. The application will require bi-directional streaming which sounded like a good opportunity to look into websockets.
My first inclination was to use something already existing, and the example applications from mod-pywebsocket have proved valuable. Unfortunately their API doesn't appear to easily lend itself to extension, and it is Python2.
Looking around the blogosphere many people have written their own websocket server for earlier versions of the websocket protocol, most don't implement the security key hash so dont' work.
Reading RFC 6455 I decided to take a stab at it myself and came up with the following:
#!/usr/bin/env python3
"""
A partial implementation of RFC 6455
http://tools.ietf.org/pdf/rfc6455.pdf
Brian Thorne 2012
"""
import socket
import threading
import time
import base64
import hashlib
def calculate_websocket_hash(key):
magic_websocket_string = b"258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
result_string = key + magic_websocket_string
sha1_digest = hashlib.sha1(result_string).digest()
response_data = base64.encodestring(sha1_digest)
response_string = response_data.decode('utf8')
return response_string
def is_bit_set(int_type, offset):
mask = 1 << offset
return not 0 == (int_type & mask)
def set_bit(int_type, offset):
return int_type | (1 << offset)
def bytes_to_int(data):
# note big-endian is the standard network byte order
return int.from_bytes(data, byteorder='big')
def pack(data):
"""pack bytes for sending to client"""
frame_head = bytearray(2)
# set final fragment
frame_head[0] = set_bit(frame_head[0], 7)
# set opcode 1 = text
frame_head[0] = set_bit(frame_head[0], 0)
# payload length
assert len(data) < 126, "haven't implemented that yet"
frame_head[1] = len(data)
# add data
frame = frame_head + data.encode('utf-8')
print(list(hex(b) for b in frame))
return frame
def receive(s):
"""receive data from client"""
# read the first two bytes
frame_head = s.recv(2)
# very first bit indicates if this is the final fragment
print("final fragment: ", is_bit_set(frame_head[0], 7))
# bits 4-7 are the opcode (0x01 -> text)
print("opcode: ", frame_head[0] & 0x0f)
# mask bit, from client will ALWAYS be 1
assert is_bit_set(frame_head[1], 7)
# length of payload
# 7 bits, or 7 bits + 16 bits, or 7 bits + 64 bits
payload_length = frame_head[1] & 0x7F
if payload_length == 126:
raw = s.recv(2)
payload_length = bytes_to_int(raw)
elif payload_length == 127:
raw = s.recv(8)
payload_length = bytes_to_int(raw)
print('Payload is {} bytes'.format(payload_length))
"""masking key
All frames sent from the client to the server are masked by a
32-bit nounce value that is contained within the frame
"""
masking_key = s.recv(4)
print("mask: ", masking_key, bytes_to_int(masking_key))
# finally get the payload data:
masked_data_in = s.recv(payload_length)
data = bytearray(payload_length)
# The ith byte is the XOR of byte i of the data with
# masking_key[i % 4]
for i, b in enumerate(masked_data_in):
data[i] = b ^ masking_key[i%4]
return data
def handle(s):
client_request = s.recv(4096)
# get to the key
for line in client_request.splitlines():
if b'Sec-WebSocket-Key:' in line:
key = line.split(b': ')[1]
break
response_string = calculate_websocket_hash(key)
header = '''HTTP/1.1 101 Switching Protocols\r
Upgrade: websocket\r
Connection: Upgrade\r
Sec-WebSocket-Accept: {}\r
\r
'''.format(response_string)
s.send(header.encode())
# this works
print(receive(s))
# this doesn't
s.send(pack('Hello'))
s.close()
s = socket.socket( socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('', 9876))
s.listen(1)
while True:
t,_ = s.accept()
threading.Thread(target=handle, args = (t,)).start()
Using this basic test page (which works with mod-pywebsocket):
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Web Socket Example</title>
<meta charset="UTF-8">
</head>
<body>
<div id="serveroutput"></div>
<form id="form">
<input type="text" value="Hello World!" id="msg" />
<input type="submit" value="Send" onclick="sendMsg()" />
</form>
<script>
var form = document.getElementById('form');
var msg = document.getElementById('msg');
var output = document.getElementById('serveroutput');
var s = new WebSocket("ws://"+window.location.hostname+":9876");
s.onopen = function(e) {
console.log("opened");
out('Connected.');
}
s.onclose = function(e) {
console.log("closed");
out('Connection closed.');
}
s.onmessage = function(e) {
console.log("got: " + e.data);
out(e.data);
}
form.onsubmit = function(e) {
e.preventDefault();
msg.value = '';
window.scrollTop = window.scrollHeight;
}
function sendMsg() {
s.send(msg.value);
}
function out(text) {
var el = document.createElement('p');
el.innerHTML = text;
output.appendChild(el);
}
msg.focus();
</script>
</body>
</html>
This receives data and demasks it correctly, but I can't get the transmit path to work.
As a test to write "Hello" to the socket, the program above calculates the bytes to be written to the socket as:
['0x81', '0x5', '0x48', '0x65', '0x6c', '0x6c', '0x6f']
Which match the hex values given in section 5.7 of the RFC. Unfortunately the frame never shows up in Chrome's Developer Tools.
Any idea what I'm missing? Or a currently working Python3 websocket example?
When I try talking to your python code from Safari 6.0.1 on Lion I get
Unexpected LF in Value at ...
in the Javascript console. I also get an IndexError exception from the Python code.
When I talk to your python code from Chrome Version 24.0.1290.1 dev on Lion I don't get any Javascript errors. In your javascript the onopen() and onclose() methods are called, but not the onmessage(). The python code doesn't throw any exceptions and appears to have receive message and sent it's response, i.e exactly the behavior your seeing.
Since Safari didn't like the trailing LF in your header I tried removing it, i.e
header = '''HTTP/1.1 101 Switching Protocols\r
Upgrade: websocket\r
Connection: Upgrade\r
Sec-WebSocket-Accept: {}\r
'''.format(response_string)
When I make this change Chrome is able to see your response message i.e
got: Hello
shows up in the javascript console.
Safari still doesn't work. Now it raise's a different issue when I attempt to send a message.
websocket.html:36 INVALID_STATE_ERR: DOM Exception 11: An attempt was made to use an object that is not, or is no longer, usable.
None of the javascript websocket event handlers ever fire and I'm still seeing the IndexError exception from python.
In conclusion. Your Python code wasn't working with Chrome because of an extra LF in your header response. There's still something else going on because the code the works with Chrome doesn't work with Safari.
Update
I've worked out the underlying issue and now have the example working in Safari and Chrome.
base64.encodestring() always adds a trailing \n to it's return. This is the source of the LF that Safari was complaining about.
call .strip() on the return value of calculate_websocket_hash and using your original header template works correctly on Safari and Chrome.
I am trying to decode h264 stream from rtsp server and render it on iPhone.
I found some libraries and read some articles about it.
Libraries are from dropCam for iPhone called RTSPClient and DecoderWrapper.
But I can not decode frame data with DecodeWrapper that using on ffmpeg.
Here are my code.
VideoViewer.m
- (void)didReceiveFrame:(NSData*)frameData presentationTime:(NSDate*)presentationTime
{
[VideoDecoder staticInitialize];
mConverter = [[VideoDecoder alloc] initWithCodec:kVCT_H264 colorSpace:kVCS_RGBA32 width:320 height:240 privateData:nil];
[mConverter decodeFrame:frameData];
if ([mConverter isFrameReady]) {
UIImage *imageData =[mConverter getDecodedFrame];
if (imageData) {
[mVideoView setImage:imageData];
NSLog(#"decoded!");
}
}
}
---VideoDecoder.m---
- (id)initWithCodec:(enum VideoCodecType)codecType
colorSpace:(enum VideoColorSpace)colorSpace
width:(int)width
height:(int)height
privateData:(NSData*)privateData {
if(self = [super init]) {
codec = avcodec_find_decoder(CODEC_ID_H264);
codecCtx = avcodec_alloc_context();
// Note: for H.264 RTSP streams, the width and height are usually not specified (width and height are 0).
// These fields will become filled in once the first frame is decoded and the SPS is processed.
codecCtx->width = width;
codecCtx->height = height;
codecCtx->extradata = av_malloc([privateData length]);
codecCtx->extradata_size = [privateData length];
[privateData getBytes:codecCtx->extradata length:codecCtx->extradata_size];
codecCtx->pix_fmt = PIX_FMT_RGBA;
#ifdef SHOW_DEBUG_MV
codecCtx->debug_mv = 0xFF;
#endif
srcFrame = avcodec_alloc_frame();
dstFrame = avcodec_alloc_frame();
int res = avcodec_open(codecCtx, codec);
if (res < 0)
{
NSLog(#"Failed to initialize decoder");
}
}
return self;
}
- (void)decodeFrame:(NSData*)frameData {
AVPacket packet = {0};
packet.data = (uint8_t*)[frameData bytes];
packet.size = [frameData length];
int frameFinished=0;
NSLog(#"Packet size===>%d",packet.size);
// Is this a packet from the video stream?
if(packet.stream_index==0)
{
int res = avcodec_decode_video2(codecCtx, srcFrame, &frameFinished, &packet);
NSLog(#"Res value===>%d",res);
NSLog(#"frame data===>%d",(int)srcFrame->data);
if (res < 0)
{
NSLog(#"Failed to decode frame");
}
}
else
{
NSLog(#"No video stream found");
}
// Need to delay initializing the output buffers because we don't know the dimensions until we decode the first frame.
if (!outputInit) {
if (codecCtx->width > 0 && codecCtx->height > 0) {
#ifdef _DEBUG
NSLog(#"Initializing decoder with frame size of: %dx%d", codecCtx->width, codecCtx->height);
#endif
outputBufLen = avpicture_get_size(PIX_FMT_RGBA, codecCtx->width, codecCtx->height);
outputBuf = av_malloc(outputBufLen);
avpicture_fill((AVPicture*)dstFrame, outputBuf, PIX_FMT_RGBA, codecCtx->width, codecCtx->height);
convertCtx = sws_getContext(codecCtx->width, codecCtx->height, codecCtx->pix_fmt, codecCtx->width,
codecCtx->height, PIX_FMT_RGBA, SWS_FAST_BILINEAR, NULL, NULL, NULL);
outputInit = YES;
frameFinished=1;
}
else {
NSLog(#"Could not get video output dimensions");
}
}
if (frameFinished)
frameReady = YES;
}
The console shows me as follows.
2011-05-16 20:16:04.223 RTSPTest1[41226:207] Packet size===>359
[h264 # 0x5815c00] no frame!
2011-05-16 20:16:04.223 RTSPTest1[41226:207] Res value===>-1
2011-05-16 20:16:04.224 RTSPTest1[41226:207] frame data===>101791200
2011-05-16 20:16:04.224 RTSPTest1[41226:207] Failed to decode frame
2011-05-16 20:16:04.225 RTSPTest1[41226:207] decoded!
2011-05-16 20:16:04.226 RTSPTest1[41226:207] Packet size===>424
[h264 # 0x5017c00] no frame!
2011-05-16 20:16:04.226 RTSPTest1[41226:207] Res value===>-1
2011-05-16 20:16:04.227 RTSPTest1[41226:207] frame data===>81002704
2011-05-16 20:16:04.227 RTSPTest1[41226:207] Failed to decode frame
2011-05-16 20:16:04.228 RTSPTest1[41226:207] decoded!
2011-05-16 20:16:04.229 RTSPTest1[41226:207] Packet size===>424
[h264 # 0x581d000] no frame!
2011-05-16 20:16:04.229 RTSPTest1[41226:207] Res value===>-1
2011-05-16 20:16:04.230 RTSPTest1[41226:207] frame data===>101791616
2011-05-16 20:16:04.230 RTSPTest1[41226:207] Failed to decode frame
2011-05-16 20:16:04.231 RTSPTest1[41226:207] decoded!
. . . . .
But the simulator shows nothing.
What's wrong with my code.
Help me solve this problem.
Thanks for your answers.
I've had a similar problem with H264 and FFmpeg.
My problem was that some devices are not sending the sequence (SPS) and picture parameter sets (PPS) with every frame, so I've needed to slightly modify my frame data.
Maybe this post will help: FFmpeg can't decode H264 stream/frame data