I try to make a sip video call using Pjsip/Pjsua on my raspberry pi 3.
Before coding, I'm using the main sample app to test different options. Everything seems to work (registering, audio calling,..) but when I try to start a video call, the programs stops with the following message :
pjsua-armv7l-unknown-linux-gnueabihf: ../src/pjmedia-videodev/v4l2_dev.c:737: vid4lin_stream_get_frame_mmap: Assertion `!"frame buffer is too small for v4l2"' failed.
I've searched a lot, including the source code :
/* get frame from mmap */
static pj_status_t vid4lin_stream_get_frame_mmap(vid4lin_stream *stream, pjmedia_frame *frame)
{
struct v4l2_buffer buf;
pj_time_val time;
pj_status_t status = PJ_SUCCESS;
pj_bzero(&buf, sizeof(buf));
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
status = xioctl(stream->fd, VIDIOC_DQBUF, &buf);
if (status != PJ_SUCCESS)
return status;
if (frame->size < buf.bytesused) {
/* supplied buffer is too small */
pj_assert(!"frame buffer is too small for v4l2");
status = PJ_ETOOSMALL;
goto on_return;
}
So I understand that the pjmedia_frame has a "size" inferior to the v4l2 buffer, resulting to my failure.
My question is simple : how can i change this setting ?
I tried evetything in the sample app : changing resolution, bitrate, fps,..
I found some ressources saying to change the h264 profile level.. ok, but where do i set it ? Is it within the v4l2 manager ? or directly in the app ? How can i do it ?
I played with different options in v4l2 to reduce the bitrate/resolution in order to have a small buffer, but still getting the same error.
At this point I'm completely clueless.
For info, I compiled PJsip using openh264 (no libx264) as suggested by PjSip.
Thanks for your help/ideas ;)
According to your question about profile level, you can try with:
const pj_str_t codec_id = {"H264", 4};
pjmedia_vid_codec_param param;
pj_status_t status;
status = pjsua_vid_codec_get_param(&codec_id, ¶m);
param.dec_fmtp.param[0].name = pj_str("profile-level-id");
param.dec_fmtp.param[0].val = pj_str("42e01f");
status = pjsua_vid_codec_set_param(&codec_id, ¶m);
do this anywhere after pjsua_start(). Last two characters in val property are profile level. Description of levels can be found here (link). More information about h264 profile here (link).
I'm not an expert of v4l2, but have little experience with encoding video on rpi3, and I suggest you to use FFmpeg instead of pure openh264, beacuse of support of hardware acceleration (link).
Good luck!
Related
I'm gonna to read/write under the modbus-tcp specification.
So, I'm trying to code the client and server in the linux environment.
(I would communicate with the windows program(as a client) using the modbus-tcp.)
but it doesn't work as I want, so I ask you here.
I'm testing the client code for linux as a client and the easymodbus as a server.
I used the libmodbus code.
I'd like to read coil(0x01) and write coil(0x05).
When the code is executed using the libmodbus, 'ff' is printed out from the Unit ID part.(according to the manual, 01 should be output for modbus-tcp.
I don't know why 'ff' is printed(photo attached).
Wrong result:
Expected result:
'[00] [00] .... [00]' ; Do you know where to control this part?
Do you have or do you know the sample code that implements the 'read/write' function using the libmodbus?
please let me know the information, if you know that.
ctx = modbus_new_tcp("192.168.0.99", 502);
modbus_set_debug(ctx, TRUE);
if (modbus_connect(ctx) == -1) {
fprintf(stderr, "Connection failed: %s\n",
modbus_strerror(errno));
modbus_free(ctx);
return -1;
}
tab_rq_bits = (uint8_t *) malloc(nb * sizeof(uint8_t));
memset(tab_rq_bits, 0, nb * sizeof(uint8_t));
tab_rp_bits = (uint8_t *) malloc(nb * sizeof(uint8_t));
memset(tab_rp_bits, 0, nb * sizeof(uint8_t));
nb_loop = nb_fail = 0;
/* WRITE BIT */
rc = modbus_write_bit(ctx, addr, tab_rq_bits[0]);
if (rc != 1) {
printf("ERROR modbus_write_bit (%d)\n", rc);
printf("Address = %d, value = %d\n", addr, tab_rq_bits[0]);
nb_fail++;
} else {
rc = modbus_read_bits(ctx, addr, 1, tab_rp_bits);
if (rc != 1 || tab_rq_bits[0] != tab_rp_bits[0]) {
printf("ERROR modbus_read_bits single (%d)\n", rc);
printf("address = %d\n", addr);
nb_fail++;
}
}
printf("Test: ");
if (nb_fail)
printf("%d FAILS\n", nb_fail);
else
printf("SUCCESS\n");
free(tab_rq_bits);
free(tab_rp_bits);
/* Close the connection */
modbus_close(ctx);
modbus_free(ctx);
return 0;
That FF you see right before the Modbus function is actually correct. Quoting the Modbus Implementation Guide, page 23:
On TCP/IP, the MODBUS server is addressed using its IP address; therefore, the
MODBUS Unit Identifier is useless. The value 0xFF has to be used.
So libmodbus is just sticking to the Modbus specification. I'm assuming, then, that the problem is in easymodbus, which is apparently expecting you to use 0x01as the unit id in your queries.
I imagine you don't want to mess with easymodbus, so you can fix this problem pretty easily from libmodbus: just change the default unit id:
modbus_set_slave(ctx, 1);
You could also go with:
rc = modbus_set_slave(ctx, MODBUS_BROADCAST_ADDRESS);
ASSERT_TRUE(rc != -1, "Invalid broadcast address");
to make your client address all slaves within the network, if you have more than one.
You have more info and a short explanation of where this problem is coming from in the libmodbus man page for modbus_set_slave function.
For a very comprehensive example, you can check libmodbus unit tests
And regarding your question number 5, I don't know how to answer it, the zeros you mean are supposed to be the states (true or false) you want to write (or read) to the coils. For writing you can change them with the value field of function modbus_write_bit(ctx, address, value).
I'm very grateful for your reply.
I tested the read/write function using the 'unit-test-server/client' code you recommended.
I've reviewed the code, but there are still many things I don't know.
However, there is an address value that acts after testing each other with unit-test-server/client code and there is an address value that does not work
(Do you know why?).
-Checked and found that the UT_BITS_ADDRESS (address value) value operates from 0x130 to 0x150
-'error Illegal data address' occurs at values below -0x130 and above 0x150
-The address I want to read/write is 0x0001 to 0x0004(Do you know how to do?).
I want to know how to process and transmit data like the TX part of the right picture.
enter image description here
I'm running both client and server in my Linux environment and I'm doing read/write testing.
Among the wrong pictures...[06][FF]... <-- I want to know how to modify FF part (to change the value to 01 as shown in the picture)
enter image description here
and "modbus_set_slave" is the function for modbus rtu?
I'd like to communicate PC Program and Linux device in the end.
so Which part do I use that function?
I thanks for your concern again.
I capture the image from an IP-Camera and I work with the frames. My programm reads when there is a movement, and then, it makes a photo and save it on the computer.
It works perfectly at first, but when it is running like 2-3 hours, it usually get an error, and I do not find a explanation for this. Because, if it is an error on getting the image or the processing, it should happens since first, shouldn't it?
The error I get is the next:
Exception in thread "main" java.lang.NullPointerException
at com.googlecode.javacv.IPCameraFrameGrabber.grab(IPCameraFrameGrabber.java:105)
at Llamada.main(Llamada.java:34)
I have looked for the error nº105 but I have not found anything.
The program is the next:
public class Llamada {
public static void main(String[] args) throws Exception {
IPCameraFrameGrabber grabber = new IPCameraFrameGrabber("http://192.168.2.102:80/mjpg/video.mjpg");
//OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(0);
grabber.start();
IplImage frame = grabber.grab();
IplImage image = null;
IplImage prevImage = null;
IplImage diff = null;
Date data = new Date();
String output = "";
int i=0, j=0;
CanvasFrame canvasFrame = new CanvasFrame("IP Camera");
canvasFrame.setCanvasSize(frame.width(), frame.height());
CvMemStorage storage = CvMemStorage.create();
while (canvasFrame.isVisible() && (frame = grabber.grab()) != null) {
cvSmooth(frame, frame, CV_GAUSSIAN, 9, 9, 2, 2);
if (image == null) {
image = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
cvCvtColor(frame, image, CV_RGB2GRAY);
} else {
prevImage = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
prevImage = image;
image = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
cvCvtColor(frame, image, CV_RGB2GRAY);
}
if (diff == null) {
diff = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
}
if (prevImage != null) {
// perform ABS difference
cvAbsDiff(image, prevImage, diff);
// do some threshold for wipe away useless details
cvThreshold(diff, diff, 64, 255, CV_THRESH_BINARY);
canvasFrame.showImage(diff);
// recognize contours
CvSeq contour = new CvSeq(null);
cvFindContours(diff, storage, contour, Loader.sizeof(CvContour.class), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
while (contour != null && !contour.isNull()) {
if (contour.elem_size() > 0) {
output = data.toString();
if (data != null)
output = output.substring(0,10);
if(i%300 == 0)
cvSaveImage((j++)+" "+ output +"-capture.jpg", frame);
CvBox2D box = cvMinAreaRect2(contour, storage);
// test intersection
if (box != null) {
CvPoint2D32f center = box.center();
CvSize2D32f size = box.size();
}
i++;
}
contour = contour.h_next();
}
}
}
grabber.stop();
canvasFrame.dispose();
}
}
Thank you for everything!
Have you tried using a debugger and setting a break point? I understand that waiting around for 2-3 hours isn't fun, but maybe it'd help you get a handle on what's going on.
That seems to be in your while loop's second conditional part. Something inside the method grab on the grabber object is throwing a NullPointerException.
Probably the way you've initialized the grabber has led it to do this.
And it would be useful to know which version of the IPCameraFrameGrabber class you're using and what the author of that class really expected. Namely it's initialized to respond to a particular camera's url. In reading the class, it would appear this makes no claim to work with all IP cameras' MJPEG streams.
Let's look at one example comment in there:
foscam url http://host/videostream.cgi?user=username&pwd=password
http://192.168.0.59:60/videostream.cgi?user=admin&pwd=password
android ipcam http://192.168.0.57:8080/videofeed
And compare that to your url:
http://192.168.2.102:80/mjpg/video.mjpg
I gather it is not a foscam videostream.cgi url nor an android ipcam videofeed url, which would appear to be the only tested urls. It reminds me of an Axis camera url. More on that later.
In a recent version of that class (also in the older one actually), there seems to be some hackish attempt at reading only to the end of a subheader that is always delimited by crlfcrlf which could have been done just as well with a buffered input reader reading lines until it gets an empty line. What I do see here that seems likely to cause an npe is:
When your url's http server's response does not contain the content-length header, which is quite possible, the returned readImage() byte[] is null.
Since javax.imageio.ImageIO specifies that it will throw an IllegalArgumentException when it gets a null input, I'm guessing it's the ByteArrayInputStream constructor in the grabBufferedImage method that's throwing this, the IplImage.createFrom(null) in the old version, or the b.length in the newer version that is.
None of the line numbers of these versions line up with the error message you've shown that you're getting, so maybe your version of the library is yet again different, and broken differently. Try using the debugger, edit and patch the source of the IPCameraFrameGrabber to better support your mjpeg over http "device" based on what you find out is really in the input stream of the http response.
Since the url format reminds me of an Axis camera, I tried this with one running firmware v5.50 with the boa server built in:
$ curl -I http://user:pass#10.10.10.10:8080/mjpg/video.mjpg
HTTP/1.0 200 OK
Cache-Control: no-cache
Pragma: no-cache
Expires: Thu, 01 Dec 1994 16:00:00 GMT
Connection: close
Content-Type: multipart/x-mixed-replace; boundary=myboundary
So you can see the content length is missing there. However, you do say you're getting frames initially for hours, then then, so I'm kind of at a loss with that part. I mean it sounds as though EITHER the input stream is getting closed, or the java implementation wrapping the stream, implemented in the http protocol handler, runs out of some kind of total space or open connection timer for some reason. I know this seems vague.
Another thing that seems odd is that from what I read in the two example classes of IPCameraFrameGrabber linked, every call to grab reads the input stream looking for headers first, which doesn't make sense to me right now, and I feel as though I must be misreading that.
I am learning how to use Sphinx4 using the Maven plug-in for Eclipse.
I took the transcribe demo found on GitHub and altered it to process a file of my own. The audio file is 16bit, mono, 16khz. It is approximately 13 seconds long. I noticed that it sounds like it is in slow motion.
The words spoken in the file are, "also make sure it's easy for you to access the recording files so you could upload it if asked".
I am attempting to transcribe the file and my results are horrendous. My attempts at finding forum posts or links that thoroughly explain how to improve the results, or what I am not doing correctly have lead me no where.
I am looking to strengthen the accuracy of the transcription, but would like to avoid having to train a model myself due to the variance in the type of data that my current project will have to deal with. Is this not possible, and is the code I am using off?
CODE
(NOTE: Audio file available at https://instaud.io/8qv)
public class App {
public static void main(String[] args) throws Exception {
System.out.println("Loading models...");
Configuration configuration = new Configuration();
// Load model from the jar
configuration
.setAcousticModelPath("resource:/edu/cmu/sphinx/models/en-us/en-us");
// You can also load model from folder
// configuration.setAcousticModelPath("file:en-us");
configuration
.setDictionaryPath("resource:/edu/cmu/sphinx/models/en-us/cmudict-en-us.dict");
configuration
.setLanguageModelPath("resource:/edu/cmu/sphinx/models/en-us/en-us.lm.dmp");
StreamSpeechRecognizer recognizer = new StreamSpeechRecognizer(
configuration);
FileInputStream stream = new FileInputStream(new File("/home/tmscanlan/workspace/example/vocaroo_test_revised.wav"));
// stream.skip(44); I commented this out due to the short length of my file
// Simple recognition with generic model
recognizer.startRecognition(stream);
SpeechResult result;
while ((result = recognizer.getResult()) != null) {
// I added the following print statements to get more information
System.out.println("\ngetWords() before loop: " + result.getWords());
System.out.format("Hypothesis: %s\n", result.getHypothesis());
System.out.print("\nThe getResult(): " + result.getResult()
+ "\nThe getLattice(): " + result.getLattice());
System.out.println("List of recognized words and their times:");
for (WordResult r : result.getWords()) {
System.out.println(r);
}
System.out.println("Best 3 hypothesis:");
for (String s : result.getNbest(3))
System.out.println(s);
}
recognizer.stopRecognition();
// Live adaptation to speaker with speaker profiles
stream = new FileInputStream(new File("/home/tmscanlan/workspace/example/warren_test_smaller.wav"));
// stream.skip(44); I commented this out due to the short length of my file
// Stats class is used to collect speaker-specific data
Stats stats = recognizer.createStats(1);
recognizer.startRecognition(stream);
while ((result = recognizer.getResult()) != null) {
stats.collect(result);
}
recognizer.stopRecognition();
// Transform represents the speech profile
Transform transform = stats.createTransform();
recognizer.setTransform(transform);
// Decode again with updated transform
stream = new FileInputStream(new File("/home/tmscanlan/workspace/example/warren_test_smaller.wav"));
// stream.skip(44); I commented this out due to the short length of my file
recognizer.startRecognition(stream);
while ((result = recognizer.getResult()) != null) {
System.out.format("Hypothesis: %s\n", result.getHypothesis());
}
recognizer.stopRecognition();
System.out.println("...Printing is done..");
}
}
Here is the output (a photo album I took): http://imgur.com/a/Ou9oH
As Nikolay says, the audio sounds odd, probably because you haven't resampled it in the right way.
To downsample the audio from the original 22050 Hz to the desired 16kHz, you can run the following command:
sox Vocaroo.wav -r 16000 Vocaroo16.wav
The Vocaroo16.wav will sounds much better and it will (probably) give you better ASR results.
Context: iOS5 AUSampler AudioUnit
I've been digging around trying to determine is there is a programmatic way to determine the number of presets in a DLS or sf2 file. I was hoping it would be available either through 'AudioUnitGetProperty' or 'AudioUnitGetParameter' for an AUSampler. Then of course I want to be able to switch presets on the fly. The Docs don't indicate if this is possible or not.
I'm using the standard code for loading DLS/sf2 per TechNote TN2283. The problem is that with lots of sf2 files it is a trial and error process to find out what the presets are.
-(OSStatus) loadFromDLSOrSoundFont: (NSURL *)bankURL withPatch: (int)presetNumber
OSStatus result = noErr;
// fill out a bank preset data structure
AUSamplerBankPresetData bpdata;
bpdata.bankURL = (CFURLRef) bankURL;
bpdata.bankMSB = kAUSampler_DefaultMelodicBankMSB;
bpdata.bankLSB = kAUSampler_DefaultBankLSB;
bpdata.presetID = (UInt8) presetNumber;
// set the kAUSamplerProperty_LoadPresetFromBank property
result = AudioUnitSetProperty(self.mySamplerUnit,
kAUSamplerProperty_LoadPresetFromBank,
kAudioUnitScope_Global,
0,
&bpdata,
sizeof(bpdata));
// check for errors
NSCAssert (result == noErr,
#"Unable to set the preset property on the Sampler. Error code:%d '%.4s'",
(int) result,
(const char *)&result);
return result;
}
OK - had an answer from an Apple Core Audio engineer:
"There is no API to retrieve the number of presets. The Sampler AU only loads a single instrument at a time from any SF2 or DLS bank, so it does not "digest" the entire bank file (and so has no knowledge of its complete contents)."
I find Apple's documentation quite limited on AudioFileStreamSeek and I cannot find any examples of actual usage anywhere. I have a working streaming audio player, but I just can't seem to get AudioFileStreamSeek to work as advertised...
Any help tips or a little example would be greatly appreciated!
I am told this works:
AudioQueueStop(audioQueue, true);
UInt32 flags = 0;
err = AudioFileStreamParseBytes(audioFileStream, length, bytes,
kAudioFileStreamParseFlag_Discontinuity);
OSStatus status = AudioFileStreamSeek(audioFileStream, framePacket.mPacket,
¤tOffset, &flags);
NSLog(#"Setting next byte offset to: %qi, flags: %d", (long long)currentOffset, flags);
// then read data from the new offset set by AudioFileStreamSeek
[fileHandle seekToFileOffset:currentOffset];
NSData* data = "" readDataOfLength:4096];
flags = kAudioFileStreamParseFlag_Discontinuity;
status = AudioFileStreamParseBytes( stream, [data length], [data bytes], flags);
if (status != noErr)
{
NSLog(#"Error parsing bytes: %d", status);
}
Unless I'm mistaken, this is only available in the 3.0 SDK, and therefore under NDA. Maybe you should take this to the Apple Beta forums?
I stand corrected. AudioFileStreamSeek doesn't show up if you do a search in the online 2.2.1 documentation. You have to manually dig into the docs to find it.
Don't forget to add the data offset (kAudioFileStreamProperty_DataOffset) to the byte offset returned by AudioFileStreamSeek. The return value is an offset into the audio data and ignores the data offset.
It's also a good idea to stop and then re-start the AudioQueue before/after seeking.
Matt Gallagher uses AudioFileStreamSeek in his example "Streaming and playing an MP3 stream".
Look at Matt's code AudioStreamer.m:
SInt64 seekPacket = floor(newSeekTime / packetDuration);
err = AudioFileStreamSeek(audioFileStream, seekPacket, &packetAlignedByteOffset, &ioFlags);