Getting RGB pixel data from CGImage - iphone

I am trying to access pixel data from a CGImage. I want to be able to access the RGB values as integers. I think I am nearly there with this code:
UIImage* theImage = [UIImage imageNamed:#"rgb.png"];
CGImageRef cgImageRef = CGImageRetain(theImage.CGImage);
CFDataRef* imageData = CGDataProviderCopyData(CGImageGetDataProvider(cgImageRef));
NSLog(#"the data = %#", imageData);
This then logs:
the data = <010002fe fffdff02 0200fe04 0003fc>
The image is a 5x1 png containing a black, white, red, green and blue pixel in that order.
I don't really understand what I am looking at here. How can I get an array of RGB values, or something similar so I can work with them.
Thanks,
Rich

Here's some sample code that will print a bunch of relevant info about an image, as well as a dump of the image's pixel data. It will work on images with alpha channels, as well as images without. The code will even work on images that are not in the RGB color space, though I doubt you'd be likely to get any of those on iOS.
Copy-paste it into your project and try to run it against a few of your image files, and then perhaps you can adapt it to your needs
-(void)imageDump:(NSString*)file
{
UIImage* image = [UIImage imageNamed:file];
CGImageRef cgimage = image.CGImage;
size_t width = CGImageGetWidth(cgimage);
size_t height = CGImageGetHeight(cgimage);
size_t bpr = CGImageGetBytesPerRow(cgimage);
size_t bpp = CGImageGetBitsPerPixel(cgimage);
size_t bpc = CGImageGetBitsPerComponent(cgimage);
size_t bytes_per_pixel = bpp / bpc;
CGBitmapInfo info = CGImageGetBitmapInfo(cgimage);
NSLog(
#"\n"
"===== %# =====\n"
"CGImageGetHeight: %d\n"
"CGImageGetWidth: %d\n"
"CGImageGetColorSpace: %#\n"
"CGImageGetBitsPerPixel: %d\n"
"CGImageGetBitsPerComponent: %d\n"
"CGImageGetBytesPerRow: %d\n"
"CGImageGetBitmapInfo: 0x%.8X\n"
" kCGBitmapAlphaInfoMask = %s\n"
" kCGBitmapFloatComponents = %s\n"
" kCGBitmapByteOrderMask = 0x%.8X\n"
" kCGBitmapByteOrderDefault = %s\n"
" kCGBitmapByteOrder16Little = %s\n"
" kCGBitmapByteOrder32Little = %s\n"
" kCGBitmapByteOrder16Big = %s\n"
" kCGBitmapByteOrder32Big = %s\n",
file,
(int)width,
(int)height,
CGImageGetColorSpace(cgimage),
(int)bpp,
(int)bpc,
(int)bpr,
(unsigned)info,
(info & kCGBitmapAlphaInfoMask) ? "YES" : "NO",
(info & kCGBitmapFloatComponents) ? "YES" : "NO",
(info & kCGBitmapByteOrderMask),
((info & kCGBitmapByteOrderMask) == kCGBitmapByteOrderDefault) ? "YES" : "NO",
((info & kCGBitmapByteOrderMask) == kCGBitmapByteOrder16Little) ? "YES" : "NO",
((info & kCGBitmapByteOrderMask) == kCGBitmapByteOrder32Little) ? "YES" : "NO",
((info & kCGBitmapByteOrderMask) == kCGBitmapByteOrder16Big) ? "YES" : "NO",
((info & kCGBitmapByteOrderMask) == kCGBitmapByteOrder32Big) ? "YES" : "NO"
);
CGDataProviderRef provider = CGImageGetDataProvider(cgimage);
NSData* data = (id)CGDataProviderCopyData(provider);
[data autorelease];
const uint8_t* bytes = [data bytes];
printf("Pixel Data:\n");
for(size_t row = 0; row < height; row++)
{
for(size_t col = 0; col < width; col++)
{
const uint8_t* pixel =
&bytes[row * bpr + col * bytes_per_pixel];
printf("(");
for(size_t x = 0; x < bytes_per_pixel; x++)
{
printf("%.2X", pixel[x]);
if( x < bytes_per_pixel - 1 )
printf(",");
}
printf(")");
if( col < width - 1 )
printf(", ");
}
printf("\n");
}
}
Here's some sample output on two images I tried. They're both 5x3 rgb. The "a.png" image has an alpha channel, while the "b.rgb" does not.
===== a.png =====
CGImageGetHeight: 5
CGImageGetWidth: 3
CGImageGetColorSpace: <CGColorSpace 0x4d08ff0> (kCGColorSpaceDeviceRGB)
CGImageGetBitsPerPixel: 32
CGImageGetBitsPerComponent: 8
CGImageGetBytesPerRow: 20
CGImageGetBitmapInfo: 0x00000003
kCGBitmapAlphaInfoMask = YES
kCGBitmapFloatComponents = NO
kCGBitmapByteOrderMask = NO
kCGBitmapByteOrderDefault = NO
kCGBitmapByteOrder16Little = NO
kCGBitmapByteOrder32Little = NO
kCGBitmapByteOrder16Big = NO
kCGBitmapByteOrder32Big = NO
Pixel Data:
(00,00,00,FF), (FF,FF,FF,FF), (FF,00,00,FF), (00,FF,00,FF), (00,00,FF,FF)
(00,00,00,FF), (FF,FF,FF,FF), (FF,00,00,FF), (00,FF,00,FF), (00,00,FF,FF)
(FF,FF,FF,00), (FF,FF,FF,00), (FF,FF,FF,00), (FF,FF,FF,00), (FF,FF,FF,00)
===== b.png =====
CGImageGetHeight: 5
CGImageGetWidth: 3
CGImageGetColorSpace: <CGColorSpace 0x4d08ff0> (kCGColorSpaceDeviceRGB)
CGImageGetBitsPerPixel: 24
CGImageGetBitsPerComponent: 8
CGImageGetBytesPerRow: 15
CGImageGetBitmapInfo: 0x00000000
kCGBitmapAlphaInfoMask = NO
kCGBitmapFloatComponents = NO
kCGBitmapByteOrderMask = NO
kCGBitmapByteOrderDefault = NO
kCGBitmapByteOrder16Little = NO
kCGBitmapByteOrder32Little = NO
kCGBitmapByteOrder16Big = NO
kCGBitmapByteOrder32Big = NO
Pixel Data:
(00,00,00), (FF,FF,FF), (FF,00,00), (00,FF,00), (00,00,FF)
(00,00,00), (FF,FF,FF), (FF,00,00), (00,FF,00), (00,00,FF)
(00,00,00), (FF,FF,FF), (FF,00,00), (00,FF,00), (00,00,FF)

Related

Running SDL/OpenGLES application on a specific DISPLAY in XServer

I am trying to port an application to an embedded system that I am trying to design. The embedded system is Raspberry Pi Zero W - based, and uses a custom Yocto build.
The application to be ported is written with SDL / OpenGLES to my understanding. I have a hard time understanding how to make a connection similar to the following depiction:
SDL APP -----> XServer ($DISPLAY) -------> Framebuffer /dev/fb1 ($FRAMEBUFFER)
System has two displays: One HDMI on /dev/fb0 and One TFT on /dev/fb1. I am trying to run the SDL application on TFT. The following are the steps I do:
First, start an XServer on DISPLAY=:1 that is connected to /dev/fb1:
FRAMEBUFFER=/dev/fb1 xinit /etc/X11/Xsession -- /usr/bin/Xorg :1 -br -pn -nolisten tcp -dpi 100
The first step seems like it's working. I can see LXDE booting up on my TFT screen. Checking the display, I get the correct display resolution:
~/projects# DISPLAY=:1 xrandr -q
xrandr: Failed to get size of gamma for output default
Screen 0: minimum 320 x 240, current 320 x 240, maximum 320 x 240
default connected 320x240+0+0 0mm x 0mm
320x240 0.00*
Second, I would like to start SDL-written application using x11. I am thinking that should work in seeing the application on the TFT. In order to do so, I try:
SDL_VIDEODRIVER=x11 SDL_WINDOWID=1 DISPLAY=:1 ./SDL_App
No matter which display number I choose, it starts on my HDMI display and not on the TFT. So now I am thinking the person who wrote the application hardcoded somethings in the application code:
void init_ogl(void)
{
int32_t success = 0;
EGLBoolean result;
EGLint num_config;
static EGL_DISPMANX_WINDOW_T nativewindow;
DISPMANX_ELEMENT_HANDLE_T dispman_element;
DISPMANX_DISPLAY_HANDLE_T dispman_display;
DISPMANX_UPDATE_HANDLE_T dispman_update;
VC_DISPMANX_ALPHA_T alpha;
VC_RECT_T dst_rect;
VC_RECT_T src_rect;
static const EGLint attribute_list[] =
{
EGL_RED_SIZE, 8,
EGL_GREEN_SIZE, 8,
EGL_BLUE_SIZE, 8,
EGL_ALPHA_SIZE, 8,
EGL_SURFACE_TYPE, EGL_WINDOW_BIT,
EGL_NONE
};
EGLConfig config;
// Get an EGL display connection
display = eglGetDisplay(EGL_DEFAULT_DISPLAY);
assert(display!=EGL_NO_DISPLAY);
// Initialize the EGL display connection
result = eglInitialize(display, NULL, NULL);
assert(EGL_FALSE != result);
// Get an appropriate EGL frame buffer configuration
result = eglChooseConfig(display, attribute_list, &config, 1, &num_config);
assert(EGL_FALSE != result);
// Create an EGL rendering context
context = eglCreateContext(display, config, EGL_NO_CONTEXT, NULL);
assert(context!=EGL_NO_CONTEXT);
// Create an EGL window surface
success = graphics_get_display_size( 0 /* LCD */ , &screen_width, &screen_height);
printf ("Screen width= %d\n", screen_width);
printf ("Screen height= %d\n", screen_height);
assert( success >= 0 );
int32_t zoom = screen_width / GAMEBOY_WIDTH;
int32_t zoom2 = screen_height / GAMEBOY_HEIGHT;
if (zoom2 < zoom)
zoom = zoom2;
int32_t display_width = GAMEBOY_WIDTH * zoom;
int32_t display_height = GAMEBOY_HEIGHT * zoom;
int32_t display_offset_x = (screen_width / 2) - (display_width / 2);
int32_t display_offset_y = (screen_height / 2) - (display_height / 2);
dst_rect.x = 0;
dst_rect.y = 0;
dst_rect.width = screen_width;
dst_rect.height = screen_height;
src_rect.x = 0;
src_rect.y = 0;
src_rect.width = screen_width << 16;
src_rect.height = screen_height << 16;
dispman_display = vc_dispmanx_display_open( 0 /* LCD */ );
dispman_update = vc_dispmanx_update_start( 0 );
alpha.flags = DISPMANX_FLAGS_ALPHA_FIXED_ALL_PIXELS;
alpha.opacity = 255;
alpha.mask = 0;
dispman_element = vc_dispmanx_element_add ( dispman_update, dispman_display,
0/*layer*/, &dst_rect, 0/*src*/,
&src_rect, DISPMANX_PROTECTION_NONE, &alpha, 0/*clamp*/, DISPMANX_NO_ROTATE/*transform*/);
nativewindow.element = dispman_element;
nativewindow.width = screen_width;
nativewindow.height = screen_height;
vc_dispmanx_update_submit_sync( dispman_update );
surface = eglCreateWindowSurface( display, config, &nativewindow, NULL );
assert(surface != EGL_NO_SURFACE);
// Connect the context to the surface
result = eglMakeCurrent(display, surface, surface, context);
assert(EGL_FALSE != result);
eglSwapInterval(display, 1);
glGenTextures(1, &theGBTexture);
glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, theGBTexture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 256, 256, 0, GL_RGBA, GL_UNSIGNED_BYTE, (GLvoid*) NULL);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrthof(0.0f, screen_width, screen_height, 0.0f, -1.0f, 1.0f);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glViewport(0.0f, 0.0f, screen_width, screen_height);
quadVerts[0] = display_offset_x;
quadVerts[1] = display_offset_y;
quadVerts[2] = display_offset_x + display_width;
quadVerts[3] = display_offset_y;
quadVerts[4] = display_offset_x + display_width;
quadVerts[5] = display_offset_y + display_height;
quadVerts[6] = display_offset_x;
quadVerts[7] = display_offset_y + display_height;
glVertexPointer(2, GL_SHORT, 0, quadVerts);
glEnableClientState(GL_VERTEX_ARRAY);
glTexCoordPointer(2, GL_FLOAT, 0, kQuadTex);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glClear(GL_COLOR_BUFFER_BIT);
}
void init_sdl(void)
{
if (SDL_Init(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER) < 0)
{
Log("SDL Error Init: %s", SDL_GetError());
}
theWindow = SDL_CreateWindow("Gearboy", 0, 0, 0, 0, 0);
if (theWindow == NULL)
{
Log("SDL Error Video: %s", SDL_GetError());
}
...
}
At first glance, I discovered two lines: vc_dispmanx_display_open( 0 /* LCD */ ); and graphics_get_display_size( 0 /* LCD */ , &screen_width, &screen_height);. I tried changing the display parameter to 1, thinking that it refers to DISPLAY=:1, but it did not do anything. I added logs for screen resolution, and I get 1920x1080, which is the resolution of the HDMI display. I think there must be something with the EGL portion of the code that I'm missing. What should I do right now? Is my logic fair enough or am I missing something?
Any requirements, please let me know. Any guidance regarding the issue is much appreciated.
EDIT: I saw that some people use the following, but raspberry pi zero can not find EGL/eglvivante.h for fb functions so I am unable to compile it:
int fbnum = 1; // fbnum is an integer for /dev/fb1 fbnum = 1
EGLNativeDisplayType native_display = fbGetDisplayByIndex(fbnum);
EGLNativeWindowType native_window = fbCreateWindow(native_display, 0, 0, 0, 0);
display = eglGetDisplay(native_display);

Bluetooth low energy, how to parse R-R Interval value?

My application is receiving information from smart heart device. Now i can see pulse value. Could you please help me to parse R-R Interval value? How can i check device support R-R Interval value or Not ?
Any advise from you
Thanks
Have you checked the Bluetooth spec? The sample code below is in C#, but I think it shows the way to parse the data in each heart rate packet.
//first byte of heart rate record denotes flags
byte flags = heartRateRecord[0];
ushort offset = 1;
bool HRC2 = (flags & 1) == 1;
if (HRC2) //this means the BPM is un uint16
{
short hr = BitConverter.ToInt16(heartRateRecord, offset);
offset += 2;
}
else //BPM is uint8
{
byte hr = heartRateRecord[offset];
offset += 1;
}
//see if EE is available
//if so, pull 2 bytes
bool ee = (flags & (1 << 3)) != 0;
if (ee)
offset += 2;
//see if RR is present
//if so, the number of RR values is total bytes left / 2 (size of uint16)
bool rr = (flags & (1 << 4)) != 0;
if (rr)
{
int count = (heartRateRecord.Length - offset)/2;
for (int i = 0; i < count; i++)
{
//each existence of these values means an R-Wave was already detected
//the ushort means the time (1/1024 seconds) since last r-wave
ushort value = BitConverter.ToUInt16(heartRateRecord, offset);
double intervalLengthInSeconds = value/1024.0;
offset += 2;
}
}
This post is a little old but a full answer has not been given.
As I run into this post and it did help me at the end, I would like to share my final code. Hopefully it will help others.
The code provided by Daniel Judge is actually right, but as he already wrote, it is C#. HIs code is a bit better compared to what Simon M came up with at the end as the code of Daniel Judge takes into account there can be more than two RR-values within one message.
Here is the actual spec of the Heart_rate_measurement characteristic
I have translated Daniel Judge his code to Objective-C:
// Instance method to get the heart rate BPM information
- (void) getHeartBPMData:(CBCharacteristic *)characteristic error:(NSError *)error
{
// Get the BPM //
// https://developer.bluetooth.org/gatt/characteristics/Pages/CharacteristicViewer.aspx?u=org.bluetooth.characteristic.heart_rate_measurement.xml //
// Convert the contents of the characteristic value to a data-object //
NSData *data = [characteristic value];
// Get the byte sequence of the data-object //
const uint8_t *reportData = [data bytes];
// Initialise the offset variable //
NSUInteger offset = 1;
// Initialise the bpm variable //
uint16_t bpm = 0;
// Next, obtain the first byte at index 0 in the array as defined by reportData[0] and mask out all but the 1st bit //
// The result returned will either be 0, which means that the 2nd bit is not set, or 1 if it is set //
// If the 2nd bit is not set, retrieve the BPM value at the second byte location at index 1 in the array //
if ((reportData[0] & 0x01) == 0) {
// Retrieve the BPM value for the Heart Rate Monitor
bpm = reportData[1];
offset = offset + 1; // Plus 1 byte //
}
else {
// If the second bit is set, retrieve the BPM value at second byte location at index 1 in the array and //
// convert this to a 16-bit value based on the host’s native byte order //
bpm = CFSwapInt16LittleToHost(*(uint16_t *)(&reportData[1]));
offset = offset + 2; // Plus 2 bytes //
}
NSLog(#"bpm: %i", bpm);
// Determine if EE data is present //
// If the 3rd bit of the first byte is 1 this means there is EE data //
// If so, increase offset with 2 bytes //
if ((reportData[0] & 0x03) == 1) {
offset = offset + 2; // Plus 2 bytes //
}
// Determine if RR-interval data is present //
// If the 4th bit of the first byte is 1 this means there is RR data //
if ((reportData[0] & 0x04) == 0)
{
NSLog(#"%#", #"Data are not present");
}
else
{
// The number of RR-interval values is total bytes left / 2 (size of uint16) //
NSUInteger length = [data length];
NSUInteger count = (length - offset)/2;
NSLog(#"RR count: %lu", (unsigned long)count);
for (int i = 0; i < count; i++) {
// The unit for RR interval is 1/1024 seconds //
uint16_t value = CFSwapInt16LittleToHost(*(uint16_t *)(&reportData[offset]));
value = ((double)value / 1024.0 ) * 1000.0;
offset = offset + 2; // Plus 2 bytes //
NSLog(#"RR value %lu: %u", (unsigned long)i, value);
}
}
}
EDIT:
this work for me, i get the correct rr values:
In some cases you can find two values at the same time for rr.
- (void) updateWithHRMData:(NSData *)datas {
const uint8_t *reportData = [datas bytes];
uint16_t bpm = 0;
uint16_t bpm2 = 0;
if ((reportData[0] & 0x04) == 0)
{
NSLog(#"%#", #"Data are not present");
}
else
{
bpm = CFSwapInt16LittleToHost(*(uint16_t *)(&reportData[2]));
bpm2 = CFSwapInt16LittleToHost(*(uint16_t *)(&reportData[4]));
if (bpm != 0 || bpm2 != 0) {
NSLog(#"%u", bpm);
if (bpm2 != 0) {
NSLog(#"%u", bpm2);
}
}
}
}
in #Brabbeldas solution i had to use a different flag to get rri values. but might depend on device used.
if ((reportData[0] & 0x10) == 0)
instead of
if ((reportData[0] & 0x04) == 0)
Parse heart rate parameters in "C"
I uploaded the sample application to GitHub Heart-Rate-Bluegiga
void ble_evt_attclient_attribute_value(const struct ble_msg_attclient_attribute_value_evt_t *msg)
{
if (msg->value.len < 2) {
printf("Not enough fields in Heart Rate Measurement value");
change_state(state_finish);
}
// Heart Rate Profile defined flags
const unsigned char HEART_RATE_VALUE_FORMAT = 0x01;
const unsigned char ENERGY_EXPENDED_STATUS = 0x08;
const unsigned char RR_INTERVAL = 0x10;
unsigned char current_offset = 0;
unsigned char flags = msg->value.data[current_offset];
int is_heart_rate_value_size_long = ((flags & HEART_RATE_VALUE_FORMAT) != 0);
int has_expended_energy = ((flags & ENERGY_EXPENDED_STATUS) != 0);
int has_rr_intervals = ((flags & RR_INTERVAL) != 0);
current_offset++;
uint16 heart_rate_measurement_value = 0;
if (is_heart_rate_value_size_long)
{
heart_rate_measurement_value = (uint16)((msg->value.data[current_offset + 1] << 8) +
msg->value.data[current_offset]);
current_offset += 2;
}
else
{
heart_rate_measurement_value = msg->value.data[current_offset];
current_offset++;
}
printf("Heart rate measurment value: %d ", heart_rate_measurement_value);
uint16 expended_energy_value = 0;
if (has_expended_energy)
{
expended_energy_value = (uint16)((msg->value.data[current_offset + 1] << 8) +
msg->value.data[current_offset]);
current_offset += 2;
printf(" Expended energy value: %d ", expended_energy_value);
}
uint16 rr_intervals[10] = {0};
if (has_rr_intervals)
{
printf(" Rr intervals: ");
int rr_intervals_count = (msg->value.len - current_offset) / 2;
for (int i = 0; i < rr_intervals_count; i++)
{
int raw_rr_interval = (uint16)((msg->value.data[current_offset + 1] << 8) +
msg->value.data[current_offset]);
rr_intervals[i] = ((double)raw_rr_interval / 1024) * 1000;
current_offset += 2;
printf("%d ", rr_intervals[i]);
}
printf("\n");
}
}

cv::matchShapes method implementation giving error

My objective is to find the pattern in the image captured from camera for this I found the cv::matchShapes method of opencv to implement it.
When I have implemented this method it is making the app crash on matchShapes method.Might be I have done something wrong:-
I exactly do not know how to use this method in order to find the match shape in my query image.This is what I tried.
Here is my code:-
- (void)tryingMatchShapes:(cv::Mat)_image _image1:(cv::Mat)_image1
{
std::vector<std::vector<cv::Point> > squares;
cv::Mat pyr, timg, gray0(_image.size(), CV_8U), gray;
int thresh = 50, N = 11;
cv::pyrDown(_image, pyr, cv::Size(_image.cols/2, _image.rows/2));
cv::pyrUp(pyr, timg, _image.size());
std::vector<std::vector<cv::Point> > contours;
for( int c = 0; c < 3; c++ ) {
int ch[] = {c, 0};
mixChannels(&timg, 1, &gray0, 1, ch, 1);
for( int l = 0; l < N; l++ ) {
if( l == 0 ) {
cv::Canny(gray0, gray, 0, thresh, 5);
cv::dilate(gray, gray, cv::Mat(), cv::Point(-1,-1));
}
else {
gray = gray0 >= (l+1)*255/N;
}
cv::findContours(gray, contours, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
}
}
cv::Mat pyr1, timg1, gray2(_image1.size(), CV_8U), gray1;
cv::pyrDown(_image1, pyr1, cv::Size(_image1.cols/2, _image1.rows/2));
cv::pyrUp(pyr1, timg1, _image1.size());
std::vector<std::vector<cv::Point> > contours1;
for( int c = 0; c < 3; c++ ) {
int ch[] = {c, 0};
mixChannels(&timg1, 1, &gray2, 1, ch, 1);
for( int l = 0; l < N; l++ ) {
if( l == 0 ) {
cv::Canny(gray2, gray1, 0, thresh, 5);
cv::dilate(gray1, gray1, cv::Mat(), cv::Point(-1,-1));
}
else {
gray1 = gray2 >= (l+1)*255/N;
}
cv::findContours(gray1, contours1, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
}
}
for( size_t i = 0; i < contours.size(); i++ )
{
double value= cv::matchShapes(contours[i], contours1[i], CV_CONTOURS_MATCH_I1, 0);
NSLog(#"%f",value);
}
}
UIImage *testImage = [UIImage imageNamed:#"note_with_marks (1).png"];
[self.imageView setImage:testImage];
cv::Mat forground = [testImage CVMat];
UIImage *testImage2 = [UIImage imageNamed:#"1.png"];
cv::Mat forground2 = [testImage2 CVMat];
[self tryingMatchShapes:forground _image1:forground2];
The app is getting crashed.
Error:-
OpenCV Error: Assertion failed (contour1.checkVector(2) >= 0 && contour2.checkVector(2) >= 0 && (contour1.depth() == CV_32F || contour1.depth() == CV_32S) && contour1.depth() == contour2.depth()) in matchShapes, file /Users/Aziz/Documents/Projects/opencv_sources/trunk/modules/imgproc/src/contours.cpp, line 1705
Please some body help me in implementing this method from coding point of view.
Please I need some coding help.I have already gone through I lot of theoretical concepts.
Thanks in advance!
std::vector<std::vector<cv::Point> > contours1;
You can't apply matchShapes to such type. You can apply it for each (one) contour, not for group (array) of contours. For example:
cv::matchShapes(contours[0], contours1[0], CV_CONTOURS_MATCH_I1, 0);

How to encode using the FFMpeg in Android (using H263)

I am trying to follow the sample code on encoding in the ffmpeg document and successfully build a application to encode and generate a mp4 file but I face the following problems:
1) I am using the H263 for encoding but I can only set the width and height of the AVCodecContext to 176x144, for other case (like 720x480 or 640x480) it will return fail.
2) I can't play the output mp4 file by using the default Android player, isn't it support H263 mp4 file? p.s. I can play it by using other player
3) Is there any sample code on encoding other video frame to make a new video (which mean decode the video and encode it back in different quality setting, also i would like to modify the frame content)?
Here is my code, thanks!
JNIEXPORT jint JNICALL Java_com_ffmpeg_encoder_FFEncoder_nativeEncoder(JNIEnv* env, jobject thiz, jstring filename){
LOGI("nativeEncoder()");
avcodec_register_all();
avcodec_init();
av_register_all();
AVCodec *codec;
AVCodecContext *codecCtx;
int i;
int out_size;
int size;
int x;
int y;
int output_buffer_size;
FILE *file;
AVFrame *picture;
uint8_t *output_buffer;
uint8_t *picture_buffer;
/* Manual Variables */
int l;
int fps = 30;
int videoLength = 5;
/* find the H263 video encoder */
codec = avcodec_find_encoder(CODEC_ID_H263);
if (!codec) {
LOGI("avcodec_find_encoder() run fail.");
}
codecCtx = avcodec_alloc_context();
picture = avcodec_alloc_frame();
/* put sample parameters */
codecCtx->bit_rate = 400000;
/* resolution must be a multiple of two */
codecCtx->width = 176;
codecCtx->height = 144;
/* frames per second */
codecCtx->time_base = (AVRational){1,fps};
codecCtx->pix_fmt = PIX_FMT_YUV420P;
codecCtx->codec_id = CODEC_ID_H263;
codecCtx->codec_type = AVMEDIA_TYPE_VIDEO;
/* open it */
if (avcodec_open(codecCtx, codec) < 0) {
LOGI("avcodec_open() run fail.");
}
const char* mfileName = (*env)->GetStringUTFChars(env, filename, 0);
file = fopen(mfileName, "wb");
if (!file) {
LOGI("fopen() run fail.");
}
(*env)->ReleaseStringUTFChars(env, filename, mfileName);
/* alloc image and output buffer */
output_buffer_size = 100000;
output_buffer = malloc(output_buffer_size);
size = codecCtx->width * codecCtx->height;
picture_buffer = malloc((size * 3) / 2); /* size for YUV 420 */
picture->data[0] = picture_buffer;
picture->data[1] = picture->data[0] + size;
picture->data[2] = picture->data[1] + size / 4;
picture->linesize[0] = codecCtx->width;
picture->linesize[1] = codecCtx->width / 2;
picture->linesize[2] = codecCtx->width / 2;
for(l=0;l<videoLength;l++){
//encode 1 second of video
for(i=0;i<fps;i++) {
//prepare a dummy image YCbCr
//Y
for(y=0;y<codecCtx->height;y++) {
for(x=0;x<codecCtx->width;x++) {
picture->data[0][y * picture->linesize[0] + x] = x + y + i * 3;
}
}
//Cb and Cr
for(y=0;y<codecCtx->height/2;y++) {
for(x=0;x<codecCtx->width/2;x++) {
picture->data[1][y * picture->linesize[1] + x] = 128 + y + i * 2;
picture->data[2][y * picture->linesize[2] + x] = 64 + x + i * 5;
}
}
//encode the image
out_size = avcodec_encode_video(codecCtx, output_buffer, output_buffer_size, picture);
fwrite(output_buffer, 1, out_size, file);
}
//get the delayed frames
for(; out_size; i++) {
out_size = avcodec_encode_video(codecCtx, output_buffer, output_buffer_size, NULL);
fwrite(output_buffer, 1, out_size, file);
}
}
//add sequence end code to have a real mpeg file
output_buffer[0] = 0x00;
output_buffer[1] = 0x00;
output_buffer[2] = 0x01;
output_buffer[3] = 0xb7;
fwrite(output_buffer, 1, 4, file);
fclose(file);
free(picture_buffer);
free(output_buffer);
avcodec_close(codecCtx);
av_free(codecCtx);
av_free(picture);
LOGI("finish");
return 0; }
H263 accepts only certain resolutions:
128 x 96
176 x 144
352 x 288
704 x 576
1408 x 1152
It will fail with anything else.
The code supplied in the question (I used it myself at first) seems to only generate a very rudimentary, if any, container format.
I found that this example, http://cekirdek.pardus.org.tr/~ismail/ffmpeg-docs/output-example_8c-source.html, worked much better as it creates a real container for the video and audio streams. My video is now displayable on the Android device.

Encoding images to video with ffmpeg

I am trying to encode series of images to one video file. I am using code from api-example.c, its works, but it gives me weird green colors in video. I know, I need to convert my RGB images to YUV, I found some solution, but its doesn't works, the colors is not green but very strange, so thats the code:
// Register all formats and codecs
av_register_all();
AVCodec *codec;
AVCodecContext *c= NULL;
int i, out_size, size, outbuf_size;
FILE *f;
AVFrame *picture;
uint8_t *outbuf;
printf("Video encoding\n");
/* find the mpeg video encoder */
codec = avcodec_find_encoder(CODEC_ID_MPEG2VIDEO);
if (!codec) {
fprintf(stderr, "codec not found\n");
exit(1);
}
c= avcodec_alloc_context();
picture= avcodec_alloc_frame();
/* put sample parameters */
c->bit_rate = 400000;
/* resolution must be a multiple of two */
c->width = 352;
c->height = 288;
/* frames per second */
c->time_base= (AVRational){1,25};
c->gop_size = 10; /* emit one intra frame every ten frames */
c->max_b_frames=1;
c->pix_fmt = PIX_FMT_YUV420P;
/* open it */
if (avcodec_open(c, codec) < 0) {
fprintf(stderr, "could not open codec\n");
exit(1);
}
f = fopen(filename, "wb");
if (!f) {
fprintf(stderr, "could not open %s\n", filename);
exit(1);
}
/* alloc image and output buffer */
outbuf_size = 100000;
outbuf = malloc(outbuf_size);
size = c->width * c->height;
#pragma mark -
AVFrame* outpic = avcodec_alloc_frame();
int nbytes = avpicture_get_size(PIX_FMT_YUV420P, c->width, c->height);
//create buffer for the output image
uint8_t* outbuffer = (uint8_t*)av_malloc(nbytes);
#pragma mark -
for(i=1;i<77;i++) {
fflush(stdout);
int numBytes = avpicture_get_size(PIX_FMT_YUV420P, c->width, c->height);
uint8_t *buffer = (uint8_t *)av_malloc(numBytes*sizeof(uint8_t));
UIImage *image = [UIImage imageNamed:[NSString stringWithFormat:#"10%d", i]];
CGImageRef newCgImage = [image CGImage];
CGDataProviderRef dataProvider = CGImageGetDataProvider(newCgImage);
CFDataRef bitmapData = CGDataProviderCopyData(dataProvider);
buffer = (uint8_t *)CFDataGetBytePtr(bitmapData);
avpicture_fill((AVPicture*)picture, buffer, PIX_FMT_RGB8, c->width, c->height);
avpicture_fill((AVPicture*)outpic, outbuffer, PIX_FMT_YUV420P, c->width, c->height);
struct SwsContext* fooContext = sws_getContext(c->width, c->height,
PIX_FMT_RGB8,
c->width, c->height,
PIX_FMT_YUV420P,
SWS_FAST_BILINEAR, NULL, NULL, NULL);
//perform the conversion
sws_scale(fooContext, picture->data, picture->linesize, 0, c->height, outpic->data, outpic->linesize);
// Here is where I try to convert to YUV
/* encode the image */
out_size = avcodec_encode_video(c, outbuf, outbuf_size, outpic);
printf("encoding frame %3d (size=%5d)\n", i, out_size);
fwrite(outbuf, 1, out_size, f);
free(buffer);
buffer = NULL;
}
/* get the delayed frames */
for(; out_size; i++) {
fflush(stdout);
out_size = avcodec_encode_video(c, outbuf, outbuf_size, NULL);
printf("write frame %3d (size=%5d)\n", i, out_size);
fwrite(outbuf, 1, outbuf_size, f);
}
/* add sequence end code to have a real mpeg file */
outbuf[0] = 0x00;
outbuf[1] = 0x00;
outbuf[2] = 0x01;
outbuf[3] = 0xb7;
fwrite(outbuf, 1, 4, f);
fclose(f);
free(outbuf);
avcodec_close(c);
av_free(c);
av_free(picture);
printf("\n");
Please give me advice how to fix that problem.
You can see article http://unick-soft.ru/Articles.cgi?id=20. But it is article on Russian, but it includes code samples and VS Example.
Has anyone found a fix for this? I am seeing the green video problem on the decode side. That is, when I decode incoming PIX_FMT_YUV420 packets and then swsscale them to PIX_FMT_RGBA.
Thanks!
EDIT:
The green images are probably due to an arm optimization backfiring. I used this to fix the problem in my case:
http://ffmpeg-users.933282.n4.nabble.com/green-distorded-output-image-on-iPhone-td2231805.html
I guess the idea is to not specify any architecture (the config will you a warning about the architecture being unknown but you can continue to 'make' anyway). That way, the arm optimizations are not used. There maybe a slight performance hit (if any), but atleast it works! :)
I think the problem is most likely that you are using PIX_FMT_RGB8 as your input pixel format. This does not mean 8 bits per channel like the commonly used 24-bit RGB or 32-bit ARGB. It means 8 bits per pixel, meaning that all three color channels are housed in a single byte. I am guessing that this is not the format of your image since it is quite uncommon, so you need to use PIX_FMT_RGB24 or PIX_FMT_RGB32 depending on whether or not your input image has an alpha channel. See this documentation page for info on the pixel formats.