I am having great success with zxing on .NET and trying to get the best speed for decoding QR Barcodes (I have a lot to do -- 1.8M). The code I am using (well bits of it):
// Create Barcode decoder
BarcodeReader q = new BarcodeReader();
q.PossibleFormats = new List<BarcodeFormat>();
q.PossibleFormats.Add(BarcodeFormat.QR_CODE);
q.AutoRotate = true; // Not necessary for QR?
q.TryHarder = false;
// Decode result
Result[] r = q.DecodeMultiple(imageFile);
My code is a little smarter in that it is in a loop and tries harder if it doesn't find it the first time.
Is there a way to add a zone, ROI or smaller area to speed up the detection?
Any other recommendations to improve performance?
The fastest way with ZXing.Net for QR codes is the following:
// Create Barcode decoder
BarcodeReader q = new BarcodeReader();
q.PossibleFormats = new List<BarcodeFormat>();
q.PossibleFormats.Add(BarcodeFormat.QR_CODE);
q.AutoRotate = false;
q.TryHarder = false;
// Decode result
Result r = q.Decode(imageFile);
But it decodes only the first QR code which is found.
Avoid DecodeMultiple if you don't need it.
All other options should only be used if really necessary.
AutoRotate isn't necessary for QR code decoding.
If your images are really big shrink them before decoding.
For most cases there is no need for images with a bigger resolution than 1000 pixels.
The only exceptions are really tiny QR codes.
Another good optimization is the use of an image source which gives grayscale images.
A lot of CPU cycles are needed for the calculation of the luminance values from
RGB images. The fastest option are 8 bit grayscale images.
Related
Setup
I am using a custom RenderBox to draw.
The canvas object in the code below comes from the PaintingContext in the paint method.
Drawing
I am trying to render pixels individually by using Canvas.drawRect.
I should point out that these are sometimes larger and sometimes smaller than the pixels on screen they actually occupy.
for (int i = 0; i < width * height; i++) {
// in this case the rect size is 1
canvas.drawRect(
Rect.fromLTWH(index % (width * height),
(index / (width * height)).floor(), 1, 1), Paint()..color = colors[i]);
}
Storage
I am storing the pixels as a List<List<Color>> (colors in the code above). I tried differently nested lists previously, but they did not cause any noticable discrepancies in terms of performance.
The memory on my Android Emulator test device increases by 282.7MB when populating the list with a 999x999 image. Note that it only temporarily increases by 282.7MB. After about half a minute, the increase drops to 153.6MB and stays there (without any user interaction).
Rendering
With a resolution of 999x999, the code above causes a GPU max of 250.1 ms/frame and a UI max of 1835.9 ms/frame, which is obviously unacceptable. The UI freezes for two seconds when trying to draw a 999x999 image, which should be a piece of cake (I would guess) considering that 4k video runs smoothly on the same device.
CPU
I am not exactly sure how to track this properly using the Android profiler, but while populating or changing the list, i.e. drawing the pixels (which is the case for the above metrics as well), CPU usage goes from 0% to up to 60%. Here are the AVD performance settings:
Cause
I have no idea where to start since I am not even sure what part of my code causes the freezing. Is it the memory usage? Or the drawing itself?
How would I go about this in general? What am I doing wrong? How should I store these pixels instead.
Efforts
I have tried so much that did not help at all that I will try to only point out the most notable ones:
I tried converting the List<List<Color>> to an Image from the dart:ui library hoping to use Canvas.drawImage. In order to do that, I tried encoding my own PNG, but I have not been able to render more than a single row. However, it did not look like that would boost performance. When trying to convert a 9999x9999 image, I ran into an out of memory exception. Now, I am wondering how video is rendered as all as any 4k video will easily take up more memory than a 9999x9999 image if a few seconds of it are in memory.
I tried implementing the image package. However, I stopped before completing it as I noticed that it is not meant to be used in Flutter but rather in HTML. I would not have gained anything using that.
This one is pretty important for the following conclusion I will draw: I tried to just draw without storing the pixels, i.e. is using Random.nextInt to generate random colors. When trying to randomly generate a 999x999 image, this resulted in a GPU max of 1824.7 ms/frames and a UI max of 2362.7 ms/frame, which is even worse, especially in the GPU department.
Conclusion
This is the conclusion I reached before trying my failed attempt at rendering using Canvas.drawImage: Canvas.drawRect is not made for this task as it cannot even draw simple images.
How do you do this in Flutter?
Notes
This is basically what I tried to ask over two months ago (yes, I have been trying to resolve this issue for that long), but I think that I did not express myself properly back then and that I knew even less what the actual problem was.
The highest resolution I can properly render is around 10k pixels. I need at least 1m.
I am thinking that abandoning Flutter and going for native might be my only option. However, I would like to believe that I am just approaching this problem completely wrong. I have spent about three months trying to figure this out and I did not find anything that lead me anywhere.
Solution
dart:ui has a function that converts pixels to an Image easily: decodeImageFromPixels
Example implementation
Issue on performance
Does not work in the current master channel
I was simply not aware of this back when I created this answer, which is why I wrote the "Alternative" section.
Alternative
Thanks to #pslink for reminding me of BMP after I wrote that I had failed to encode my own PNG.
I had looked into it previously, but I thought that it looked to complicated without sufficient documentation. Now, I found this nice article explaining the necessary BMP headers and implemented 32-bit BGRA (ARGB but BGRA is the order of the default mask) by copying Example 2 from the "BMP file format" Wikipedia article. I went through all sources but could not find an original source for this example. Maybe the authors of the Wikipedia article wrote it themselves.
Results
Using Canvas.drawImage and my 999x999 pixels converted to an image from a BMP byte list, I get a GPU max of 9.9 ms/frame and a UI max of 7.1 ms/frame, which is awesome!
| ms/frame | Before (Canvas.drawRect) | After (Canvas.drawImage) |
|-----------|---------------------------|--------------------------|
| GPU max | 1824.7 | 9.9 |
| UI max | 2362.7 | 7.1 |
Conclusion
Canvas operations like Canvas.drawRect are not meant to be used like that.
Instructions
First of, this is quite straight-forward, however, you need to correctly populate the byte list, otherwise, you are going to get an error that your data is not correctly formatted and see no results, which can be quite frustrating.
You will need to prepare your image before drawing as you cannot use async operations in the paint call.
In code, you need to use a Codec to transform your list of bytes into an image.
final list = [
0x42, 0x4d, // 'B', 'M'
...];
// make sure that you either know the file size, data size and data offset beforehand
// or that you edit these bytes afterwards
final Uint8List bytes = Uint8List.fromList(list);
final Codec codec = await instantiateImageCodec(bytes));
final Image image = (await codec.getNextFrame()).image;
You need to pass this image to your drawing widget, e.g. using a FutureBuilder.
Now, you can just use Canvas.drawImage in your draw call.
I have a video written with the Xvid MPEG4-Codec. Unfortunately, each frame has an annoying bar to the right and to the bottom of a couple of pixels. What I would like to do is to remove this bar, preferably using MATLAB.
For this purpose, I have written this code to try this out:
function [] = changeVideo(in_path, out_path, reqSize)
videoList = dir(strcat(in_path, '\*.avi'));
for ii = 1:numel(videoList)
vidReader = VideoReader(strcat([in_path '\'], videoList(ii).name));
vidWriter = VideoWriter(strcat([out_path '\'], videoList(ii).name),'MPEG-4');
open(vidWriter);
while hasFrame(vidReader)
% here I would change the size of the frame
writeVideo(vidWriter,readFrame(vidReader));
end
close(vidWriter);
close(vidReader);
end
end
Unfortunately, this does not seem to work. The resulting video has a different memory footprint and quality than the original video. Since MATLAB can read the original video, I was hoping that there is a way to replicate this video, but only change each frame slightly in its size. Is this possible?
Thank you!
I am currently recording on a single camera the images, one aside of the other one, of the same sample out of a microscope.
I have 2 issues with that, and I figured out that in post procesing with Matlab I could arrange these questions.
-First, the 2 images on the camera are supposed to have the same pixel size, or one is just a litle bigger than the other one, probably because of optical pathways. What is the adapted Matlab function or way to correlate the two images so they will have exactly the same pixel size in X and Y ?
Two images on same camera , one bigger or smaller compared to the other one
-Secondly, my sample is moving a litle during the recording ( while still staying in my field of view of course ). To make my analysis easier, it would be suitable that I could correct the images so the sample remain at the same place as in the first image, to perform calculations on it easier. What would be the adapted Matlab function or way to correct this movement in the image ?
Sample moving in the image on the camera
Sorry for the poor quality of my drawings !
Thank you very much for your advices and help.
First zero-pad the images to a sufficient degree, to get them both to double the size of the bigger one.
size_padding = max(size(fig1),size(fig2));
fig1_pad = padarray(fig1,size_padding-size(fig1),'post');
fig2_pad = padarray(fig2,size_padding-size(fig2),'post');
Assuming the sample is the only feature present in the images, the best way to proceed would be to use the xcorr2() function and find the lag corresponding to the maximum correlation, to get the space shift between the two images:
xc = xcorr2(fig1_pad,fig2_pad);
[max_cc, imax] = max(abs(xc(:)));
[ypeak, xpeak] = ind2sub(size(xc),imax(1));
corr_offset = [ (ypeak-size(fig2_pad,1)) (xpeak-size(fig2_pad,2)) ];
You then use circshift() to shift one of the images using the lag you obtained in the last step.
fig2_shift = circshift(fig2_pad,corr_offset);
You now have two images of the same size, where hopefully the sample is in the same position. If you want to remove the padding zeroes, crop the images to your liking with respect to the center using imcrop().
My goal is to write directly to pixels in a texture/sprite, to be able to update the game's details.
I was successful in doing this in Cocos2d-x with C++ (sample code below). The question is, "How can I do the equivalent in Javascript?"
//C++ WORKS:
board_texture= *sprite->getTexture();
// initialize pixels (array of GLubyte)
for (i=0 ; i<num_pix; i++){
pixels[i] = (GLubyte) 0;
}
// editing colors at specific positions:
pixels[i]=(GLubyte) 255; // r
pixels[i+1]=(GLubyte) 0; // g
pixels[i+2]=(GLubyte) 0;// b
pixels[i+3]=(GLubyte) 255; // a
board_texture.updateWithData(pixels,0,0,board_w,board_h);
sprite->setTexture(&board_texture);
This runs extremely fast on an iPad updating a 400x400 texture easily at 60FPS.
I've looked at Cocos2d-JS, however, and can find NO equivalent working functions for "updateWithData". I was able to get the texture2D into a local variable, but can find no way to update it's pixels (yet)
//JS does NOT work
var t;
t= this.map_sprite.getTexture(); // OK
// but does NOT allow me to updateWithData
t.setTexture(pixels) // fails, with Invalid Native Format. What's the proper format then?
Why? Cocos2d's developers are saying C++ is not supported in the future as much as Javascript (Cocos2d-js), as evidenced by the lack of JS support in the Cocos IDE. So, since I'm just starting out, I figured, I should do it in JS, not C++.
I am working with images of size 2 to 4MB. I am working with images of resolution 1200x1600 by performing scaling, translation and rotation operations. I want to add another image on that and save it to photo album. My app is crashing after i successfully edit one image and save to photos. Its happening because of images size i think. I want to maintain the 90% of resolution of the images.
I am releasing some images when i get memory warning. But still it crashes as i am working with 2 images of size 3MB each and context of size 1200x1600 and getting a image from the context at the same time.
Is there any way to compress images and work with it?
I doubt it. Even compressing and decompressing an image without doing anything to it loses information. I suspect that any algorithms to manipulate compressed images would be hopelessly lossy.
Having said that, it may be technically possible. For instance, rotating a Fourier transform also rotates the original image. But practical image compression isn't usually as simple as just computing a Fourier transform.
Alternatively, you could write piecemeal algorithms that chop the image up into bite-sized pieces, transform the pieces and reassemble them afterwards. You might also provide a real-time view of the process by applying the same transform to a smaller version of the full image.
The key will be never to full decode the entire image into memory at full size.
If you need to display the image, there's no reason to do that at full size -- the display on the iPhone is too small to take advantage of that. For image objects that are for display, decode the image in scaled down form.
For processing, you will need to write custom code that works on a stream of pixels rather than an in-memory array. I don't know if this is available on the iPhone already, but you can write it yourself by writing to the libpng library API directly.
For example, your code right now probably looks something like this (pseudo code)
img = ReadImageFromFile("image.png")
img2 = RotateImage(img, 90)
SaveImage(img2, "image2.png")
The key thing to understand, is that in this case, img is not the data in the PNG file (2MB), but the fully uncompressed image (~6mb). RotateImage (or whatever it's called) returns another image of about this same size. If you are scaling up, it's even worse.
You want code that looks more like this (but there might not be any API's for you to do it -- you might have to write it yourself)
imgPixelGetter = PixelDecoderFromFile("image.png")
imgPixelSaver = OpenImageForAppending("image2.png")
w = imgPixelGetter.Width
h = imgPixelGetter.Height
// set up a 90 degree rotate
imgPixelSaver.Width = h
imgPixelSaver.Height = w
// read each vertical scanline of pixels
for (x = 0; x < w; ++x) {
pixelRect = imgPixelGetter.ReadRect(x, 0, 1, h) // x, y, w, h
pixelRect.Rotate(90); // it's now got a width of h and a height of 1
imgPixelSaver.AppendScanLine(pixelRect)
}
In this algorithm, you never had the entire image in memory at once -- you read it out piece by piece and saved it. You can write similar algorithms for scaling and cropping.
The tradeoff is that it will be slower than just decoding it into memory -- it depends on the image format and the code that's doing the ReadRect(). Unfortunately, PNG is not designed for this kind of access to the pixels.