Basically my question is like this one, but from PNG instead of JPEG.
More specifically, I have a bunch of PNG images concatenated together and I want to discover their lengths, so I can split the stream in pieces corresponding to individual images.
I do not need to decode or validate the images. I can assume that the input stream is composed of valid PNG images and do not want to verify that. Instead, it is useful for me to do this as quickly as possible, so the less amount of decoding operations are required, the better.
Here you have a perl script, based on this answer
#!/usr/bin/perl
undef $/;
$_ = <>;
$n = 0;
for $match (split(/(?=\x{89}PNG\x{0d}\x{0a}\x{1a}\x{0a})/)) {
open(O, sprintf('>temp%04d.png',++$n));
print O $match;
close(O);
}
Save this as, say splitpng.pl and run perl splitpng.pl < myfile
This is not 100% foolproof (the rigourous way would be to count chunks sizes, as per Jongware's comment), but the probabilty of having that signature inside a PNG should be small.
Related
I have a fixed width file format (original was input for a Fortran routine). Several lines of the file look like the below:
1078.0711005.481 932.978 861.159 788.103 716.076
How this actually should read:
1078.071 1005.481 932.978 861.159 788.103 716.076
I have tried various methods, textscan, fgetl, fscanf etc, however the problem I have is, as seen above, sometimes because of the fixed width of the original files there is no whitespace between some of the numbers. I cant seem to find a way to read them directly and I cant change the original format.
The best I have come up with so far is to use fgetl which reads the whole line in, then I reshape the result into an 8,6 array
A=fgetl
A=reshape(A,8,6)
which generates the following result
11
009877
703681
852186
......
049110
787507
118936
So now I have the above and thought I might be able to concatenate the rows of that array together to form each number, although that is seeming difficult as well having tried strcat, vertcat etc.
All of that seems a long way round so was hoping for some better suggestions.
Thanks.
If you can rely on three decimal numbers you can use a simple regular expression to generate the missing blanks:
s = '1078.0711005.481 932.978 861.159 788.103 716.076';
s = regexprep(s, '(\.\d\d\d)', '$1 ');
c = textscan(s, '%f');
Now c{1} contains your numbers. This will also work if s is in fact the whole file instead of one line.
You haven't mentioned which class of output you needed, but I guess you need to read doubles from the file to do some calculations. I assume you are able to read your file since you have results of reshape() function already. However, using reshape() function will not be efficient for your case since your variables are not fixed sized (i.e 1078.071 and 932.978).
If I did't misunderstand your problem:
Your data is squashed in some parts (i.e 1078.0711005.481 instead
of 1078.071 1005.481).
Fractional part of variables have 3 digits.
First of all we need to get rid of spaces from the string array:
A = A(~ismember(A,' '));
Then using the information that fractional parts are 3 digits:
iter = length(strfind(A, '.'));
for k=1:iter
[stat,ind] = ismember('.', A);
B(k)=str2double(A(1:ind+3));
A = A(ind+4:end);
end
B will be an array of doubles as a result.
I have a singe image.tiff file, a video sequence exported as 32 bit tiff. I would like to open it as an image stack in MATLAB, and be able to navigate frame by frame. I believe implay() is the way to do this in matlab. If I try this I get "Error occurred while attempting to read file: image.tiff Details of error: Incorrect chunk size information in AVI file." Does implay() only work with the .avi format? do I need to covert this 32 tiff to a .avi before i can use implay()? or is there maybe some other (non-implay()) way of opening this as a stack?
Thanks
You could try to create an image stack and use implay to view it. The function accepts multiple types of arguments, for grayscale images it should be provided with an array of size N x M x K where K is the number of frames, (N,M) is the image size. For color images an array of size NxMx3xK is expected.
To create the array for the case with multiple files, each containing a frame you have multiple options, the simplest is probably to use the cat function for concatenation:
image_stack = [];
for i = 1: num_frames
curr_image = imread(sprintf('frame_%04d_color.tif', i));
image_stack = cat(4, image_stack, curr_image);
end
implay(image_stack);
This solution is a bit slower, than if the image_stack is allocated beforehand though.
For your case with a single TIFF file, the frames need to be extracted in a manner suitable for the storage format, but this is a separate problem from the video replay.
I found the following ImageMagick command in some forum and works nicely for compare images.
convert image1 image2 -compose Difference -composite -format '%[fx:mean*100]' info:
The result is one floating point number and low values (like 0.5 and such) mean: the images are similar.
Using the attached images, it produces the number: 0.0419167. (the images are very similar)
I want to use Image::Magick (perlmagick). The problem is i don't know how to achieve the same result with perlmagick. The following works, and creates the composite, (black image using the attached images)
#!/usr/bin/env perl
use 5.014;
use strict;
use warnings;
use Data::Dumper;
use Image::Magick;
my $i1 = Image::Magick->new;
$i1->Read('s1.jpg');
my $i2 = Image::Magick->new;
$i2->Read('s2.jpg');
$i1->Composite(image => $i2, compose=>'Difference');
$i1->Display();
The question is, how to convert the result to an number, e.g. how to achieve the
... -format '%[fx:mean*100]' info:
part of the above command in PerlMagick for getting only the above "number"?
Is someone want test, attaching two images:
I am guessing you want to call
my $format = $iI->Fx( expression=>'mean*100' );
This should do the same thing as what you had on command line.
see here for more detailed documentation of fx in PerlMagick
( there is an example fx line on the page )
On the same page: search for #statistics.
Seems to me the mean is accessible via
my #stats = $i1->Statistics;
my $mean = $stats[3]; # hash would be nice, mean is 4th according to docs
print "$mean\n"; # outputs something like .0413 for me
Not sure if this is what you need, but that is how I found the 'mean',
whether this is precisely what fx mean does I am not certain and honestly not willing to understand the entire doc on the fx method ;)
BTW, the script I still had was based on Randall Schwartz's post
Just found the answer.
Image::Magick has the Statistics method, so calling:
my #stat = $i1->Statistics();
say Dumper \#stat
prints image stats for each channel, like:
$VAR1 = [
'8',
'0',
'0.168627450980392',
'0.0418661437908497', <--- mean RED
'0.0236850206077085',
'-0.250788990333716',
'0.312238727773058',
'8',
'0',
'0.133333333333333',
'0.0385273202614379', <- mean GREEN
'0.0193189321997568',
'-0.435325792918546',
'0.0686645009183608',
'8',
'0',
'0.23921568627451',
'0.0453563725490196', <- mean BLUE
'0.0301331898766906',
'0.309072091600589',
'0.66336367830764'
];
e.g. averaging the 3 numbers, got the wanted number: 0.0419166122004357
use List::Util qw(sum)
my $s = sum #stat[3,10,17];
say $s/3;
From the docs: misc methods
Statistics() returns the image statistics for each channel in the
image. The returned values are an array of depth, minima, maxima,
mean, standard deviation, kurtosis, skewness, and entropy values in
RGB, CMYK, RGBA, or CMYKA order (depending on the image type).
I have a square matrix file, and I want to load it into Perl. I want to operate like 2 dimensional array which in C is like matrix[14][50], and then it goes directly to row 14 column 50. Is there a way to do this?
And can I modify the input file directly or I have to load it into a variable, do the operation and then write it out?
I have written a module which is likely able to do what you need. Tie::Array::CSV. It creates a magical array of arrayrefs (a Perl 2D array), which allows row and element r/w access to a file. Depending on your column separator you might need to adjust the options (CSV is default).
It works very similarly with different syntax.
>matrix <- array(1:10000, dim=c(100,100)) #50 X 50 matrix
>somevariable <- matrix[14,51] #somevariable will now be 5014
Turn this into a R script
and call it from perl such as
my $var = `rscript arguments`; # $var is now the output of your rscript
I am currently working on building a classifier using c5.0. I have a dataset of 8000 entries and each entry has its own i.d number (1-8000). When testing the performance of the classifier I had to make 5sets of 10:90 (training data: test data) splits. Of course any training cases cannot appear again in the test cases, and duplicates cannot occur in either set.
To solve the problem of picking examples at random for the training data, and making sure the same cannot be picked for the test data I have developed a horribly slow method;
fill a file with numbers from 1-8000 on separate lines.
randomly pick a line number (from a range of 1-8000) and use the contents of the line as the id number of the training example.
write all unpicked numbers to a new file
decrement the range of the random number generator by 1
redo
Then all unpicked numbers are used as test data. It works but its slow. To speed things up I could use List::Util 'shuffle' to just 'randomly' shuffle and array of these numbers. But how random is 'shuffle'? It is essential that the same level of accuracy is maintained. Sorry about the essay, but does anyone know how 'shuffle' actually works. Any help at all would be great
Here is the shuffle algorithm used in List::Util::PP
sub shuffle (#) {
my #a=\(#_);
my $n;
my $i=#_;
map {
$n = rand($i--);
(${$a[$n]}, $a[$n] = $a[$i])[0];
} #_;
}
Which looks like a Fisher-Yates shuffle.