I'm trying to convert an image to a sound where you can see the image if you were to view the spectrogram of that sound. Kind of like the aphex twin had done in window licker.
So far I have written an iPhone app that takes a photograph and then converts it to grayscale. I then use this gray scale as a magnitude which I'd like to plug back through an inverse FFT.
The problem I have, though, is how do I go from magnitude into the imaginary and real parts.
mag = sqrtf( (imag * imag) + (real * real));
Obviously I can't solve for 2 unknowns. Furthermore I can't find out if those real and imaginary parts are negative or not.
So I'm at a bit of a loss. It must be possible. Can anyone point me in the direction of some useful information?
A spectrogram contains no phase information, so you can just set the imaginary parts to 0 and set the real parts equal to the magnitude. Remember that you need to maintain complex conjugate symmetry if you want to end up with a purely real time domain signal after you have applied the inverse FFT.
The math wonks are right about regenerating from greyscale, but why limit yourself thus? Have you considered keeping a portion of the phase information in the color channels?
Specifically, why not process the LEFT channel into BLUE, the RIGHT channel into RED, and for the GREEN color element, run the transform again on (LEFT-RIGHT), so that you have three spectra.
In one version of "Surround Sound", L-R encodes the rear channel - there is good stuff there.
When regenerating your sound, assign the "real" values to the corresponding channels.
Try the following (formulas - but this editor insists on calling them code..)
LEFT.real=+BLUE
RIGHT.real=+RED
LEFT.imag=+GREEN
RIGHT.imag=-GREEN
Experiment with variations on this, while listening thru some sort of surround sound setup, to see which provides the most pleasing results. Make sure not to drive the thing into clipping, since phase changes occur, regeneration of a complex saturated signal is likely to create clipping.
Related
I have several images that i would like to correct from artifacts. They show different animals but they appear to look like they were folded (look at the image attached). The folds are straight and they go through the wings as well, they are just hard to see but they are there. I would like to remove the folds but at the same time preserve the information from the picture (structure and color of the wings).
I am using MATLAB right now and i have tried several methods but nothing seems to work.
Initially i tried to see if i can see anything by using an FFT but i do not see a structure in the spectrum that i can remove. I tried to use several edge detection methods (like Sobel, etc) but the problem is that the edge detection always finds the edges of the wings (because they are stronger)
rather than the straight lines. I was wondering if anyone has any ideas about how to proceed with this problem? I am not attaching any code because none of the methods i have tried (and described) are working.
Thank you for the help in advance.
I'll leave this bit here for anyone that knows how to erase those lines without affecting the quality of the image:
a = imread('https://i.stack.imgur.com/WpFAA.jpg');
b = abs(diff(a,1,2));
b = max(b,[],3);
c = imerode(b,strel('rectangle',[200,1]));
I think you should use a 2-dimensional Fast Fourier Transform
It might be easier to first use GIMP / Photoshop if a filter can resolve it.
I'm guessing the CC sensor got broken (it looks to good for old scanner problems). Maybe an electric distortion while it was reading the camera sensor. Such signals in theory have a repeating nature.
I dont think this was caused by a wrong colordepth/colorspace translation
If you like to code, then you might also write a custom pixel based filter in which you take x vertical pixels (say 20 or so) compare them to the next vertical row of 20 pixels. Compare against HSL (L lightnes), not RGB.
From all pixels calculate brightness changes this way.
Then per pixel check H (heu) is within range of nearby pixels take slope average of their brightness(ea take 30 pixels horizontal, calculate average brightnes of first 10 and last 10 pixels apply that brightness to center pixel 15,... //30, 15, 10 try to find what works well
Since you have whole strokes that apear brighter/darker such filter would smooth that effect out, the difficulty is to remain other patterns (the wings are less distorted), knowing what color space the sensor had might allow for a better decision as HSL, maybe HSV or so..
The specs for the web audio API dynamic compressor node refer to some curve being drawn over various decibel values. How can I visualize that curve?
For filter nodes, the web audio API provides a getFrequencyResponse method that produces data that can be visualized on a 2D canvas.
Is there a similar method for the dynamic processor node? Or are there well-known formulas used to compute the magnitude of the node's effect on various dB values?
I'm not sure exactly how to calculate the curve for knee, but I'm pretty sure it shouldn't be super difficult. Ignoring the knee, here's what you'd need:
First, you'd start out with a line that has a slope of 1 (45 degree angle, up and to the right). Another way of saying that is that output = input
Then, when you hit threshold, you change the slope of the line to match your compression ratio. So if your ratio is 2.3:1, your slope above the threshold would be output = input / 2.3.
Anyway, I'm sure if you do some searching, you can figure out how to factor in the knee. It's probably just a parabola that joins the two slopes (with a vertex at the point where they would normally intersect if the knee was 0). Then you just need to figure out what the value does, but if you read the Web Audio spec, the unit for knee is dB – which leads me to believe this isn't really implementation-specific. I think there probably is a Right Way™ to do it.
Unfortunately there is no way to examine easily the effect of the dynamic compressor node. And the actual implementation isn't specified in the WebAudio spec. The only way to know the effect is to examine the source code. Or perhaps feed a sine wave of different frequencies to the node and examine the output to see what is happening, experimentally. This might be hard to capture the effect of all of the parameters.
I'm trying to write a code The helps me in my biology work.
Concept of code is to analyze a video file of contracting cells in a tissue
Example 1
Example 2: youtube.com/watch?v=uG_WOdGw6Rk
And plot out the following:
Count of beats per min.
Strenght of Beat
Regularity of beating
And so i wrote a Matlab code that would loop through a video and compare each frame vs the one that follow it, and see if there was any changes in frames and plot these changes on a curve.
Example of My code Results
Core of Current code i wrote:
for i=2:totalframes
compared=read(vidObj,i);
ref=rgb2gray(compared);%% convert to gray
level=graythresh(ref);%% calculate threshold
compared=im2bw(compared,level);%% convert to binary
differ=sum(sum(imabsdiff(vid,compared))); %% get sum of difference between 2 frames
if (differ ~=0) && (any(amp==differ)==0) %%0 is = no change happened so i dont wana record that !
amp(end+1)=differ; % save difference to array amp wi
time(end+1)=i/framerate; %save to time array with sec's, used another array so i can filter both later.
vid=compared; %% save current frame as refrence to compare the next frame against.
end
end
figure,plot(amp,time);
=====================
So thats my code, but is there a way i can improve it so i can get better results ?
because i get fealing that imabsdiff is not exactly what i should use because my video contain alot of noise and that affect my results alot, and i think all my amp data is actually faked !
Also i actually can only extract beating rate out of this, by counting peaks, but how can i improve my code to be able to get all required data out of it ??
thanks also really appreciate your help, this is a small portion of code, if u need more info please let me know.
thanks
You say you are trying to write a "simple code", but this is not really a simple problem. If you want to measure the motion accuratly, you should use an optical flow algorithm or look at the deformation field from a registration algorithm.
EDIT: As Matt is saying, and as we see from your curve, your method is suitable for extracting the number of beats and the regularity. To accuratly find the strength of the beats however, you need to calculate the movement of the cells (more movement = stronger beat). Unfortuantly, this is not straight forwards, and that is why I gave you links to two algorithms that can calculate the movement for you.
A few fairly simple things to try that might help:
I would look in detail at what your thresholding is doing, and whether that's really what you want to do. I don't know what graythresh does exactly, but it's possible it's lumping different features that you would want to distinguish into the same pixel values. Have you tried plotting the differences between images without thresholding? Or you could threshold into multiple classes, rather than just black and white.
If noise is the main problem, you could try smoothing the images before taking the difference, so that differences in noise would be evened out but differences in large features, caused by motion, would still be there.
You could try edge-detecting your images before taking the difference.
As a previous answerer mentioned, you could also look into motion-tracking and registration algorithms, which would estimate the actual motion between each image, rather than just telling you whether the images are different or not. I think this is a decent summary on Wikipedia: http://en.wikipedia.org/wiki/Video_tracking. But they can be rather complicated.
I think if all you need is to find the time and period of contractions, though, then you wouldn't necessarily need to do a detailed motion tracking or deformable registration between images. All you need to know is when they change significantly. (The "strength" of a contraction is another matter, to define that rigorously you probably would need to know the actual motion going on.)
What are the structures we see in the video? For example what is the big dark object in the lower part of the image? This object would be relativly easy to track, but would data from this object be relevant to get data about cell contraction?
Is this image from a light microscop? At what magnification? What is the scale?
From the video it looks like there are several motions and regions of motion. So should you focus on a smaller or larger area to get your measurments? Per cell contraction or region contraction? From experience I know that changing what you do at the microscope might be much better then complex image processing ;)
I had sucsess with Gunn and Nixons Dual Snake for a similar problem:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.64.6831
I placed the first aproximation in the first frame by hand and used the segmentation result as starting curv for the next frame and so on. My implementation for this is from 2000 and I only have it on paper, but if you find Gunn and Nixons paper interesting I can probably find my code and scan it.
#Matt suggested smoothing and edge detection to improve your results. This is good advise. You can combine smoothing, thresholding and edge detection in one function call, the Canny edge detector.Then you can dialate the edges to get greater overlap between frames. Little overlap will probably mean a big movement between frames. You can use this the same way as before to find the beat. You can now make a second pass and add all the dialated edge images related to one beat. This should give you an idea about the area traced out by the cells as they move trough a contraction. Maybe this can be used as a useful measure for contraction of a large cluster of cells.
I don't have access to Matlab and the Image Processing Toolbox now, so I can't give you tested code. Here are some hints: http://www.mathworks.se/help/toolbox/images/ref/edge.html , http://www.mathworks.se/help/toolbox/images/ref/imdilate.html and http://www.mathworks.se/help/toolbox/images/ref/imadd.html.
I've done work on software used for controlling imaging hardware, such as microscopes, that are sometimes hard to get time on. This means it is difficult to test out new/different algorithms which would require access to the instrument. I'd like to create a synthetic instrument that could be used for some of these testing purposes, and I was thinking of using some kind of fractal image generation to create the synthetic images. The key would be to be able to generate features at many different 'magnifications' and locations in some sort of deterministic manner. This is because some of the algorithms being tested may need to pan/zoom and relocate previously 'imaged' areas. Onto these base images I can then apply whatever instrument 'defects' are appropriate (focus, noise, saturation, etc.).
I'm at a bit of a loss on how to select/implement a good fractal algorithm for the base image. Any help would be appreciated. Preferably it would have the following qualities:
Be fast at rendering new image areas.
Fairly wide 'feature' coverage at as many locations and scales as possible.
Be deterministic (but initialized from random starting parameters).
Ability to tune to make images look more like 'real' images.
Item 2 is important, for example a mandelbrot set, with its large smooth/empty regions, might not be good since the software controlling the synthetic scope might fall into one of these areas.
So far I've thought of using something like a mandelbrot, but randomly shifting/rotating/scaling and merging two or more fractal sets to get more complete 'feature' coverage.
I've also seen images of the fractal flame algorithms and they seem to generate images that might be useful (and nice to look at).
Finally, I've thought of using some sort of paused particle simulation run to generate images that are more cell-like (my current imaging target), but I'm not sure if this approach can be made to work with the other requirements.
Edit:
#Jeffrey - So it sounds like some kind of terrain generation might be the way to go, as long as I have complete control over the PSRNG. Perhaps I can use some stored initial seed + x position + y position to generate my random numbers? But then I am unsure of how to consistently generate the terrains across scales, except, as you mentioned, to create the base terrain at the coursest scale, and at certain pre-determined 'magnifications' add new deterministic pseudo-random variations to this base. I'd also have to be careful about when to generate the next level of terrain, since if I'm too aggressive I'd have to generate and integrate the results appropriately for display at the coarser level... This is why I initially was leaning toward a more 'traditional' fractal, since this integration from finer scales would be handled more implicitly (I think).
The idea behind a fractal terrain creation algorithm is to build the image at each scale separately. For a landscape it's easy: just make a small array of height values, and set them randomly. Then scale it up to a larger array, averaging the values so that the contour is smooth, and then add small random amounts to those values. Then scale it up, etc. The original small bumps have become mountains, and they are filled with complex terrain.
There are two particular difficulties with the problem posed here, though. First, you don't want to store any of these values, since it would be potentially huge. Secondly, the features at each scale are of a different kind than the features at other scales.
These problems are not insurmountable.
Basically, you would divide the image up into a grid, and using deterministic psedorandom numbers establish the key features of each square in the grid. For example, each square could have a certain density of cell types.
At the next level of magnification, subdivide each square into another grid, apply a gradiant of values across the grid that is based on the values of the containing square and its surrounding squares. Then apply pseudorandom variations to that seeded with the containing square's grid coordinates. For the random seed, always use the coordinates of the immediately containing square of the subdivision under consideration regardless of where the image is cropped, in order to ensure that it is recreated correctly accross multiple runs.
At some level of magnification the random values go from being densities of paticles types to particle locations. Then for each particle, there are partical features. Then features on those features.
Although arbitrary left/right and up/down scrolling will be desired, the image at all levels of magnification above the current scene will have to be calculated each time the frame is shifted to ensure that all necessary features are included. This way the image can be scrolled from one cell to another without loss of consistancy. Partical simulations can be used to ensure that cells or cell features don't overlap. This could be done in a repeatable, deterministic manner.
And don't forget to apply a smoothing gradient based on averages of surrounding squares at higher levels before adding in the random variations. Otherwise, the abrupt changes will make the squares themselves appear in the images!
This answer is somewhat rambling and probably confusing, but that is best I can explain it right now. I hope it helps!
I'm looking to do background subtracting on an image. I'm new to MATLAB and new to image processing/analysis, so sorry if any of this sounds stupid. 1) Other than imsubtract() are there other ways to do background subtracting (besides comparing one image to another)? 2) In the Math Works explanation for imsubtract() why do they make their structuring element a disk? This seems rather difficult so far because every time I try something, I end up not only subtracting the noisy background but also losing the parts of the image I want to look at!
What kind of image do you work with?
Background subtraction is easy. If you want to subtract a constant value, or a background with the same size as your image, you simply write img = img - background. imsubtract simply makes sure that the output is zero wherever the background is larger than the image.
Background estimation is hard. There you need to know what kind of image you're looking at, otherwise, the background estimation will fail.
If you have, for example, spot or line features that are either all dark on bright or bright on dark background, you can pass through with a local maximum filter (imdilate) or a local minimum filter (imerode), respectively, that is larger than your features, so that wherever you place the filter mask, there are some pixels that cover background. Also, you want the filter to have somewhat similar shape as the features. In your case, if you lose part of your image, you may want to try and make the filter larger (but not too large).
Instead of subtracting maximum or minimum, subtracting the median can work well, though you have to choose the filter size such that there's usually a majority of background pixels inside the filter mask. Unfortunately, median filtering is rather slow.
To subtract the background image, you need a model of the background. The simplest model is an image captured as the background along with some allowable deviation (+/- 0-255). Then, background subtraction in MATLAB is pretty simple:
image( find(abs(image-background) <= threshold) ) = 0;
It becomes more difficult when you use a statistical model, but essentially subtracting the background is pretty easy. imsubtract is NOT background subtraction; it is a subtraction filter like you would find in photoshop. It does not care about background versus foreground, which then defeats the point.
Since background subtraction itself is pretty easy, the question becomes more about background estimation. This is a bit more complicated, and generally requires more frames and training to build statistical models of the background (for example, looking at pixels as Gaussian distributions or mixtures of Gaussians, or looking instead at optical flow to determine what's not moving).
If you have access to technical articles (via work or school), "Pfinder: Real-Time Tracking of the Human Body" by Wren and others gives a pretty simple approach. Or you can just search google for single Gaussian background subtraction. There are a number of methods implemented with OpenCV here --> http://dparks.wikidot.com/source-code <-- that you might find useful.
The Computer Vision System Toolbox has the vision.ForegroundDetector object, which implements a variant of Stauffer and Grimson's GMM background subtraction. The implementation is very fast, leveraging multiple cores. Check out this example of how to use background subtraction as a building block of a system for tracking multiple objects.