My supervisor has asked me to convert the parts of our Perl scripts that use PerlMagick to instead pipe and use the command line version of ImageMagick (for various unrelated reasons).
Using the our existing interface (crop, scale, save, etc) I'm building up a list of operations the user wants to perform on an image, constructing the statement to pipe and then executing it.
What I would like to know is:
Are convert operations performed from left to right? ie the order I pass them
What happens if I pass the same option twice? Are they performed separately?
Obviously the order in which operations are performed on an image is vital, so I'm trying to work out if I can perform all of the operations in one go (possibly gaining efficiency?) or if it I'm going to have to perform each operation separately.
Thanks
Unfortunately, the accepted answer to this question is not yet complete... :-)
Three (major) classes of parameters
Assuming, your ImageMagick version is a recent one, here is an important amendment to it:
you should differentiate between 3 major classes of command line parameters:
Image Settings
Image Operators
Image Sequence Operators
These three classes do behave differently:
Image Settings
An image setting persists as it appears on the command line.
It may affect all subsequent processing (but not previous processing):
processing such as reading an image or more images later in the command line;
processing done by a following image operator;
processing conducted by writing an image as output.
An image setting stays in effect...
...either until it is reset or replaced by a different setting of the same type,
...or until the command line terminates.
Image Operators
An image operator is applied to a (single) image and forgotten.
It differs from an image setting because it affects the image immediately as it appears on the command line.
(Remember: an image setting which persists until the command line terminates, or until it is reset.)
If you need to apply the same image operator to a second image in the same commandline, you have to repeat that exact operator on the commandline.
Strictly speaking, in compliance with the new architecture of ImageMagick command lines, all image operators should be written after the loading of the image it is meant for.
However, the IM developers compromised:
in the interest of backward compatibility, image operators can still appear before loading an image -- they will then be applied to the first image that is available to them.
Image Sequence Operators
An image sequence operator is applied to all currently loaded images (and then forgotten).
It differs from a simple image operator in that it does not only affect a single image.
(Some operators only make sense if their operation has multiple images for consumption: think of -append, -combine, -composite, -morph...)
From above principles you can already conclude: the order of the command line parameters is significant in most cases. (If you know what they do, you also know which order you need to use applying them.)
(For completeness' sake I should add: there is another class of miscellanious, or other parameters, which do not fall into any of the above listed categories. Think -debug, -verbose or -version.)
Unfortunately, the clear distinction between the 3 classes of IM command line paramaters is not yet common knowledge among (otherwise quite savvy) IM users. So it merits to get much more exposure.
This clear differentiation was introduced with ImageMagick major version 6. Before, it was more confusing: some settings' semantics changed with context and also with the order they were given. Results from complex commands were not always predictable and sometimes surprising and illogical. (Now they may be surprising too sometimes, but when you closely look at them, understanding the above, they are always quite logical.)
Which is which ?!?
Whenever you are not sure, which class one particular parameter belongs to, run
convert -help | less
Search for your parameter. Once found, scroll back: you should then find the "heading" under which it appears. Now you can be sure which type it is: an Image Setting, an Image Operator, or an Image Sequence Operator -- and take into account what I've said about them above.
Some more advice
If your job is to port your ImageMagick interface from PerlMagick to CLI, you should be aware of one other trick: You can insert
+write output-destination
anywhere on the command line (even multiple times). This will then write out the currently loaded image (or the currently loaded image sequence) in its currently processed state to the given output destination. (Think of it as something similar as the tee-command for shell/terminal usage, which re-directs a copy of <stdout> into a file.) Output destination can be a file, or show: or whatever else is valid for IM outputs. After writing to the output, processing of the complete command will continue.
Of course, it only makes sense to insert +write after the first (or any other) image operator -- otherwise the current image list will not have changed.
Should there by multiple output images (because the current image list consists of more than one image), then ImageMagick will automatically assign index numbers to the respective filename.
This is a great help with debugging (or optimizing, streamlining, simplifying...) complex command setups.
Are convert operations performed from left to right? ie the order I
pass them
Yes. If I take the following two examples, which are identical except for the operations order, I can expect different results based on the left to right.
convert rose: -sample 300% -wave 5x10 rose_post_wave.png
convert rose: -wave 5x10 -sample 300% rose_pre_wave.png
You can see the effects of the wave operation impact the image after, or before the sampling of the image.
What happens if I pass the same option twice? Are they performed
separately?
The will be executed twice. No special locking, or automatic operation counting exists.
convert rose: -blur 0.5x0.5 -scale 300% rose_blur1.png
convert rose: -blur 0.5x0.5 -blur 0.5x0.5 -scale 300% rose_blur2.png
Related
My problem is the following:
I have very many (~1000) mutually calling Matlab scripts, which are very poorly written, regularly damage each other's environments and generally became unmanageable.
One of the reasons I even got this problem is that I need to write a testsuite covering a big part of them. Luckily, for most of them the main criterion of 'correctness' is 'they don't crash'.
Just running them one by one in a loop is generally not an option, because they regularly call clear classes, close all, clc, shadow built-in functions and operators, et cetera.
So my original aim was to find a way to run a matlab script in sort of an 'isolated environment', but I didn't find a good way to do it. (Suggestions welcome, but it is not the main question.)
Since I will need to convert them all to functions anyway, I am looking for some way to do it auto-magically, or at least semi-automatically.
What I can mean semi-automatically:
Just add a line function varargout = $filename( varargin ) as the first line of the file, and end as the last one. This will at least make them runnable as functions with feval and all such functions and (more importantly) prevent them from damaging the test-runner.
Do point 1 and scan the file for referencing undeclared variables and add them as function arguments. This should be also doable, since the names of the variables are known. This will not help identifying output variables, but will still be a lot of help. For example, we could pack the whole workspace into one big output structure.
Do a runtime version of point 2. This way the 'magical converter' can actually track execution environments (workspaces) and identify which variables are implicitly used as 'input arguments' of a script, and which would be later used 'output arguments'. This option looks EXPHARD, but for a small number of calls should be not too bad in practice.
Point 1 I can implement myself using sed, as I also can get rid of all clear classes and clc, but the options 2 and 3 seem much harder. Is there anything at least remotely resembling options 2 or 3?
I want to have a std.m file for the standard deviation. It is in data fun toolbox, but, by mistake, I changed the code and the std command is not working anymore. How can I run the original std (standard deviation) command?
Taking all the comments out, the function std.m is actually extremely simple:
function y = std(varargin)
y = sqrt(var(varargin{:}));
This is the definition of the standard deviation: the square root of the Variance.
Set built-in functions to Read-Only
Now don't go breaking the var.m file because it is more complex and I wonder if there would be copyright issue to display the listing here.
To avoid the problem of breaking built-in files, it is advisable to set all your Matlab toolbox files as Read Only. I remember old Matlab installer giving the option to do that at install time. I don't know if the installer still offers the option, but if not it is extremely easy to do it manually (on Windows, just select your folders, right-click Properties, tick read only and accept to propagate the property to all subfolders and files).
Overloading
Once this is done, your built-in files are safe. You can still modify the default behavior of a built-in function by overloading it. This consist in writing a function with the same name and arrange for it to be called before the default function (your overload function becomes the default one).
This article explain how to overload user functions.
Matlab does not recommend to directly overload the built-in functions (rather call it another name like mySTD.m for example), but if you insist it is perfectly feasible and still a much better option than modifying the built-in function... at least the original function is still intact somewhere.
I am defining a variable in the beginning of my source code in MATLAB. Now I would like to know at which lines this variable effects something. In other words, I would like to see all lines in which that variable is read out. This wish does not only include all accesses in the current function, but also possible accesses in sub-functions that use this variable as an input argument. In this way, I can see in a quick way where my change of this variable takes any influence.
Is there any possibility to do so in MATLAB? A graphical marking of the corresponding lines would be nice but a command line output might be even more practical.
You may always use "Find Files" to search for a certain keyword or expression. In my R2012a/Windows version is in Edit > Find Files..., with the keyboard shortcut [CTRL] + [SHIFT] + [F].
The result will be a list of lines where the searched string is found, in all the files found in the specified folder. Please check out the options in the search dialog for more details and flexibility.
Later edit: thanks to #zinjaai, I noticed that #tc88 required that this tool should track the effect of the name of the variable inside the functions/subfunctions. I think this is:
very difficult to achieve. The problem of running trough all the possible values and branching on every possible conditional expression is... well is hard. I think is halting-problem-hard.
in 90% of the case the assumption that the output of a function is influenced by the input is true. But the input and the output are part of the same statement (assigning the result of a function) so looking for where the variable is used as argument should suffice to identify what output variables are affected..
There are perverse cases where functions will alter arguments that are handle-type (because the argument is not copied, but referenced). This side-effect will break the assumption 2, and is one of the main reasons why 1. Outlining the cases when these side effects take place is again, hard, and is better to assume that all of them are modified.
Some other cases are inherently undecidable, because they don't depend on the computer states, but on the state of the "outside world". Example: suppose one calls uigetfile. The function returns a char type when the user selects a file, and a double type for the case when the user chooses not to select a file. Obviously the two cases will be treated differently. How could you know which variables are created/modified before the user deciding?
In conclusion: I think that human intuition, plus the MATLAB Debugger (for run time), and the Find Files (for quick search where a variable is used) and depfun (for quick identification of function dependence) is way cheaper. But I would like to be wrong. :-)
This question is really in two parts. To briefly introduce the issue, we have a requirement to take a CNC file (used with a Roland milling machine) that has been produced using a tool called ArtCam, and modify it to shift the absolute position of the pattern being cut.
The software produces, and the machine accepts, input files in the following form:
;;^IN;
!MC1;
!RC5000;
V50.0;
^PR;Z0,0,10500;
^PA;
V49.8;
Z0,0,1000;
V39.8;
Z0,0,100;
Z10,0,99;
Z1000,0,-13;
Z10,0,-124;
Z0,0,-125;
...thousands more Zx,y,z; instructions...
The first part to my question is, can anyone actually tell me what this file format is called? It's clearly not G-Code, and I haven't been able to find any reference or documentation for it anywhere.
The second part is, does anyone know how we might easily modify the absolute position of the pattern that these files cut. Obviously the Z lines are X,Y,Z position commands but I don't know if they're absolute or relative, and I don't know in what coordinate space/system they are. For all I know there might be a simple command we can add at the top that shifts the starting point, or we might need to rewrite all the Z lines, but without some information on the file format I'm at a dead end.
Thanks!
I realise this is an old question and you maybe already have an answer (or have no need for one now) but it looks like it's RML-1, assuming my searches were correct.
I first found this which showed very similar code to your example. It mentions ArtCAM and output for the MDX-540, a Roland machine.
Searching Roland's milling machines for information was a bit useless, but going through their 3D products for the MDX-540 mentions that the control command sets is "RML-1 and NC codes".
Then searching for RML-1 gives a result for a PDF manual.
Reading that PDF it looks like the single letter commands are "Mode 1", the ^ is used to select Mode2 and the 2 letter commands are Mode2 commands. !xx commands are common to both Mode1 and Mode2.
^PR sets the movement to relative mode.
^PA sets the movement to absolute mode.
Z moves.
Looking at your code sample it appears as if most positions are absolute and you'd need to re-write them all.
I am optimizing a very time/memory consuming program by running it over a dataset and under multiple parameters. For each "run", I have a csv file, "setup.csv" set up with "runNumber","Command" for each run. I then import this into a perl script to read the command for the run number I would like, extrapolate the variables, then execute it on the system via the system command. Should I be worried about the potential for this to be exploited, (I am worried right now)? If so, what can I do to protect our server? My plan now is to change the file permissions of the "setup.csv" to read only and ownership to root, then go in as root whenever I need to append another run to the list.
Thank you very much for your time.
Run your code in taint mode with -T. That will force you to carefully launder your data. Only pass through strings that are ones you are expecting. Do not launder with .*, but rather check against a list of good strings.
Ideally, there a list of known acceptable values, and you validate against that.
Either way, you want to avoid the shell by using the multi-argument form of system or by using IPC::System::Simple's systemx.
If you can't avoid the shell, you must properly convert the text to pass to the command into shell literals.
Even then, you have to be careful of values that start with -. Lots of tools accept -- to denote the end options, allowing other values to be passed safely.
Finally, you might want to make sure the args don't contain the NUL character (\0).
systemx('tool', '--', #args)
Note: Passing arbitrary strings is not possible in Windows. Extra validation is required.