Difference between % and %% in ipython magic commands - ipython

What difference does it make to use %timeit and %%timeit in ipython? Because when I read the documentation using ?%timeit and ?%%timeit it was the same documentation. So, what difference does adding % as prefix make?

In general, one percentage sign is referred to as line magic and applies just to code that follows it on that same line. Two percentage signs is referred to as cell magic and applies to everything that follows in that entire cell.
As nicely put in The Data Science Handbook:
Magic commands come in two flavors: line magics, which are denoted by
a single % prefix and operate on a single line of input, and cell
magics, which are denoted by a double %% prefix and operate on
multiple lines of input.
Some magic commands, like timeit, can work as line magic or cell magic:
Used as line magic:
%timeit y = 2 if x < 3 else 4
Used as cell magic:
%%timeit
if x < 3:
y=2
else:
y=4

Related

Matlab "auto-squeeze" plot

Matlab plot requires the data to be of the same dimension. Meaning, you cannot plot a 1x10 vector with a 1x1x10 vector. This is sometimes necessary. For those purposes, you can use the squeeze function to get rid of the singleton dimensions.
However, this is kind of a hassle. For the plot function specifically, it would be useful to have the argument always squeezed. How would one go about creating a new function, lets call it splot which squeezes every input and passes it onto plot. Here is an attempt (that doesn't work)
function splot(varargin)
for i=1:length(varargin)
varargin{i}=squeeze(varargin{i});
end
plot(varargin)
end
plot(varargin) part fails, because that is simply not how matlab syntax works. But is there any way to achieve what I want? I guess I could write a long if elseif chain where I manually write the case with every possible number of input arguments like:
if length(varargin)==2
plot(varargin{1},varargin{2})
if length(varargin)==3
plot(varargin{1},varargin{2},varargin{3})
But this is going to be very annoying. Any better ideas.
This question is similar to Is there any mechanism to auto squeeze in Matlab / Octave , however, not similar enough, because the other question is for squeezing every vector, which is a bad idea. Here I am asking a way to squeeze only the inputs to the plot function and requiring syntax help.
From the docs there are several ways to call plot. Generally
Just numeric arrays, these can be on their own, or one or more pairs
plot(Y), plot(X,Y) or plot(X1,Y1,...,Xn,Yn)
Numeric arrays as before, with a char array giving the line spec
plot(X,Y,LineSpec) or plot(Y,LineSpec)
Either of the previous two, plus name-value pair options
plot(___,Name,Value)
In any of these cases, you want to squeeze the first N inputs which are numeric, since either of the optional additions have the first non-plottable input as a char.
We can achieve that with the following code, see the comments for details:
function h = splot( varargin )
% Check if there are any optional inputs, which will either be
% LineSpec (which is a char) or name-value pairs (which the
% first of will be a char)
bNumericArg = cellfun( #isnumeric, varargin );
% By default, assume all inputs are arrays to plot
lastArrayArg = numel(varargin);
if ~all(bNumericArg)
% In this case, there are some optional inputs, get last array index
lastArrayArg = find( ~bNumericArg, 1 ) - 1;
end
% Squeeze the arrays
for ii = 1:lastArrayArg
varargin{ii} = squeeze(varargin{ii});
end
% Plot with all inputs, optional output
if nargout > 0
h = plot( varargin{:} );
else
plot( varargin{:} );
end
end
There are two possible cases I've not handled here which the plot function can handle,
Having the first input as the target axes i.e. plot(ax,___), could be achieved by altering the loop slightly to start from 1 or 2 depending if the first input is an axes object
Having pairs of arrays each with their own line spec argument i.e. plot(X1,Y1,LineSpec1,...,Xn,Yn,LineSpecn). The later pairs will be ignored. This would be trickier to handle since you'd likely have to parse all inputs and check whether a char is just a line spec or if you're messing with name-value pairs. Maybe a heuristic to do with "two arrays then a char, repeated". I've never used this syntax so omitting the over-complication for now.

Multi-line fprintf() using one element from each array per line

I've hit some unexpected behaviour using the fprintf() function in MATLAB. I'm trying to print a multi-line file using the contents of a cell array and a numerical array. I know that I can use the fprintf() function as follows to print out the contents of a cell array:
myCellArray = {'one','two','three'};
fprintf('%s\n',myCellArray{:})
This results in the following output:
one
two
three
I can also print out a numerical array as follows:
myNumericalArray = [1,2,3];
fprintf('%i\n',myNumericalArray)
This results in:
1
2
3
However, the weird behaviour appears if I try to mix these, as follows:
fprintf('%s is %i\n',myCellArray{:},myNumericalArray)
This results in:
one is 116
wo is 116
hree is 1
I think this happens because MATLAB tries to print the next entry in myCellArray in the place of the %i, rather than using the first entry in myNumericalArray. This is evident if I type the following:
fprintf('%s %s\n',myCellArray{:},myCellArray{:})
Which results in:
one two
three one
two three
...Is there some way to ensure that only one element from each array is used per line?
I agree with your idea. So, I could only think of circumventing this by creating a combined cell array with alternating values from your two initial arrays, see the following code:
myCombinedArray = [myCellArray; mat2cell(myNumericalArray, 1, ones(1, numel(myNumericalArray)))];
fprintf('%s is %i\n', myCombinedArray{:})
Gives the (I assume) desired output:
one is 1
two is 2
three is 3
fprintf(formatSpec,A1,...,An) will print all the element of A1 in column order, then all the element of A2 in column order... and size(A1) is not necessarily equal to size(A2).
So in your case the easiest solution is IMO the for loop:
for ii = 1:length(myCellArray)
fprintf('%s is %d\n',myCellArray{ii},myNumericalArray(ii))
end
For the small explanation foo(cell{:}) is similar to the splat operator (python, ruby,...) so matlab will interpret this command as foo(cell{1},cell{2},...,cell{n}) and this is why your two arguments are not interpreted pair-wise.
This is similar to the loop solution, only more compact:
arrayfun(#(c,n) fprintf('%s is %i\n', c{1}, n), myCellArray, myNumericalArray)

How to dump variables as MATLAB source code?

Is there a way to dump a MATLAB variable as the source code for the corresponding literal initializer? IOW, I'm looking for some function x such that, for example:
>> A = zeros(2);
>> x(A)
ans =
[0 0; 0 0]
>> class(x(A))
ans =
char
Is there such a function, or an easy way to achieve the same effect? (I realize that literal initializers may not exist for some MATLAB items; for such items the problem is intrinsically unsolvable.)
I am aware of the fact that MATLAB offers many ways to save data to files, but none of the ways I've found produce MATLAB source code, which is what I'm after.
For simple numeric values (and also char arrays), the mat2str function does what you're looking for.
For example, (from the MATLAB documentation):
Consider the matrix
x = [3.85 2.91; 7.74 8.99]
x =
3.8500 2.9100
7.7400 8.9900
The statement
A = mat2str(x)
produces
A =
[3.85 2.91;7.74 8.99]
where A is a string of 21 characters, including the square brackets, spaces, and a semicolon.
Further, passing the string 'class' as the second argument ensure that the answer will be case to the correct numeric type.
See the MATLAB documentation for mat2str, or run
doc mat2str
in MATLAB, for more information.
I know you are looking for a function that can do this, rather than an interactive procedure, but for anyone else who wants to do this manually...
The MATLAB variable editor/viewer has built-in code generation functionality. Open the variable in the editor, click the save icon, and choose MATLAB Script (*.m) file type (default is .mat):
The resulting MatrixCode.m:
% -------------------------------------------------------------------
% Generated by MATLAB on 3-Mar-2014 17:35:49
% MATLAB version: 8.3.0.73043 (R2014a)
% -------------------------------------------------------------------
M = ...
[16 2 3 13;
5 11 10 8;
9 7 6 12;
4 14 15 1];
Maybe someone with Java and reverse engineering skills can figure out how to call this GUI operation from the command line.
As Sam Roberts commented, matlab.io.saveVariablesToScript is now the ultimate method to convert any datatype to a script. This method was introduced in 2014a and works for struct, cell, and all primitive datatypes.
chappjc's method is also correct, but it uses a MATLAB GUI frontend to the saveVariablesToScript method.

Is there a way to automatically suppress Matlab from printing big matrices in command window?

Is there an option in matlab or a plugin/app or a trick such that if you are in an interactive command session, every time it would print out a matrix way too big for a human to look through, it redacts the output to either a warning of how big the matrix is or a summary (only a few rows and columns) of the matrix?
There are many times where I want to examine a matrix in the command window, but I didn't realize how big it was, so I accidentally printed the whole thing out. Or some place inside a function I did not code myself, someone missed a semicolon and I handed it a big matrix, and it dumps the whole thing in my command window.
It make sense that in 99.99% of the time, people do not intend to print a million row matrix in their interactive command window, right? It completely spams their scroll buffer and removes all useful information that you had on screen before.
So it makes much more sense for matlab to automatically assume that the user in interactive sessions want to output a summary of a big matrix, instead of dumping the whole thing into the command window. There should at least be such an option in the settings.
One possibility is to overload the display function, which is called automatically when you enter an expression that is not terminated by ;. For example, if you put the following function in a folder called "#double" anywhere on your MATLAB path, the default display behavior will be overridden for double arrays (this is based on Mohsen Nosratinia's display.m for displaying matrix dimensions):
% #double/display.m
function display(v)
% DISPLAY Display a variable, limiting the number of elements shown.
name = inputname(1);
if isempty(name)
name = 'ans';
end
maxElementsShown = 500;
newlines = repmat('\n',1,~strcmp(get(0,'FormatSpacing'),'compact'));
if numel(v)>maxElementsShown,
warning('display:varTooLong','Data not displayed because of length.');
% OR show the first N=maxElementsShown elements
% builtin('disp', v(1:maxElementsShown));
elseif numel(v)>0,
fprintf([newlines '%s = \n' newlines], name);
builtin('disp', v);
end
end
For example,
>> xx=1:10
xx =
1 2 3 4 5 6 7 8 9 10
>> xx=1:1e4
Warning: Data not displayed because of length.
> In double.display at 17
EDIT: Updated to respect 'compact' and 'loose' output format preference.
EDIT 2: Prevent displaying an empty array. This makes whos and other commands avoid an unnecessary display.

What is your favourite MATLAB/Octave programming trick? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I think everyone would agree that the MATLAB language is not pretty, or particularly consistent. But nevermind! We still have to use it to get things done.
What are your favourite tricks for making things easier? Let's have one per answer so people can vote them up if they agree. Also, try to illustrate your answer with an example.
Using the built-in profiler to see where the hot parts of my code are:
profile on
% some lines of code
profile off
profile viewer
or just using the built in tic and toc to get quick timings:
tic;
% some lines of code
toc;
Directly extracting the elements of a matrix that satisfy a particular condition, using logical arrays:
x = rand(1,50) .* 100;
xpart = x( x > 20 & x < 35);
Now xpart contains only those elements of x which lie in the specified range.
Provide quick access to other function documentation by adding a "SEE ALSO" line to the help comments. First, you must include the name of the function in all caps as the first comment line. Do your usual comment header stuff, then put SEE ALSO with a comma separated list of other related functions.
function y = transmog(x)
%TRANSMOG Transmogrifies a matrix X using reverse orthogonal eigenvectors
%
% Usage:
% y = transmog(x)
%
% SEE ALSO
% UNTRANSMOG, TRANSMOG2
When you type "help transmog" at the command line, you will see all the comments in this comment header, with hyperlinks to the comment headers for the other functions listed.
Turn a matrix into a vector using a single colon.
x = rand(4,4);
x(:)
Vectorizing loops. There are lots of ways to do this, and it is entertaining to look for loops in your code and see how they can be vectorized. The performance is astonishingly faster with vector operations!
Anonymous functions, for a few reasons:
to make a quick function for one-off uses, like 3x^2+2x+7. (see listing below) This is useful for functions like quad and fminbnd that take functions as arguments. It's also convenient in scripts (.m files that don't start with a function header) since unlike true functions you can't include subfunctions.
for closures -- although anonymous functions are a little limiting as there doesn't seem to be a way to have assignment within them to mutate state.
.
% quick functions
f = #(x) 3*x.^2 + 2*x + 7;
t = (0:0.001:1);
plot(t,f(t),t,f(2*t),t,f(3*t));
% closures (linfunc below is a function that returns a function,
% and the outer functions arguments are held for the lifetime
% of the returned function.
linfunc = #(m,b) #(x) m*x+b;
C2F = linfunc(9/5, 32);
F2C = linfunc(5/9, -32*5/9);
Matlab's bsxfun, arrayfun, cellfun, and structfun are quite interesting and often save a loop.
M = rand(1000, 1000);
v = rand(1000, 1);
c = bsxfun(#plus, M, v);
This code, for instance, adds column-vector v to each column of matrix M.
Though, in performance critical parts of your application you should benchmark these functions versus the trivial for-loop because often loops are still faster.
LaTeX mode for formulas in graphs: In one of the recent releases (R2006?) you add the additional arguments ,'Interpreter','latex' at the end of a function call and it will use LaTeX rendering. Here's an example:
t=(0:0.001:1);
plot(t,sin(2*pi*[t ; t+0.25]));
xlabel('t');
ylabel('$\hat{y}_k=sin 2\pi (t+{k \over 4})$','Interpreter','latex');
legend({'$\hat{y}_0$','$\hat{y}_1$'},'Interpreter','latex');
Not sure when they added it, but it works with R2006b in the text(), title(), xlabel(), ylabel(), zlabel(), and even legend() functions. Just make sure the syntax you are using is not ambiguous (so with legend() you need to specify the strings as a cell array).
Using xlim and ylim to draw vertical and horizontal lines. Examples:
Draw a horizontal line at y=10:
line(xlim, [10 10])
Draw vertical line at x=5:
line([5 5], ylim)
Here's a quick example:
I find the comma separated list syntax quite useful for building function calls:
% Build a list of args, like so:
args = {'a', 1, 'b', 2};
% Then expand this into arguments:
output = func(args{:})
Here's a bunch of nonobvious functions that are useful from time to time:
mfilename (returns the name of the currently running MATLAB script)
dbstack (gives you access to the names & line numbers of the matlab function stack)
keyboard (stops execution and yields control to the debugging prompt; this is why there's a K in the debug prompt K>>
dbstop error (automatically puts you in debug mode stopped at the line that triggers an error)
I like using function handles for lots of reasons. For one, they are the closest thing I've found in MATLAB to pointers, so you can create reference-like behavior for objects. There are a few neat (and simpler) things you can do with them, too. For example, replacing a switch statement:
switch number,
case 1,
outargs = fcn1(inargs);
case 2,
outargs = fcn2(inargs);
...
end
%
%can be turned into
%
fcnArray = {#fcn1, #fcn2, ...};
outargs = fcnArray{number}(inargs);
I just think little things like that are cool.
Using nargin to set default values for optional arguments and using nargout to set optional output arguments. Quick example
function hLine=myplot(x,y,plotColor,markerType)
% set defaults for optional paramters
if nargin<4, markerType='none'; end
if nargin<3, plotColor='k'; end
hL = plot(x,y,'linetype','-', ...
'color',plotColor, ...
'marker',markerType, ...
'markerFaceColor',plotColor,'markerEdgeColor',plotColor);
% return handle of plot object if required
if nargout>0, hLine = hL; end
Invoking Java code from Matlab
cellfun and arrayfun for automated for loops.
Oh, and reverse an array
v = 1:10;
v_reverse = v(length(v):-1:1);
conditional arguments in the left-hand side of an assignment:
t = (0:0.005:10)';
x = sin(2*pi*t);
x(x>0.5 & t<5) = 0.5;
% This limits all values of x to a maximum of 0.5, where t<5
plot(t,x);
Know your axis properties! There are all sorts of things you can set to tweak the default plotting properties to do what you want:
set(gca,'fontsize',8,'linestyleorder','-','linewidth',0.3,'xtick',1:2:9);
(as an example, sets the fontsize to 8pt, linestyles of all new lines to all be solid and their width 0.3pt, and the xtick points to be [1 3 5 7 9])
Line and figure properties are also useful, but I find myself using axis properties the most.
Be strict with specifying dimensions when using aggregation functions like min, max, mean, diff, sum, any, all,...
For instance the line:
reldiff = diff(a) ./ a(1:end-1)
might work well to compute relative differences of elements in a vector, however in case the vector degenerates to just one element the computation fails:
>> a=rand(1,7);
>> diff(a) ./ a(1:end-1)
ans =
-0.5822 -0.9935 224.2015 0.2708 -0.3328 0.0458
>> a=1;
>> diff(a) ./ a(1:end-1)
??? Error using ==> rdivide
Matrix dimensions must agree.
If you specify the correct dimensions to your functions, this line returns an empty 1-by-0 matrix, which is correct:
>> diff(a, [], 2) ./ a(1, 1:end-1)
ans =
Empty matrix: 1-by-0
>>
The same goes for a min-function which usually computes minimums over columns on a matrix, until the matrix only consists of one row. - Then it will return the minimum over the row unless the dimension parameter states otherwise, and probably break your application.
I can almost guarantee you that consequently setting the dimensions of these aggregation functions will save you quite some debugging work later on.
At least that would have been the case for me. :)
The colon operator for the manipulation of arrays.
#ScottieT812, mentions one: flattening an array, but there's all the other variants of selecting bits of an array:
x=rand(10,10);
flattened=x(:);
Acolumn=x(:,10);
Arow=x(10,:);
y=rand(100);
firstSix=y(1:6);
lastSix=y(end-5:end);
alternate=y(1:2:end);
In order to be able to quickly test a function, I use nargin like so:
function result = multiply(a, b)
if nargin == 0 %no inputs provided, run using defaults for a and b
clc;
disp('RUNNING IN TEST MODE')
a = 1;
b = 2;
end
result = a*b;
Later on, I add a unit test script to test the function for different input conditions.
Using ismember() to merge data organized by text identfiers. Useful when you are analyzing differing periods when entries, in my case company symbols, come and go.
%Merge B into A based on Text identifiers
UniverseA = {'A','B','C','D'};
UniverseB = {'A','C','D'};
DataA = [20 40 60 80];
DataB = [30 50 70];
MergeData = NaN(length(UniverseA),2);
MergeData(:,1) = DataA;
[tf, loc] = ismember(UniverseA, UniverseB);
MergeData(tf,2) = DataB(loc(tf));
MergeData =
20 30
40 NaN
60 50
80 70
Asking 'why' (useful for jarring me out of a Matlab runtime-fail debugging trance at 3am...)
Executing a Simulink model directly from a script (rather than interactively) using the sim command. You can do things like take parameters from a workspace variable, and repeatedly run sim in a loop to simulate something while varying the parameter to see how the behavior changes, and graph the results with whatever graphical commands you like. Much easier than trying to do this interactively, and it gives you much more flexibility than the Simulink "oscilloscope" blocks when visualizing the results. (although you can't use it to see what's going on in realtime while the simulation is running)
A really important thing to know is the DstWorkspace and SrcWorkspace options of the simset command. These control where the "To Workspace" and "From Workspace" blocks get and put their results. Dstworkspace defaults to the current workspace (e.g. if you call sim from inside a function the "To Workspace" blocks will show up as variables accessible from within that same function) but SrcWorkspace defaults to the base workspace and if you want to encapsulate your call to sim you'll want to set SrcWorkspace to current so there is a clean interface to providing/retrieving simulation input parameters and outputs. For example:
function Y=run_my_sim(t,input1,params)
% runs "my_sim.mdl"
% with a From Workspace block referencing I1 as an input signal
% and parameters referenced as fields of the "params" structure
% and output retrieved from a To Workspace block with name O1.
opt = simset('SrcWorkspace','current','DstWorkspace','current');
I1 = struct('time',t,'signals',struct('values',input1,'dimensions',1));
Y = struct;
Y.t = sim('my_sim',t,opt);
Y.output1 = O1.signals.values;
Contour plots with [c,h]=contour and clabel(c,h,'fontsize',fontsize). I usually use the fontsize parameter to reduce the font size so the numbers don't run into each other. This is great for viewing the value of 2-D functions without having to muck around with 3D graphs.
Vectorization:
function iNeedle = findClosest(hay,needle)
%FINDCLOSEST find the indicies of the closest elements in an array.
% Given two vectors [A,B], findClosest will find the indicies of the values
% in vector A closest to the values in vector B.
[hay iOrgHay] = sort(hay(:)'); %#ok must have row vector
% Use histogram to find indices of elements in hay closest to elements in
% needle. The bins are centered on values in hay, with the edges on the
% midpoint between elements.
[iNeedle iNeedle] = histc(needle,[-inf hay+[diff(hay)/2 inf]]); %#ok
% Reversing the sorting.
iNeedle = iOrgHay(iNeedle);
Using persistent (static) variables when running an online algorithm. It may speed up the code in areas like Bayesian machine learning where the model is trained iteratively for the new samples. For example, for computing the independent loglikelihoods, I compute the loglikelihood initially from scratch and update it by summing this previously computed loglikelihood and the additional loglikelihood.
Instead of giving a more specialized machine learning problem, let me give a general online averaging code which I took from here:
function av = runningAverage(x)
% The number of values entered so far - declared persistent.
persistent n;
% The sum of values entered so far - declared persistent.
persistent sumOfX;
if x == 'reset' % Initialise the persistent variables.
n = 0;
sumOfX = 0;
av = 0;
else % A data value has been added.
n = n + 1;
sumOfX = sumOfX + x;
av = sumOfX / n; % Update the running average.
end
Then, the calls will give the following results
runningAverage('reset')
ans = 0
>> runningAverage(5)
ans = 5
>> runningAverage(10)
ans = 7.5000
>> runningAverage(3)
ans = 6
>> runningAverage('reset')
ans = 0
>> runningAverage(8)
ans = 8
I'm surprised that while people mentioned the logical array approach of indexing an array, nobody mentioned the find command.
e.g. if x is an NxMxO array
x(x>20) works by generating an NxMxO logical array and using it to index x (which can be bad if you have large arrays and are looking for a small subset
x(find(x>20)) works by generating list (i.e. 1xwhatever) of indices of x that satisfy x>20, and indexing x by it. "find" should be used more than it is, in my experience.
More what I would call 'tricks'
you can grow/append to arrays and cell arrays if you don't know the size you'll need, by using end + 1 (works with higher dimensions too, so long as the dimensions of the slice match -- so you'll have to initialize x to something other than [] in that case). Not good for numerics but for small dynamic lists of things (or cell arrays), e.g. parsing files.
e.g.
>> x=[1,2,3]
x = 1 2 3
>> x(end+1)=4
x = 1 2 3 4
Another think many people don't know is that for works on any dim 1 array, so to continue the example
>> for n = x;disp(n);end
1
2
3
4
Which means if all you need is the members of x you don't need to index them.
This also works with cell arrays but it's a bit annoying because as it walks them the element is still wrapped in a cell:
>> for el = {1,2,3,4};disp(el);end
[1]
[2]
[3]
[4]
So to get at the elements you have to subscript them
>> for el = {1,2,3,4};disp(el{1});end
1
2
3
4
I can't remember if there is a nicer way around that.
-You can make a Matlab shortcut to an initialization file called startup.m. Here, I define formatting, precision of the output, and plot parameters for my Matlab session (for example, I use a larger plot axis/font size so that .fig's can be seen plainly when I put them in presentations.) See a good blog post from one of the developers about it http://blogs.mathworks.com/loren/2009/03/03/whats-in-your-startupm/ .
-You can load an entire numerical ascii file using the "load" function. This isn't particularly fast, but gets the job done quickly for prototyping (shouldn't that be the Matlab motto?)
-As mentioned, the colon operator and vectorization are lifesavers. Screw loops.
x=repmat([1:10],3,1); % say, x is an example array of data
l=x>=3; % l is a logical vector (1s/0s) to highlight those elements in the array that would meet a certain condition.
N=sum(sum(l));% N is the number of elements that meet that given condition.
cheers -- happy scripting!