I have a struct (of matrices) in Matlab that has been saved on the harddisk. I currently use load to load these files inside my functions. Do you have any suggestions of doing this someother way that is much faster?
(Yes, I can pass the struct as a variable to my function but that is not possible due to memory issues!). Thanks! This would be a great help!
A = struct('local', randn(200000,14), 'usd', randn(200000,14), ...
'ttm', randn(180000,14), 'avg', randn(190000,14), ...
'ttm1yr', randn(190000,14), 'avg1yr', randn(190000,14)) ;
save('A.mat', 'A') ; clear all;clc
tic, load A.mat, A=A.local; toc %--> Takes 1.05 seconds
It appears that you're interested only in specific chunks of your saved file. I would suggest changing the format of the saved data so that individual variables can be loaded using the
S = load(filename, variables)
form of load. This will speed up the loading significantly, since you'll avoid copying all of the unwanted data from disk into memory, just to free it right away.
If your data is already in struct form, you can use this form of save (from the online docs):
save(filename, '-struct', structName, fieldNames) stores the fields of
the specified scalar structure as individual variables in the file. If
you include the optional fieldNames, the save function stores only the
specified fields of the structure. You cannot specify variables and
the '-struct' keyword in the same call to save.
Starting in R2011a you can access the contents of a Mat file without using load via the matfile object. help matfile for details.
The real advantage is that this allows read/write of portions of large arrays without loading or saving the entire array.
This is of limited use to you with your current structure, since fields of structures cannot be indexed this way. But a relatively small re-factor would allow you to take advantage of these features, depending on what the rest of your application looks like.
Related
I have a project consisting of multiple nested functions.
For debugging purpose I want to save all internal variables in one way or another, in order to display figures, replay parts of code etc...
I also want to keep this as transparent as possible regarding calculation time.
My first thought was to create a global variable, and store programmatically at the end of each function the inputs and outputs inside the variable as a structure :
globalVariable.nameOfParentfunction_NameOfFunction.nameInput1 = valueInput1;
globalVariable.nameOfParentfunction_NameOfFunction.nameInput2 = valueInput2;
...
globalVariable.nameOfParentfunction_NameOfFunction.nameOutput1 = valueOutput1;
...
Is it possible to use some kind of reflection to get the name and value of inputs/outputs without necessarily parse the file where the function is written?
I found a good but maybe outdated topic about parsing
How do I collect internal signals?
The simple solution is to use save, which will save all variables in the current workspace (the function’s context) to file.
If you want to keep the values in memory, not in a file, you can use names = who to get a list of all variables defined in the current workspace, then use val = eval(names{i}) to get the value of the variable called name{i}.
I would recommend putting all of that in a separate function, which you can call from any other function to store its variables, to avoid repeating code. This function would use evalin('caller',…) to get names and values of variables in the workspace of the calling function.
Note that using eval or evalin prevents MATLAB from optimizing code using its JIT. They also are dangerous to use, since they can execute arbitrary code, however in this case you control what is being executed so that is no a concern.
Is there a way to access data generated in one .m file from another. What I am trying to do is I have one .m call it A.m where I have loaded a large amount of data from a .txt file and broken it up into a structure with various fields. Since this takes up a large amount of space in the script I would like to create another .m file call it B.m, in which I can access the structure created in A.m and plot and perform calculations in B.m. So, basically I want to access a structure created in A.m from B.m. Is this possible?
-Thanks
There are some things to think of here. First, to limit scope, do not use scripts: use functions instead. Calling a script from another script mainly add code to the first script and have nothing to do with scope. However, by using this method, your code becomes hard to read and understand. If you want that all code to be in the same scope I would recommend you keep all the code in the same m-file.
A function however have a function scope and unless a variable is declared global it can only be passed into this scope by using function input arguments. Also, the only way to return values is to use function output arguments.
function [out1, out2,...] = myFun(in1,in2,...)
out1 = in1*in2;
out2 = in2.^2;
...
Now to the tricky part. The variable passed into the function is passed as "copy on write", meaning that the variables are always passed as a reference unless they are modified inside the script. When using structs, only the field that is modified is copied. This can have consequences for your program as well. Since you say your data is large, changing too many fields in the struct in the same function may cause memory overflow.
Anyway, if you only uses script you do not need to pass any data, since the scope is not affected. However, I recommend you to use functions and pass the struct as an input argument. If this was not what you asked for, please comment on this answer.
I have multiple vectors (+100) that have been loaded into MATLAB workspace, I would like to write a script that can plot and save them all, but for that I need their name, my question is: is there a way to automatically get the name vectors saved in the workspace.
thanks in advance.
Step one: whoever gave you a *.mat file with 100+ named variables in it, [censored for strong language and scenes some viewers may find upsetting]. I am only partly joking here; if you find yourself in this sort of situation normally it is because something has gone terribly wrong upstream. We can work around it, though.
Step two: use who with the filename to get a list of variables in that file
names = who('-file', 'all');
Step three: load the variables (or a subset of them) into a struct
data = load('all.mat');
Step four: use dynamic structure naming to extract data:
for n = 1:length(names);
plot(data.(names{n})); % or whatever you want to do with this data
end
I would probably just use the loop to dump the data in a cell array so as to make further processing simpler and avoid further use of dynamic field names or worse, eval.
You can use who, which lists all variables alphabetically in the active workspace.
I have two .mat files. And I want to read these data of two mat files and store in variables A and B. This is my code, but I think it is not good. Can you help me store it without using matArray in matlab? (Varibale matArray is born when you call load function)
load input1.mat;
A=matArray;
load input2.mat;
B=matArray
Thank you so much
You had it right. The variable name that you have when you save the file is the name that will appear in the workspace when you load the file again. Best you can do:
load('input1.mat');
A=matArray;
load('input2.mat');
B=matArray;
clear matArray
At least you'll get the space back at the end. There is, to my knowledge, no "rename" function in Matlab...
Of course if you know what you want to name the variable when you read it in, you should save it as such:
A = matArray;
save('input1.mat', 'A');
etc
Use an output argument with the load function.
A = load('input1.mat');
B = load('input2.mat');
The two arrays will now be fields of the structures A and B:
size(A.matArray);
plot(B.matArray);
If you choose to copy these into simpler variables, or stick with your current copying approach, you should know that the copy operation is extremely efficient. When you do A = matArray; A shares the data of matArray until one of them is modified. Therefore, if you delete matArray before modifying A, no extra memory is consumed by the copy.
I need to analyse several sets of data which are associated with different parameter sets (one single set of parameters for each set of data). I'm currently struggling to find a good way to store these parameters such that they are readily available when analysing a specific dataset.
The first thing I tried was saving them in a script file parameters.m in the data directory and load them with run([path_to_data,'/parameters.m']). I understand, however, that this is not good coding practice and it also gave me scoping problems (I think), as changes in parameters.m were not always reflected in my workspace variables. (Workspace variables were only changed after Clear all and rerunning the code.)
A clean solution would be to define a function parameters() in each data directory, but then again I would need to add the directory to the search path. Also I fear I might run into namespace collisions if I don't give the functions unique names. Using unique names is not very practical on the other hand...
Is there a better solution?
So define a struct or cell array called parameters and store it in the data directory it belongs in. I don't know what your parameters look like, but ours might look like this:
parameters.relative_tolerance = 10e-6
parameters.absolute_tolerance = 10e-6
parameters.solver_type = 3
.
.
.
and I can write
save('parameter_file', 'parameters')
or even
save('parameter_file', '-struct', 'parameters', *fieldnames*)
The online help reveals how to use -struct to store fields from a structure as individual variables should that be useful to you.
Once you've got the parameters saved you can load them with the load command.
To sum up: create a variable (most likely a struct or cell array) called parameters and save it in the data directory for the experiment it refers to. You then have all the usual Matlab tools for reading, writing and investigating the parameters as well as the data. I don't see a need for a solution more complicated than this (though your parameters may be complicated themselves).