I need to analyse several sets of data which are associated with different parameter sets (one single set of parameters for each set of data). I'm currently struggling to find a good way to store these parameters such that they are readily available when analysing a specific dataset.
The first thing I tried was saving them in a script file parameters.m in the data directory and load them with run([path_to_data,'/parameters.m']). I understand, however, that this is not good coding practice and it also gave me scoping problems (I think), as changes in parameters.m were not always reflected in my workspace variables. (Workspace variables were only changed after Clear all and rerunning the code.)
A clean solution would be to define a function parameters() in each data directory, but then again I would need to add the directory to the search path. Also I fear I might run into namespace collisions if I don't give the functions unique names. Using unique names is not very practical on the other hand...
Is there a better solution?
So define a struct or cell array called parameters and store it in the data directory it belongs in. I don't know what your parameters look like, but ours might look like this:
parameters.relative_tolerance = 10e-6
parameters.absolute_tolerance = 10e-6
parameters.solver_type = 3
.
.
.
and I can write
save('parameter_file', 'parameters')
or even
save('parameter_file', '-struct', 'parameters', *fieldnames*)
The online help reveals how to use -struct to store fields from a structure as individual variables should that be useful to you.
Once you've got the parameters saved you can load them with the load command.
To sum up: create a variable (most likely a struct or cell array) called parameters and save it in the data directory for the experiment it refers to. You then have all the usual Matlab tools for reading, writing and investigating the parameters as well as the data. I don't see a need for a solution more complicated than this (though your parameters may be complicated themselves).
Related
I have a project consisting of multiple nested functions.
For debugging purpose I want to save all internal variables in one way or another, in order to display figures, replay parts of code etc...
I also want to keep this as transparent as possible regarding calculation time.
My first thought was to create a global variable, and store programmatically at the end of each function the inputs and outputs inside the variable as a structure :
globalVariable.nameOfParentfunction_NameOfFunction.nameInput1 = valueInput1;
globalVariable.nameOfParentfunction_NameOfFunction.nameInput2 = valueInput2;
...
globalVariable.nameOfParentfunction_NameOfFunction.nameOutput1 = valueOutput1;
...
Is it possible to use some kind of reflection to get the name and value of inputs/outputs without necessarily parse the file where the function is written?
I found a good but maybe outdated topic about parsing
How do I collect internal signals?
The simple solution is to use save, which will save all variables in the current workspace (the function’s context) to file.
If you want to keep the values in memory, not in a file, you can use names = who to get a list of all variables defined in the current workspace, then use val = eval(names{i}) to get the value of the variable called name{i}.
I would recommend putting all of that in a separate function, which you can call from any other function to store its variables, to avoid repeating code. This function would use evalin('caller',…) to get names and values of variables in the workspace of the calling function.
Note that using eval or evalin prevents MATLAB from optimizing code using its JIT. They also are dangerous to use, since they can execute arbitrary code, however in this case you control what is being executed so that is no a concern.
I would like to use two versions of the same Matlab software package at the same time (I would like to compare their outputs for testing). The package modifies the path so that functions in subdirectories can be found. This seems problematic since the package assumes that it is the only copy running on the machine. The path is essentially a global variable which is unintentionally shared between the two copies of the code.
Example simplified code structure:
/main_code.m
/compare_results.m
/code_a/somefn.m
/code_a/submethods/
/code_b/somefn.m
/code_b/submethods/
Note that somefn.m adds the submethods directory to the path, and calls code from the submethods folder by relying on the path.
Example of code that I would like to run:
for i = 1:1000000
% Run version A:
result_a = code_a.somefn(i);
% Run version B:
result_b = code_b.somefn(i);
% Compare the output from the two versions:
compare_results(a,b);
end
One solution that I can think of is to manually update the Matlab path every time that I want to switch to a different version of the package. This seems like unnecessary coding overhead, and potentially a performance problem (due to switching the path so often).
Another solution might be to rewrite the code to be object oriented, so that the functions are attached to objects, and I can create objects of different versions. The problem with this is that in reality the code package contains hundreds of files, and I was not the original author so rewriting would be a huge task .
(Yet another option would be to change directory all the time, so that the code to run is always in the current directory. This would be so much of a headache due to the number of subfolders that I do not think this is a serious solution. It also has potential performance overhead drawbacks similar to always changing the path.)
Is there a cleaner way to handle this? Can I somehow specify the folder of the code that I want to run? What is the best way to design such a code package so that this problem does not come up?
Just create packages, which can contain functions with the same names: Packages Create Namespaces
Basically create two folders named, say, +package1 and +package2. The "+" in the folder name is important. Then place your functions under both of them, say, foo.m. Then, you can call each separately without messing with MATLAB path as:
>> package1.foo
>> package2.foo
You can use private functions. Change the directory structure as:
/main_code.m
/compare_results.m
/+code_a/somefn.m
/+code_a/private/
/+code_b/somefn.m
/+code_b/private/
Each somefn has access to the functions contained in its private sub-folder. So there is no need to create global variable and to add private sub-folders to the path.
I have hundreds of Catia V5 catparts and catproducts in a folder on hard disc. I want to know if a particular catpart is used in some catproduct or not. If it is not used in any product, I want to delete it and clean my hard disc. One way to do it is to open all catproducts one by one and check carefully they contain this model. This is cumbersome process and can lead to serious mistakes. Is there some automatic way to check it? If not, is it possible to write some macro for that purpose?
It is possible with a VBA script. If it's just Catpart file that your looking for in a product, then your script would work as follows
query your folder(s) for all catparts and catproducts.(use 2 dictionaries or arrays, one for each file type each)
Via a loop, Individually open and load each catproduct and essentially walk the tree and compare each child Catpart to your compiled list of catparts. If a match is found, movethe part to a new "white list"(dictionary or array)
Close the catproduct and check the next one.
Then, when all done, your original list(dictionary or array) will be your unused parts.
I'm not sure exactly how your models are built, but you may need to check for additional references/links in your catproducts (additional logic) before doing something like this.
I have multiple vectors (+100) that have been loaded into MATLAB workspace, I would like to write a script that can plot and save them all, but for that I need their name, my question is: is there a way to automatically get the name vectors saved in the workspace.
thanks in advance.
Step one: whoever gave you a *.mat file with 100+ named variables in it, [censored for strong language and scenes some viewers may find upsetting]. I am only partly joking here; if you find yourself in this sort of situation normally it is because something has gone terribly wrong upstream. We can work around it, though.
Step two: use who with the filename to get a list of variables in that file
names = who('-file', 'all');
Step three: load the variables (or a subset of them) into a struct
data = load('all.mat');
Step four: use dynamic structure naming to extract data:
for n = 1:length(names);
plot(data.(names{n})); % or whatever you want to do with this data
end
I would probably just use the loop to dump the data in a cell array so as to make further processing simpler and avoid further use of dynamic field names or worse, eval.
You can use who, which lists all variables alphabetically in the active workspace.
I have a struct (of matrices) in Matlab that has been saved on the harddisk. I currently use load to load these files inside my functions. Do you have any suggestions of doing this someother way that is much faster?
(Yes, I can pass the struct as a variable to my function but that is not possible due to memory issues!). Thanks! This would be a great help!
A = struct('local', randn(200000,14), 'usd', randn(200000,14), ...
'ttm', randn(180000,14), 'avg', randn(190000,14), ...
'ttm1yr', randn(190000,14), 'avg1yr', randn(190000,14)) ;
save('A.mat', 'A') ; clear all;clc
tic, load A.mat, A=A.local; toc %--> Takes 1.05 seconds
It appears that you're interested only in specific chunks of your saved file. I would suggest changing the format of the saved data so that individual variables can be loaded using the
S = load(filename, variables)
form of load. This will speed up the loading significantly, since you'll avoid copying all of the unwanted data from disk into memory, just to free it right away.
If your data is already in struct form, you can use this form of save (from the online docs):
save(filename, '-struct', structName, fieldNames) stores the fields of
the specified scalar structure as individual variables in the file. If
you include the optional fieldNames, the save function stores only the
specified fields of the structure. You cannot specify variables and
the '-struct' keyword in the same call to save.
Starting in R2011a you can access the contents of a Mat file without using load via the matfile object. help matfile for details.
The real advantage is that this allows read/write of portions of large arrays without loading or saving the entire array.
This is of limited use to you with your current structure, since fields of structures cannot be indexed this way. But a relatively small re-factor would allow you to take advantage of these features, depending on what the rest of your application looks like.