MATLAB has a representation of a directed/undirected graph. I would like to deserialize a graph with many node and edge attributes serialized via MATLAB's save function into Python. I know about scipy.io.loadmat and h5py's File (for MATLAB v7.3 saved files), but neither seems to produce a representation in Python that actually holds intelligible vertex/edge data.
How do I do this? I'm concerned with this and the inverse operation, i.e writing an object from Python to a format MATLAB load can read. Is there a bytewise data description of a serialized MATLAB object and/or a Graph looks like somewhere?
For example, in MATLAB I could:
s = [1 1 2 2 3];
t = [2 4 3 4 4];
G = digraph(s,t);
G.Edges.Rand = rand(size(G.Edges)); % Add an edge attribute
G.Nodes.Val = rand(size(G.Nodes)); % Add a node attribute
save('loadmat.mat', 'G'); % Readable by scipy.io.loadmat
save('h5py.mat', 'G', '-v7.3'); % Readable by h5py.File
then, in Python I could read these
from scipy.io import loadmat
G0 = loadmat('loadmat.mat')
from h5py import File
G1 = File('h5py.mat')
Neither seems to give me the vertex/edge data or am I just missing it?
Thanks
The lengthy but certain way to do this is to define a schema for G in something like Google Protocol Buffers. With that schema you can automatically generate serialiser source code for Python and Java (that you could use in Matlab). This would allow you exchange G between Matlab and Python, or indeed between anything else.
You would probably have to hand write code to translate between how G is stored in Matlab and however protoc chose to represent in Java the message you had defined in the schema. You won't (I'm fairly certain) be able to do Ggpb = G. If both Java and Matlab supported type reflection then you could probably write something neat to do it automatically...
The digraph object is an user-defined object type, not one of the fundamental MATLAB types; no wonder python doesn't understand it. Please note that even MATLAB will not understand the saved object layout if it doesn't have a working definition of the saved object's class, accessible in the path.
You might want to save the adjacency matrix:
A = full(adjacency(G));
save('adjacency.mat', 'A');
or the incidence matrix:
I = full(incidence(G));
save('incidence.mat', 'I');
whatever suits you better.
Late edit
Another way is to force the object to become a POD (plain old data) that has better chances to be understood by loadmat:
S = struct(G);
save('pod_digraph.mat', 'S');
But mind that you'll have access to all information; the dependent properties will be saved as such; you'll need to recreate the class' interface by yourself in order to maintain consistency (e.g. the adjacency matrix and the incidence matrix can be both constructed on-the-fly from the same internal information, which may look like neither of them). Also, one cannot convert a POD to the original object unless the constructor knows how to do this.
Related
I have a very huge 3D matrix, the data was written into disk for future use. Writing the matrix into a bin is easy, reading it back however have some issue.
Write to bin:
z=repmat(complex(rand(5),rand(5)),[1 1 5])
z_imag = imag(z);
z_real = real(z);
adjacent = [z_real z_imag];
fileID = fopen('complex.bin','w');
fwrite(fileID,adjacent,'double')
And now, I try to read it back using memmapfile:
m = memmapfile('complex.bin', 'Offset', 0, 'Format', {'double' [5,5,5] 'x'});
complexValues = complex(m.Data(:).x(1,:), m.Data(:).x(2,:)); %this line doesn't work though, just for explanation's sake
It gave me an error saying that
Error using memmapfile/subsref (line 764) A subscripting operation on
the Data field attempted to create a comma-separated list. The
memmapfile class does not support the use of comma-separated lists
when subscripting.
I was referring to the solution here, the suggested solution used the reshape to shape the matrix beforehand (as contrast to my method above). I try to avoid using reshape in my code as I'm dealing with very huge data and that might computationally expensive and takes a long time. Is there an alternative/better way to do this?
Thanks in advance!
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 4 years ago.
Improve this question
I want to get the content of a variable from its name as a string:
%suppose that I have structures variables in the workspace:
struct_1=struct('field1' ,"str",'field2',0:5)
struct_2=struct('field1' ,"str",'field2',0:10)
struct_3=struct('field1' ,"str",'field2',0:20)
%and I have an other variable like:
a=5
var2='hello'
%and I want to concatenate all these structures in the same structure
%So i wan to get their name.
%I don't know if there is an other way to get just the structure variables
structures=who(str*)
array=[]
for i=1:length(structures)
array=[array; structures(i). field2]; % here I get an error because the content of structures are string representing the variable name
end
%create the new struct
str='newstr_value'
struct_4=struct('field1',str, 'field2',array)
How to fix this error and is there any way to do this better ?
While I would still highly recommend going back to the point of origin of these dynamically named structures (shame 🔔🔔🔔 on the toolbox creator if this is actually the output...) and fixing this problem there, there is an approach that does not require the use of eval.
Similar to my approach for a tangentially related question, you can save the desired structures to a temporary *.mat file and take advantage of load's optional structure output to consolidate the data structures for access in a more robust programmatic matter.
For example:
struct_1=struct('field1' ,"str",'field2',0:5);
struct_2=struct('field1' ,"str",'field2',0:10);
struct_3=struct('field1' ,"str",'field2',0:20);
save('tmp.mat', '-regexp', '^struct_');
clear
oldworkspace = load('tmp.mat');
varnames = fieldnames(oldworkspace);
nvars = numel(varnames);
% Get original structure fields
% The subsequent steps assume that all input structs have the same fields
fields = fieldnames(oldworkspace.(varnames{1}));
nfields = numel(fields);
% Initialize output struct
for ii = 1:nfields
newstruct.(fields{ii}) = [];
end
newstruct(nvars) = newstruct;
for ii = 1:nvars
for jj = 1:nfields
newstruct(ii).(fields{jj}) = oldworkspace.(varnames{ii}).(fields{jj});
end
end
Gives us:
Yay.
Not sure if your situation absolutely requires dynamic calls of variables by their string names. eval can solve your problem. However, eval is slow. At times, it is unreliable. You can easily have conflicting code as you build various components of your project. And in my experience, there have always been alternatives to eval. In your case, Adriaan outlined an example in his comment.
Since you are only trying to interface with a third-party toolbox, if the number of variables you read is reasonable, you can do the followings with eval.
array=struct;
for i=1:length(structures)
eval(['array(',num2str(i),').field2=',structures{i},';'])
end
If you know all the structs are named struct_#, you can also do
array=struct;
num=3; % the number of outputs from your toolbox
for i=1:num
eval(['array(',num2str(i),').field2=struct_',num2str(i),'.field2;'])
end
Note that your field2 values are all arrays. You can't just create an array and expect each "cell" to carry another array value. Arrays, or rather matrices, in Matlab are a special data type. They only take numerical values. (Matrices are highly optimized in Matlab in various ways. That is why we use Matlab.)
Also note that when you set equality of fields of structs, the equality is by reference. There is no deep copy of the data. Modifying one will modify another. (Just beware of this but don't necessarily rely on it. Deciding when to deep copy things is one of the things we let Matlab handle.)
As well, be careful with the result from who. You need to be absolutely clear with what is in the local scope. Above is assuming your result from calling structures=who(str*) is correct for your application.
I'm using a lot of MATLAB for a Control class and something is really bumming me out. They want us to use a lot of Simulink even though I find the visual representation not that helpful and the interface between Simulink and MATLAB scripts hard to figure out in general.
So I have a model and I have added scopes for sinks which can save data directly to the workspace when ran from Simulink. However when I use the command sim in the script to directly use the model according to some parameters (stopTime, solving method, etc.) the results are buried in an object which is poorly documented to say the least so say I have:
simout = sim('lab','StopTime','100','Solver','ode1','FixedStep','2');
Now I have an object in my workspace but to access the data I want I need to go 3, hell 4 layers deep sometimes in the members of simout. My first question is :
Is there a way to access directly or at least know what those members are without using the tedious use of who.
I don't want to compile code to access its documentation! And help is not really helpful for those situations.
Why doesn't the Simulink model save the data when invoked like prescribed in the sink properties. I know that the line of code I mentioned overides some of the simulink block prescriptions (e.g. the solving method used).
How to know how simulink models interact with matlab scripts, granted I am a noob in coding in general, but the documentation doesn't really tell me what are the formal definitions of the model and the way it is used in matlab. I am scared at some points the default settings of 'sim' will overide some settings I set up in an earlier model which would prove to be a nasty buisiness to debug.
TL;DR Is there a quick way to access deeply buried members of an object? For example right now I have to do:
simout = sim('lab','StopTime','100','Solver','ode1','FixedStep','2');
who(simout)
ScopeData = (simout.get('ScopeData'))
signals = (ScopeData.signals)
time = (ScopeData.time)
Can I do something more C-ish of the sort of (Simout->ScopeData).signals?
And finally, why is the MATLAB suite presented like it was an app for day-traders when it is used a lot by EE people who in general need to know their coding? Why are libraries with headers and good documentation for what you are importing in your code (e.g. boost, string etc.) not used? This last option might be less pretty by hiding the mechanics, but to be able to write code properly I feel like I have to know most of the underlying mechanics of the code.
Invariably when most people start off using MATLAB or Simulink, they hate it. The main reasons I see for this is people are taught MATLAB very poorly which prevents them from understanding the power of MATLAB and when it should be used.
Before I get into describing how the MATLAB workspace, m files and Simulink are all related let's first define what each of those is separately and a few of the things you can do with them.
The MATLAB Workspace
The MATLAB workspace contains all of the variables you have created in MATLAB either explicitly via the command window or implicitly through running a .m file (don't worry I'll come back to these). The simplest method to add a variable to the MATLAB workspace is to directly type into the command window like
>> A = 1
A =
1
which adds the variable A to the workspace and assigns it a value of 1. Even simpler though is to simple type 1 like
>> 1
ans =
1
which adds the variable ans to the workspace with a value of 1. The default workspace variable for any command executed by MATLAB that is not explicitly assigned to a variable is ans.
The workspace browser in MATLAB displays information about all of the variables currently in the workspace. For instance if we were to execute, in the command window,
>> A = 5;
>> B = [6 8];
>> C = [3 6 7; 9 11 12];
>> D = eye(max(size(C)));
the workspace browser would look something like
Some Helpful Workspace Commands
The save and clear commands are helpful commands that work with the MATLAB workspace. save lets you save variables from the workspace to a file whilst clear lets you remove variables from the workspace, thus freeing up memory. clear all will clear every variable from the workspace.
m Files
An m-file (or script file) is a simple text file with a .m file extension. In it you type MATLAB commands which can be executed sequentially by executing just the m-file in the command window. For instance, lets copy our commands from above and paste them into an m-file called mfile_test.m. Now lets add the command C - A to the bottom of the m-file (note do not append a ; after the command). We can execute this m-file from the command window and see the output of all of commands within it executed sequentially
>> mfile_test.m
ans =
-2 1 2
4 6 7
m Files and Functions
m files can also contain functions. For instance we could write a function that calculates the area and circumference of a circle like
function [A, C] = circle(r)
A = pi*r*r;
C = 2*pi*r;
end
and put it in an m-file called circle.m. We could then use this like
>> circle(5)
ans =
78.5398
where ans now holds the area of a circle with a radius of 5 or
>> [A, C] = circle(5)
A =
78.5398
C =
31.4159
where A holds the area of the circle and C the circumference.
Simulink Models
Simulink is a graphical tool that can be used to model various different types of systems as well as modelling dynamic system behaviour. Simulink is closely embedded into the MATLAB ecosystem which adds additional power to it (again I'll come back to this later when describing the similarities).
Simulink models contain blocks which all exhibit different behaviour. There is also the functionality to create your own Simulink blocks should you need. Models in Simulink are made by connecting blocks of different kinds to emulate the system you want to model.
Simulink is quite extensive and I don't recommend learning it on your own. The best way to learn Simulink is invariably as part of a course, Control Systems or Systems Analysis type courses are ideal for this purpose. For that reason I'm not going to dwell on it for much longer here.
How are the MATLAB Workspace, m Files and Simulink all Related?
Well, the MATLAB workspace is available to both m files and Simulink. Variables in the MATLAB workspace can be used in both m files and Simulink.
m files that contain only scripts (and not functions) will write every variable created in them to the MATLAB workspace. m files that contain only functions won't write any of the new variables used in those functions to the MATLAB workspace. The only way a function will alter variables in the MATLAB workspace is by assigning an output value to ans or if you explicitly declare output arguments when calling the function.
Simulink and the MATLAB workspace have a very similar relationship to m files and the MATLAB workspace. Any variable in the MATLAB workspace is available for use in Simulink (in any part, including configuration). For instance in the below configuration I use the MATLAB workspace variables start_time, stop_time and step_time to set up the parameters of model. Typically I would define these in an m-file before I run my Simulink model with sim(). This is how we can relate all 3 together.
Simulink can write variables to the MATLAB workspace by adding output arguments to the sim() command. However, as you've found this can be quite nasty to navigate and extract what you really want! A better approach would be to use the Data Import/Export options in Simulink coupled with a To Workspace block to grab the outputs that you are concerned with so that you can ignore everything else.
Below I have a screenshot of the Simulink Data Import/Export pane. You can see there are options in here for us to send variables to the MATLAB workspace. Typically, the most common you would need is tout which will be a range given by start_time:step_time:stop_time from the earlier configuration pane. Other areas that would be of primary interest in this screen would be xInitial and xout which are used in Root Locus analysis.
However, all that aside one of the best blocks in Simulink is the To Workspace block. This block can be used to store variables directly in the MATLAB workspace and is one of the keys to being able to link MATLAB and Simulink. You get the power of Simulink but the computational and plotting abilities of MATLAB. I've included a screenshot of it below as well as a typical configuration I would use. The default Save Format for this block is Timeseries, however, I recommend changing this to Array as it will make your life much easier.
OK, where was I? I feel like I'm giving a lecture instead of writing an answer!
A Practical Explanation
Simulink Model
So now let's put everything we've learned into practice with a simple example. First we are going to create a simple Simulink model like this
Now we'll set up our configuration for this, as before for the Solver section
We'll also untick Limit data points to last: in the Data Import/Export section
And, that's it. Our simple Simulink model has been created and setup. For the purposes of this answer, I'm saving mine as stackoverflow_model.slx.
MATLAB Script
Now we'll create simple MATLAB script (m-file) called stackoverflow_script.m that will set up the necessary variables for our Simulink model by adding them to the MATLAB workspace. We'll then call our Simulink model and check what new variables it added to the workspace. And, finally we'll generate a simple plot to show the benefits of this approach.
So here is the MATLAB script
% Script developed to describe the relationship between the MATLAB
% workspace, m-files and Simulink
close all
clear all
% Initialise variables
start_time = 0;
stop_time = 10;
step_time = (stop_time - start_time) / 1000; % Creates 1000 + 1 points
% Choose k
k = 60;
% Execute Simulink model
sim('stackoverflow_model');
whos % To display variables returned from Simulink
% Plot results
figure;
plot(tout, yout, 'r');
title('Sample Plot');
xlabel('Time (s)');
ylabel('Output');
Putting it All Together
Now we execute this script in the command window with stackoverflow_script and sit back and marvel at the POWER of MATLAB and Simulink combined.
>> stackoverflow_script
Name Size Bytes Class Attributes
k 1x1 8 double
start_time 1x1 8 double
step_time 1x1 8 double
stop_time 1x1 8 double
tout 1001x1 8008 double
yout 1001x1 8008 double
We can see from the output above that all of the variables Simulink needs (k, start_time, step_time and stop_time) are all in the MATLAB workspace. We can also see that Simulink adds 2 new variables to the MATLAB workspace tout and yout which are simple 1001x1 vectors of doubles. No nasty structs to navigate.
And finally, this produces a nice plot
And so that concludes our whistle stop tour through MATLAB, m files and Simulink! I hope you've enjoyed it as much as I did writing it!
P.S. I haven't checked this for spelling or grammar mistakes so edits are very much welcome ;)
General
Well to put it short workspace is the variable-environement you are working in. If you run a script, your workspace is 'base', which is the same the console uses. So Matlab does have different environements, one is a kind of included environement known as path, the other one is for variables, known as workspace.
Simulink uses a different one, which prevents shadowing variable names I guess.
To check check members in the current workspace use who
To write members to another workspace use assignin
To run something in a specified workspace use evalin
Your Questions
1.
Who lists all the variables in the current workspace you don't need it for the thing you wanna do.
The whole simulink documentation isn't that good...
2.
It does...
3.
If you run a script and define variables they are defined in the base workspace. When you specify a variable in simulink by just entering its name (for example a), it does load it from the base workspace (hence this way arround no problems).
The other way arround is to either use the given export blocks, or specify export values in your own blocks by either using global or assignin.
4.
If you open the scope block and hit the options-buttion (the little gear), you can select an export option. You can aswell specify the type you wan't to use. You seem to use the struct with time option, which is the one with the most lvls, I'd suggest to use the array-type if your problem is just the fact that it is a struct.
You can also just use the Outputblock to specify the export type and name.
So I'd go with:
sim('modelname');
signals=ScopeData.signals;
time=ScopeData.time;
Or when specified as an array:
sim('modelname');
signals=ScopeData(:,2);
time=ScopeData(:,1);
In the example above I don't specify the way the model is run, however you can also specify it as you posted.
Mathworks has done it again: my ancient R2012 (as bestowed by my company) returns a nice set of doubles identifying the figure window numbers in response to
currhandles=findall(0,'type','figure');
Now I got a fellow remotely IM-ing me 'cause the code I gave him fails under R2015 because findall now returns a structure for the figure handle. I can't play w/ his system (no RDC) and the mathworks documentation pages don't appear to specify the elements of the figure handle structure. In particular, I'd like to know if I can still retrieve the figure window number. Anyone know?
Of course.
currhandles(:).Number
will return all numbers as a comma-separated list.
Or specify a number you want:
currhandles(1).Number
The order appears to be the inverse order of initialization.
Alternatively you can define two anonymous functions to get an array directly:
figure(1); figure(2); figure(42);
getNumbers = #(x) [x.Number];
getFigureNumbers = #() getNumbers([findall(0,'type','figure')]);
getFigureNumbers()
ans =
42 2 1
I am trying to load feature vectors into classifiers such as a k-nearest neighbors classifier.
I have my code for GLCM, so I get contrast, correlation, energy, homogeneity in numbers (feature vectors).
My question is, how can I save every set of feature vectors from all the training images? I have seen somewhere that people had a .set file to load into classifiers (may be it is a special case for the particular classifier toolbox).
load 'mydata.set';
for example.
I suppose it does not have to be a .set file.
I'd just need a way to store all the feature vectors from all the training images in a separate file that can be loaded.
I've google,
and I found this that may be useful
but I am not entirely sure.
Thanks for your time and help in advance.
Regards.
If you arrange your feature vectors as the columns of an array called X, then just issue the command
save('some_description.mat','X');
Alternatively, if you want the save file to be readable, say in ASCII, then just use this instead:
save('some_description.txt', 'X', '-ASCII');
Later, when you want to re-use the data, just say
var = {'X'}; % <-- You can modify this if you want to load multiple variables.
load('some_description.mat', var{:});
load('some_description.txt', var{:}); % <-- Use this if you saved to .txt file.
Then the variable named 'X' will be loaded into the workspace and its columns will be the same feature vectors you computed before.
You will want to replace the some_description part of each file name above and instead use something that allows you to easily identify which data set's feature vectors are saved in the file (if you have multiple data sets). Your array of feature vectors may also be called something besides X, so you can change the name accordingly.