I am working on this project which requires analyzing a large (>50GB) dataset in a server, both in Stata and MATLAB. Both parts are required and I cannot use only one of them.
My ultimate goal is to generate a .tex file named something like commands.tex which looks like this:
\newcommand{\var1}{val1}
\newcommand{\var2}{val2} % MATLAB file matlab_file.m on DD/MM/YYYY
\newcommand{\var3}{val3} % Stata file stata_file.m on DD/MM/YYYY
...
where variables are ordered alphabetically and each of the values is most probably a number. Note that the commands in the comments would help me trace where did I generate the values. The usage of the file is so that after a preamble I can use LaTeX on the following way:
<preamble>
\input{commands.tex}
\begin{document}
Variable 1 has a value of \var1 and variable 2 has a value of \var2.
\end{document}
The purpose of this is so that I can analyze locally (or remotely) a sample, say of 0.1 or 10 percent of the total observations, write a report with those, and then run the analysis again with a bigger size. I want to completely eliminate the chances of me copying a number wrong.
I am trying to write some code both in MATLAB and Stata, but I think that is beyond my expertise, and would be very grateful if someone could help me figure out how to do it. To be honest, I feel I would be able to do the MATLAB part but the Stata I have no idea.
Stata code
What I am trying to do is to generate a command that takes as an input a name and a scalar and as an output defines the corresponding variable in my commands.tex file detailed above. My goal is to be able to generate something like this:
sysuse auto
reg price weight
define_variable PriceWeight = _b[weight], format(%4.2f)
and what I hope the code to do is that:
If \newcommand{\PriceWeight} does not exist in commands.tex then it adds its value to the list, preserving the alphabetical order.
If the variable exists then it deletes its value and rewrites above it, with the value given in the scalar.
I know how to give the values to a program in Stata, but I do not exactly know how to use those values and perform the necessary commands. The syntax is something like:
program define define_variable
syntax anything = X, [format(string)]
<other code>
end
Note: Of course, I need something way deeper than regression coefficients, but as a simple example this would suffice.
MATLAB code
This seems to be easier in MATLAB, but I do not know exactly how to automate the process. In MATLAB what I want to be able to do is something like:
clc; clear;
PriceWeight = 3
define_variable('PriceWeight',PriceWeight,format)
again where it automatically goes to the single file and updates it accordingly. Any hel[p with be very much appreciated.
Based on your comments and assuming that your file with all relevant variables is not huge, I would suggest getting your data from Stata to Matlab, and update your variables there as necessary (using functions such as exist or strcmp if you have a list of names). A quick google search gives me this link for Stata to Matlab.
To make it easy to process you might want to create a cell (I will call C), where one column contains all variable names and one column contains the scalar values.
Then, once you have assembled all your variables, you can sort your cell alphabetically and write it to a file using this.. Of course you would write a .tex file, and then iterate over your cell with something like
fprintf(fID,'\newcommand{\%s}{%f} ',C{i,1},C{i,2})
I hope this is understandable and helps.
Related
I am trying to use the MATLAB scripts shipped with Dymola to post-process the output result of Dymola. But in some cases, the output data in the .mat file only have 2 elements, how could I get the data between 10s and 100s in this kind of cases?
It's a parameter or variable that is not time depending so it's stored in a compact way. I understand the mechanism, but it is not user-friendly when post-processing the data in MATLAB, I have to find the "wrong" dimensional data. How could I fix this issue?
I recommend creating some simple logic that looks at the size of the variable and then automatically puts it into some dictionary, list, etc. From there you can manipulate the variable. I know you are asking for Matlab but here is a Python solution that I have used which may help you get started:
varNames_param_base=[]
varNames_var_base=[]
for i, val in enumerate(r.varNames()):
if np.size(r.values(val)) == 4:
varNames_param_base.append(val)
else:
varNames_var_base.append(val)
I used those lines in this file.
In the example r.varNames() is a list of all the variable names (i.e., strings) which are read from the resulting Dymola .mat file. r.values gets the value of the variable name currently being used in the for loop (i.e., val).
You may also consider converting your result file to SDF (a simple HDF5 representation), because that format does not use any clever storage options (if I remember correctly).
How can I create a function with MATLAB so I can call it any where in my code?
I'm new to MATLAB so I will write a PHP example of the code I want to write in MATLAB!
Function newmatlab(n){
n=n+1;
return n;
}
array=array('1','2','3','4');
foreach($array as $x){
$result[]=newmatlab($x);
}
print_f($result);
So in nutshell, I need to loop an array and apply a function to each item in this array.
Can some one show me the above function written in MATLAB so I can understand better?
Note: I need this because I wrote a code that analyzes a video file and then plots data on a graph. I then and save this graph into Excel and jpg. My problem is that I have more than 200 video to analyze, so I need to automate this code to loop inside folders and analyze each *.avi file inside and etc.
As others have said, the documentation covers this pretty thoroughly, but perhaps we can help you understand.
There are a handful of ways that you can define functions in Matlab, but probably the most useful for you to get started is to define one in an m-file. I'll use your example code. You can do this by creating a file called newmatlab.m in your project's directory that looks something like this
% newmatlab.m
function result = newmatlab(array)
result = array + 1
Note that the function has the same name as the file and that there is no explicit return statement - it figures that out by what you've named the output parameter(s) (result in this case).
Then, in the same directory, you can create a script (or another function) that calls your newmatlab function by that name:
% main.m (or whatever)
a = [1 2 3 4];
b = newmatlab(a)
That's it! This is a simplified explanation, but hopefully enough to get you started and then the documentation can help more.
PS: There's no "include" in Matlab; any functions that are defined in m-files in the current path are visible. You can find out what's in the path by using the path command. Roughly, it's going to consist of
Matlab's own directory
The MATLAB subdirectory of your Documents directory
The current working directory
Is there any way to create an empty .mat file from a terminal session? Basically, what I am doing is brain graph analysis. The software I am using, if an entire brain is scrubbed (ie, if the displacement of the brain is greater than a certain threshold) the output file will be left out or will be very small. When analyzing, however, I need to be able to eliminate both subjects from the analysis if the entire brain is scrubbed/too much of the brain is scrubbed. To accomplish this, the easiest way would be to simply check the dimensions of the output file within matlab, and if they are below the arbitrary threshold I decide then both subjects will just be skipped over for analysis. The issue is, I can easily check if a file contains too few remaining frames, however, if the resulting file contains no frames, it will entirely just not exist. As the outputs are all sorted, the only thing I need to do is check consecutive files' dimensions, and if one of the files does not contain enough values, then I can simply skip over it entirely. Simply touching a blank file obviously will not work, since it will not contain any encoding. I hope this is a good explanation for my motivation to do this, and if any of you know of any suggestions, please let me know.
A simple solution would be to create an empty file from Matlab and duplicate the file when needed from the console.
Just open Matlab, set to the destination folder and type this:
clear all
save empty.mat
Then, when needed, copy the file from the console. :)
Saving the contents of an empty struct creates an empty .mat file:
emptyStruct = struct;
save('myFile.mat','-struct','emptyStruct');
there are datasets in .mat format in the this site: http://www.cs.nyu.edu/~roweis/data.html
I want to change the format to .csv.
Can someone tell me how to change the format to create the .csv file.
Thanks!
Suppose that the .mat files from the site are available already. In the command window in Matlab, you may write, for example:
load('C:\Users\YourUserName\Downloads\mnist_all.mat');
to load the .mat file; the result should be a set of matrices test0, test1, ..., train0, train1 ... created in your workspace, which you want saved as CSV files. Because they're different size, you need to save one CSV per variable, e.g. (also in the command window):
csvwrite('C:\Users\YourUserName\Downloads\mnist_test0.csv', test0);
Repeat the command for each variable, and do not forget to change also the name of the output file to avoid overwriting.
Did you tried the csvwrite function in Matlab?
Just load your .mat files with the load function and then write them with csvwrite!
I do not have a Matlab license so I installed GNU Octave 4.2.1 (2017) on Windows 10 (thank you to John W. Eaton and others). I was not fully successful using the csvwrite so I used the following workaround. (BTW, I am totally incompetent in the Octave world. csvwrite worked for simple data structures).
In the Command Window I used the following two commands
load myfile.mat
save("-text","myfile.txt","variablename")
When the "myfile.mat" is loaded, the variable names for the data vectors loaded are displayed in the workspace window. This is the name(s) to use in the save command. Some .mat files will load several data structures.
The "-text" option is the default, so you may not need to include this option in the command.
The output file lists the .mat file contents in text format as single column (of potentially sequential variables). It should be easy to use you text editor to massage this data into the original matrix structure for use in whatever app you are comfortable with.
Had a similar issue. Needed to convert a series of .mat files that had two columns of numerical data into standard data files (ascii text). Note that I don't really ever use csv, but everything here could be adapted by using csvwrite instead of the standard save.
Using Octave 4.2.1 ....
load myfile.mat
LI = [L, I] ## L and I are column vectors representing my data
save myfile.txt LI
Note that L and I appear to be default variable names chosen by Octave for the two columns vectors in my original data file. Ideally a script that iterated over all files with the .mat extension in my directory would be ideal, but this got the job done. It saves the data as two space separated columns of data.
*** Update
The following script works on Octave 4.2.1 for a series of data files with the .mat extension that are in the same directory. It will iterate over them and write the data out to text files with the same name but with the extension .dat . Note that this is not efficient, so if you have a lot of files or if they are large it can take a while to run. I would suggest that you run it from the command line using octave mat2dat.m so you can actually watch it go.
I make no guarantees that this will work for you, but it did for me. I also am NOT proficient in Octave or Matlab, so I'm sure a better solution exists.
# mat2dat.m
dirlist = glob("*.mat")
for i=1:length(dirlist)
filename = dirlist{i,1}
load(filename, "L", "I")
LI = [L,I]
tmpname = filename(1:length(filename)-3)
txtname = strcat(tmpname, 'dat')
save(txtname, "LI")
end
I need to create a matlab mfile that will run another matlab file with default values given in a txt file. It's ment to be useful for testing programs, so that user may specify values in a txt files and instead of inputing values every time he starts the program, my script will give the program default values and user will only see the result.
My idea is to load tested file into a variable, change 'variable=input('...');' for variable = default_variable;, save it to tmp file, execute, and than delete tmp file. Is this going to do the job?
I have only two problems:
1) How to eliminate the problem of duplicated variable names - i mean this must work for all scripts, i don't know the names of variables used in tested script.
2) As I wrote before - is this going to work fine? Or maybe I missed a easier way to do it - for example maybe I don't have to create a tmp file?
I really need your help!
Thanks in advance!
If the person who has to edit the default values has access to Matlab, I would recommend saveing the values in a mat file and loading them when needed. Otherwise you could just write a smalls cript that contains the assignment to certain variables, but make sure to keep this small. For example:
maxRuns = 100;
clusters = 12;
So much for setting up the defaults. Regarding the process my main advice is to wrap the thing that you want to test into a function. This way variables used in the code to call the 'script' will not interfere as a function gets its own separate workspace. Check doc function if you are not familiar with them.