Related
I want to be able to run something like the following:
f:{[dt] syms:`sym1;eval parse"select from tbl where date = dt, sym=syms"}
f[.z.D]
Given the following :
tbl:([] date:2022.01.01 2022.01.01; Id:1000000 2000000; sym:`sym1`sym2;price:10 20;qty:3 4)
f:{[dt] syms:`sym1; ?[tbl;((=;`date;`dt);(=;`sym;`syms));0b;()]}
f1:{[dt] syms:`sym1; (?) . (tbl;((=;`date;`dt);(=;`sym;`syms));0b;())}
f2:{[dt] syms:`sym1; value (?;tbl;((=;`date;`dt);(=;`sym;`syms));0b;())}
f[.z.D] // works
f1[.z.D] // Gives Error - dt not recognized/out of scope
f2[.z.D] // Gives Error - dt not recognized/out of scope
Value within a function seems to not be able to detect the local variables and surprisingly (?) . also fails. (maybe because this in itself is a function and dt is not defined here?)
Is there any work around for this?
For context, I have a function that takes a select string/functional select, parses it, does some checks and manipulations on the functional form and returns a modified functional form.
I want users to be able to call this function from their own functions and that parameters they have defined in their function can be in the outputted functional form and that functional form can be valued some how.
I don't want users to be forced to pass more variables into my function etc.
What you need to do here is remove the backtick for dt and syms
I would also recommend using a backtick when calling your table name.
Further, you should make sure syms is enlisted if it is only one symbol.
So your function should be:
f:{[dt] syms:(),`sym1; ?[`tbl;((=;`date;dt);(=;`sym;syms));0b;()]}
If you parse your select statement you can see the correct form for functional selects:
q)parse "select from tbl where date=2022.01.01,sym=`sym1"
?
`tbl
,((=;`date;2022.01.01);(=;`sym;,`sym1)) // comma in front of `sym1 means enlist
0b
()
The backtick is not needed as this is a variable, defined in your function, it would be the same as doing:
?[`tbl;((=;`date;2022.01.01);(=;`sym;enlist `sym1));0b;()]
This should allow you to use your function correctly:
q)f[2022.01.01]
date Id sym price qty
---------------------------------
2022.01.01 1000000 sym1 10 3
For more information, see the kx documentation
I have used the following code pattern to access my *.accdb files:
accdb_path='C:\path\to\accdb\file\wbe3.accdb';
accdb_url= [ 'jdbc:odbc:Driver={Microsoft Access Driver (*.mdb, *.accdb)};DSN='''';DBQ=' accdb_path ];
conn = database('','','','sun.jdbc.odbc.JdbcOdbcDriver',accdb_url);
If instead I want to create a new *.accdb file, how would I do that? There is much on the web about how to connect, but I haven't found how to create the *.accdb file itself.
In case it matters, I want to be able to execute SQL 92 syntax. I am using Matlab 2015b. I do not want to use the Matlab GUI for exploring databases.
Actually, what you are attempting to do can be very tricky to achieve. It may require a direct interface to Access through an ActiveX control and I'm not even sure it can be done. It seems that the web is lacking a solid information pool concerning Access interoperability.
One quick workaround I can suggest you, althrough miserable, is to manually create an empty ACCDB file that you can use as template and then duplicate it whenever a new database must be created:
conn = CreateDB('C:\PathB\wbe3.accdb');
function accdb_conn = CreateDB(accdb_path)
status = copyfile('C:\PathA\template.accdb',accdb_path,'f');
if (status)
accdb_url = ['jdbc:odbc:Driver={Microsoft Access Driver (*.mdb, *.accdb)};DSN='';DBQ=' accdb_path];
accdb_conn = database('','','','sun.jdbc.odbc.JdbcOdbcDriver',accdb_url);
else
accdb_conn = [];
error(['Could not duplicate the ACCDB template to the directory "' accdb_path '".']);
end
end
The following example is based on Tommaso's answer, which provides code for copying an empty *.accdb file and connecting to the copy. Based on an afternoon of trial, error, perusing of the web/help, I've expanded on that to create a database table and export a Matlab table to it. I've also embedded comments showing where modifications are needed, presumably due to my older 2015b version of Matlab, error catching constructs, and caveats in the file copy.
srcPath = [pwd '/emptyFile.accdb']; % Source
tgtPath = [pwd '/new.accdb']; % Target
cpyStatOk = copyfile( srcPath, tgtPath );
% No warning B4 clobber target file
if cpyStatOk
accdb_url= [ ...
'jdbc:odbc:Driver={Microsoft Access Driver (*.mdb, *.accdb)};DSN='''';DBQ=' ...
tgtPath ];
conn = database('','','','sun.jdbc.odbc.JdbcOdbcDriver',accdb_url);
else
error('Couldn''t copy %s to %s',srcPath,tgtPath);
end % if cpyStatOk
try
% conn.Execute(['CREATE TABLE tstMLtbl2accdb ' ... Not for 2015b
curs = conn.exec(['CREATE TABLE tstMLtbl2accdb ' ...
'( NumCol INTEGER, StrCol VARCHAR(255) );']);
if ~isempty( curs.Message )
% fprintf(2,'%s: %s\n',mfilename,curs.Message);
error('%s: %s\n',mfilename,curs.Message);
% Trigger `catch` & close(conn)
end %if
% sqlwrite( conn, 'tstMLtbl2accdb', ...Not supported in 2015b
datainsert( conn, 'tstMLtbl2accdb', {'NumCol','StrCol'}, ...
table( floor(10*rand(5,1)), {'abba';'cadabra';'dog';'cat';'mouse'}, ...
'VariableNames',{'NumCol','StrCol'} ) );
catch xcptn
close(conn)
fprintf(2,'Done `catch xcptn`\n');
rethrow(xcptn);
end % try
%
% Other database manipulations here
%
close(conn)
disp(['Done ' mfilename]);
This has been immensely educational for myself, and I hope it is useful for others considering the use of SQL as an alternative to the more code-heavy Matlab counterpart to relational database manipulations. With this amount of overhead, I'd have to say that it is not attractive to perform SQL manipulations on data residing in the Matlab workspace except where one really needs the hyperoptimization of relational database query engines.
To those savvy with interfacing to Access, your comment on the purpose of the field names argument of the datainsert function would be appreciated. It is dubbed colnames in the documentation. From testing, the field names and number of columns must match between the existing target table in Access and the source table in Matlab. So the field names argument doesn't seem to serve any purpose. The help documentation isn't all that helpful.
AFTERNOTE: I've composed a "specification" for the colnams argument based on examples from TMW. TMW has confirmed this explanation:
The colnames argument tells the external database environment the names and order of fields of the data container supplied via the data argument. These field names are used to match the fields of the transferred data with fields in the table tablename residing within the external database environment. Because of this explicit name matching, the order of the fields in data do not have to match the order of the fields in tablename.
If I find any departures of empirical behaviour from the above "specification", I will update this answer.
I am trying to write a function that finds all the variables in a function which contains computing operations. I do this to find out which variables this block of calculations requires as input arguments to perform the calculations.
A calculation function always gets a table with different parameters as input, accesses certain parameters of that table to calculate certain metrics.
For example my data table T contains the double arrays Power and Time. This table is passed to the function calc_energy which then calculates Energy:
function [ T ] = calc_energy( T )
T.Energy = T.Power .* T.Time;
end
Say whenever a new data table is generated, different calculations as the above one are run automatically. Now if Power itself is calculated by the function calc_power, but calc_energy is run before or parallel to calc_power, then I have a problem because calc_energy doesn't find the required variables in T.
In order to prevent such an error, I want my function check_required_variables to be called inside the calc_energy and check beforehand whether T.Power exists. The thing is that check_required_variables should be a general function that is called in every single calculation function and therefore doesn't know the required variables. It has to find them in the function that it is called by.
function [ T ] = calc_energy( T )
OK = check_required_variables( T, 'calc_energy.m' );
if OK == 1
T.Energy = T.Power .* T.Time;
else
error('Required fields not found');
end
end
Are there any functions that spot the variables Energy, Power and Time used in my code? And are there functions that maybe analyse the dependencies between these variables (Energy dependent on Power and Time)? What are generally clever ways to figure out the ocurring variables just from the code? Any ideas?
I know this is a tough one so I'm thankful for any suggestions.
use isfield
function [ T ] = calc_energy( T )
if all(isfield(T,{'Power','Time'}))
T.Energy = T.Power .* T.Time;
else
error('Nooooope')
end
end
However, knowing which ones are required seems a bit harder.... You can always try to read the .m file, and regexp for T.____, then put the input of that to isfield.
This however, just hints of bad software design. A function should know what it requires to run. What happens if OK is false? It just skips the computation? You can then cascade to hundreds of calls (depend on the application) because the original structure failed to have a variable, or had a typo. I'd take the radically opposite approach to software design:
function [ T ] = calc_energy( T )
assert(all(isfield(T,{'Power','Time'}))); %error if there are not.
T.Energy = T.Power .* T.Time;
end
I have a cell array of stucts, each containing the personalia of a person. I put it into this function to get them listed in a text file of a chosen name.
function store( filename, persons )
fid = fopen(filename,'w');
for i=1:length(persons)
fprintf(fid, '%s',serialize_person(persons{i}));
end
Now this function works fine: I enter a <1x3 cell> and get out a text file with three listed persons. However, I want to call this function from another:
function process_store()
list=input('Write in the list of persons you want listed: ');
fprintf('\n')
newfile=input('Give the text file a name: ','s');
store(filename,list)
end
Here I enter the name of the <1x3 cell> as before, but I get a error message "Error using input,Undefined function or variable 'persons'."
Why is this? Am I not using the exact same data as Im using in 'store'?
The problem is that the variable persons isn't accessible inside the function process_store. In Matlab (and most other programming languages) functions can't access variables defined in their calling functions. To understand this better, I recommend having a read of the Wikipedia article on levels of scope.
You essentially have two options here:
Make persons a global variable, both in your workspace and in the function process_store, by using the declaration global persons. I wouldn't recommend this.
Use the function evalin to allow process_store to access variables in its parent workspace.
I'd go with option 2 if I were you. It's a little tricky, so let me explain how it works. Let's create a variable persons in the global workspace.
>> persons = {'John', 'Jack', 'Jill'};
Now say we have the following function
function example()
x = input('Give me a variable name: ');
disp(x)
end
What happens if we try to use it?
>> example()
Give me a variable name: persons
Error using input
Undefined function of variable 'persons'
Error in example (line 2)
x = input('Give me a variable name: ');
Oh dear. That's because the function example doesn't have access to the global workspace, which is where persons is defined. But instead, we can store the name of the variable we want to access, and then check out its value in the global workspace by using evalin, like this
function example()
s = input('Give me a variable name: ', 's');
x = evalin('caller', s);
disp(x)
end
Now if we use it:
>> example()
Give me a variable name: persons
'John' 'Jack' 'Jill'
It works as we expected! Great!
Massive disclaimer
There is almost never a good reason to use functions like evalin (or eval, or assignin or any other function that messes around executing strings as code). There's almost certainly a better way of doing what you want to do. But without knowing what it is you're trying to do, it's hard to give you better advice.
At the prompt
Write in the list of persons you want listed:
if you typed
persons
then you would get exactly that error message if the variable persons was not defined.
I am trying to create a function call graph for around 500 matlab src files. I am unable to find any tools which could help me do the same for multiple src files.
Is anyone familiar with any tools or plugins?
In case any such tools are not available, any suggestions on reading 6000 lines of matlab code
without documentation is welcome.
Let me suggest M2HTML, a tool to automatically generate HTML documentation of your MATLAB m-files. Among its feature list:
Finds dependencies between functions and generates a dependency graph (using the dot tool of GraphViz)
Automatic cross-referencing of functions and subfunctions with their definition in the source code
Check out this demo page to see an example of the output of this tool.
I recommend looking into using the depfun function to construct a call graph. See http://www.mathworks.com/help/techdoc/ref/depfun.html for more information.
In particular, I've found that calling depfun with the '-toponly' argument, then iterating over the results, is an excellent way to construct a call graph by hand. Unfortunately, I no longer have access to any of the code that I've written using this.
I take it you mean you want to see exactly how your code is running - what functions call what subfunctions, when, and how long those run for?
Take a look at the MATLAB Code Profiler. Execute your code as follows:
>> profile on -history; MyCode; profile viewer
>> p = profile('info');
p contains the function history, From that same help page I linked above:
The history data describes the sequence of functions entered and exited during execution. The profile command returns history data in the FunctionHistory field of the structure it returns. The history data is a 2-by-n array. The first row contains Boolean values, where 0 means entrance into a function and 1 means exit from a function. The second row identifies the function being entered or exited by its index in the FunctionTable field. This example [below] reads the history data and displays it in the MATLAB Command Window.
profile on -history
plot(magic(4));
p = profile('info');
for n = 1:size(p.FunctionHistory,2)
if p.FunctionHistory(1,n)==0
str = 'entering function: ';
else
str = 'exiting function: ';
end
disp([str p.FunctionTable(p.FunctionHistory(2,n)).FunctionName])
end
You don't necessarily need to display the entrance and exit calls like the above example; just looking at p.FunctionTable and p.FunctionHistory will suffice to show when code enters and exits functions.
There are already a lot of answers to this question.
However, because I liked the question, and I love to procrastinate, here is my take at answering this (It is close to the approach presented by Dang Khoa, but different enough to be posted, in my opinion):
The idea is to run the profile function, along with a digraph to represent the data.
profile on
Main % Code to be analized
p = profile('info');
Now p is a structure. In particular, it contains the field FunctionTable, which is a structure array, where each structure contains information about one of the calls during the execution of Main.m. To keep only the functions, we will have to check, for each element in FunctionTable, if it is a function, i.e. if p.FunctionTable(ii).Type is 'M-function'
In order to represent the information, let's use a MATLAB's digraph object:
N = numel(p.FunctionTable);
G = digraph;
G = addnode(G,N);
nlabels = {};
for ii = 1:N
Children = p.FunctionTable(ii).Children;
if ~isempty(Children)
for jj = 1:numel(Children)
G = addedge(G,ii,Children(jj).Index);
end
end
end
Count = 1;
for ii=1:N
if ~strcmp(p.FunctionTable(ii).Type,'M-function') % Keep only the functions
G = rmnode(G,Count);
else
Nchars = min(length(p.FunctionTable(ii).FunctionName),10);
nlabels{Count} = p.FunctionTable(ii).FunctionName(1:Nchars);
Count = Count + 1;
end
end
plot(G,'NodeLabel',nlabels,'layout','layered')
G is a directed graph, where node #i refers to the i-th element in the structure array p.FunctionTable where an edge connects node #i to node #j if the function represented by node #i is a parent to the one represented by node #j.
The plot is pretty ugly when applied to my big program but it might be nicer for smaller functions:
Zooming in on a subpart of the graph:
I agree with the m2html answer, I just wanted to say the following the example from the m2html/mdot documentation is good:
mdot('m2html.mat','m2html.dot');
!dot -Tps m2html.dot -o m2html.ps
!neato -Tps m2html.dot -o m2html.ps
But I had better luck with exporting to pdf:
mdot('m2html.mat','m2html.dot');
!dot -Tpdf m2html.dot -o m2html.pdf
Also, before you try the above commands you must issue something like the following:
m2html('mfiles','..\some\dir\with\code\','htmldir','doc_dir','graph','on')
I found the m2html very helpful (in combination with the Graphviz software). However, in my case I wanted to create documentation of a program included in a folder but ignoring some subfolders and .m files. I found that, by adding to the m2html call the "ignoreddir" flag, one can make the program ignore some subfolders. However, I didn't find an analogue flag for ignoring .m files (neither does the "ignoreddir" flag do the job). As a workaround, adding the following line after line 1306 in the m2html.m file allows for using the "ignoreddir" flag for ignoring .m files as well:
d = {d{~ismember(d,{ignoredDir{:}})}};
So, for instance, for generating html documentation of a program included in folder "program_folder" but ignoring "subfolder_1" subfolder and "test.m" file, one should execute something like this:
m2html( 'mfiles', 'program_folder', ... % set program folder
'save', 'on', ... % provide the m2html.mat
'htmldir', './doc', ... % set doc folder
'graph', 'on', ... % produce the graph.dot file to be used for the visualization, for example, as a flux/block diagram
'recursive', 'on', ... % consider also all the subfolders inside the program folders
'global', 'on', ... % link also calls between functions in different folders, i.e., do not link only the calls for the functions which are in the same folder
'ignoreddir', { 'subfolder_1' 'test.m' } ); % ignore the following folders/files
Please note that all subfolders with name "subfolder_1" and all files with name "test.m" inside the "program_folder" will be ignored.