MATLAB: overriding table() methods - matlab

SETUP Win7 64b, R2015b, 16 GB of RAM, CPU i7-2700
The table() is a fundamental Matlab class which is also sealed, hence I cannot subclass it.
I want to fix some methods of this class and add new ones.
For instance, table.disp() is fundamentally broken, e.g. try NOT disp(table(rand(1e7,1))), or forget the ; in the command window. The variable takes only 76 MB in RAM but the display is unbuffered and it will stall your system!
Can I override methods like table.disp() without writing into matlabroot\toolbox\matlab\datatypes\#table?
Can I extend the table class with a new method under C:\MATLAB\#table\ismatrixlike.m? Why do I get
ismatrixlike(table)
Undefined function 'ismatrixlike' for input arguments of type 'table'.
Obviously, I did
addpath C:\MATLAB\
rehash toolboxcache
I also tried clear all.
The path has (alphabetic) precedence over matlabroot, but is missing a table.m class definition. If I add the native class defition to C:\MATLAB\#table, then I can run my new method (after a clear all). However:
>> methods(table)
Methods for class table:
classVarNames ismatrixlike table varfun
convertColumn renameVarNames unstack
only lists the methods in the new \#table folder, even though (some of) the old methods still work, e.g.
size(table)
This partly solves the problem, since now, the native \#table\private folder is not accessible anymore and therefore many native methods are broken!
Why am I doing this? Because I do not want to wait another 2 years before the table() is fixed. I already lost entire days because I simply forgot a ; in the command window and I cannot force a restart on my pc if it is running multiday simulations, but I have to wait for the disk-swap to end :(.
APPENDIX
More context about disp(table(rand(1e7,1))). This is what happens when I hit it (and luckily I am fast enough to CTRL-C out of it):
The culprit is line 172 of table.disp() which converts the numeric array into a cellstring (with the padding too!):
[cells, err, isLeft] = sprintfc(f, x, b);

After experimenting with several alternatives, I adopted the solution that intereferes the least with Matlab's native #table implementation and it's easily removed if things go awry.
The solution:
copy the whole #table folder, i.e. fullfile(matlabroot,'toolbox','matlab','datatypes','#table'), into a destination where you have write permissions.
I picked the destination to be fullfile(matlabroot,'toolbox','local','myfiles') since I do not have to bother with OS cross-compatibility, i.e. matlabroot takes care of that for me.
paste into the destination your #table folder with the new, overloaded and overriding methods (partially overwriting the copied original files)
add the destination to the matlab path, before the original #table, e.g. addpath your_destination -begin
Effects, pros and cons:
The native #table class/methods are now shadowed, try e.g. which table -all. However, this effect is quite clear, easily detectable and easily removed (delete destintation and remove path);
No weird conflicts between native #table (now shadowed) and new #table;
All methods, new and old, are visible, try methods(table);
Private table methods are accessible...
... but you are forced to use them.
Exposing the new methods (user-implemented) to the private ones requires more maintenance and direct handling of version conflicts in the table implementations.
You need write permissions on some eligible destination.
For those interested about the details, you can look into, https://github.com/okomarov/tableutils. Specifically the install_tableutils (the readme might not be updated).

The following works for me:
Define a modified disp function, say disp_modified.m, as follows, and put it in your path:
function disp_modified(t)
if istable(t)
%// Do whatever you want to display tables
builtin('disp', '''disp'' function intercepted!')
else
%// For non-tables, call `disp` normally
builtin('disp', t)
end
Define disp as a function handle to the modifed function (you can do that in startup.m to always have it by default):
disp = #disp_modified;
After this, in the command window I get
>> disp(1:5)
1 2 3 4 5
>> disp({1 2 3 'bb'})
[1] [2] [3] 'bb'
>> disp(table(rand(1e3,1)))
'disp' function intercepted!

Depending on the usage of the new class perhaps you could follow a cleaner approach. The proposed approach described in your post has the drawback that perhaps code used in your updated environment would not be easily portable to a new environment, or a program executed in your environment may demonstrate different behavior in a different environment.
Some questions you could consider (and perhaps clarify) would be: How do you intend to use the new class? Do you want to replace all the existing table uses? Do you want to be able to use it instead of a table class argument? Or do you want to alter the table so that each usage of the original table class in your environment uses the new class.
If you just need a new improved table for your usage, you could consider encapsulating the original table class in a new class. E.g MyTable, delegate all the methods you do not need to the original table methods, replace the methods you would like to improve or add new ones.
Update: Just saw the complete solution in Github and understood what you intended to do. Nice work. I will leave the post in case anyone finds it useful.

Related

Exploit Matlab copy-on-write by ensuring function arguments are read-only?

Background
I'm planning to create a large number of Matlab table objects once, so that I can quickly refer to their contents repeatedly. My understanding is that each table variable/column is treated in copy-on-write manner. That is, if a table column is not modified by a function, then a new copy is not created.
From what I recall of C++ as of 1.5 decades ago, I could ensure that the code for a function does not modify its argument's data by using constant-correctness formalism.
The specific question
I am not using C++ in these days, but I would like to achieve a similar effect of ensuring that the code for my Matlab function doesn't change the data for selected arguments, either inadvertently or otherwise. Does anyone know of a nonburensome way to do this, or just as importantly, whether this is an unrealistic expectation?
I am using R2015b.
P.S. I've web searched and came across various relevant articles, e.g.:
http://www.mathworks.com/matlabcentral/answers/359410-is-it-possible-to-avoid-copy-on-write-behavior-in-functions-yet
http://blogs.mathworks.com/loren/2007/03/22/in-place-operations-on-data
(which I need clarification on to fully understand, but it isn't my priority just now)
However, I don't believe that I am prematurely optimizing. I know that I don't want to modify the tables. I just need a way to enforce that without having to go through contortions like creating a wrapper class.
I've posted this at:
* Stack Overflow
* Google groups
There is no way of making variables constants in MATLAB, except by creating a class with a constant (and static?) member variable. But even then you can do:
t = const_table_class.table;
t(1,1) = 0; % Created and modified a copy!
The reason that a function does not need to mark its inputs as const is because arguments are always passed by value. So a local modification does not modify data in the caller’s workspace. const is something that just doesn’t exist in the MATLAB language.
On the other hand, you can be certain that your data will not be modified by any of the functions you call. Thus, as long as the function that owns the tables does not modify them, they will remain constant. Any function you pass these tables to, if they attempt to modify them, they will create a local copy to be modified. This is only locally a problem. The memory used up by this copy will be freed upon function exit. It will be a bug in the function, but not affect code outside this function.
You can define a handle class that contains a table as it's preperty. Define a property set listener that triggers and generates error/warning when the value of the property changes.
classdef WarningTable < handle
properties (SetObservable)
t
end
methods
function obj = WarningTable(varargin)
obj.t = table(varargin);
addlistener(obj,'t','PreSet',...
#(a,b)warning('table changed!'));
end
end
end
This should generate warning:
mytable = WarningTable;
mytable.t(1,1) = 0;

'statsfminbx' error with MATLAB built-in factoran()

Could someone please assist in debugging the following error, while performing factor analysis on my data?
factoran(x,2) ; %where x is <250*10> double =>
Undefined function 'statsfminbx' for input arguments of type 'cell'.
Error in factoran
[Psi1, nll, lagrange, err, output] = ...
After looking into the codes (from line 269), it sounds like for some reasons maximum likelihood estimate of the loadings could not be obtained, thus I look for a way around to overcome this matter.
Thanks in advance
statsfminbx is a private function to factoran, and can only be called by factoran itself.
If you create a folder called private and put code in it, that code can only be called from code in the immediate parent folder (it's actually only on the path at all, for code in the immediate parent folder). This is a useful technique to avoid polluting the namespace.
The original factoran is in toolbox\stats\stats\factoran.m, and can therefore call toolbox\stats\stats\private\statsfminbx.m.
When you made a copy of factoran and placed it outside the default folder, your copy can no longer access the private function statsfminbx. You may try making your own copy of the private folder, placing it in the appropriate place beneath your copy of factoran, and putting a copy of statsfminbx in it. (On the other hand, you may find that you subsequently get a different error related to another similar problem with another private function).
PS unless you have a good reason for doing this, it's typically not a good idea to be making your own copies of toolbox code - it may well break, for these sort of reasons.
PPS you can tell that this is the problem by typing which statsfminbx, you'll see it's not found - but type which -all statsfminbx and you'll see it finds it, telling you that it's a private function to the stats folder.

Can I use Eclipse templates to insert methods and also call them?

I'm doing some competitions on a website called topcoder.com where the objective is to solve algorithmic problems. I'm using Eclipse for this purpose, and I code in Java, it would be help me to have some predefined templates or macros that I can use for common coding tasks. For example I would like to write methods to be able to find the max value in and int[] array, or the longest sequence in an int[] array, and so on (there should be quite many of these). Note I can't write these methods as libraries because as part of the competition I need to submit everything in one file.
Therefore ideally, I would like to have some shortcut available to generate code both as a method and as a calling statement at once. Any ideas if this is possible?
Sure you can - I think that's a nifty way to auto-insert boilerplate or helper code. To the point of commenters, you probably want to group the code as a helper class, but the general idea sounds good to me:
You can see it listed in your available templates:
Then as you code your solution, you can Control+Space, type the first few characters of the name you gave your template, and you can preview it:
And then you can insert it. Be sure if you use a class structure to position it as an inner class:
Lastly - if you want to have a template inserts a call to method from a template, I think you would just use two templates. One like shown above (to print the helper code) and another that might look like this, which calls a util method and drops the cursor after it (or between the parentheses if you'd like, etc):
MyUtils.myUtilMethod1();${cursor}

Find indirect calls to specific built-in MATLAB function

What do I want?
I am looking for a way to detect all points in my code where a specific function is called.
Why do I want it?
Some examples:
Some output comes out sorted or randomized, and I want to know where this happens
I am considering to change/overload a function and want to know in which part of my code this could have impact
What have I tried?
I tried placing a breakpoint in the file that was called. This only works for non builtin functions which are called from short running code that always executes everything.
I tried 'find files', this way I can easily find direct calls to sort but it is not so easy to find a call to sort invoked by unique for example.
I have tried depfun, it tells me:
whether something will be called
from where non-builtin functions will be called
I thought of overloading the builtin function, but feels like a last resort for me as I am afraid to make a mess. | Edit: Also it probably won't help due to function precedence.
The question
What is the best way to track all potential (in)direct function calls from a specific function to a specific (built-in)function.
I don't exactly understand your use case, but I guess most of the information you want can be obtained using dbstack, which gives you the call-stack of all the parent functions calling a certain function. I think the easiest way is to overload built-in functions something like this (I tried to overload min):
function varargout = min(varargin)
% print info before function call
disp('Wrapped function called with inputs:')
disp(varargin)
[stack,I] = dbstack();
disp('Call stack:')
for i=1:length(stack)
fprintf('level %i: called from line %i in file %s\n', ...
i, stack(i).line, stack(i).file);
end
% call original function
[varargout{1:nargout}] = builtin('min', varargin{:});
% print info after function call
disp('Result of wrapped function:')
disp(varargout)
I tried to test this, but I could not make it work unfortunately, matlab keeps on using the original function, even after playing a lot with addpath. Not sure what I did wrong there, but I hope this gets you started ...
Built-in functions take precedence over functions in local folder or in path. There are two ways you can overload a built-in for direct calls from your own code. By putting your function in a private folder under the same directory where your other MATLAB functions are. This is easier if you are not already using private folder. You can rename your private folder once you are done investigating.
Another way is to use packages and importing them. You put all your override functions in a folder (e.g. +do_not_use). Then in the function where you suspect built-in calls are made add the line "import do_not_use.*;". This will make calls go to the functions in +do_not_use directory first. Once you are done checking you can use "clear import" to clear all imports. This is not easy to use if you have too many functions and do not know in which function you need to add import.
In addition to this, for each of the function you need to follow Bas Swinckels answer for the function body.
Function precedence order.
Those two methods does not work for indirect calls which are not from your own code. For indirect calls I can only think of one way where you create your own class based on built-in type. For example, if you work only on double precision types, you need to create your own class which inherits from double and override the methods you want to detect. Then pass this class as input to your code. Your code should work fine (assuming you are not using class(x) to decide code paths) since the new class should behave like a double data type. This option will not work if your output data is not created from your input data. See subclassing built-in types.
Did you try depfun?
The doc shows results similar to the ones you request.
doc depfun:
...
[list, builtins, classes, prob_files, prob_sym, eval_strings, called_from, java_classes] = depfun('fun') creates additional cell arrays or structure arrays containing information about any problems with the depfun search and about where the functions in list are invoked. The additional outputs are ...
Looks to me you could just filter the results for your function.
Though need to warn you - usually it takes forever to analyze code.

lua - capturing variable assignments

Ruby has this very interesting functionality in which when you create a class with 'Class.new' and assign it to a constant (uppercase), the language "magically" sets up the name of the class so it matches the constant.
# This is ruby code
MyRubyClass = Class.new(SuperClass)
puts MyRubyClass.name # "MyRubyClass"
It seems ruby "captures" the assignment and inserts sets the name on the anonymous class.
I'd like to know if there's a way to do something similar in Lua.
I've implemented my own class system, but for it to work I've got to specify the same name twice:
-- This is Lua code
MyLuaClass = class('MyLuaClass', SuperClass)
print(MyLuaClass.name) -- MyLuaClass
I'd like to get rid of that 'MyLuaClass' string. Is there any way to do this in Lua?
When assigning to global variables you can set a __newindex metamethod for the table of globals to catch assignments of class variables and do whatever is needed.
You can eliminate one of the mentions of MyLuaClass...
> function class(name,superclass) _G[name] = {superclass=superclass} end
> class('MyLuaClass',33)
> =MyLuaClass
table: 0x10010b900
> =MyLuaClass.superclass
33
>
Not really. Lua is not an object-orientated language. It can behave like one sometimes. But far from every time. Classes are not special values in Lua. A table has the value you put in it, no more. The best you can do is manually set the key in _G from the class function and eliminate having to take the return value.
I guess that if it REALLY, REALLY bothers you, you could use debug.traceback(), get a stack trace, find the calling file, and parse it to find the variable name. Then set that. But that's more than a little overkill.
With respect at least to Lua 5.2: You can capture assignments to A) the global table of a Lua State, as mentioned in a previous reply, and also B) to any other Lua Object whose __index and __newindex metamethods have been substituted (by replacing the metatable), this I can confirm as I'm currently using both these techniques to hook and redirect assignments made by Lua scripts to external C/C++ resource management.
There is a gotcha with regards to reading them back though, the trick is to NOT let the values be set in a Lua State.
As soon as they exist there, your hooks will fail to be called, so if you want to go down this path, you need to capture ALL get/set attempts, and NEVER store the values in a Lua State.