I've been working with matrices over GF(2) in Matlab. Well, I've been working with 0/1 matrices that I've been treating as being defined over GF(2). I was surprised/happy to see that Matlab provides some functionality in the Communications System Toolbox for working over finite fields. In particular, if I want to find the rank of a matrix over a finite field, there are a couple of methods: (1) use gfrank on the matrices that I already have defined, or (2) use rank on a Galois field array (created with gf). For matrices over GF(2), the former method seems to be significantly faster; however, there's a problem...
The documentation for gfrank says that the function doesn't work over fields of the form GF(2^m). I double checked on a toy example, and specifying GF(2) as the field to work over seems to output correct results. Moreover, the function's m-file specifies GF(2) as the default field (by specifying the second argument as 2 if nargin < 2). Something has to be wrong here, and it seems to be the documentation. However, I'd hate to assume that the documentation is wrong only to find out much later that the computation doesn't always work over GF(2^m). Does anybody know for sure what's wrong here? Thanks for your help.
Related
I have a lengthy symbolic expression that involves rational polynomials (basic arithmetic and integer powers). I'd like to simplify it into a single (simple) rational polynomial.
numden does it, but it seems to use some expensive optimization, which probably addresses a more general case. When tried on my example below, it crashed after a few hours--out of memory (32GB).
I believe something more efficient is possible even if I don't have a cpp access to matlab functionality (e.g. children).
Motivation: I have an objective function that involves polynomials. I manually derived it, and I'd like to verify and compare the derivatives: I subtract the two expressions, and the result should vanish.
Currently, my interest in this is academic since practically, I simply substitute some random expression, get zero, and it's enough for me.
I'll try to find the time to play with this as some point, and I'll update here about it, but I posted in case someone finds it interesting and would like to give it a try before that.
To run my function:
x = sym('x', [1 32], 'real')
e = func(x)
The function (and believe it or not, this is just the Jacobian, and I also have the Hessian) can't be pasted here since the text limit is 30K:
https://drive.google.com/open?id=1imOAa4VS87WDkOwAK0NoFCJPTK_2QIRj
I have a very big sparse csc_matrix x. I want to do elementwise exp() on it. Basically what I want is to get the same result as I would have got with numpy.exp(x.toarray()). But I can't do that(my memory won't allow me to convert the sparse matrix into an array). Is there any way out? Thanks in advance!
If you don't have the memory to hold x.toarray(), you don't have the memory to hold the output you're asking for. The output won't be sparse; in fact, unless your input has negative infinities in it, the output probably won't have a single 0.
It'd probably be better to compute exp(x)-1, which is as simple as
x.expm1()
If you want to do something on nonzeros only: the data attribute is writable at least in some representations including csr and csc. Some representations allow for duplicate entries, so make sure you are acting on a "normalised" form.
To change non-zero elements, maybe this would work for you:
x = some big sparse matrix
np.exp( x.data, out=x.data ) # ask np.exp() to store results in existing x.data
presumably slower:
# above seems more efficient (no new memory alloc).
x.data = np.exp( x.data )
I've been wrestling with how to get an element-wise log2() of each non-zero array element. I ended up doing smth like:
np.log2( x.data, out=x.data )
The following two techniques seem like exactly what I was looking for. My matrix is sparse but it still plenty of non-zero elements.
Credit to #DSM here for the idea of directly changing x.data, I think that is a superb insight about sparse matrices.
Credit to #Mike Müller for the idea of using "out" as itself. In the same thread, #kmario23 points out an important caveat about promoting .data to floats (inputs could be int or smth) so it is compatible with the .exp() or whatever function, I would want to do that if I was writing smth for general use.
note: I'm just starting to learn about sparse matrices, so would like to know if this is a bad idea for reason(s) I'm not seeing. Please do let me know if I'm on thin ice with this.
Normally I wouldn't mess with private attributes, but .data shows up pretty clearly in the attributes documentation for the various sparse matrices I've looked at.
I have two matrices S and T which have n columns and a row vector v of length n. By my construction, I know that S does not have any duplicates. What I'm looking for is a fast way to find out whether or not the row vector v appears as one of the rows of S. Currently I'm using the test
if min([sum(abs(S - repmat(f,size(S,1),1)),2);sum(abs(T - repmat(v,size(dS_new,1),1)),2)]) ~= 0 ....
When I first wrote it, I had a for loop testing each (I knew this would be slow, I was just making sure the whole thing worked first). I then changed this to defining a matrix diff by the two components above and then summing, but this was slightly slower than the above.
All the stuff I've found online says to use the function unique. However, this is very slow as it orders my matrix after. I don't need this, and it's a massively waste of time (it makes the process really slow). This is a bottleneck in my code -- taking nearly 90% of the run time. If anyone has any advice as how to speed this up, I'd be most appreciative!
I imagine there's a fairly straightforward way, but I'm not that experienced with Matlab (fairly, just not lots). I know how to use basic stuff, but not some of the more specialist functions.
Thanks!
To clarify following Sardar_Usama's comment, I want this to work for a matrix with any number of rows and a single vector. I'd forgotten to mention that the elements are all in the set {0,1,...,q-1}. I don't know whether that helps or not to make it faster!
You may want this:
ismember(v,S,'rows')
and replace arguments S and v to get indices of duplicates
ismember(S,v,'rows')
Or
for test if v is member of S:
any(all(bsxfun(#eq,S,v,2))
this returns logical indices of all duplicates
all(bsxfun(#eq,S,v),2)
I would like to ensure that the input arguments to a user-defined MATLAB function (contained in an m-file) are of a certain type. I understand that MATLAB automatically assigns data types to variables (to the liking of some and the dismay of others), but I would like to know if there is an option of "strict data typing" in MATLAB, or something of the sort, especially for input arguments for a user-defined function.
I found a helpful explanation of MATLAB's "fundamental classes" (data types) at these two webpages:
http://www.mathworks.com/help/matlab/matlab_prog/fundamental-matlab-classes.html
http://www.mathworks.com/help/matlab/data-types_data-types.html
However, I have been unable to find an answer to the question of strict data typing, particularly for function input arguments. I thought it would be a pretty basic question that already had been answered in numerous places, but after extensive searching I have not yet found a conclusive answer. For now, I have been manually checking the data type using the is[TYPE]() functions and displaying an error message if it does not comply, though this seems sloppy and I wish I could just get MATLAB to enforce it for me.
Below is an example of a function in which I would like to specify the input argument data type. It resides in a file called strict_data_type_test.m in MATLAB's current path. In this function, I would like to force the variable yes_or_no to be of MATLAB's logical data type. I know I can use the islogical() function to manually check, but my question is if it is possible to have MATLAB enforce it for me. I also know that any non-zero double evaluates to true and a zero evaluates to false, but I want to force the user to send a logical to make sure the wrong argument was not sent in by accident, for example. Here is the example function:
function y = strict_data_type_test( x, yes_or_no )
% manual data type check can go here, but manual check is not desirable
if (yes_or_no)
y = 2 .* x;
else
y = -5 .* x;
end
end
Adding the data type before the input argument variable name (like in most programming languages) treats the data type text as another variable name instead of a data type identifier. From that it would seem that strict data typing is not possible in MATLAB by any means, but maybe one of you many gurus knows a useful trick, feature, or syntax that I have not been able to find.
validateattributes might also work for you, if there is an appropriate attribute for your case. For example if you want to enforce that yes_or_no is a logical scalar, you could try:
validateattributes(yes_or_no,{'logical'},{'scalar'})
Otherwise maybe an attribute like 'nonempty'.
I've gotten some great responses so I can't pick just one as the "accepted answer", but to summarize what I've learned from you all so far:
No, MATLAB does not have built-in strict data typing for function input arguments
MATLAB compiles the code before running, so manual validation checking should not be very taxing on performance (the profiler can be used to assess this)
Many helpful methods of doing the manual validation checking exist, listed here in order of most relevant to least relevant for what I was trying to do:
inputParser class
validateattributes()
Error/exception handling (throw(), error(), assert(), etc.)
MATLAB's built-in state detection functions (a.k.a predicate functions)
I can look through some MathWorks-provided MATLAB functions (or Statistics toolbox functions) for ideas on how to validate input arguments by typing edit followed by the function name. Two suggested functions to look at are normpdf() (from the Statistics toolbox) and integral(). Some other functions I found helpful to look at are dot() and cross().
Other thoughts:
It would appear that the inputParser class was the overall concensus on the most professional way to validate input arguments. It was noted on a related (but not duplicate) stackoverflow post that the newer MathWorks functions tend to make use of this class, suggesting that it may be the best and most up-to-date choice.
Since the MathWorks-provided MATLAB functions do not appear to enforce strict input argument data typing, this further suggests that even if it was possible to do so, it may not be a recommended approach.
MATLAB seems to regard "error handling" and "exception handling" as two different concepts. For example, here are two links to MATLAB's Documentation Center that show how MathWorks considers "error handling" and "exception handling" differently: MathWorks Documentation Center article on Error Handling, MathWorks Documentation Center article on Exception Handling. A relevant StackOverflow post has been made on this topic and can be found here (link). I contacted MathWorks and added some new information about this topic to that post, so if you are interested you may read more by following that link.
Matlab provides an 'inputParser' which allows to check inputs. Besides this you can use assertions:
assert(islogical(yes_or_no),'logical input expected')
To ensure the correct number of input arguments, use narginchk.
btw: Take a look in some Matlab functions like edit integral and check how tmw deals with this.
You may find writing this sort of code tedious or worry that it degrades performance:
if ~islogical(yes_or_no) && ~isscalar(yes_or_no)
error('test:NotLogicalType','Second argument must be logical (Boolean).');
end
if yes_or_no
y = 2 .* x;
else
y = -5 .* x;
end
Recall, however, that Matlab compiles the code before running so even if you need to test many conditions it will be quite fast. Run the profiler to see.
Another option in some cases (maybe not your example) is to use a lazier method. This option lets your code run with whatever inputs were provided, but uses a try/catch block to trap any error:
try
if yes_or_no
y = 2 .* x;
else
y = -5 .* x;
end
catch me
...
error('test:NotLogicalType','Second argument must be logical (Boolean).');
% rethrow(me);
end
The code above would produce an error if yes_or_no was a cell array for example (it will still allow non-Boolean, non-scalar, etc. values for yes_or_no though, but Matlab is often overly permissive). You can then either generate a custom error message, detect, what kind of error was thrown and try something else, etc. Many of the functions in the Statistics toolbox use this approach (e.g., type edit normpdf in your command window) for better or worse.
I have 2 sets of data of float numbers, set A and set B. Both of them are matrices of size 40*40. I would like to find out which set is closer to the normal distribution. I know how to use probplot() in matlab to plot the probability of one set. However, I do not know how to find out the level of the fitness of the distribution is.
In python, when people use problot, a parameter ,R^2, shows how good the distribution of the data is against to the normal distribution. The closer the R^2 value to value 1, the better the fitness is. Thus, I can simply use the function to compare two set of data by their R^2 value. However, because of some machine problem, I can not use the python in my current machine. Is there such parameter or function similar to the R^2 value in matlab ?
Thank you very much,
Fitting a curve or surface to data and obtaining the goodness of fit, i.e., sse, rsquare, dfe, adjrsquare, rmse, can be done using the function fit. More info here...
The approach of #nate (+1) is definitely one possible way of going about this problem. However, the statistician in me is compelled to suggest the following alternative (that does, alas, require the statistics toolbox - but you have this if you have the student version):
Given that your data is Normal (not Multivariate normal), consider using the Jarque-Bera test.
Jarque-Bera tests the null hypothesis that a given dataset is generated by a Normal distribution, versus the alternative that it is generated by some other distribution. If the Jarque-Bera test statistic is less than some critical value, then we fail to reject the null hypothesis.
So how does this help with the goodness-of-fit problem? Well, the larger the test statistic, the more "non-Normal" the data is. The smaller the test statistic, the more "Normal" the data is.
So, assuming you have converted your matrices into two vectors, A and B (each should be 1600 by 1 based on the dimensions you provide in the question), you could do the following:
%# Build sample data
A = randn(1600, 1);
B = rand(1600, 1);
%# Perform JB test
[ANormal, ~, AStat] = jbtest(A);
[BNormal, ~, BStat] = jbtest(B);
%# Display result
if AStat < BStat
disp('A is closer to normal');
else
disp('B is closer to normal');
end
As a little bonus of doing things this way, ANormal and BNormal tell you whether you can reject or fail to reject the null hypothesis that the sample in A or B comes from a normal distribution! Specifically, if ANormal is 1, then you fail to reject the null (ie the test statistic indicates that A is probably drawn from a Normal). If ANormal is 0, then the data in A is probably not generated from a Normal distribution.
CAUTION: The approach I've advocated here is only valid if A and B are the same size, but you've indicated in the question that they are :-)