Dears,
I have a model with two boolean decision variable, avere only one can be equal ti 1.
Is it the solver faster if I use AddAllDifferent or It i use a simple constraint (ADD) x+y=1?
In this case, i would drop both ideas and stick to boolean-clauses:
x.Not() OR y.Not()
AND
x OR y
or if ortools supports it:
x XOR y
(= pseudocode -> not necessarily any sane syntax)
This is as simple as it gets and can be very efficiently reasoned about by unit-propagation. Furthermore, 2-SAT things might be at work internally (implication-graph and so on).
No need to reason about integer-arithmetic (including potential channeling of bool/int) or global-constraints.
The above is something which is sometimes even used in SAT when decomposing all-different by prohibit all possible pairs.
(ortools might be clever to analyze your approaches to get to the same boiled-down formulation; but maybe not -> why not help if it's so simple)
I have two matrices S and T which have n columns and a row vector v of length n. By my construction, I know that S does not have any duplicates. What I'm looking for is a fast way to find out whether or not the row vector v appears as one of the rows of S. Currently I'm using the test
if min([sum(abs(S - repmat(f,size(S,1),1)),2);sum(abs(T - repmat(v,size(dS_new,1),1)),2)]) ~= 0 ....
When I first wrote it, I had a for loop testing each (I knew this would be slow, I was just making sure the whole thing worked first). I then changed this to defining a matrix diff by the two components above and then summing, but this was slightly slower than the above.
All the stuff I've found online says to use the function unique. However, this is very slow as it orders my matrix after. I don't need this, and it's a massively waste of time (it makes the process really slow). This is a bottleneck in my code -- taking nearly 90% of the run time. If anyone has any advice as how to speed this up, I'd be most appreciative!
I imagine there's a fairly straightforward way, but I'm not that experienced with Matlab (fairly, just not lots). I know how to use basic stuff, but not some of the more specialist functions.
Thanks!
To clarify following Sardar_Usama's comment, I want this to work for a matrix with any number of rows and a single vector. I'd forgotten to mention that the elements are all in the set {0,1,...,q-1}. I don't know whether that helps or not to make it faster!
You may want this:
ismember(v,S,'rows')
and replace arguments S and v to get indices of duplicates
ismember(S,v,'rows')
Or
for test if v is member of S:
any(all(bsxfun(#eq,S,v,2))
this returns logical indices of all duplicates
all(bsxfun(#eq,S,v),2)
I have a linear system to solve, written as Ax=b
A is a 175 by 175 symmetric square, with ones at it's diagonal (i.e., aii=1), and other entries ranges from 0 to 1(i.e., 0
A is very ill conditioned, and not positive definite, its rank is 162 and its condition number is 3.5869e+16
I spent several days to solve this in MATLAB, I've tried almost every method I can find, including \, pcg, bicg, bicgstab, bicgstabl, cgs, gmres, lsqr, minres, qmr, symmlq, tfqmr
These methods gave me some solutions. But I don't know how to trust them, or which solution to trust. Is there a criteria to determine?
I'll appreciate someone who can give me a solution that I can trust.
Thanks!
A and b are stored in .mat files and can be download from dropbox links:
https://www.dropbox.com/s/s6xlbq68juqs6xi/A.mat?dl=0
https://www.dropbox.com/s/pxl0hdup20hf2lr/b.mat?dl=0
use like this:
load('A.mat');
load('b.mat');
x = A\b;
Not sure if this will help, but give it a go:
Tikhonov regularization
Basically, when the following is hard to compute due to ill-conditiones:
You minimize the following instead
Being \Gamma generally the identity matrix.
In the end, you get the following equation for x:
To add to that, you will generally want to add an "hyperparameter", to control how much you regularize the problem. So \Gamma instead of being just the identity matrix, it will be a number (i.e. 0.001) multiplied by the identity matrix of size(A).
The last equation should be straightforward to try in Matlab. Give it a go.
NOTE: this is not the answer. Actually probably there is no unique answer to solving an ill posed problem. This is just a way to go.
Just starting with Modelica and having trouble understanding how it works.
In the below 'method' of the model, qInflow and qOutflow are used in the second line to evaluate der(h), but they have not received a value yet! (they were not defined in the 'data' of the method)? In what order is the code executed.
equation
assert(minV >= 0, "minV must be greater or equal to zero");
der(h)=(qInflow - qOutflow)/area;
qInflow=if time > 150 then 3*flowLevel else flowLevel;
qOutflow=Functions.LimitValue(minV, maxV, -flowGain*outCtr);
error=ref - h;
der(x)=error/T;
outCtr=K*(error + x);
end FlatTank;
From http://www.mathcore.com/resources/documents/ie_tank_system.pdf
This is an understandable point of confusion when coming from languages and systems that utilize imperative semantics. But Modelica doesn't work like that.
When working with Modelica it is important to understand that an equation section contains equations, not assignments. Consider this, if I gave you the following equations:
x + y = 3;
x + 2*y = 5;
If you understand that this is a mathematical context, you can then determine that x must have a value of 1 and y must have a value of 2. In other words, you have to solve a system of simultaneous equations. You'll note that the left hand side of these equations are not variables (in general), they are expressions. An equation is simply a relationship that equates one expression, on the left hand side, with another expression, on the right hand side. Furthermore, this relationship is always true and so order is irrelevant.
This is quite different from imperative programming languages with imperative semantics. But it is also very powerful because you can state these relationships (linear systems of equations, non-linear systems of equations, implicit equations, etc) and the compiler will work out the most efficient way to solve them.
Getting back to your example, when you look at the code in your question you are interpreting those equations as assignment statements. This notion is reinforced because they just happen to have variables on the left hand sides. But they are really equations. In an equation based system, you do not worry about whether a given variable has been assigned to previously. Instead, the requirement is simply that for every variable there exists (somewhere) an equation and that there are no extra equations. In other words, you should have the same number of variables as unknowns and that the system of equations has a unique solution. That is all that Modelica requires.
Now, Modelica supports the kind of imperative semantics you are used to. But they are only to be used in special cases because they constrain the interpretation of the mathematical behavior in such a way that it interferes with the symbolic manipulation that allows Modelica compilers to generate really fast code. So it is more than a question of style. You should use equations if at all possible and algorithms in Modelica should only be used as a last resort.
One last note. Some people may be wondering "Are you telling me that these equations will be put into some giant system of equations and solved by matrix inversion or Newton-Raphson or something? Why make it so complicated when it could obviously be solved in a much easier way!" But it will not be solved as a giant system of equations. If it can be solved as a simple set of assignments it will. That is one (among many) of the different symbolic manipulation techniques that will be applied. In fact, this is a key point about Modelica...you don't need to worry about optimizing the solution method, the tool will take care of that. And more importantly, if you connect components in such a way that a simultaneous system does arise, you don't need to worry about that either. Modelica tools can handle such "algebraic loops" for you, they will optimize it to find the most computationally efficient formulation and won't depend on you reformulating your model for those cases.
Does that help?
You cannot know the execution order of the equations in a Modelica model until you run a Modelica tool on it (you can re-order any equation in the source model and get the same result). And then the order is only true for this tool with the settings you used.
This was the order chosen by the OpenModelica compiler (omc +s +simCodeTarget=Dump model.mo):
error = ref - h;
outCtr = K * (error + x);
der(x) = DIVISION(error, T, #SHARED_LITERAL_2(String#);
qOutflow = LimitValue(minV, maxV, (-flowGain) * outCtr);
qInflow = if time > 150.0 then 3.0 * flowLevel else flowLevel;
der(h) = DIVISION(qInflow - qOutflow, area, #SHARED_LITERAL_3(String#);
This example was a little boring because the left and right sides of no equation changed place (h = error - ref would be viable if h was not chosen as a state variable, etc).
I programmed in MATLAB for many years, but switched to using R exclusively in the past few years so I'm a little out of practice. I'm interviewing a candidate today who describes himself as a MATLAB expert.
What MATLAB interview questions should I ask?
Some other sites with resources for this:
"Matlab interview questions" on Wilmott
"MATLAB Questions and Answers" on GlobaleGuildLine
"Matlab Interview Questions" on CoolInterview
This is a bit subjective, but I'll bite... ;)
For someone who is a self-professed MATLAB expert, here are some of the things that I would personally expect them to be able to illustrate in an interview:
How to use the arithmetic operators for matrix or element-wise operations.
A familiarity with all the basic data types and how to convert effortlessly between them.
A complete understanding of matrix indexing and assignment, be it logical, linear, or subscripted indexing (basically, everything on this page of the documentation).
An ability to manipulate multi-dimensional arrays.
The understanding and regular usage of optimizations like preallocation and vectorization.
An understanding of how to handle file I/O for a number of different situations.
A familiarity with handle graphics and all of the basic plotting capabilities.
An intimate knowledge of the types of functions in MATLAB, in particular nested functions. Specifically, given the following function:
function fcnHandle = counter
value = 0;
function currentValue = increment
value = value+1;
currentValue = value;
end
fcnHandle = #increment;
end
They should be able to tell you what the contents of the variable output will be in the following code, without running it in MATLAB:
>> f1 = counter();
>> f2 = counter();
>> output = [f1() f1() f2() f1() f2()]; %# WHAT IS IT?!
We get several new people in the technical support department here at MathWorks. This is all post-hiring (I am not involved in the hiring), but I like to get to know people, so I give them the "Impossible and adaptive MATLAB programming challenge"
I start out with them at MATLAB and give them some .MAT file with data in it. I ask them to analyze it, without further instruction. I can very quickly get a feel for their actual experience.
http://blogs.mathworks.com/videos/2008/07/02/puzzler-data-exploration/
The actual challenge does not mean much of anything, I learn more from watching them attempt it.
Are they making scripts, functions, command line or GUI based? Do they seem to have a clear idea where they are going with it? What level of confidence do they have with what they are doing?
Are they computer scientists or an engineer that learned to program. CS majors tend to do things like close their parenthesis immediately, and other small optimizations like that. People that have been using MATLAB a while tend to capture the handles from plotting commands for later use.
How quickly do they navigate the documentation? Once I see they are going down the 'right' path then I will just change the challenge to see how quickly they can do plots, pull out submatrices etc...
I will throw out some old stuff from Project Euler. Mostly just ramp up the questions until one of us is stumped.
Floating Point Questions
Given that Matlab's main (only?) data type is the double precision floating point matrix, and that most people use floating point arithmetic -- whether they know it or not -- I'm astonished that nobody has suggested asking basic floating point questions. Here are some floating point questions of variable difficulty:
What is the range of |x|, an IEEE dp fpn?
Approximately how many IEEE dp fpns are there?
What is machine epsilon?
x = 10^22 is exactly representable as a dp fpn. What are the fpns xp
and xs just below and just above x ?
How many dp fpns are in [1,2)? How many atoms are on an edge of a
1-inch sugar cube?
Explain why sin(pi) ~= 0, but cos(pi) = -1.
Why is if abs(x1-x2) < 1e-10 then a bad convergence test?
Why is if f(a)*f(b) < 0 then a bad sign check test?
The midpoint c of the interval [a,b] may be calculated as:
c1 = (a+b)/2, or
c2 = a + (b-a)/2, or
c3 = a/2 + b/2.
Which do you prefer? Explain.
Calculate in Matlab: a = 4/3; b = a-1; c = b+b+b; e = 1-c;
Mathematically, e should be zero but Matlab gives e = 2.220446049250313e-016 = 2^(-52), machine epsilon (eps). Explain.
Given that realmin = 2.225073858507201e-308, and Matlab's u = rand gives a dp fpn uniformly distributed over the open interval (0,1):
Are the floating point numbers [2^(-400), 2^(-100), 2^(-1)]
= 3.872591914849318e-121, 7.888609052210118e-031, 5.000000000000000e-001
equally likely to be output by rand ?
Matlab's rand uses the Mersenne Twister rng which has a period of
(2^19937-1)/2, yet there are only about 2^64 dp fpns. Explain.
Find the smallest IEEE double precision fpn x, 1 < x < 2, such that x*(1/x) ~= 1.
Write a short Matlab function to search for such a number.
Answer: Alan Edelman, MIT
Would you fly in a plane whose software was written by you?
Colin K would not hire me (and probably fire me) for saying "that
Matlab's main (only?) data type is the double precision floating
point matrix".
When Matlab started that was all the user saw, but over the years
they have added what they coyly call 'storage classes': single,
(u)int8,16,32,64, and others. But these are not really types
because you cannot do USEFUL arithmetic on them. Arithmetic on
these storage classes is so slow that they are useless as types.
Yes, they do save storage but what is the point if you can't do
anything worthwhile with them?
See my post (No. 13) here, where I show that arithmetic on int32s is 12 times slower than
double arithmetic and where MathWorkser Loren Shure says "By
default, MATLAB variables are double precision arrays. In the olden
days, these were the ONLY kind of arrays in MATLAB. Back then even
character arrays were stored as double values."
For me the biggest flaw in Matlab is its lack of proper types,
such as those available in C and Fortran.
By the way Colin, what was your answer to Question 14?
Ask questions about his expertise and experience in applying MATLAB in your domain.
Ask questions about how he would approach designing an application for implementation in MATLAB. If he refers to recent features of MATLAB, ask him to explain them, and how they are different from the older features they replace or supplement, and why they are preferable (or not).
Ask questions about his expertise with MATLAB data structures. Many of the MATLAB 'experts' I've come across are very good at writing code, but very poor at determining what are the best data structures for the job in hand. This is often a direct consequence of their being domain experts who've picked up MATLAB rather than having been trained in computerism. The result is often good code which has to compensate for the wrong data structures.
Ask questions about his experience, if any, with other languages/systems and invite him to expand upon his observations about the relative strengths and weaknesses of MATLAB.
Ask for top tips on optimising MATLAB programs. Expect the answers: vectorisation, pre-allocation, clearing unused variables, etc.
Ask about his familiarity with the MATLAB profiler, debugger and lint tools. I've recently discovered that the MATLAB 'expert' over in the corner here had never, in 10 years using the tool, found the profiler.
That should get you started.
I. I think this recent SO question
on indexing is a very good question
for an "expert".
I have a 2D array, call it 'A'. I have
two other 2D arrays, call them 'ix'
and 'iy'. I would like to create an
output array whose elements are the
elements of A at the index pairs
provided by x_idx and y_idx. I can do
this with a loop as follows:
for i=1:nx
for j=1:ny
output(i,j) = A(ix(i,j),iy(i,j));
end
end
How can I do this without the loop? If
I do output = A(ix,iy), I get the
value of A over the whole range of
(ix)X(iy).
II. Basic knowledge of operators like element-wise multiplication between two matrices (.*).
III. Logical indexing - generate a random symmetric matrix with values from 0-1 and set all values above T to 0.
IV. Read a file with some properly formatted data into a matrix (importdata)
V. Here's another sweet SO question
I have three 1-d arrays where elements
are some values and I want to compare
every element in one array to all
elements in other two.
For example:
a=[2,4,6,8,12]
b=[1,3,5,9,10]
c=[3,5,8,11,15]
I want to know if there are same
values in different arrays (in this
case there are 3,5,8)
Btw, there's an excellent chance your interviewee will Google "MATLAB interview questions" and see this post :)
Possible question:
I have an array A of n R,G,B triplets. It is a 3xn matrix. I have another array B in the form 1xn which stores an index value (association to a cluster) for each triplet.
How do I plot the triplets of A in 3D space (using plot3 function), coloring each triplet according to its index in B? (The goal is to qualitatively evaluate my clustering)
Really, really good programmers who are MATLAB novices won't be able to give you an efficient (== MATLAB style) solution. However, it is a very simple problem if you do know your MATLAB.
Depends a bit what you want to test.
To test MATLAB fluency, there are several nice Stack Overflow questions that you could use to test e.g. array manipulations (example 1, example 2), or you could use fix-this problems like this question (I admit, I'm rather fond of that one), or look into this list for some highly MATLAB-specific stuff. If you want to be a bit mean, throw in a question like this one, where the best solution is a loop, and the typical MATLAB-way-of-thinking solution would just fill up the memory.
However, it may be more useful to ask more general programming questions that are related to your area of work and see whether they get the problem solved with MATLAB.
For example, since I do image analysis, I may ask them to design a class for loading images of different formats (a MATLAB expert should know how to do OOP, after all, it has been out for two years now), and then ask follow-ups as to how to deal with large images (I want to see a check on how much memory would be used - or maybe they know memory.m - and to hear about how MATLAB usually works with doubles), etc.