Performance of Octave compared to Matlab and Scilab - matlab

Question

I test the following command in Octave, Scilab and Matlab prompt.
>> A = rand(10000,10000);
>> B = rand(10000,1);
>> tic,A\B;, toc
The timings were around, respectively, 40, 15.8 and 15.7 sec. For comparison Mathematica's performance was
In[7]:= A = RandomReal[{0, 1}, {10000, 10000}];
In[9]:= B = RandomReal[{0, 1}, 10000];
In[10]:= Timing[LinearSolve[A, B];]
Out[10]= {14.125, Null}
Does this indicate that Octave is not so capable as the rest of the softwares in the field of linear equations?

Answer 1

I think that your tests are flawed. The algorithms behind A\B make use of the special patterns and structures in the systems of equations, so execution time depends very much on what random(10000,10000) has generated. On three different runs with Octave 4.0.0 on my machine, your code returns 7.1s, 95.1s and 16.4s. That indicates that the first matrix generated by random was probably sparse, and that could have been the case when you tested your code with Scilab and Matlab. So unless you make sure that the algorithms are evaluating the same thing, or unless you average the execution time in a sound manner (that is not very trivial to find for me), then it doesn't make sense to compare them as you did.

Answer 2

#dimitris thanks for the question, I found your method quite helpful for a quick comparison, and the somewhat different answers I get now are interesting. I didn't have the problems alluded to by the other respondents, the answers (timings) were consistent and helpful.
Here are the times on my Windows machine (Asus with Ryzen 7 4700U) where I've printed 5 runs for each of Scilab 6.1.0, Octave 6.2.0, Python 3.8.7 and Julia 1.5.3) - I've been exploring the possible use of them for a job I currently have. I don't have (can't afford..) either Matlab or Mathematica, so no results from these..
Octave
>> for i=1:5;A=rand(10000,10000);B=rand(10000,1);tic;A\B;toc,end;
Elapsed time is 9.79964 seconds.
Elapsed time is 9.78249 seconds.
Elapsed time is 9.70953 seconds.
Elapsed time is 9.73357 seconds.
Elapsed time is 9.69932 seconds.
Scilab
--> for i=1:5;A=rand(10000,10000);B=rand(10000,1);tic;A\B;toc,end;
ans = 10.407628
ans = 10.706747
ans = 10.490241
ans = 10.773073
ans = 10.517951
Julia
m=10000;
for i=1:5;
A=rand(m,m);B=rand(m,1);
t=time();A\B;
println(time()-t)
end;
7.833999872207642
7.7170000076293945
7.675999879837036
7.76200008392334
7.740999937057495
Python
from pylab import *
import numpy as np
import datetime as dt
N = 10000
for i in range(5):
A = np.random.random((N,N))
B = np.random.random((N,1))
tic=dt.datetime.now()
np.linalg.solve(A,B)
print((dt.datetime.now()-tic))
0:00:05.567395
0:00:05.703859
0:00:05.050467
0:00:04.995202
0:00:05.294050

Answer 3

You should have run the tests in each one probably around a thousand times or more. Also note they should use the same algorithms but somewhat less fine-tuned. A more sensible approach is to test across many cases over many different dimensions and average the results.
Most matrix math comes from LAPACK. the difference is that Matlab has dlls with fortran and C++ that may be slightly better. I believe that they make a little bit better use of your math coprocessor. It is called the Intel MKL kernel.
The actual algorithm changes dependent on the structure of the matrix and size.