I am trying to solve a complex optimization problem using Matlab's Particle Swarm Optimization. The objective function called in PSO solves an interior optimization problem using matlab's fsolve() function, where the solution from fsolve is used to compute the function evaluation that PSO will try to minimize.
Typically, without PSO wrapped around the problem, we run fsolve() in parallel to compute the central differences to perform gradient calculations; however, I am aware that I cannot do this if I want to run PSO in parallel since there cannot be nested parallel functions.
The next thought was to run fsolve in serial, when computing the gradients, and free up the parallel pool to allow PSO to evaluate the objective function (which involves solving the interior optimization problem) on multiple cores at the same time.
i.e.: if there are 5 particles in the swarm, I want to run each interior optimization problem on a different core simulataneously for each iteration. My question is then, is it possible (even in serial) to run fsolve on different cores simultaneously?
Unforunately, due to the nature of the work, I am unable to share the code for this work, but moreso need to figure out if this is possible before moving forward with the current direction.
Any and all thoughts on this would be greatly appreciated. Thanks!
I have tried to implement the described thought above, but am getting errors that I am unable to diagnose at the moment. I expected each particle iteration to run slower since fsolve cannot use multiple cores to compute the gradients, but overall, I thought that allowing PSO to compute each particles solution individually on different cores would still be more efficient than running all of the iterations in serial with fsolve running in parallel.
Edit:
Despite what I say below, I wasn't able to get fsolve to work inside a parfor loop or using parfeval. I get an Undefined function 'fsolve' ... error. It appears that parallel execution of fsolve is unsupported. Even if you only call it once per worker, it will still cause that worker to fail. Any iteration of fsolve inside each worker (and hence my answer below) is irrelevant.
It shouldn’t matter how many times a function is called inside each parallel worker. Each iteration is separated from the others in time. As long as the overall code block is compatible with parallel execution, the instruction stream within is executed as normal MATLAB code on each worker.
Related
I have a laptop with a multi-core processor and I would like to run a lengthy loop in which Simulink simulations are performed. Is it beneficial to split the loop into two parts (it is possible in my case), open the Matlab application twice, and run a Matlab script in each of them?
Someone told me that Matlab/Simulink always uses one core per opened Matlab application. Is that correct?
MATLAB splits some builtin functions across multiple cores, but standard MATLAB code uses just one core. Generally, if you are running several independent iterations, then the computation time can benefit from parallelization. You can do this easily using either parfor (if the have the Parallel Computing Toolbox), or batch_job.
I have implemented a Naive Bayes classifier. On Matlab, my classify function takes 2 minutes to run while octave takes 25 minutes to run the same code. Does anyone know what causes ocatve to run slower so that I can tweak my code accordingly?
PS: I have to submit to a server which runs octave and not Matlab.
Matlab does a lot of "hidden" optimization when running your code (Octave probably, too, but different ones). Many of these optimizations e.g. concern that parameters to functions are not copied if you do not modify these parameters in the function, but instead passed by reference. This can significantly speed up calculations when you e.g. pass around large matrices, since otherwise most of your computational time is spend on copying. There are many, many similar optimizations, and not all of them are documented at all.
Without specific knowledge of what you are computing, it's hard to guess where the difference comes from. I am not aware if octave has an equivalence to the matlab profiler, but if, I would use this to find out where octave spends all the time. For debugging, I would also recommend to download Octave to your PC and debug there.
I'm having a trouble doing MCMC(Monte Carlo Markov Chain). So for MCMC, say I will run 10000 iterations, then within each iteration, I will draw some parameters. But in each iteration, I have some individual data which are independently, so I can do parfor. However, the problem is, it seems the time to finish one iteration just grows quickly as MCMC goes on. Soon, it's extremely time consuming.
My question is: is there any efficient way to combine parfor and while loop?
I have the following pseudo-code:
r=1;
while r<10000
parfor i=1:I
make draws from proposal distribution
calculate acceptance rate
accept or reject current draw
end
r=r+1;
end
Launching lots of separate parfor loops can be inefficient if each loop duration is small. Unfortunately, as you are probably aware, you cannot break out of a parfor loop. One alternative might be to use parfeval. The idea would be to make many parfeval calls (but not too many), and then you can terminate when you have sufficient results.
This (fairly long) blog article shows an example of using parfeval in a situation where you might wish to terminate the computations early.
I have a kinetic monte carlo code. Now its kinetic and hence each loop updates the current state to a future state, making it to be a dependent for loop.
I want to use parallel computing feature of matlab, but it seems the famous 'parfor' command only works for independent loops.
So my question, is it possible to use parallel computing in matlab to parallelize code where loops are not independent?
Usually these kinds of calculations are done on a grid, and the grid is distributed across the workers, each worker having its own part of the grid to calculate. This can't be done independently in general because the value at one point on the grid will depend on neighbouring points. These boundary values are communicated between the workers using some mechanism such as message passing or shared memory.
In MATLAB you can either use spmd or communicating jobs with the labSend and labReceive functions or you can use distributed arrays.
I wrote a code to solve a system using ode45 and ode15s in matlab. I am wondering if I can improve the speed of the code using multiple core (or parallel code) in my script.
Anyone have tried this ??
Thanks
No, you can't.
All numerical integrators, ode45 and friends included, use some form of iterative scheme to solve the user-implemented (coupled) non-linear (partial) differential equations.
Each new step in the iterative schemes of ode45/15s/.. (to compute the new state of the system) depends on the previous step (the old state of the system), therefore, these numerical integrators cannot be parallelized effectively.
The only speedup you can do that's likely to have a big impact is to optimize your implementation of the differential equation.
From my experience, the only way to use multiple cores for ODE suite solvers in MATLAB is to use "parfor loop" to start multiple computations together at the same time, your single computation not be any faster, but you can start many with different parameters and have multiple solutions after that long wait. So if you need to start ODE many times that might speed up your work.
To speed up one ODE function it also a good idea to play with RelTol and AbsTol settings (changes time form seconds to hours), using Jpattern option can also be very helpful (my almost tridiagonal pattern made it run twice as fast). If your differential equation is simple maybe try to compile it first, or at least vectorize (I used to write some part of code in Java and then point MATLAB to use compiled .class file). Obviously the length of your solution vector plays important role, so don't make it more than a few hounded.