SciPy provides two functions for nonlinear least squares problems:
optimize.leastsq() uses the Levenberg-Marquardt algorithm only.
optimize.least_squares() allows us to choose the Levenberg-Marquardt, Trust Region Reflective, or Trust Region Dogleg algorithm.
Should we always use least_squares() instead of leastsq()?
If so, what purpose does the latter serve?
Short answer
Should we always use least_squares() instead of leastsq()?
Yes.
If so, what purpose does the latter serve?
Backward compatibility.
Explanation
The least_squares function is new in 0.17.1. Its documentation refers to leastsq as
A legacy wrapper for the MINPACK implementation of the Levenberg-Marquadt algorithm.
The original commit introducing least_squares actually called leastsq when the method was chosen to be 'lm'. But the contributor (Nikolay Mayorov) then decided that
least_squares might feel more solid and homogeneous if I write a new wrapper to MINPACK functions, instead of calling leastsq.
and so he did. So, leastsq is no longer required by least_squares, but I'd expect it to be kept at least for a while, to avoid breaking old code.
Related
Blocks and functions in Modelica have some similarities and differences. In blocks, output variables are most likely expressed in terms of input variables using equations, whereas in functions output variables are expressed in terms of input variables using assignments. Given a relationship y = f(u) that can be expressed using both notions, I am interested in knowing which notion shall you favour in which situation?
Personally,
Blocks can be better integrated in block diagrams using input/output connectors
Equations in blocks can be most likely better treated by compilers for symbolic manipulation, optimization, and evaluating analytical derivatives required for Jacobian evaluation. So I guess blocks are likely less sensitive to numerical errors in some boundary cases. For functions, derivatives are likely to be evaluated using finite difference methods, if they are not explicitly provided.
on the other hand a set of assignments in a function will be most likely treated as a single equation. The same set of assignments if expressed in terms of a larger set of equations in a block will result in a model of larger size probably leading to a decrease in runtime performance
although a block with an algorithmic section is kind of equivalent to a function with the same assignments set, the syntax of a function call is favored in couple of situations
One can establish hierarchies of blocks types and do all of sort of things of object oriented modelings. Functions are kind of limited. It is not possible to extend from a non-abstract function that contains an algorithm section. But it is possible to have (an) abstract function(s) that act(s) as (an) interface(s) out of which implemented functions can be established etc.
Some of the above arguments are dependent on the way a specific simulation environment treats a block or a function. These might be low-level details not necessarily known.
The list in your "question" is already a pretty good summary. Still there are some additional things that should be considered:
Regarding the differentiation of functions, the developer at least needs to define how often the assignments can be differentiated (here is a nice read on this), as e.g. Dymola will not do it automatically. Alternatively the differentiated function can be specified manually (here). By the way, a partial derivative can be defined as well, see Language Specification, Sec. 12.7.2.
When it is necessary to invert a function, it can be necessary to define it manually. This is described in the Language Specification, Sec. 12.8.
Also it could be important that code from a function can be inlined, which should overcome some of the issues mentioned above, see Language Specification, Sec. 18.3.
Generally I would go for blocks whenever there is no very strong reason for a function. Some that come to my mind are the need for procedural execution, or for-loops.
This is just my two cents - more opinions welcome...
You might be interested in the opposite: calling a block as if it was a function:
https://github.com/modelica/ModelicaSpecification/issues/1512
The advantage of using function syntax is that you don't need to declare + connect components:
Block b;
equation
connect(x, b.in1);
connect(y, b.in2);
connect(z, b.out1);
vs
z = Block(x, y);
Of course right now, this syntax does not exist yet. And you really want to use blocks when you can. Algorithmic blocks might as well be functions as they are shorter and easier to write and will introduce fewer trajectories in your result-file (good unless you want to debug what happens inside the function call I guess).
The noEvent operator in Modelica doesn't use iteration to find the precise time instant in which the event was triggered.
It seems this would cause calculation error, here is an example I find on the following website
https://mbe.modelica.university/behavior/discrete/decay/
So Do I have to ensure the function is smooth when using noEvent operator?
What's the purpose of using noEvent operator if it can't ensure accuracy?
Although the question is already answered I would like to add some points, as I think it could be useful for many.
There are some common reasons to use the noEvent() statement:
Guarding expressions: This is used to prevent a function from being evaluated outside of their validity range. A typical example is der(x) = if x>=0 then sqrt(x) else 0; which would work perfectly in most common programming languages. This doesn't work always in Modelica for the following reason: When searching for the time when the condition x>=0 becomes false, it is possible that both branches are evaluated with values of x varying around 0. The same fact is mentioned in the screenshot posted by marvel This results in a crash if the square root of a negative x is evaluated. Therefore der(x) = if noEvent(x>=0) then -sqrt(x) else 0; Is used to suppress the iteration to search for the crossing time, leaving the handling of the discontinuity to the solver (often referred to as "expressions are taken literally instead of generating crossing functions"). In case of a variable step-size solver being used, this makes the solver reduce the step-size to meet it's relative error tolerance, which will likely result in degraded performance. Additionally this can be critical if the function described is not smooth enough resulting in non-precise or even instable simulations.
Continuous Expressions: When a function is continuous there is actually no event necessary. This comes down to the fact, that events are used to describe discontinuities. So if there is none, usually the event is simply superfluous and can therefore be suppressed. This is actually covered by the smooth() operator in Modelica, but the specification says, that a tool is free to still generate events. To my experience, tools generate events if the change to the function is relatively big. Therefore it can make sense to have a noEvent() within a smooth().
Avoid chattering: noEvent can help here but actually chattering is a more general problem. Therefore I'd recommend to solve issues related to chattering by re-building the model.
If none of the above is true the use of noEvent should be considered carefully.
I think the Modelica Language Specification Version 3.4 Section 3.7.3.2. and Section 8.5. will help you out here (in case you have not already checked this).
From what i know it should only be used for efficiency reasons and in most cases one should use smooth() instead or in conjunction.
Based on the two different ways of dealing with the event. If using noEvent operator, there is no halt of the integration, but the numerical solver assumes that the function should be smooth, with unsmooth functions, there would be numerical errors.
so I have the following Integral that i need to do numerically:
Int[Exp(0.5*(aCosx + bSinx + cCos2x + dSin2x))] x=0..2Pi
The problem is that the output at any given value of x can be extremely large, e^2000, so larger than I can deal with in double precision.
I havn't had much luck googling for the following, how do you deal with large numbers in fortran, not high precision, i dont care if i know it to beyond double precision, and at the end i'll just be taking the log, but i just need to be able to handle the large numbers untill i can take the log..
Are there integration packes that have the ability to handle arbitrarily large numbers? Mathematica clearly can.. so there must be something like this out there.
Cheers
This is probably an extended comment rather than an answer but here goes anyway ...
As you've already observed Fortran isn't equipped, out of the box, with the facility for handling such large numbers as e^2000. I think you have 3 options.
Use mathematics to reduce your problem to one which does (or a number of related ones which do) fall within the numerical range that your Fortran compiler can compute.
Use Mathematica or one of the other computer algebra systems (eg Maple, SAGE, Maxima). All (I think) of these can be integrated into a Fortran program (with varying degrees of difficulty and integration).
Use a library for high-precision (often called either arbitray-precision or multiple-precision too) arithmetic. Your favourite search engine will turn up a number of these for you, some written in Fortran (and therefore easy to integrate), some written in C/C++ or other languages (and therefore slightly harder to integrate). You might start your search at Lawrence Berkeley or the GNU bignum library.
(Yes I know that I wrote that you have 3 options, but your question suggests that you aren't ready to consider this yet) You could write your own high-/arbitrary-/multiple-precision functions. Fortran provides everything you need to construct such a library, there is a lot of work already done in the field to learn from, and it might be something of interest to you.
In practice it generally makes sense to apply as much mathematics as possible to a problem before resorting to a computer, that process can not only assist in solving the problem but guide your selection or construction of a program to solve what's left of the problem.
I agree with High Peformance Mark that the best option here numerically is to use analytics to scale or simplify the result first.
I will mention that if you do want to brute force it, gfortran (as of 4.6, with the libquadmath library) has support for quadruple precision reals, which you can use by selecting the appropriate kind. As long as your answers (and the intermediate results!) don't get too much bigger than what you're describing, that may work, but it will generally be much slower than double precision.
This requires looking deeper at the problem you are trying to solve and the behavior of the underlying mathematics. To add to the good advice already provided by Mark and Jonathan, consider expanding the exponential and trig functions into Taylor series and truncating to the desired level of precision.
Also, take a step back and ask why you are trying to accomplish by calculating this value. As an example, I recently had to debug why I was getting outlandish results from a property correlation which was calculating vapor pressure of a fluid to see if condensation was occurring. I spent a long time trying to understand what was wrong with the temperature being fed into the correlation until I realized the case causing the error was a simulation of vapor detonation. The problem was not in the numerics but in the logic of checking for condensation during a literal explosion; physically, a condensation check made no sense. The real problem was the code was asking an unnecessary question; it already had the answer.
I highly recommend Forman Acton's Numerical Methods That (Usually) Work and Real Computing Made Real. Both focus on problems like this and suggest techniques to tame ill-mannered computations.
Matlab Coder is a recently released MathWorks product. My understanding is that it is a Matlab-to-C compiler with the biggest advantage over previous solutions being that the resulting program does not need to be linked against a Matlab shared library.
Can someone with access to this product confirm the above? What are the dependencies of the translated programs and what kind of performance are we talking about? Also I would really like to see some example outputs, to know if the resulting C programs can be understood and improved without access to the Matlab source.
If done right this could be very powerful, allowing rapid prototyping in Matlab and instantaneous conversion to C when things are getting serious. I kind of whish it doesn't work well so that Python+Numpy+Scipy.weave is still superior ^^.
MATLAB Coder can allocate memory using malloc, so you can generate C code from MATLAB functions that operate on dynamically sized data. You can also choose the option of static allocation with a maximum size for variables.
RE: using BLAS for matrix multiplication – while the generated C code doesn’t automatically include any processor/platform specific optimizations, there is a feature called Target Function Library, which that allows users to write their own implementation of primitive operations (such as matrix multiplication), and include them in the generated code. You can hook up BLAS libraries to MATLAB Coder via that method. There’s also an ability to include optimized processor specific calls for larger functions through custom code integration and conditional compilation that lets you specify one set of code to use for code generation, and another set for simulation (for example, an optimized FIR function for an Texas Instruments DSP and functionally equivalent code for simulation that can execute on your PC written in C or in MATLAB).
Hope this is helpful --
Arvind Ananthan; Product Manager for MATLAB Coder; MathWorks
I am using Matlab Coder with Real Time Workshop (RTW) in order to generate self standind standard C code.
First of all you are asked to use a Matlab subset called "Embedded Matlab" you can find the doc about it on the web
You also have to avoid any dynamic exploitation of variables and you can't obviously generate c code for plots or figures.
The code it generates could be a mess to understand but it works.
you should actually not try to understand it. In a certain way it is as you would try to understand the assembler your compiler generates from a C code you wrote, quite pointless.
another thing you should take care of is to declare persistent big data types (vectors, big arryes, etc.) otherwise they will be allocated into your stack...
good luck!
Although many of you will have a decent idea of what I'm aiming at, just from reading the title -- allow me a simple introduction still.
I have a Fortran program - it consists of a program, some internal subroutines, 7 modules with its own procedures, and ... uhmm, that's it.
Without going into much detail, for I don't think it's necessary at this point, what would be the easiest way to use MATLAB's plotting features (mainly plot(x,y) with some customizations) as an interactive part of my program ? For now I'm using some of my own custom plotting routines (based on HPGL and Calcomp's routines), but just as part of an exercise on my part, I'd like to see where this could go and how would it work (is it even possible what I'm suggesting?). Also, how much effort would it take on my part ?
I know this subject has been rather extensively described in many "tutorials" on the net, but for some reason I have trouble finding the really simple yet illustrative introductory ones. So if anyone can post an example or two, simple ones, I'd be really grateful. Or just take me by the hand and guide me through one working example.
platform: IVF 11.something :) on Win XP SP2, Matlab 2008b
The easiest way would be to have your Fortran program write to file, and have your Matlab program read those files for the information you want to plot. I do most of my number-crunching on Linux, so I'm not entirely sure how Windows handles one process writing a file and another reading it at the same time.
That's a bit of a kludge though, so you might want to think about using Matlab to call the Fortran program (or parts of it) and get data directly for plotting. In this case you'll want to investigate Creating Fortran MEX Files in the Matlab documentation. This is relatively straightforward to do and would serve your needs if you were happy to use Matlab to drive the process and Fortran to act as a compute service. I'd look in the examples distributed with Matlab for simple Fortran MEX files.
Finally, you could call Matlab from your Fortran program, search the documentation for Calling the Matlab Engine. It's a little more difficult for me to see how this might fit your needs, and it's not something I'm terribly familiar with.
If you post again with more detail I may be able to provide more specific tips, but you should probably start rolling your sleeves up and diving in to MEX files.
Continuing the discussion of DISLIN as a solution, with an answer that won't fit into a comment...
#M. S. B. - hello. I apologize for writing in your answer, but these comments are much too short, and answering a question in the form of an answer with an answer is ... anyway ...
There is the Quick Plot feature of DISLIN -- routine QPLOT needs only three arguments to plot a curve: X array, Y array and number N. See Chapter 16 of the manual. Plus only several additional calls to select output device and label the axes. I haven't used this, so I don't know how good the auto-scaling is.
Yes, I know of Quickplot, and it's related routines, but it is too fixed for my needs (cannot change anything), and yes, it's autoscaling is somewhat quircky. Also, too big margins inside the graf.
Or if you want to use the power of GRAF to setup your graph box, there is subroutine GAXPAR to automatically generate recommended values. -2 as the first argument to LABDIG automatically determines the number of digits in tick-mark labels.
Have you tried the routines?
Sorry, I cannot find the GAXPAR routine you're reffering to in dislin's index. Are you sure it is called exactly like that ?
Reply by M.S.B.: Yes, I am sure about the spelling of GAXPAR. It is the last routine in Chapter 4 of the DISLIN 9.5 PDF manual. Perhaps it is a new routine? Also there is another path to automatic scaling: SETSCL -- see Chapter 6.
So far, what I've been doing (apart from some "duck tape" solutions) is
use dislin; implicit none
real, dimension(5) :: &
x = [.5, 2., 3., 4., 5.], &
y = [10., 22., 34., 43., 15.]
real :: xa, xe, xor, xstp, &
ya, ye, yor, ystp
call setpag('da4p'); call metafl('xwin');
call disini(); call winkey('return');
call setscl(x,size(x),'x');
call setscl(y,size(y),'y')
call axslen(1680,2376) !(8/10)*2100 and 2970, respectively
call setgrf('name','name','line','line')
call incmrk(1); call hsymbl(3);
call graf(xa, xe, xor, xstp, ya, ye, yor, ystp); call curve(x,y,size(x))
call disfin()
end
which will put the extreme values right on the axis. Do you know perhaps how could I go to have one "major tick margin" on the outside, as to put some area between the curve and the axis (while still keeping setscl's effects) ?
Even if you don't like the built-in auto-scaling, if you are already using DISLIN, rolling your own auto-scaling will be easier than calling Fortran from MATLAB. You can use the Fortran intrinsic functions minval and maxval to find the smallest and largest values in the data, than write a subroutine to round outwards to "nice" round values. Similarly, a subroutine to decide on the tick-mark spacing.
This is actually not so easy to accomplish (and ideas to prove me wrong will be gladly appreciated). Or should I say, it is easy if you know the rough range in which your values will lie. But if you don't, and you don't know
whether your values will lie in the range of 13-34 or in the 1330-3440, then ...
... if I'm on the wrong track completely here, please, explain if you ment something different. My english is somewhat lacking, so I can only hope the above is understandable.
Inside a subroutine to determine round graph start/end values, you could scale the actual min/max values to always be between 1 and 10, then have a table to pick nice round values, then unscale back to the correct range.
--
Dump Matlab because its proprietary, expensive, bloated/slow and codes are not easy to parallelize.
What you should do is use something on the lines of DISLIN, PLplot, GINO, gnuplotfortran etc.