I am currently learning the usage of the MIT Drake library for optimization and would like to formulate a nonlinear model predictive control problem. I notice that the pydrake examples show the methods to add nonlinear costs and constraints in a nonlinear program problem. However, I am not sure how to integrate nonlinear (and maybe even switching) system dynamics into the optimization problem. Are there any related examples or useful hints? Any help would be much appreciated.
Thanks,
Keran
We have a set of "trajectory optimization" classes that make it easy to set up a MathematicalProgram for this case. I'd recommend looking at http://underactuated.csail.mit.edu/trajopt.html.
The switching case needs more care, depending on exactly how you formulate it. I would need more details to advise. But planning through contact is similar: http://underactuated.csail.mit.edu/contact.html .
Related
In Dymola, I often meet a nonlinear system initialization failure or maybe a stiff system that is hard to solve in the large thermo-fluid system, but for a simple system, there wouldn't be this kind of problem. My questions are:
So I am wondering how much is the largest capability of solving a nonlinear system model? For example, how many nonlinear equations I could include in my model at most?
Is there any setting in Dymola which allows increasing the capability of solving nonlinear system?
How could I decrease the number of nonlinear equations in the model without damage to the accuracy of the model?
These are pretty difficult questions to be answered in a generally valid fashion. Still I'll try to share some of my experience with Dymola and non-linear systems.
There is no hard number which will limit the size. It depends more on how strongly non-linear the equations are than on their number. I have simulated models with non-linear systems of size 150, which are pretty stable while others of size 10 can brake...
There are multiple perspectives to this
I have worked on some models that made the C-Compiler run out of memory during compilation. If you have this sort of problem, forcing 64Bit compilation by setting Advanced.CompileWith64=2 can help. Then you shouldn't run out of memory any more. This only refers to the size only.
Performance for non-linear systems can be improved by activating DAE-mode by setting Advanced.Define.DAEsolver=true. This does not work with all solvers though.
Additionally to the above it can help to set Advanced.MoveEquationsToDynamics=true, for which the manual states: "It forces the integrator to solve the nonlinear
equations each integrator step and thereby it also updates the initial guesses more often."
As mentioned by Erik, the homotopy()-operator can be very important as it helps the solver converging in case of difficult initialization.
This is very specific to the model. Decoupling can help, e.g. by splitting the system in smaller systems by adding energy storing elements/states. This can be done based on physics of the system and is the preferable solution if possible. As an (more artificial) alternative filter/delays can be added. Usually this has a negative effect on accuracy.
I very much agree with Markus's advice but would also like to remind you about Modelica's homotopy operator. A well-chosen simplified model can greatly help Dymola to initialize a model with a large and difficult non-linear system.
In general good initial guesses are very important when solving non-linear systems. Using homotopy is simply an implicit way to provide these good guesses.
I am currently running a multiple linear regression using MATLAB's LinearModel.fit function, and I am bit confused in regards to how to properly add interaction terms to the model by hand. As I am aware, LinearModel.fit does not standardize variables on its own, so I have been doing so manually.
So far, the way I have done it has been to
Standardize the observations for each variables
Multiply corresponding standardized values from specific variables to create the interaction terms and then add these new variables to the set of regression data
Run the regression
Is this the correct way to go about doing this? Should I standardize the interaction term variables also after calculating the 'raw' terms? Any help would be greatly appreciated!
Whether or not to standardize interaction terms probably depends on what you intend to do with the model. Standardization typically does not affect model performance as much as it allows for more straightforward model interpretation as your learned coefficients will be on similar scales. I suspect whether to do this or not is largely a matter of opinion. Here is a relevant stats.stackexchange post that may help.
My intuition would be the same as how you have described your process so far.
Could anyone please help in providing an example showing how ANCOVA (analysis of covariance) can be done in scipy/statsmodel, with python?
I am not sure if I am asking too much, but a quick search showed me this which is not informative enough for me.
Thanks!
Statsmodels uses the linear model, OLS, to estimate ANOVA. So, having additional continuous regressors as in ANCOVA does not change the analysis.
Here are a few links to the relevant documentation
Anova helper functions and examples for ANCOVA interactions
http://statsmodels.sourceforge.net/devel/examples/generated/example_interactions.html
using formulas to create the design matrix
http://statsmodels.sourceforge.net/devel/example_formulas.html
the core OLS model
http://statsmodels.sourceforge.net/devel/generated/statsmodels.regression.linear_model.OLS.html
I have to solve a multiobjective problem but I don't know if I should use CPLEX or Matlab. Can you explain the advantage and disadvantage of both tools.
Thank you very much!
This is really a question about choosing the most suitable modeling approach in the presence of multiple objectives, rather than deciding between CPLEX or MATLAB.
Multi-criteria Decision making is a whole sub-field in itself. Take a look at: http://en.wikipedia.org/wiki/Multi-objective_optimization.
Once you have decided on the approach and formulated your problem (either by collapsing your multiple objectives into a weighted one, or as series of linear programs) either tool will do the job for you.
Since you are familiar with MATLAB, you can start by using it to solve a series of linear programs (a goal programming approach). This page by Mathworks has a few examples with step-by-step details: http://www.mathworks.com/discovery/multiobjective-optimization.html to get you started.
Probably this question is not a matter of your current concern. However my answer is rather universal, so let me post it here.
If solving a multiobjective problem means deriving a specific Pareto optimal solution, then you need to solve a single-objective problem obtained by scalarizing (aggregating) the objectives. The type of scalarization and values of its parameters (if any) depend on decision maker's preferences, e.g. how he/she/you want(s) to prioritize different objectives when they conflict with each other. Weighted sum, achievement scalarization (a.k.a. weighted Chebyshev), and lexicographic optimization are the most widespread types. They have different advantages and disadvantages, so there is no universal recommendation here.
CPLEX is preferred in the case, where (A) your scalarized problem belongs to the class solved by CPLEX (obviously), e.g. it is a [mixed integer] linear/quadratic problem, and (B) the problem is complex enough for computational time to be essential. CPLEX is specialized in the narrow class of problems, and should be much faster than Matlab in complex cases.
You do not have to limit the choice of multiobjective methods to the ones offered by Matlab/CPLEX or other solvers (which are usually narrow). It is easy to formulate a scalarized problem by yourself, and then run appropriate single-objective optimization (source: it is one of my main research fields, see e.g. implementation for the class of knapsack problems). The issue boils down to finding a suitable single-objective solver.
If you want to obtain some general information about the whole Pareto optimal set, I recommend to start with deriving the nadir and the ideal objective vectors.
If you want to derive a representation of the Pareto optimal set, besides the mentioned population based-heuristics such as GAs, there are exact methods developed for specific classes of problems. Examples: a library implemented in Julia, a recently published method.
All concepts mentioned here are described in the comprehensive book by Miettinen (1999).
Can cplex solve a pareto type multiobjective one? All i know is that it can solve a simple goal programming by defining the lexicographical objs, or it uses the weighted sum to change weights gradually with sensitivity information and "enumerate" the pareto front, which highly depends on the weights and looks very subjective.
You can refer here as how cplex solves the bi-objetive one, which seems not good.
For a true pareto way which includes the ranking, i only know some GA variants can do like NSGA-II.
A different approach would be to use a domain-specific modeling language for mathematical optimization like YALMIP (or JUMP.jl if you like to give Julia a try). There you can write your optimization problem with Matlab with some extra YALMIP functionalities and use CPLEX (or any other supported solver as a backend) without restricting to one solver.
I'm quite new with this topic so any help would be great. What I need is to optimize a neural network in MATLAB by using GA. My network has [2x98] input and [1x98] target, I've tried consulting MATLAB help but I'm still kind of clueless about what to do :( so, any help would be appreciated. Thanks in advance.
Edit: I guess I didn't say what is there to be optimized as Dan said in the 1st answer. I guess most important thing is number of hidden neurons. And maybe number of hidden layers and training parameters like number of epochs or so. Sorry for not providing enough info, I'm still learning about this.
If this is a homework assignment, do whatever you were taught in class.
Otherwise, ditch the MLP entirely. Support vector regression ( http://www.csie.ntu.edu.tw/~cjlin/libsvm/ ) is much more reliably trainable across a broad swath of problems, and pretty much never runs into the stuck-in-a-local-minima problem often hit with back-propagation trained MLP which forces you to solve a network topography optimization problem just to find a network which will actually train.
well, you need to be more specific about what you are trying to optimize. Is it the size of the hidden layer? Do you have a hidden layer? Is it parameter optimization (learning rate, kernel parameters)?
I assume you have a set of parameters (# of hidden layers, # of neurons per layer...) that needs to be tuned, instead of brute-force searching all combinations to pick a good one, GA can help you "jump" from this combination to another one. So, you can "explore" the search space for potential candidates.
GA can help in selecting "helpful" features. Some features might appear redundant and you want to prune them. However, say, data has too many features to search for the best set of features by some approaches such as forward selection. Again, GA can "jump" from this set candidate to another one.
You will need to find away to encode the data (input parameters, features...) fed to GA. For finding a set of input paras or a good set of features, I think binary encoding should work. In addition, choosing operators for GA to reproduce offsprings is also important. Yet GA needs to be tuned, too (early stopping which can also be applied to ANN).
Here are just some ideas. You might want to search for more info about GA, feature selection, ANN pruning...
Since you're using MATLAB already I suggest you look into the Genetic Algorithms solver (known as GATool, part of the Global Optimization Toolbox) and the Neural Network Toolbox. Between those two you should be able to save quite a bit of figuring out.
You'll basically have to do 2 main tasks:
Come up with a representation (or encoding) for your candidate solutions
Code your fitness function (which basically tests candidate solutions) and pass it as a parameter to the GA solver.
If you need help in terms of coming up with a fitness function, or encoding of candidate solutions then you'll have to be more specific.
Hope it helps.
Matlab has a simple but great explanation for this problem here. It explains both the ANN and GA part.
For more info on using ANN in command line see this.
There is also plenty of litterature on the subject if you google it. It is however not related to MATLAB, but simply the results and the method.
Look up Matthew Settles on Google Scholar. He did some work in this area at the University of Idaho in the last 5-6 years. He should have citations relevant to your work.