If I am using, say guided local search for a VRP, I keep getting slightly different results even when I set a seed to generate my data. Is this normal because it is a heuristic and not an algorithm or is there something else I am missing?
That is normal, and yes it is because it uses heuristics, not a "real" optimization since VRP is an NP-hard problem. Even if you run it on a different system that has different hardware configurations, especially the CPU, there is a possibility that it will produce different results. Everything depends on how complex is the VRP problem, the more complex the problem the probability of producing different results will increase.
Related
I am currently developing a deep reinforcement learning network however, I have a small doubt about the number of q-values I will have at the output of the NN. I will have a total of 150 q-values, which personally seems excessive to me. I have read on several papers and books that this could be a problem. I know that it will depend from the kind of NN I will build, but do you guys think that the number of q-values is too high? should I reduce it?
There is no general principle what is "too much". Everything depends solely on the problem and throughput one can get in learning. In particular number of actions does not have to matter as long as internal parametrisation of Q(a, s) is efficient. To give some example lets assume that the neural network is actually of form NN(a, s) = Q(a, s), in other words it accepts action as an input, together with the state, and outputs the Q value. If such an architecture can be trained in a problem considered, than it might be able to scale to big action spaces; on the other hand if the neural net basically has independent output per action, something of form NN(s)[a] = Q(a, s) then many actions can lead to relatively sparse learning signal for the model and thus lead to slow convergence.
Since you are asking about reducing action space it sounds like the true problem has complex control (maybe it is a continuous control domain?) and you are looking for some discretization to make it simpler to learn. If this is the case you will have to follow the typical approach of trial and error - try with simple action space, observe the dynamics, and if the results are not satisfactory - increase the complexity of the problem. This allows making iterative improvements, as opposed to going in the opposite direction - starting with too complex setting to get any results and than having to reduce it without knowing what are the "reasonable values".
Simulating models with dymola, I get different results depending on the chosen integration method. So my question is: why choose which method?
Ideally the choice of method should be based on which one most quickly gives a result close enough to the exact result.
But we don't know the exact result, and in this case at least some of the solvers (likely all) don't generate a result close enough to it.
One possibility is to first try to simulate with a lot stricter tolerance - e.g. 1e-9. (Note: for the fixed step-size solvers, Euler and Rkfix* it would be smaller step-size, but don't start using them.)
Hopefully the difference between the solvers will decrease and the different solvers give more similar results (which should be close to the exact one).
You can then always use this stricter tolerance. Or in case one solver already gave the same result with less strict tolerance - then you can use that one at less strict tolerance; but you also have to make a trade-off between the accuracy and simulation-time.
Sometimes this does not happen and even the same solver generate different results for different tolerances; without converging to a true solution.
(Assumedly the solution are close together at the start but then quickly diverge.)
It is likely that the model is chaotic in that case. That is a bit more complicated to handle and there are several options:
It could be due to a modeling error that can be corrected
It could be that the model is correct, but the system could be changed to become smoother
It could be that the important outputs converge regardless of differences in other variables
It could also be some other error (including problems during initialization), but that requires a more complete example to investigate.
Choose the solver that matches best the exact solution.
Maybe Robert Piché's work on numerical solvers gives some more clues.
I am reading people's implementation of DCGAN, especially this one in tensorflow.
In that implementation, the author draws the losses of the discriminator and of the generator, which is shown below (images come from https://github.com/carpedm20/DCGAN-tensorflow):
Both the losses of the discriminator and of the generator don't seem to follow any pattern. Unlike general neural networks, whose loss decreases along with the increase of training iteration. How to interpret the loss when training GANs?
Unfortunately, like you've said for GANs the losses are very non-intuitive. Mostly it happens down to the fact that generator and discriminator are competing against each other, hence improvement on the one means the higher loss on the other, until this other learns better on the received loss, which screws up its competitor, etc.
Now one thing that should happen often enough (depending on your data and initialisation) is that both discriminator and generator losses are converging to some permanent numbers, like this:
(it's ok for loss to bounce around a bit - it's just the evidence of the model trying to improve itself)
This loss convergence would normally signify that the GAN model found some optimum, where it can't improve more, which also should mean that it has learned well enough. (Also note, that the numbers themselves usually aren't very informative.)
Here are a few side notes, that I hope would be of help:
if loss haven't converged very well, it doesn't necessarily mean that the model hasn't learned anything - check the generated examples, sometimes they come out good enough. Alternatively, can try changing learning rate and other parameters.
if the model converged well, still check the generated examples - sometimes the generator finds one/few examples that discriminator can't distinguish from the genuine data. The trouble is it always gives out these few, not creating anything new, this is called mode collapse. Usually introducing some diversity to your data helps.
as vanilla GANs are rather unstable, I'd suggest to use some version
of the DCGAN models, as they contain some features like convolutional
layers and batch normalisation, that are supposed to help with the
stability of the convergence. (the picture above is a result of the DCGAN rather than vanilla GAN)
This is some common sense but still: like with most neural net structures tweaking the model, i.e. changing its parameters or/and architecture to fit your certain needs/data can improve the model or screw it.
When running the GlobalSearch solver on a nonlinear constrained optimization problem I have, I often get very different solutions each run. For the cases that I have an analytical solution, the numerical results are less dispersed than the non-analytical cases but are still different each run. It would be nice to get the same results at least for these analytical cases so that I know the optimization routine is working properly. Is there a good explanation of this in the Global Optimization Toolbox User Guide that I missed?
Also, why does GlobalSearch use a different number of local solver runs each run?
Thanks!
A full description of how the GlobalSearch algorithm works can be found Here.
In summary the GlobalSearch method iteratively performs convex optimization. Basically it starts out by using fmincon to search for a local minimum near the initial conditions you have provided. Then a bunch of "trial points", based on how good the initial result was, are generated using the "scatter search algorithm." Then there is some more convex optimization and rating of "how good" the minima around these points are.
There are a couple of things that can cause the algorithm give you different answers:
1. Changing the initial conditions you give it
2. The scatter search algorithm itself
The fact that you are getting different answers each time likely means that your function is highly non-convex. The best thing that I know of that you can do in this scenario is just to try the optimization algorithm at several different initial conditions and see what result you get back the most frequently.
It also looks like there is something called the 'PlotFcns' property which would allow you get a better idea what the functions the solver is generating for you look like.
You can use the ga or gamulti objective functions within the GlobalSearch api. I would recommend this. Convex optimizers wont be able to solve a non-linear problem. Even then Genetic Algorithms dont gaurantee the solution. If you run the ga and then use its final minimum as the start of your fmincon search then it should result in the same answer consistently. There may be better ones but if the search space is unknown you may never know.
Iam writting my thesis and using software called Wingen3 and I am facing problem in determing How many replication should I put when using the program to generate Data?
Some says 5, some says 10,000 but is there a rule or a formula to determine how many replication?
Nobody can give you more than a hand-waving guess without knowing more about your specific case. Note: I know absolutely nothing about "Wingen3", but sample size questions are (or at least ought to be) a function of the statistical properties of your estimators, not of the software.
In general you replicate simulations when they are stochastic to estimate the distributional behavior of the output measures. How many replications depends entirely on what type of measure you're trying to determine and what margin of error you're willing to tolerate in the estimates. One fairly common technique is to make a small initial run and estimate the sample variability of your performance measure. Then project how large a sample will get you down to the desired margin of error. This works fairly well if you're estimating means, medians, or quartiles, but not at all well for estimating quantiles in the tail of your distribution. For instance, if you want to determine the 99.9%-ile, you're seeking extremes that happen one time in a thousand on average and you may need tens or even hundreds of thousands of replications to accurately assess such rare events.