Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I'm trying to solve a SAT problem that consists of one-hot constraints only. Right now I'm using the one-hot encoding proposed by Claessen in Sec. 4.2 of [1] and MiniSAT.
I wonder if there is a better way of solving such a problem, e.g. a class of solvers that is a better match for this kind of problems compared to CNF-based SAT solvers. I've searched a bit but can't find much information on SAT and one-hot constraints at all.
[1] Successful SAT Encoding Techniques. Magnus Bjiirk. 25th July 2009
Satisfiability modulo theorem solvers are often a good choice for expressing mathematical problems that would otherwise end up using one-hot CNF encodings. A key insight is that with CNF and DPLL-based solvers alone you're limited to what unit propagation can discover efficiently. At a higher level of abstraction there are theories of numbers and sets that can be used to prune the search space before selectively applying a SAT solver to more appropriate subproblems. That's what SMT gives you, along with more expressive power and a much more pleasant language in which express relationships.
But if you're stuck with one-hot constraints with no associated higher level math problem, there are more efficient ways to encode onehot constraints that require only a linear number of new clauses to express the relation instead of the usual quadratic increase. See
Efficient CNF Encoding for Selecting 1 from N Objects by Klieber and Kwon.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed last year.
Improve this question
I have a lot of constraints(in millions) and want to solve a mixed integer problem. Tried using PuLP and OR-Tools(with SCIP solver), but they failed to solve the problem. I tried using DuaLip for parallel/distributed computing using spark, but it doesn't seem to have good documentation for usage. How should I proceed further?
AFAIK there do not exist continuous LP solvers that do distributed computing. That would require some decomposition scheme. As LPs solve very fast we don't use those techniques anymore. An LP with a few million constraints is not very large these days. Quite often interior-point algorithms do quite well on these large problems (relative to Simplex algorithms). We solve LPs of this size on a completely routine basis on fairly standard hardware.
Your PuLP problem is likely more related to PuLP having problems generating the problem than the solver solving it. (Note that PuLP is not a solver). Python based modeling tools may be slower in generating easy LPs than the solver needs to solve it.
Note: the situation for MIP solvers is very different.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
This question is not about how to use matlab, but trying to find out what is happening when matab is solving a differential equation. Results are often different when using different numerical methods. i wonder which numerical method is used in matlab.
Matlab has all kinds of numerical solvers available. The basic set can be found at the bottom of this page:
https://www.mathworks.com/help/matlab/math/choose-an-ode-solver.html
If you'd like to know about a particular solver (say ode45) you can scroll to the bottom of the documentation for the given solver (for ode45 it's found here: https://www.mathworks.com/help/matlab/ref/ode45.html). For this solver the paper which explains it is linked. It may, however, be a little obtuse if you are unfamiliar with the general idea behind numerical solvers, so you might consider checking out a more pedagogical text in this area, such as the one at http://numerical.recipes/. You may also consider googling less complicated solvers like the Euler Method or the Runge-Kutta method -- both give you the flavor for how the numerical solvers work.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
After using Wolfram Alpha and MATLAB's Symbolic Math Toolbox for solving integrals, ODEs and PDEs, I got curious to know how would I implement an analytical(closed-form) integration(or non-trivial equation) solver.
For example, how could i programmatically solve the following integral analytically?
Integrals are solved by (very complicated) pattern matching. If the integrand looks like the square root of something, then the integral is ...; if it looks like a rational function, then the integral is ..., if exponential, then ...., etc etc etc. There are at least two major difficulties. One is recognizing that an integrand matches one pattern or another, the other is constructing the solution once you have a pattern match. The paper by Lichtblau cited above is about the second part. As to pattern matching on expressions, try a web search for "pattern matching" or "unification". As it happens, pattern matching is most naturally expressed in Lisp, but it can also be handled in other programming languages, usually by reinventing a subset of Lisp.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
I have dataset consisting of about 300 objects with 84 features for each object. The objects are already separated into two classes. With PCA I´m able to reduce the dimensionality to about 24. I´m using 3 principle components covering about 96% of the variance of the original data. The problem I have is that PCA doesnt care about the ability to separate the classes from each other. Is there a way to combine PCA for reducing feature space and LDA for finding a discriminance function for those two classes ?
Or is there a way to use LDA for finding the features that separate two classes in threedimensional space in the best manner ?
I´m kind of irritated because I found this paper but I´m not really understanding. http://faculty.ist.psu.edu/jessieli/Publications/ecmlpkdd11_qgu.pdf
Thanks in advance.
You should have a look at this article on principle component regression (PCR, what you want if the variable to be explained is scalar) and partial least squares regression (PLSR) with MATLAB's statistics toolbox. In PCR essentially, you choose the principal components as they most explain the dependent variable. They may not be the ones with the largest variance.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I find this question a little tricky. Maybe someone knows an approach to answer this question. Imagine that you have a dataset(training data) which you don't know what it is about. Which features of training data would you look at in order to infer classification algorithm to classify this data? Can we say anything whether we should use a non-linear or linear classification algorithm?
By the way, I am using WEKA to analyze the data.
Any suggestions?
Thank you.
This is in fact two questions in one ;-)
Feature selection
Linear or not
add "algorithm selection", and you probably have three most fundamental questions of classifier design.
As an aside note, it's a good thing that you do not have any domain expertise which would have allowed you to guide the selection of features and/or to assert the linearity of the feature space. That's the fun of data mining : to infer such info without a priori expertise. (BTW, and while domain expertise is good to double-check the outcome of the classifier, too much a priori insight may make you miss good mining opportunities). Without any such a priori knowledge you are forced to establish sound methodologies and apply careful scrutiny to the results.
It's hard to provide specific guidance, in part because many details are left out in the question, and also because I'm somewhat BS-ing my way through this ;-). Never the less I hope the following generic advice will be helpful
For each algorithm you try (or more precisely for each set of parameters for a given algorithm), you will need to run many tests. Theory can be very helpful, but there will remain a lot of "trial and error". You'll find Cross-Validation a valuable technique.
In a nutshell, [and depending on the size of the available training data], you randomly split the training data in several parts and train the classifier on one [or several] of these parts, and then evaluate the classifier on its performance on another [or several] parts. For each such run you measure various indicators of performance such as Mis-Classification Error (MCE) and aside from telling you how the classifier performs, these metrics, or rather their variability will provide hints as to the relevance of the features selected and/or their lack of scale or linearity.
Independently of the linearity assumption, it is useful to normalize the values of numeric features. This helps with features which have an odd range etc.
Within each dimension, establish the range within, say, 2.5 standard deviations on either side of the median, and convert the feature values to a percentage on the basis of this range.
Convert nominal attributes to binary ones, creating as many dimensions are there are distinct values of the nominal attribute. (I think many algorithm optimizers will do this for you)
Once you have identified one or a few classifiers with a relatively decent performance (say 33% MCE), perform the same test series, with such a classifier by modifying only one parameter at a time. For example remove some features, and see if the resulting, lower dimensionality classifier improves or degrades.
The loss factor is a very sensitive parameter. Try and stick with one "reasonnable" but possibly suboptimal value for the bulk of the tests, fine tune the loss at the end.
Learn to exploit the "dump" info provided by the SVM optimizers. These results provide very valuable info as to what the optimizer "thinks"
Remember that what worked very well wih a given dataset in a given domain may perform very poorly with data from another domain...
coffee's good, not too much. When all fails, make it Irish ;-)
Wow, so you have some training data and you don't know whether you are looking at features representing words in a document, or genese in a cell and need to tune a classifier. Well, since you don't have any semantic information, you are going to have to do this soley by looking at statistical properties of the data sets.
First, to formulate the problem, this is more than just linear vs non-linear. If you are really looking to classify this data, what you really need to do is to select a kernel function for the classifier which may be linear, or non-linear (gaussian, polynomial, hyperbolic, etc. In addition each kernel function may take one or more parameters that would need to be set. Determining an optimal kernel function and parameter set for a given classification problem is not really a solved problem, there are only useful heuristics and if you google 'selecting a kernel function' or 'choose kernel function', you will be treated to many research papers proposing and testing various approaches. While there are many approaches, one of the most basic and well travelled is to do a gradient descent on the parameters-- basically you try a kernel method and a parameter set , train on half your data points and see how you do. Then you try a different set of parameters and see how you do. You move the parameters in the direction of best improvement in accuracy until you get satisfactory results.
If you don't need to go through all this complexity to find a good kernel function, and simply want an answer to linear or non-linear. then the question mainly comes down to two things: Non linear classifiers will have a higher risk of overfitting (undergeneralizing) since they have more dimensions of freedom. They can suffer from the classifier merely memorizing sets of good data points, rather than coming up with a good generalization. On the other hand a linear classifier has less freedom to fit, and in the case of data that is not linearly seperable, will fail to find a good decision function and suffer from high error rates.
Unfortunately, I don't know a better mathematical solution to answer the question "is this data linearly seperable" other than to just try the classifier itself and see how it performs. For that you are going to need a smarter answer than mine.
Edit: This research paper describes an algorithm which looks like it should be able to determine how close a given data set comes to being linearly seperable.
http://www2.ift.ulaval.ca/~mmarchand/publications/wcnn93aa.pdf