I understand how to prove that the halting problem is undecidable. However, I'm confused as to why it is NP-hard. Is a problem being undecidable equivalent to saying it's in NP-hard?
A problem is said to be "NP-hard" if there is a poly-time reduction from every NP problem to it.
The halting problem is indeed NP-hard, because of the following reason:
Let L be an NP problem, since L is in NP, it is decidable - thus there is a deterministic turing machine that decides L, denote it as M. Now we can create M', that simulates M and halts i.f.f M accepts x (where x is the input). And the result of the reduction on input x is (M', x).
Denote the reduction as R and indeed x is in L i.f.f R(x) is in H.
(M(x) = 1 => M'(x) halts => (M', x) is in H.
M(x) = 0 => M'(x) doesn't halt => (M', x) is not in H)
Why is the reduction polynomial? the size of the description of M may be very large, but it's FIXED. So it takes constant time to create M' (one can look at it as M' being "hardcoded" in R), so the reduction is linear and so polynomial.
Because of the transitivity of reduction (i.e., if there is a (polynomial/ computational) reduction from L1 to L2 and from L2 to L3 (MUST BE SAME REDUCTION TYPE!) there is a reduction from L1 to L3) it is enough to show a reduction from a known NP-hard problem to a new problem to prove that the new problem is NP-hard, and that's the common way to do so. In this example, it was convenient to prove directly from the definiton.
As of your other question, the answer is no. there exists an undecidable problem which is not NP-hard.
You can see a proof here:
https://math.stackexchange.com/questions/642726/are-all-undecidable-problems-np-hard
Note that if P=NP, every problem (except for the trivial sets) is NP-hard. so The proof assumes P != NP.
Related
I’m trying to prove something in Coq using the commutativity of the logic AND operator. I coded this short example:
Axiom ax1 : A /\ B.
Theorem th1 : B /\ A.
Proof.
pose proof (ax1) as H.
symmetry.
apply H.
Qed.
I use axiom (ax1) in my proof, and I get stuck on the symmetry command. This is my current goal:
1 subgoal
H : A /\ B
______________________________________(1/1)
B /\ A
When I use the symmetry command, I get the following error:
Tactic failure: The relation and is not a declared symmetric relation. Maybe you need to require the Coq.Classes.RelationClasses library.
My current solution is to destruct hypothesis H, split the goal, and apply the right sub-hypothesis to the right subgoal. It takes a lot of space and is a bit frustrating to not be able to use AND commutativity.
So I’ve some questions: is symmetry the right command to use with a commutative relation? If yes, how can I fix my code for it to work? If not, is there a way to use the commutativity of the AND operator?
You can make the symmetry tactic work for any relation… as long as you show first that it is symmetric. This is done in the standard library for equality (which is why symmetry works for it out of the box), but not for and.
So if you want to use it, you have to do it yourself, like so:
Require Import RelationClasses.
Instance and_comm : Symmetric and.
Proof.
intros A B [].
now split.
Qed.
The RelationClasses module of the standard library declares the Symmetric typeclass, which is used by Coq when you call the symmetry tactic. Next we prove that and is indeed symmetric, and declare this as an instance of the Symmetric class. Once this is done, symmetry works as you expect it to.
The symmetry tactic is specifically for reasoning about equalities (a = b iff b = a).
I don't know of general machinery for commutative operators, but you can find the lemma you need with something like this:
Search _ (?a /\ ?b <-> ?b /\ ?a)
(I cheated: originally, I used -> and only switched to <-> when that didn't find anything!)
There's exactly one lemma of this form in the base library and it's called and_comm. You should be able to solve your remaining proof obligation with it: apply and_comm.
Let assume an equation A x = b, where A is a real n x n matrix, b is a real valued vector of length n and x is the solution vector of this linear system.
We can find the solution of x by finding the inverse of A.
B= inv(A)
And therefore
x =A^{-1}b.
x= B*b
Can I apply the same solver if A and b are complex?
EDIT: I'm looking for explanation why it should work. Thanks :)
You can do that. Better in Matlab would be x = A\b. That will usually get the answer faster and more accurately.
In short yes, it works for any matrices irrespective of the field. (Well, at least real or complex field works.)
I think your question is whether B exists or not. It does as long as the determinant of B is non-zero. Vice versa.
The cvx suite for MATLAB can solve the (seemingly innocent) optimization problem below, but it is rather slow for the large, full matrices I'm working with. I'm hoping this is because using cvx is overkill, and that the problem actually has an analytic solution, or that a clever use of some built-in MATLAB functions can more quickly do the job.
Background: It is well-known that both x1=A\b and x2=pinv(A)*b solve the least-squares problem:
minimize norm(A*x-b)
with the distinction that norm(x2)<=norm(x1). In fact, x2 is the minimum-norm solution to the problem, so norm(x2)<=norm(x) for all possible solutions x.
Defining D=norm(A*x2-b), (equivalently D=norm(A*x1-b)), then x2 solves the problem
minimize norm(x)
subject to
norm(A*x-b) == D
Problem: I'd like to find the solution to:
minimize norm(x)
subject to
norm(A*x-b) <= D+threshold
In words, I don't need norm(A*x-b) to be as small as possible, just within a certain tolerance. I want the minimum-norm solution x that gets A*x within D+threshold of b.
I haven't been able to find an analytic solution to the problem (like using the pseudoinverse in the classic least-squares problem) on the web or by hand. I've been searching things like "least squares with nonlinear constraint" and "least squares with threshold".
Any insights would be greatly appreciated, but I suppose my real question is:
What is the fastest way to solve this "thresholded" least-squares problem in MATLAB?
Interesting question. I do not know the answer to your exact question, but a working solution is presented below.
Recap
Define res(x) := norm(Ax - b).
As you state, x2 minimizes res(x). In the overdetermined case (typically A having more rows than col's), x2 is the unique minimum. In the underdetermined case, it is joined by infinitely many others*. However, among all of these, x2 is the unique one that minimizes norm(x).
To summarize, x2 minimizes (1) res(x) and (2) norm(x), and it does so in that order of priority. In fact, this characterizes (fully determines) x2.
The limit characterization
But, another characterization of x2 is
x2 := limit_{e-->0} x_e
where
x_e := argmin_{x} J(x;e)
where
J(x;e) := res(x) + e * norm(x)
It can be shown that
x_e = (A A' + e I)^{-1} A' b (eqn a)
It should be appreciated that this characterization of x2 is quite magical. The limits exists even if (A A')^{-1} does not. And the limit somehow preserves priority (2) from above.
Using e>0
Of course, for finite (but small) e, x_e will not minimize res(x)(instead it minimzes J(x;e)). In your terminology, the difference is the threshold. I will rename it to
gap := res(x_e) - min_{x} res(x).
Decreasing the value of e is guaranteed to decrease the value of the gap. Reaching a specific gap value (i.e. the threshold) is therefore easy to achieve by tuning e.**
This type of modification (adding norm(x) to the res(x) minimization problem) is known as regularization in statistics litterature, and is generally considered a good idea for stability (numerically and with respect to parameter values).
*: Note that x1 and x2 only differ in the underdetermined case
**:It does not even require any heavy computations, because the inverse in (eqn a) is easily computed for any (positive) value of e if the SVD of A has already been computed.
Which of the following is the most precise classification of a problem X?
X is in NP
X is in P
X is in O(n2)
X is in Θ(n2).
I would greatly appreciate if anyone could explain the answer of this to me?
I believe it's either NP or P, but i'm really not sure
NP or P means whether it can be solved in polynomial time in a non deterministic machine(NP) or in a deterministic machine(P). This reflects the complexity of the problem but not the complexity of an algorithm that solves the problem.
While O(n^2) means that the algorithm being analyzed to solve a problem has an upper bound of n^2 complexity when n is the input.
Theta(n^2) is also a way of expressing the complexity of an algorithm used to solve a problem but Theta(n^2) in contrast of O(n^2) means that that the lower and upper bound of complexity is n^2.
There's also another measure which is o(little-oh) which indicates the lower bound complexity of an algorithm.
Theta is more precise because like O(n^2) means just upper bound, the algorithm is also O(n^3) and O(n!).
Θ(n2) ⊂ O(n2) ⊂ P ⊆ NP
I need to calculate (a/b) mod m where a and b are very large numbers.
What I am trying to do is to calculate (a mod m) * (x mod m), where x is the modular inverse of b.
I tried using Extended Euclidean algorithm, but what to do when b and m are not co-prime?
It is specifically mentioned that b and m need to be co-prime.
I tried using the code here, and realized that for example:
3 * x mod 12 is not at all possible for any value of x, it does not exist!
What should I do? Can the algorithm be modified somehow?
Yep, you are in trouble. x has no solution in b*x = 1 mod m if b and m have a common divisor. Similarly, in your original problem a/b = y mod m, you are looking for y such that a=by mod m. If a is divisible by gcd(b,m), then you can divide out by that factor and solve for y. If not, then there is no y that can solve the equation (i.e. a/b mod m is not defined).
The reason that b and m have to be coprime is because of the Chinese Remainder Theorem. Basically the problem:
3 * x mod 12
Can be thought of as a compound problem involving
3*x mod 3 and 3*x mod 4 = 2^2
Now if b is not coprime to 12, this is like trying to divide by zero. Thus the answer doesn't exist!
This is due to field theory in abstract algebra. A field is basically a set which has addition, subtraction, multiplication, and division well-defined. A finite field is always of the form GF(p^n), where p is prime and n is a positive integer, and the operations are addition and multiplication modulo p^n. Now, 12 is not a prime power, so your ring is not a field. Thus this problem can't be solved for any b which is not coprime to m.
Check this: http://www.math.harvard.edu/~sarah/magic/topics/division
It might help.
It explains methods of modular division.