XOR is not solvable by using a single perceptron with standard scalar product and unit step function.
This article suggests using 3 perceptron to make a network:
http://toritris.weebly.com/perceptron-5-xor-how--why-neurons-work-together.html
I'm trying to run the 3-perceptron network this way but it doesn't produce correct results for XOR:
//pseudocode
class perceptron {
constructor(training_data) {
this.training_data = training_data
}
train() {
iterate multiple times over training data
to train weights
}
unit_step(value) {
if (value<0) return 0
else return 1
}
compute(input) {
weights = this.train()
sum = scalar_product(input,weights)
return unit_step(sum)
}
}
The above perceptron can solve NOT, AND, OR bit operations correctly. This is how I use 3 perceptrons to solve XOR:
AND_perceptron = perceptron([
{Input:[0,0],Output:0},
{Input:[0,1],Output:0},
{Input:[1,0],Output:0},
{Input:[1,1],Output:1}
])
OR_perceptron = perceptron([
{Input:[0,0],Output:0},
{Input:[0,1],Output:1},
{Input:[1,0],Output:1},
{Input:[1,1],Output:1}
])
XOR_perceptron = perceptron([
{Input:[0,0],Output:0},
{Input:[0,1],Output:1},
{Input:[1,0],Output:1},
{Input:[1,1],Output:0}
])
test_x1 = 0
test_x2 = 1
//first layer of perceptrons
and_result = AND_perceptron.compute(test_x1,test_x2)
or_result = OR_perceptron.compute(test_x1,test_x2)
//second layer
final_result = XOR_perceptron.compute(and_result,or_result)
The final_result above is not consistent, sometimes 0, sometimes 1. It seems I run the 2 layers wrongly. How to run these 3 perceptrons in 2 layers the correct way?
If you want to build a neural network with logical connectives (and, or, not), you have to consider the following equivalences regarding xor:
A xor B ≡ (A ∨ B) ∧ ¬(A ∧ B) ≡ (A ∨ B) ∧ (¬A ∨ ¬B) ≡ (A ∧ ¬B) ∨ (¬A ∧ B)
So you would need at least three and- or or-perceptrons and one negation if you want to use your perceptrons if I understand them correctly. In the article they use three perceprons with special weights for the xor. These are not the same as and- and or-perceptrons.
Related
I would like to use Scala's property-based testing tool ScalaCheck to express a property
if an integer n * n = 0 then n = 0
How can I write this property in ScalaCheck? I know for example
val myprop = forAll {(n: Int) => n + 1 - 1 = n}
But I do not know how to express "A implies B" in ScalaCheck (without reducing it to Not-A or B, which can look clumsy).
Use ==> (implication operator)
val prop = forAll { n: Int =>
(n * n == 0) ==> n == 0
}
(see their User Guide )
the catch is: in this particular example the condition is very hard to satisfy so ScalaCheck will give up after several tries (but at least it does tell you so, otherwise you get a false positive because your necessary condition was never checked). In that case you can provide a custom generator so that it will generate values that satisfy your condition.
I have a few True/False questions I need to answer, I don't know how to do it. Any help would be appreciated!
(a) True or false: Suppose that you develop an algorithm whose run-time is given by f(n) = 5f(n/2) + 3 for n = 2,4,8,16,32,..., and f(1) = 7. Then the run-time of your algorithm is O(n3), but it is not O(n2).
(b) True or false: f (n) = n3n satisfies the recurrence relation f (n) = 6f (n − 1) − 9f(n−2).
For part (a) you can apply the master theorem. The non recursive overhead is O(1), the 'critical' function n^{log_b(a)} is n^{\log_2(5)} = n^{2.321928}. So the algorithm runs in θ(n^{2.321928}), meaning it is not O(n^2). But, it is O(n^3).
I have a string a and on comparison with string b, if equals has an string c, else has string x. I know in the hypothesis that fun x <= fun c. How do I prove this below statement? fun is some function which takes in string and returns nat.
fun (if a == b then c else x) <= S (fun c)
The logic seems obvious but I am unable to split the if statements in coq. Any help would be appreciated.
Thanks!
Let me complement Yves answer pointing out to a general "view" pattern that works well in many situations were case analysis is needed. I will use the built-in support in math-comp but the technique is not specific to it.
Let's assume your initial goal:
From mathcomp Require Import all_ssreflect.
Variables (T : eqType) (a b : T).
Lemma u : (if a == b then 0 else 1) = 2.
Proof.
now, you could use case_eq + simpl to arrive to next step; however, you can also match using more specialized "view" lemmas. For example, you could use ifP:
ifP : forall (A : Type) (b : bool) (vT vF : A),
if_spec b vT vF (b = false) b (if b then vT else vF)
where if_spec is:
Inductive if_spec (A : Type) (b : bool) (vT vF : A) (not_b : Prop) : bool -> A -> Set :=
IfSpecTrue : b -> if_spec b vT vF not_b true vT
| IfSpecFalse : not_b -> if_spec b vT vF not_b false vF
That looks a bit confusing, the important bit is the parameters to the type family bool -> A -> Set. The first exercise is "prove the ifP lemma!".
Indeed, if we use ifP in our proof, we get:
case: ifP.
Goal 1: (a == b) = true -> 0 = 2
Goal 2: (a == b) = false -> 1 = 2
Note that we didn't have to specify anything! Indeed, lemmas of the form { A
} + { B } are just special cases of this view pattern. This trick works in many other situations, for example, you can also use eqP, which has a spec relating the boolean equality with the propositional one. If you do:
case: eqP.
you'll get:
Goal 1: a = b -> 0 = 2
Goal 2: a <> b -> 1 = 2
which is very convenient. In fact, eqP is basically a generic version of the type_dec principle.
If you can write an if-then-else statement, it means that the test expression a == b is in a type with two constructors (like bool) or (sumbool). I will first assume the type is bool. In that case, the best approach during a proof is to enter the following command.
case_eq (a == b); intros hyp_ab.
This will generate two goals. In the first one, you will have an hypothesis
hyp_ab : a == b = true
that asserts that the test succeeds and the goal conclusion has the following shape (the if-then-else is replaced by the then branch):
fun c <= S (fun c)
In the second goal, you will have an hypothesis
hyp_ab : a == b = false
and the goal conclusion has the following shape (the if-then-else is replaced by the else branch).
fun x <= S (fun c)
You should be able to carry on from there.
On the other hand, the String library from Coq has a function string_dec with return type {a = b}+{a <> b}. If your notation a == b is a pretty notation for string_dec a b, it is better to use the following tactic:
destruct (a == b) as [hyp_ab | hyp_ab].
The behavior will be quite close to what I described above, only easier to use.
Intuitively, when you reason on an if-then-else statement, you use a command like case_eq, destruct, or case that leads you to studying separately the two executions paths, remember in an hypothesis why you took each of these executions paths.
How can I use a comparison of to rational numbers in an if-statement?
if 1 = 2 then 1 else 2
1 = 2 is of course Prop and not bool.
I don't understand how dfan's answer is related to the question...
Of course, 1 = 2 is a Prop, it is the statement that 1 is equal to 2. Hopefully you don't have a proof of this statement...
What you want is a function that, given two natural numbers, 1 and 2, returns true if they are equal, and false if they aren't.
The library Coq.Arith.EqNat gives you such a function, named beq_nat.
In fact, you might want something even better, a function that returns a proof of equality or a proof of difference:
(* In Coq.Arith.Peano_dec *)
Theorem eq_nat_dec : forall n m, {n = m} + {n <> m}.
(* ^ a proof that n = m
^ or a proof that n <> m *)
if is overloaded to work with such things, so you can even write:
if eq_nat_dec 2 3 then ... else ...
Qeq_bool indeed takes two rationals and produces a bool.
Require Export QArith_base.
Eval compute in Qeq_bool (3#2) (3#2). = true: bool
Eval compute in Qeq_bool (3#2) (5#2). = false: bool
Eval compute in (if Qeq_bool (3#2) (5#2) then 7 else 9). = 9: nat
This is a follow-up on Isabelle's Code generation: Abstraction lemmas for containers?:
I want to generate code for the_question in the following theory:
theory Scratch imports Main begin
typedef small = "{x::nat. x < 10}" morphisms to_nat small
by (rule exI[where x = 0], simp)
code_datatype small
lemma [code abstype]: "small (to_nat x) = x" by (rule to_nat_inverse)
definition a_pred :: "small ⇒ bool"
where "a_pred = undefined"
definition "smaller j = [small i . i <- [0 ..< to_nat j]]"
definition "the_question j = (∀i ∈ set (smaller j). a_pred j)"
The problem is that the equation for smaller is not suitable for code generation, as it mentions the abstraction function small.
Now according to Andreas’ answer to my last question and the paper on data refinement, the next step is to introduce a type for sets of small numbers, and create a definition for smaller in that type:
typedef small_list = "{l. ∀x∈ set l. (x::nat) < 10}" by (rule exI[where x = "[]"], auto)
code_datatype Abs_small_list
lemma [code abstype]: "Abs_small_list (Rep_small_list x) = x" by (rule Rep_small_list_inverse)
definition "smaller' j = Abs_small_list [ i . i <- [0 ..< to_nat j]]"
lemma smaller'_code[code abstract]: "Rep_small_list (smaller' j) = [ i . i <- [0 ..< to_nat j]]"
unfolding smaller'_def
by (rule Abs_small_list_inverse, cases j, auto elim: less_trans simp add: small_inverse)
Now smaller' is executable. From what I understand I need to redefine operations on small list as operations on small_list:
definition "small_list_all P l = list_all P (map small (Rep_small_list l))"
lemma[code]: "the_question j = small_list_all a_pred (smaller' j)"
unfolding small_list_all_def the_question_def smaller'_code smaller_def Ball_set by simp
I can define a good looking code equation for the_question. But the definition of small_list_all is not suitable for code generation, as it mentions the abstraction morphismsmall. How do I make small_list_all executable?
(Note that I cannot unfold the code equation of a_pred, as the problem actually occurs in the code equation of the actually recursive a_pred. Also, I’d like to avoid hacks that involve re-checking the invariant at runtime.)
I don't have a good solution to the general problem, but here's an idea that will let you generate code for the_question in this particular case.
First, define a function predecessor :: "small ⇒ small with an abstract code equation (possibly using lift_definition from λn::nat. n - 1).
Now you can prove a new code equation for smaller whose rhs uses if-then-else, predecessor and normal list operations:
lemma smaller_code [code]:
"smaller j = (if to_nat j = 0 then []
else let k = predecessor j in smaller k # [k])"
(More efficient implementations are of course possible if you're willing to define an auxiliary function.)
Code generation should now work for smaller, since this code equation doesn't use function small.
The short answer is no, it does not work.
The long answer is that there are often workarounds possible. One is shown by Brian in his answer. The general idea seems to be
Separate the function that has the abstract type in covariant positions besides the final return value (i.e. higher order functions or functions returning containers of abstract values) into multiple helper functions so that abstract values are only constructed as a single return value of one of the helper function.
In Brian’s example, this function is predecessor. Or, as another simple example, assume a function
definition smallPrime :: "nat ⇒ small option"
where "smallPrime n = (if n ∈ {2,3,5,7} then Some (small n) else None)"
This definition is not a valid code equation, due to the occurrence of small. But this derives one:
definition smallPrimeHelper :: "nat ⇒ small"
where "smallPrimeHelper n = (if n ∈ {2,3,5,7} then small n else small 0)"
lemma [code abstract]: "to_nat (smallPrimeHelper n) = (if n ∈ {2,3,5,7} then n else 0)"
by (auto simp add: smallPrimeHelper_def intro: small_inverse)
lemma [code_unfold]: "smallPrime n = (if n ∈ {2,3,5,7} then Some (smallPrimeHelper n) else None)"
unfolding smallPrime_def smallPrimeHelper_def by simp
If one wants to avoid the redundant calculation of the predicate (which might be more complex than just ∈ {2,3,5,7}, one can make the return type of the helper smarter by introducing an abstract view, i.e. a type that contains both the result of the computation, and the information needed to construct the abstract type from it:
typedef smallPrime_view = "{(x::nat, b::bool). x < 10 ∧ b = (x ∈ {2,3,5,7})}"
by (rule exI[where x = "(2, True)"], auto)
setup_lifting type_definition_small
setup_lifting type_definition_smallPrime_view
For the view we have a function building it and accessors that take the result apart, with some lemmas about them:
lift_definition smallPrimeHelper' :: "nat ⇒ smallPrime_view"
is "λ n. if n ∈ {2,3,5,7} then (n, True) else (0, False)" by simp
lift_definition smallPrimeView_pred :: "smallPrime_view ⇒ bool"
is "λ spv :: (nat × bool) . snd spv" by auto
lift_definition smallPrimeView_small :: "smallPrime_view ⇒ small"
is "λ spv :: (nat × bool) . fst spv" by auto
lemma [simp]: "smallPrimeView_pred (smallPrimeHelper' n) ⟷ (n ∈ {2,3,5,7})"
by transfer simp
lemma [simp]: "n ∈ {2,3,5,7} ⟹ to_nat (smallPrimeView_small (smallPrimeHelper' n)) = n"
by transfer auto
lemma [simp]: "n ∈ {2,3,5,7} ⟹ smallPrimeView_small (smallPrimeHelper' n) = small n"
by (auto intro: iffD1[OF to_nat_inject] simp add: small_inverse)
With that we can derive a code equation that does the check only once:
lemma [code]: "smallPrime n =
(let spv = smallPrimeHelper' n in
(if smallPrimeView_pred spv
then Some (smallPrimeView_small spv)
else None))"
by (auto simp add: smallPrime_def Let_def)