I have been wanting to try and make a neural net that controlls a game.
I'm very close to my goal, but yet I feel like I have messed up somewhere!
I'm getting values that are reasonably close to what I'm expecting. But there is one problem...
I decided to try it with the Flappy Bird concept...
I have 3 inputs currently.
1) Height from ground (goes toward 1 the closer it gets)
2) Distance
to nearest wall in the top of the screen
3) ... bottom of the screen.
3 Hidden neurons, and one output.
It kind of jump when it gets to a lower wall. But it ALSO jumps when it reaches a upper wall. Thus hitting it in steath of dropping down lower.
My question really, is there some way of turning that jump into drops insteath?
Or do any of you suspect where I might start looking?
1) I know i have trouble with the sigmoid function. I have no idea how to write it actually. It says
f(x) = 1 / (1 + e^x)
Can I just swap e with a number or something? What is that character? I never really heard about it in the physics math I had...
The derivative is actually easier it seems.
But this is the right formula, right?
fd(x) = f(x) * (1 - f(x))
The function
f(x) = 1 / (1 + e^(-x))
is just an approximation of the step function that is also differentiable.
(where e is the natural logarithm.)
You can find a number of examples of Neural Networks playing Flappy Bird. For example, this developer seems to only use 2 inputs, 2 hidden, and 1 output. His inputs only seem to be distance to the top wall and distance to the bottom wall.
I think this is a better solution than your 3 inputs - "distance from the bottom of the screen" is probably not an effective feature.
Note: This is also dependent on how you are calculating distance to each wall.
Related
I am working on a simulation, which contains:
a bolt, welded uprightly to the world
a nut, connected to the bolt via ScrewJoint. The mass of a nut set to 0.02 kg, the inertia is a diagonal 1.1e-9 *I. This is configured via a .sdf file.
an iiwa manipulator, which is beside a point for now.
The problem is that the nut is very hard to manipulate and I cannot find a parameter to adjust, which could've made it more lifelike. To be more specific:
I measure the ability of force, applied tangentially in a horizontal plane to the nut, to cause the screwing motion of a joint, that joins the nut with a bolt
I'd like to have greater amount of motion at lower forces, and so far I am failing to achieve that
My interest in doing this is not idle; I am interested in more complicated simulations, which are also failing when iiwa is coming in a contact with this same joint; I've asked about those here and here. (Both answered partially). To sum up those here: when manipulator grips the nut, the nut fights the screwing in such a manner, that the schunk gripper is forced to unclasp and iiwa is thrown off-track, but the nut remains stationary.
I attach below two simpler experiments to better illustrate the issue:
1. Applying tangentially in a horizontal plane 200N force using ExternallyAppliedSpatialForce.
Graph notation: (here as well as below)
The left graph contains linear quantities (m, m/s, etc) along world's Z axis, the right graph contains angular quantities (in degrees, deg, deg/s, etc) around world's Z axis. The legend entries with a trailing apostrophe use the secondary Y-axis scale; other legend entries use the primary Y-axis scale.
Experiment summary:
This works as expected, 200 N is enough to make the nut spin on a bolt, resulting in the nut traveling vertically along the bolt for just under 1 centimeter, and spinning for over 90 degrees. Note: the externally applied force does not show up on my graph.
2. Applying tangentially in a horizontal plane force using iiwa and a simple position controller.
Experiment summary:
The force here is approximately the same as before: 70N along tz, but higher (170N) in tx and ty, though it is applied now only for a brief moment. The nut travels just a few degrees or hundredth fractions of centimeter.
The video of this unsuccessful interaction is below, the contact forces are visualized using ContactVisualizer.
Please advise me on how to make this screw_joint more compliant?
I've tried varying mass and inertia of the nut (different up to the orders of magnitude) in these experiments, this
seems to scale the contact forces, but does not affect acceleration or velocity of the nut after contact.
I like your experiment using ExternallyAppliedSpatialForce to get an idea of scales, though TBH I didn't quite get the details of this setup.
Things that caught my eye though are about scales, which you can estimate with pen and paper:
Your inertia is 1e-9 kg⋅m²?! Judging from your interaction with the iiwa I estimated a radius of 1cm and with that you'd get 2e-6 kg⋅m², three orders of magnitude larger.
A force of 200 N on a 20 grams nut would cause an acceleration of 10000 m/s². As a reference, that's 1000 times the acceleration of Earth's gravity!
Are these numbers correct? Also, if you happen to have fast interactions (do you?), you might want to estimate a time step that makes sense for your application.
Hopefully this helps!
A good thing is that I've fixed it; a bad thing: I don't understand the fix.
Let's restate a problem that I was tacking: a manipulated object reacted quite predictably to the ExternallyAppliedSpatialForce, but couldn't be moved via the contact with the manipulator.
What was done:
I've update drake from 2f340192a9dc79110410faf8a6d54a8615ddca92 (circa 22 Aug, 2022) to 42448c0af1b39f0c46f760e7ae37d77097689ad3 (circa 3 Nov, 2022)
After the update, my experimental setup broke down with assertion Actuation input port for model instance ... must be connected. [Similar to the issue raised in this question.]. My fix was like that:
bolt_n_nut_ = internal::AddAndWeldModelFrom(sdf_path, "nut_and_bolt",
plant_->world_frame(),
"bolt", X_WC, plant_);
then later in ManipulationStation::Finalize:
auto zero_torque = builder.template AddSystem<systems::ConstantVectorSource<double>>(
Eigen::VectorXd::Zero(1));
builder.Connect(zero_torque->get_output_port(),
plant_->get_actuation_input_port(bolt_n_nut_));
With changes above, a manipuland began to interact with the manipulator:
Things to note in graphs:
the distance the manipuland has moved grew from fractions of millimeter up to tens of centimeters. The video presents that a nut became manipulable.
This interaction violates the constraints of the ScrewJoint, i.e. the manipuland moves along it's axis without as much rotation
I'm trying to use GA to train an ANN whose job is to move a bar vertically so that it makes a ball bounce without hitting the wall behind the bar, in other words, a single bar pong.
I'm going to ask it directly because i think to know what the problem is.
The game window is 200x200 pixels, so i created 40000 input neurons.
The obvious doubt is: can GA handle chromosomes of 40000(input)*10(hidden)*2 elements(genes)?
Since i think the answer is no(i implemented this solution and doesn't seem to work), the solution seems simple, i feed the NN with only 4 parameters which are the coordinates x,y of bar and ball, nailed it.
Nice solution, but the problem is: how can i apply such a solution in a game like supermario where the number of enemies in the screen is not fixed? Surely i cannot create a NN with dynamic numbers of inputs.
I hope you can help me.
You have to use features to represent your state. For example, you can divide the screen in tiles and assign a value according to a function that takes into account the enemy (e.g., a boolean if the enemy is in the tile or the distance to the closest enemy).
You can still use pixels but you might need to preprocess them in order to reduce their size (e.g., use a recurrent NN).
Btw, a NN might not be able to handle 200x200 pixels, but it was able to learn to play Atari games using a representation of the state by preprocessed pixels of size 84x84x4 (see this paper).
When doing SPH, the paper by Kelagar recommends using a particular kernel for pressure induced forces between particles. The kernel it recommends is the following when the radius is within the kernel radius:
(15/(pi*h^9)) * (h - r)^3
where h is the kernel radius, and r is the radius we are interested in calculating the value of a function at.
The paper then states that the gradient of this function is
(-45/(pi*h^9))*((r_vec)/r)*(h-r)^2
where r_vec is now the vector from the center of the kernel to the point we are interested in. As length of r_vec goes to 0 from the positive direction, the paper states that this gradient approaches:
(-45/(pi*h^6))
But this is a scalar, not a vector. In order for there to be a repulsion between the two points we're interested in, there needs to be a direction to repel in.
What direction should we use for when two particles are right next to each other?
I assume that first expression is meant to be a potential. The negative gradient (derivative with respect to r) is then the force. This gradient is a vector, always pointing toward or away from the center. This appears correct for the second expression.
r_vec is, according to what you say, a vector pointing away from the origin to a point at some distance r away. (r_vec/r) is then a unit vector to specify direction. This works at every point except the origin itself, where it can be declared undefined, or declared to be zero. Zero is the average value of (r_vec/r) over all "nearby" points. This means zero force.
Normally in particle simulations with pair-wise forces, we ignore forces of a particle on itself, and of two particle at the same exact position. What about two particle very close, and you have a force law that goes like 1/r, 1/(r^2), or similar? Nobody wants a divide by zero fault. Usually there's a small radius below which the potential is constant matching the given potential formula at the boundary of that radius. Particles too close together have zero force, just so that the simulation won't crash. It may seem unphysical for the force to suddenly cease just inside that boundary when it is fiercely strong just outside it. But we strive to avoid such situations. Keep count of such incidences, and if there are too many, the simulation has gone bad. Maybe a smaller time step is needed.
Luckily you don't have a 1/r type of force, but still you have that nasty r_vec/r whose direction can swing wildly. The same technique of making force zero below a certain tiny radius will help.
But that third expression bothers me. If it's force at r=0, then starting with the force law in the second expression, I'm not sure how the third expression comes about. The problem of it looking scalar while, if it is supposed to be force, expecting vector could be resolved by understanding that it is meant to be the radial component of a force vector. Just multiply the expression by (r_vec/r), the familiar unit-magnitude vector. OTOH, it has no defined direction, so it is nonsense.
Better overall solution: start with a new potential function, one that smoothly levels off and is flat right at r=0, like exp(-r^2) or 1/(1+r^2). The given potential peaks sharply. You want something more like Instead of declaring force zero inside some small zone, the force would just naturally be zero at r=0. Find a flat-at-origin potential that approximates the given one well outside some small radius.
The following problem:
Given is an arbitrary polygon. It shall be covered 100% with the minimum number of circles of a given radius.
Note:
1) Naturally the circles have to overlap.
2) I try to solve the problem for ARBITRARY polygons. But also solutions for CONVEX polygons are appreciated.
3) As far as Im informed, this problem is NP-hard ( an algorithm to find the minimum size set cover for the Set-cover problem )
Choose: U = polygon and S1...Sk = circles with arbitrary centers.
My solution:
Ive already read some papers and tried a few things on my own. The most promising idea that I came up with was in fact one already indicated in Covering an arbitrary area with circles of equal radius.
So I guess it’s best I quickly try to describe my own idea and then refine my questions.
The picture gives you already a pretty good idea of what I do
IDEA and Problem Formulation
1. I approximate the circles with their corresponding hexagons and tessellate the whole R2, i.e. an sufficiently large area; keyword hexagonally closest packaging. (cyan … tessellation, red dotted, centers of the cyan hexagons)
2. I put the polygon somewhere in the middle of this tessellated area and compute the number of hexagons that are needed to cover the polygon.
In the following Im trying to minimize N, which is number ofhexagons needed to cover the polygon, by moving the polygon around step by step, after each step “counting” N.
Solving the problem:
So that’s when it gets difficult (for me). I don’t know any optimizers that solve this problem properly, since they all terminate after moving the polygon around a bit and not observing any change.
My solution is the following:
First note that this is a periodic problem:
1. The polygon can be moved in horizontal direction x with a period of 3*r (side length = radius r) of the hexagon.
2. The polygon can be moved in vertical direction y with a period of r^2+r^2-2*rrcos(2/3*pi) of the hexagon.
3. The polygon can be rotated phi with a period of 2/3*pi.
That means, one has to search a finite area of possible solutions to find the optimal solution.
So what I do is, I choose a stepsize for (x,y,phi) and simply brute force compute all possible solutions, picking out the optimum.
Refining my questions
1) Is the problem formulated intelligently? Right now im working on an algorithm that only tessellates a very small area, so that as little hexagons as possible have to be computed.
2) Is there a more intelligent optimizer to solve the problem?
3) FINALLY: I also have difficulties finding appropriate literature, since I don’t guess I don’t know the right keywords to look for. So if anybody can provide me with literature, it would also be appreciated a lot.
Actually I could go on about other things ive tried but I think no one of u guys wants to spend the whole afternoon just reading my question.
Thx in advance to everybody who takes the time to think about it.
mat
PS i implement my algorithms in matlab
I like your approach! When you mention your optimization I think a good way to go about it is by rotating the hexagonal grid and translating it till you find the least amount of circles that cover the region. You don't need to rotate 360 since the pattern is symmetric so just 360/6.
I've been working on this problem for a while and have just published a paper that contains code to solve this problem! It uses genetic algorithms and BFGS optimization. You can find a link to the paper here: https://arxiv.org/abs/2003.04839
Edit: Answer rewritten (there's no limitation that circles couldn't go outside the polygon).
You might be interested in Covering a simple polygon with circles. I think the algorithm works or is extendable also to complex polygons.
1.Inscribe the given polygon in a minimum sized rectangle
2.Cover the rectangle optimally by circles (algorithm is available)
I am experimenting with some new ideas in Cocos2D/Box2D on iPhone.
I want to animate a small swarm of fireflies moving on circular (random?) paths... the idea is that the user can capture a firefly with a net..
I have considered using gravity simulations for this but I believe it is over complicating things... my previous experience with using Bezier curves tells me that this isn't the solution either..
Does anyone have any bright insights for me?
Thanks so much.
Do you need the fireflies to collide with each other?
I ask, as if this isn't a requirement, Box2D is probably overkill for your needs. Cocos2d is an excellent choice for this by the sounds of it, but I think you'd be better off looking into flocking algorithms like boids
Even that may be overly complicated. Mix a few sin and cosine terms together with some random scaling factors will likely be enough.
You could have one sin/cosine combination forming an ellipse nearly the size of the screen:
x = halfScreenWidth + cos (t) * halfScreenWidth * randomFactor;
y = halfScreenHeight + sin (t) * halfScreenHeight * randomFactor;
where randomFactor would be something in the realm of 0.6 to 0.9
This will give you broad elliptical motion around the screen, then you could add a smaller sin/cos factor to make them swirl around the point on that ellipse.
By multiplying your time delta (t) by different values (negative and positive) the path of the curve will move in a less geometric way. For example, if you use
x = halfScreenWidth + cos (2*t) * halfScreenWidth * randomFactor;
the ellipse will turn into a figure 8. (i think!)
Hope this helps get you started. Good luck.
One place to look for ideas would be in the domain of artificial life. They have been simulating swarms of entities for a long time. Here is a link for some simple swarm code written in Java that should give you some ideas.
http://www.aridolan.com/ofiles/Download.aspx