MATLAB 3D lines are invisible - matlab

Hello I'am suffered from a problem
As you can see I want a draw 3D graph.
Problem is when I draw sphere lines are invisible.
Here is simple version of my source
clear all; close all; clc
n=1;
n_inner_drone=3;
n_outter_drone=2;
length=100;
initial_d = zeros(1,n);
inner_x = zeros(n_inner_drone,n);
inner_y = zeros(n_inner_drone,n);
inner_z = zeros(n_inner_drone,n);
outter_x = zeros(n_outter_drone,n);
outter_y = zeros(n_outter_drone,n);
outter_z = zeros(n_outter_drone,n);
radius= length;
disp('test');
%%%%%%%%%%%%%%%%%%%%%% Sphere
% figure()
% [x, y, z] = sphere;
% h = surfl(x*length, y*length, z*length);
% hSurf = surf(X,Y,Z,'EdgeColor','none','LineStyle','none','FaceLighting','phong');
% set(h, 'FaceAlpha', 0.05)
% surf(x*length, y*length, z*length,
% shading interp
hold on
%%%%%%%%%%%%%%%%%%%%%%%%%
for i=1:n_inner_drone
k=1;
while 1
x_temp= randi([-length, length], 1, 1);
y_temp= randi([-length, length], 1, 1);
z_temp= randi([-length, length], 1, 1);
dist = sqrt(x_temp^2 + y_temp^2 + z_temp^2);
if dist<radius
if i==1
initial_d(k) = dist;
end
inner_x(i,k) = x_temp;
inner_y(i,k) = y_temp;
inner_z(i,k) = z_temp;
k = k+1;
end
if k == n+1, break, end
end
end
ideal_direction_length = ones(1,n);
ideal_direction_length = ideal_direction_length * length;
ideal_direction_length = ideal_direction_length - initial_d;
k=1;
random_x = inner_x(1,:);
random_y = inner_y(1,:);
random_z = inner_z(1,:);
random_moving_distance = zeros(1,n);
moving_distance = 0;
trigger = 0;
while 1
if trigger == 0
direction = randi([1, 6], 1, 1);
trigger = 1;
end
if direction == 1
random_x(k) = random_x(k) + 1;
elseif direction == 2
random_x(k) = random_x(k) - 1;
elseif direction == 3
random_y(k) = random_y(k) + 1;
elseif direction == 4
random_y(k) = random_y(k) - 1;
elseif direction == 5
random_z(k) = random_z(k) + 1;
elseif direction == 6
random_z(k) = random_z(k) - 1;
end
dist = sqrt(random_x(k)^2 + random_y(k)^2 + random_z(k)^2);
moving_distance = moving_distance+1;
%%%%%%%%%% Line
plot3(random_x(n),random_y(n),random_z(n),'k+')
%%%%%%%%%%%%%%%
if dist>radius
random_moving_distance(k) = moving_distance;
k = k+1;
moving_distance = 0;
trigger = 0;
end
if k == n+1, break, end
end
plot3(inner_x(1,n),inner_y(1,n),inner_z(1,n),'r*')
for k=2:n_inner_drone
plot3(inner_x(k,n),inner_y(k,n),inner_z(k,n),'b*')
end
for k=1:n_outter_drone
plot3(outter_x(k,n),outter_y(k,n),outter_z(k,n),'k*')
end
At the first, I suspected I worngly draw lines, but without sphere I can see lines as fig2.
Those anyone who knows about this problem.
Please answer to me and I will very appericiate about it.
Thanks for reading.

I think it is because:
plot3(gravity_x(n),gravity_y(n),gravity_z(n))
is not a line. Its a single point.
plot3(gravity_x(n:n+1),gravity_y(n:n+1),gravity_z(n:n+1))
is a line.

Related

Unable to perform assignment because the left and right sides have a different number of elements. MATLAB ERROR

I try to compute a few matrices using MATLAB. After run this code I get a error "Unable to perform assignment because the left and right sides have a different number of elements."
After simple debugging i get conclusion that after comment X(j) and Y(j) program run correctly.
I looked on some similar problem but I cannot find sth that can help me with above problem
clear all
close all
clc
%stale
a1 = 1;
a2 = 1;
a3 = 1;
a4 = 1;
a5 = 1;
%% Napęd 2
X=0; Y=0; Z=0;
th1 = 0; th2 = 0; th3 = 0; th4 = 0; th5 = 0;
%alfa2 = 1;
alfa2 = 0:(pi/12):(pi);
alfa1 = 0; alfa3 = 0; alfa4 = 0; alfa5 = 0;
d5=1; d4=1; d3=1; d2=1; d1=1;
a3 = 1;
a3 = 0.1:(1/14):1;
for j = 1 : length(alfa2)
X(j) = a5*cos(th1)*cos(th4)*cos(th5) + a5*sin(th1)*sin(alfa2(j))*sin(th4)*cos(th5) - a5*cos(th1)*sin(th4)*sin(th5) + a5*sin(th1)*sin(alfa2(j))*cos(th4)*sin(th5) + d4*sin(th1)*cos(alfa2(j)) + a3*cos(th1) + alfa2*cos(th1);
Y(j) = a5*sin(th1)*sin(th4)*cos(th5) - a5*cos(th1)*sin(alfa2(j))*cos(th4)*cos(th5) - a5*sin(th1)*sin(th4)*sin(th5) - a5*cos(th1)*sin(alfa2(j))*cos(th4)*sin(th5) + d4*sin(th1)*cos(alfa2(j)) + a3*cos(th1) + alfa2*cos(th1);
Z(j) = a5*cos(alfa2(j))*sin(th4)*cos(th5) + a5*cos(alfa2(j))*cos(th4)*sin(th5) -d4*sin(alfa2(j)) + d1;
end
figure(1);
subplot(2, 2, 1); plot3(Y, X, Z); grid on; title("Przestrzen XYZ");
xlabel('y'); ylabel('X'); zlabel('Z')
subplot(2, 2, 2); plot(Y, X); grid on; xlabel('Y'); ylabel('X');
title("Plaszczyzna XY");
subplot(2, 2, 3); plot(X, Z); grid on; xlabel('X'); ylabel('Z');
title("Plaszczyzna XZ");
subplot(2, 2, 4); plot(Y, Z); grid on; xlabel('Y'); ylabel('Z');
title("Plaszczyzna YZ");
This is my workspace:
Workspace
You are using alpha2 in the end of your equations, which is a vector. You likely want alpha2(j)

Matlab - polygon and line intersection

Based on Matlab - Draw angles lines over circle and get the intersecting points
I tried the following code that should find the intersecting points and mark them. Unfortunately, it does not work. Do you have an idea how I can solve it?
Script:
clc;
clear;
close all;
r=1000;
nCircle = 1000;
t = linspace(0,2*pi,nCircle);
xCircle = 0+ r*sin(t);
yCircle = 0+ r*cos(t);
pgon = polyshape(xCircle,yCircle );
line(xCircle,yCircle );
axis equal;
hold on;
nAngles = 45;
lineLength = r+50;
for angle = 0:nAngles:359
xLine(1) = 0;
yLine(1) = 0;
xLine(2) = xLine(1) + lineLength * cosd(angle);
yLine(2) = yLine(1) + lineLength * sind(angle);
plot(xLine, yLine);
lineseg = [xLine; yLine];
intersectCoord = intersect(pgon,lineseg);
scatter(intersectCoord(2,2),intersectCoord(1,2));
end
The update script only check the last generated line but it still does not work with intersect:
clc;
clear;
close all;
r=1000;
nCircle = 1000;
t = linspace(0,2*pi,nCircle);
xCircle = 0+ r*sin(t);
yCircle = 0+ r*cos(t);
objectCircle = rot90([xCircle; yCircle]);
line(xCircle,yCircle);
axis equal;
hold on;
nAngles = 35;
lineLength = r+5;
for angle = 0:nAngles:360
disp(angle);
xLine(1) = 0;
yLine(1) = 0;
xLine(2) = xLine(1) + lineLength * cosd(angle);
yLine(2) = yLine(1) + lineLength * sind(angle);
plot(xLine, yLine);
lineseg = [xLine; yLine];
end
coefficients = polyfit([xLine(1), xLine(2)], [ yLine(1), yLine(2)], 1);
a = coefficients (1);
b = coefficients (2);
for xLineCoord = 0:lineLength
i=xLineCoord+1;
yLineCoord=a*xLineCoord +b ;
yLineCoordArray(i, :) = yLineCoord;
xLineCoordArray(i, :) = i;
end
objectLine = [xLineCoordArray, yLineCoordArray];
C = intersect(objectLine,objectCircle);

matlab generalized cross correlation and ROTH Filter

I have a problem for finding cross correlation of tho signals. No filtere version works but ROTH filter version does not work. Can you tell me the problem please ? Thanks
tho=2;
A=1;
t=-15:0.1:15;
FFTLength=length(t);
x= rect( t+1,tho,A ); % First rectangle signal
plot(t,x)
hold on
y=rect(t-4,tho,A);
plot(t,y,'r')
for i=1:length(x)
x(i)=x(i)+normrnd(0,0.1);
end
figure
plot(t,x,'r')
for i=1:length(y)
y(i)=y(i)+normrnd(0,0.1);
end
figure
plot(t,y,'r')
Rxx = xcorr(x);
Ryy = xcorr(y);
Rxy = xcorr(x,y);
Sxx = fft(Rxx,FFTLength);
Syy = fft(Ryy,FFTLength);
Sxy = fft(Rxy,FFTLength);
X=fft(x,301);
Y=fft(y,301);
%Filtering
H=1;
X2=X.*H;
Y2=Y.*H;
x_t=ifft(X2);
y_t=ifft(Y2);
[correlation,lags] = xcorr(y_t,x_t);
delay = lags(find(correlation==max(correlation)))
disp('Plain time ')
X2=fft(x).*(1./Sxx);
Y2=fft(Y).*(1./Sxx);
x2=real(ifft(X2));
y2=real(ifft(Y2));
[correlation2,lags2] = xcorr(x2,y2);
delay2 = lags2(find(correlation2==max(correlation2)))
disp('ROTH Filter ');
by the way rect is ;
function [ y ] = rect( t,tho,A )
for i = 1 : length(t)
if -(tho / 2) <= t(i) && t(i) <= (tho / 2)
y(i) = A * 1;
else
y(i) = 0;
end
end
end

Finger peak detection using MATLAB

I have to create an algorithm with Matlab that, with a image of a hand, can know the form of the hand by the number of raised fingers and the presence or absence of the thumb. So far, the algorithm is almost complete but I don't know what more I can do that could find the peaks that represents the fingers. We tried a lot of things but nothing works. The idea is to find when there is a sudden increasement but as the pixels are never completely aligned, nothing that we tried worked. Someone has any idea? Here is the code so far.
The image that he is reading is this one:
To know if the finger is relevant or not, we already have an idea that might work... but we need to find the fingers first.
clear all
close all
image=imread('mao2.jpg');
YCBCR = rgb2ycbcr(image);
image=YCBCR;
cb = image(:,:,2);
cr = image(:,:,3);
imagek(:,1) = cb(:);
imagek(:,2) = cr(:);
imagek = double(imagek);
[IDX, C] = kmeans(imagek, 2, 'EmptyAction', 'singleton');
s=size(image);
IDX= uint8(IDX);
C2=round(C);
imageNew = zeros(s(1),s(2));
temp = reshape(IDX, [s(1) s(2)]);
for i = 1 : 1 : s(1)
for j = 1 : 1 : s(2)
imageNew(i,j,:) = C2(temp(i,j));
end
end
imageNew=uint8(imageNew);
[m,n]=size(imageNew);
for i=1:1:m
for j = 1:1:n
if(imageNew(i,j)>=127)
pretobranco(i,j)=0;
else
pretobranco(i,j)=1;
end
end
end
I2=imfill(pretobranco);
imshow(I2);
imwrite(I2, 'mao1trab.jpg');
[m,n]=size(I2);
B=edge(I2);
figure
imshow(B);
hold on;
stats=regionprops(I2,'BoundingBox');
rect=rectangle('position', [stats(1).BoundingBox(1), stats(1).BoundingBox(2), stats(1).BoundingBox(3), stats(1).BoundingBox(4)], 'EdgeColor', 'r');
stats(1).BoundingBox(1)
stats(1).BoundingBox(2)
stats(1).BoundingBox(3)
stats(1).BoundingBox(4)
figure
Bound = B( stats(1).BoundingBox(2): stats(1).BoundingBox(2)+stats(1).BoundingBox(4)-1, stats(1).BoundingBox(1):stats(1).BoundingBox(1)+stats(1).BoundingBox(3)-1);
imshow(Bound)
y1 = round(stats(1).BoundingBox(2))
y2 = round(stats(1).BoundingBox(2)+stats(1).BoundingBox(4)-1)
x1 = round(stats(1).BoundingBox(1))
x2 = round(stats(1).BoundingBox(1)+stats(1).BoundingBox(3)-1)
% Bounding box contida em imagem[M, N].
[M,N] = size(Bound)
vertical=0;
horizontal=0;
if M > N
vertical = 1 %imagem vertical
else
horizontal = 1 %imagem horizontal
end
%Find thumb
MaoLeft = 0;
MaoRight = 0;
nPixelsBrancos = 0;
if vertical==1
for i = x1:1:x2
for j= y1:1:y2
if I2(j,i) == 1
nPixelsBrancos = nPixelsBrancos + 1; %Numero de pixels da mão
end
end
end
for i=x1:1:x1+30
for j=y1:1:y2
if I2(j,i) == 1
MaoLeft = MaoLeft + 1; %Number of pixels of the hand between the 30 first colums
end
end
end
for i=x2-30:1:x2
for j=y1:1:y2
if I2(j,1) == 1
MaoRight = MaoRight + 1; %Number of pixels of the hand between the 30 last colums
end
end
end
TaxaBrancoLeft = MaoLeft/nPixelsBrancos
TaxaBrancoRight = MaoRight/nPixelsBrancos
if TaxaBrancoLeft <= (7/100)
if TaxaBrancoRight <= (7/100)
Thumb = 0 %Thumb in both borders is defined as no Thumb.
else
ThumbEsquerdo = 1 %Thumb on left
end
end
if TaxaBrancoRight <= (7/100) && TaxaBrancoLeft >= (7/100)
ThumbDireito = 1 %Thumb on right
end
end
if horizontal==1
for i = x1:1:x2
for j= y1:y2
if I2(i,j) == 1
nPixelsBrancos = nPixelsBrancos + 1; %Numero de pixels da mão
end
end
end
for i=x1:1:x2
for j=y1:1:y1+30
if I2(i,j) == 1
MaoLeft = MaoLeft + 1; %Numero de pixels da mão entre as 30 primeiras colunas
end
end
end
for i=x1:1:x2
for j=y2-30:1:y2
if I2(j,1) == 1
MaoRight = MaoRight + 1; %Numero de pixels da mão entre as 30 ultimas colunas
end
end
end
TaxaBrancoLeft = MaoLeft/nPixelsBrancos
TaxaBrancoRight = MaoRight/nPixelsBrancos
if TaxaBrancoLeft <= (7/100)
if TaxaBrancoRight <= (7/100)
Thumb = 0 %Polegar nas duas bordas. Definimos como sem polegar.
else
ThumbEsquerdo = 1 %Polegar na borda esquerda
end
end
if TaxaBrancoRight <= (7/100) && TaxaBrancoLeft >= (7/100)
ThumbDireito = 1 %Polegar na borda direita
end
end
figure
imshow(I2);
%detecção da centroid
Ibw = im2bw(I2);
Ilabel = bwlabel(Ibw);
stat = regionprops(Ilabel,'centroid');
figure
imshow(I2); hold on;
for x = 1: numel(stat)
plot(stat(x).Centroid(1),stat(x).Centroid(2),'ro');
end
centroid = [stat(x).Centroid(1) stat(x).Centroid(2)] %coordenadas x e y da centroid
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Seemed like an interesting problem, so I gave it a shot. Basically you start with a Sobel filter to find the edges in your image (after slight denoising). Then clean up the resulting lines, use them to separate regions within your binary mask of the hand, use a watershed transform to find the wrist, some distance transforms to find other landmarks, then remove the palm. What you're left with is separate regions for each finger and thumb. You can count those regions easily enough or find which way they are pointing, or whatever you'd like.
imgURL = 'https://encrypted-tbn2.gstatic.com/imgs?q=tbn:ANd9GcRQsqJtlrOnSbJNTnj35Z0uG9BXsecX2AXn1vV0YDKodq-zSuqnnQ';
imgIn=imread(imgURL);
gaussfilt = fspecial('gaussian', 3, .5); % Blur starting image
blurImg = imfilter(double(img(:,:,1)), gaussfilt);
edgeImg = edge(blurImg, 'sobel'); % Use Sobel edge filter to pick out contours of hand + fingers
% Clean up contours
edgeImg = bwmorph(edgeImg, 'close', 1);
edgeImg = bwmorph(edgeImg, 'thin', Inf);
% Clean up rogue spots in corners
edgeImg([2 end-1], 2) = 0;
edgeImg([2 end-1], end-1) = 0;
% Extend lines to edge of image (correct for 'close' operation above
edgeImg([1 end],:) = edgeImg([2 end-1],:);
edgeImg(:, [1 end]) = edgeImg(:, [2 end-1]);
% Remove all but the longest line
regs = regionprops(edgeImg, 'Area', 'PixelIdxList');
regs(vertcat(regs.Area) ~= max(vertcat(regs.Area))) = [];
lineImg = false(size(edgeImg, 1), size(edgeImg, 2));
lineImg(regs.PixelIdxList) = 1;
fillImg = edgeImg;
% Close in wrist
if any(fillImg(1,:))
fillImg(1,:) = 1;
end
if any(fillImg(end,:))
fillImg(end,:) = 1;
end
if any(fillImg(:,1))
fillImg(:,1) = 1;
end
if any(fillImg(:,end))
fillImg(:,end) = 1;
end
fillImg = imfill(fillImg, 'holes');
fillImg([1 end], :) = 0;
fillImg(:, [1 end]) = 0;
fillImg([1 end],:) = fillImg([2 end-1],:);
fillImg(:, [1 end]) = fillImg(:, [2 end-1]);
% Start segmenting out hand + fingers
handBin = fillImg;
% Set lines in above image to 0 to separate closely-spaced fingers
handBin(lineImg) = 0;
% Erode these lines to make fingers a bit more separate
handBin = bwmorph(handBin, 'erode', 1);
% Segment out just hand (remove wrist)
distImg = bwdist(~handBin);
[cDx, cDy] = find(distImg == max(distImg(:)));
midWrist = distImg;
midWrist = max(midWrist(:)) - midWrist;
midWrist(distImg == 0) = Inf;
wristWatershed = watershed(imerode(midWrist, strel('disk', 10)));
whichRegion = wristWatershed(cDx, cDy);
handBin(wristWatershed ~= whichRegion) = 0;
regs = regionprops(handBin, 'Area', 'PixelIdxList');
regs(vertcat(regs.Area) ~= max(vertcat(regs.Area))) = [];
handOnly = zeros(size(handBin, 1), size(handBin, 2));
handOnly(regs.PixelIdxList) = 1;
% Find radius of circle around palm centroid that excludes wrist and splits
% fingers into separate regions.
% This is estimated as D = 1/3 * [(Centroid->Fingertip) + 2*(Centroid->Wrist)]
% Find Centroid-> Wrist distance
dist2w = wristWatershed ~= whichRegion;
dist2w = bwdist(dist2w);
distToWrist = dist2w(cDx, cDy);
% Find Centroid-> Fingertip distance
dist2FE = zeros(size(handOnly, 1), size(handOnly, 2));
dist2FE(cDx, cDy) = 1;
dist2FE = bwdist(dist2FE).*handOnly;
distToFingerEnd = max(dist2FE(:));
circRad = mean([distToFingerEnd, distToWrist, distToWrist]); % Estimage circle diameter
% Draw circle
X = bsxfun(#plus,(1:size(handOnly, 1))',zeros(1,size(handOnly, 2)));
Y = bsxfun(#plus,(1:size(handOnly, 2)),zeros(size(handOnly, 1),1));
B = sqrt(sum(bsxfun(#minus,cat(3,X,Y),reshape([cDx, cDy],1,1,[])).^2,3))<=circRad;
% Cut out binary mask within circle
handOnly(B) = 0;
% Label separate regions, where each now corresponds to a separate digit
fingerCount = bwlabel(handOnly);
% Display overlay image
figure()
imshow(imgIn)
hold on
overlayImg = imshow(label2rgb(fingerCount, 'jet', 'k'));
set(overlayImg, 'AlphaData', 0.5);
hold off
Results:
http://imgur.com/ySn1fPy

a variation of Windy gridworld game problem in reinforcement learning with my matlab code

In reinforcement learning, a typical example is the windy gridworld
And I face with a new variation of windy gridworld, which additionally has a wall and stochastic wind, I am stuck in these two new things
Figure 1 shows a standard gridworld, with start (S) and goal (G) cells, but with two dierences: there
is a wall the agent can not cross (indicated by the black cells) and there is a crosswind down and to the
left at the right edge of the grid. The available actions in each cell are the king's moves | 8 actions in
total for each cell. If any action would bring you outside the gridworld or collide with the wall, you end
up in the nearest cell (e.g. going northeast in the top left cell will bring you one cell to the right). In the
right region the resultant next cells are shifted down-left by a stochastic \wind", the mean strength of
which varies column by column. The mean strength of the wind is given below each column, in number
of cells shifted down-left.
Due to stochasticity, the wind sometimes varies by 1 from the mean values
given for each column (except if the mean is 0). That is, a third of the time you are shifted down-left
exactly according to the values indicated below the column, a third of the time you are shifted one cell
further down and left that, and another third of the time you are shifted one cell less than the mean.
For example, if you are in the row of the wall and in the middle of the opening and you move up, then
one-third of the time you end up one row west of that cell, one-third of the time you end up two colums
west, one column south of that cell, and one-third of the time you end up at the same column north of
that cell. The wind aects the cell you are in, not the cell you are going to.
Implement the Q-learning algorithm2 in the above problem with = 0:1,
= 0:9 and initial Q(s; a) = 0
for all s; a. Each action generates a reward of rs = 􀀀1, except for the actions that lead immediately to
the goal cell (rg = 10). Use the:
-Greedy action selection method with = 0:2.
Greedy action selection method with initial Q(s,a) > 0 and initial Q(s,a) < 0.
my matlab code will work.
My real problem is on the
function nextPos = GiveNextPos(curPos, actionIndex, windpowers, gridCols, gridRows), in which the agent will decide a action, and move to the next step. But there are many factors to influence the next step, such as stochastic wind and wall
so first question is about the stochastic wind, how can I program in matlab to say in 1/3 chance, it is 3, in another 1/3 chance, it is 1...
the second question is about colliding wall?should I firstly calculate the next step for king's walk and wind, and then use this next step value to check if I hit the wall or not???)
function WindyGridWorldQLearning()
fprintf('WindyGridWorldQLearning\n');
gamma = 0.9;
alpha = 0.1;
epsilon = 0.2;
gridcols = 10;
gridrows = 7;
windpowers = [0 0 0 0 1 1 2 2 1 1];
fontsize = 16;
showTitle = 1;
episodeCount = 900;
selectedEpisodes = [900];
isKing = 1;
canHold = 0;
start.row = 7;
start.col = 1;
goal.row = 1;
goal.col = 1;
selectedEpIndex = 1;
actionCount = 8;
% initialize Q with zeros
Q = zeros(gridrows, gridcols, actionCount);
a = 0; % an invalid action
% loop through episodes
for ei = 1:episodeCount,
%disp(sprintf('Running episode %d', ei));
curpos = start;
nextpos = start;
%epsilon or greedy
if(rand > epsilon) % greedy
[qmax, a] = max(Q(curpos.row,curpos.col,:));
else
a = IntRand(1, actionCount);
end
while(PosCmp(curpos, goal) ~= 0)
% take action a, observe r, and nextpos
nextpos = GiveNextPos(curpos, a, windpowers, gridcols, gridrows);
if(PosCmp(nextpos, goal) ~= 0), r = -1; else r = 10; end
% choose a_next from nextpos
[qmax, a_next] = max(Q(nextpos.row,nextpos.col,:));
if(rand <= epsilon) % explore
a_next = IntRand(1, actionCount);
end
% update Q:
curQ = Q(curpos.row, curpos.col, a);
nextQ = qmax; %Q(nextpos.row, nextpos.col, a_next);
Q(curpos.row, curpos.col, a) = curQ + alpha*(r + gamma*nextQ - curQ);
curpos = nextpos; a = a_next;
end % states in each episode
% if the current state of the world is going to be drawn ...
if(selectedEpIndex <= length(selectedEpisodes) && ei == selectedEpisodes(selectedEpIndex))
curpos = start;
rows = []; cols = []; acts = [];
for i = 1:(gridrows + gridcols) * 10,
[qmax, a] = max(Q(curpos.row,curpos.col,:));
nextpos = GiveNextPos(curpos, a, windpowers, gridcols, gridrows);
rows = [rows curpos.row];
cols = [cols curpos.col];
acts = [acts a];
if(PosCmp(nextpos, goal) == 0), break; end
curpos = nextpos;
end % states in each episode
%figure;
figure('Name',sprintf('Episode: %d', ei), 'NumberTitle','off');
DrawWindyEpisodeState(rows, cols, acts, start.row, start.col, goal.row, goal.col, windpowers, gridrows, gridcols, fontsize);
if(showTitle == 1),
title(sprintf('Windy grid-world SARSA - episode %d - (\\epsilon: %3.3f), (\\alpha = %3.4f), (\\gamma = %1.1f)', ei, epsilon, alpha, gamma));
end
selectedEpIndex = selectedEpIndex + 1;
end
end % episodes loop
function c = PosCmp(pos1, pos2)
c = pos1.row - pos2.row;
if(c == 0)
c = c + pos1.col - pos2.col;
end
function nextPos = GiveNextPos(curPos, actionIndex, windpowers, gridCols, gridRows)
nextPos = curPos;
switch actionIndex
case 1 % east
nextPos.col = curPos.col + 1;
case 2 % south
nextPos.row = curPos.row + 1;
if(nextPos.row ==4 && nextPos.col <= 4 ) nextPos.row = curPos.row; end
case 3 % west
nextPos.col = curPos.col - 1;
if(nextPos.row ==4 && nextPos.col <= 4 ) nextPos.col = curPos.col; end
case 4 % north
nextPos.row = curPos.row - 1;
if(nextPos.row ==4 && nextPos.col <= 4 ) nextPos.row = curPos.row; end
case 5 % northeast
nextPos.col = curPos.col + 1;
nextPos.row = curPos.row - 1;
if(nextPos.row ==4 && nextPos.col <= 4 ) nextPos.row = curPos.row; end
case 6 % southeast
nextPos.col = curPos.col + 1;
nextPos.row = curPos.row + 1;
if(nextPos.row ==4 && nextPos.col <= 4 ) nextPos.row = curPos.row; end
case 7 % southwest
nextPos.col = curPos.col - 1;
nextPos.row = curPos.row + 1;
if(nextPos.row ==4 && nextPos.col <= 4 ) nextPos.row = curPos.row; end
case 8 % northwest
nextPos.col = curPos.col - 1;
nextPos.row = curPos.row - 1;
if(nextPos.row ==4 && nextPos.col <= 4 ) nextPos.row = curPos.row; end
case 9 % hold
nextPos = curPos;
otherwise
disp(sprintf('invalid action index: %d', actionIndex))
end
if(curPos.col > 4)
nextPos.row = nextPos.row - windpowers(nextPos.col);
nextPos.col = nextPos.col - windpowers(nextPos.col);
end
if(nextPos.col <= 0), nextPos.col = 1; end
if(nextPos.col > gridCols), nextPos.col = gridCols; end
if(nextPos.row <= 0), nextPos.row = 1; end
if(nextPos.row > gridRows), nextPos.row = gridRows; end
function n = IntRand(lowerBound, upperBound)
n = floor((upperBound - lowerBound) * rand + lowerBound);
function DrawWindyEpisodeState(rows, cols, acts, SRow, SCol, GRow, GCol, windpowers, gridrows, gridcols, fontsize)
DrawGrid(gridrows, gridcols);
DrawTextOnCell('S', 0, SRow, SCol, gridrows, gridcols, fontsize);
DrawTextOnCell('G', 0, GRow, GCol, gridrows, gridcols, fontsize);
for i=1:length(rows),
DrawActionOnCell(acts(i), rows(i), cols(i), gridrows, gridcols, fontsize);
end
for i=1:gridcols,
[xc, yc] = FindColBaseCenter(i, gridrows, gridcols);
text(xc, yc, sprintf('%d',windpowers(i)), 'FontSize', fontsize, 'Rotation', 0);
end
function DrawEpisodeState(rows, cols, acts, SRow, SCol, GRow, GCol, gridrows, gridcols, fontsize)
DrawGrid(gridrows, gridcols);
DrawTextOnCell('S', 0, SRow, SCol, gridrows, gridcols, fontsize);
DrawTextOnCell('G', 0, GRow, GCol, gridrows, gridcols, fontsize);
for i=1:length(rows),
DrawActionOnCell(acts(i), rows(i), cols(i), gridrows, gridcols, fontsize);
end
function DrawGrid(gridrows, gridcols)
xsp = 1 / (gridcols + 2);
ysp = 1 / (gridrows + 2);
x = zeros(1, 2*(gridcols + 1));
y = zeros(1, 2*(gridcols + 1));
i = 1;
for xi = xsp:xsp:1 - xsp,
x(2*i - 1) = xi; x(2*i) = xi;
if(mod(i , 2) == 0)
y(2*i - 1) = ysp;y(2*i) = 1-ysp;
else
y(2*i - 1) = 1 - ysp;y(2*i) = ysp;
end
i = i + 1;
end
x2 = zeros(1, 2*(gridrows + 1));
y2 = zeros(1, 2*(gridrows + 1));
i = 1;
for yi = ysp:ysp:1 - ysp,
y2(2*i - 1) = yi; y2(2*i) = yi;
if(mod(i , 2) == 0)
x2(2*i - 1) = xsp;x2(2*i) = 1-xsp;
else
x2(2*i - 1) = 1 - xsp;x2(2*i) = xsp;
end
i = i + 1;
end
plot(x, y, '-');
hold on
plot(x2, y2, '-');
axis([0 1 0 1]);
axis off
set(gcf, 'color', 'white');
function DrawTextOnCell(theText, rotation, row, col, gridrows, gridcols, fontsize)
[xc, yc] = FindCellCenter(row, col, gridrows, gridcols);
text(xc, yc, theText, 'FontSize', fontsize, 'Rotation', rotation);
function DrawActionOnCell(actionIndex, row, col, gridrows, gridcols, fontsize)
rotation = 0;
textToDraw = 'o';
switch actionIndex
case 1 % east
textToDraw = '\rightarrow';
rotation = 0;
case 2 % south
textToDraw = '\downarrow';
rotation = 0;
case 3 % west
textToDraw = '\leftarrow';
rotation = 0;
case 4 % north
textToDraw = '\uparrow';
rotation = 0;
case 5 % northeast
textToDraw = '\rightarrow';
rotation = 45;
case 6 % southeast
textToDraw = '\downarrow';
rotation = 45;
case 7 % southwest
textToDraw = '\leftarrow';
rotation = 45;
case 8 % northwest
textToDraw = '\uparrow';
rotation = 45;
otherwise
disp(sprintf('invalid action index: %d', actionIndex))
end
DrawTextOnCell(textToDraw, rotation, row, col, gridrows, gridcols, fontsize);
function [x,y] = FindCellCenter(row, col, gridrows, gridcols)
xsp = 1 / (gridcols + 2);
ysp = 1 / (gridrows + 2);
x = ((2*col + 1) / 2) * xsp;
y = 1 - (((2*row + 1) / 2) * ysp);
x = x - xsp/5;
function [x,y] = FindColBaseCenter(col, gridrows, gridcols)
row = gridrows + 1;
xsp = 1 / (gridcols + 2);
ysp = 1 / (gridrows + 2);
x = ((2*col + 1) / 2) * xsp;
y = 1 - (((2*row + 1) / 2) * ysp);
x = x - xsp/5;
For the wind just generate a random number n, say between 0 and 1. If you want 3 different behaviors each with a 1/3 chance, just have conditions for n < .33 , .33 < n < .66 ... etc.
I don't quite understand what you're saying with the wall, but you should check the action the agent will take and the effect the wind will have on it and then see if this results in you hitting a wall. If so take the appropriate action.