Related
I have a very simple feed forward neural network (pytorch)
import torch
import torch.nn.functional as F
import numpy as np
class Net_1(nn.Module):
def __init__(self):
super(Net_1, self).__init__()
self.fc1 = nn.Linear(5*5, 64)
self.fc2 = nn.Linear(64, 32)
self.fc3 = nn.Linear(32, 3)
def forward(self,x):
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return F.log_softmax(x, dim=1)
net = Net_1()
and the input is this 5x5 numpy array
state = [[0, 0, 3, 0, 0],
[0, 0, 0, 0, 0],
[0, 2, 1, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]]
state = torch.Tensor(state).view(-1)
net(state) throws the following error
Dimension out of range (expected to be in range of [-1, 0], but got 1)
the problem is when F.log_softmax() is applied
at the point when you call return F.log_softmax(x, dim=1), x is a 1-dimensional tensor with shape torch.Size([3]).
dimension indexing in pytorch starts at 0, so you cannot use dim=1 for a 1-dimensional tensor, you will need to use dim=0.
replace return F.log_softmax(x, dim=1) with return F.log_softmax(x, dim=0) and you'll be good to go.
in the future you can check tensor sizes by adding print(x.shape) in forward.
You are giving a 3 element 1d array to your log_softmax function.
When saying dim=1 you are telling it to apply softmax to an axis that doesn't exist.
Just set dim=0 for a 1d array.
More on this function and what that parameter means here
I have the following code for drawing a picture:
%scamp data
[flipedBV, flipeddepth] = flipit(depthi, BVsurf_int_ens);
i=linspace(0,5,100);
edges_eps=10.^(-i);
logedg_BV= log10(fliplr(edges_eps));
[n_BV,xout_BV] = histc(histazza(log10(flipedBV)), logedg_BV);
x = logedg_BV;
%model data
[n_BVn,xout_BVn] = histc(histazza(log10(sqrt(-BVsurf_Num_ens))), logedg_BV);
BVsurfF = figure;hold on
h=area(x,n_BV/sum(n_BV),'facecolor',[1 0 0]); %%red area where the problem gonna be
legend('SCAMP')
xlabel('$$ N~[1/s]$$','Interpreter','latex','fontsize',18)
set(gca,'fontsize',14,'ygrid','on')
alpha(.5) %%translucency of the red area
%%add new data
addLineToFig('KEPflu', [0 0 1], BVsurfF, 'BVsurf_Num_ens', logedg_BV);
addLineToFig('KEPflu2', [0 1 1], BVsurfF, 'BVsurf_Num_ens', logedg_BV);
addLineToFig('GASflu', [0 0 0], BVsurfF, 'BVsurf_Num_ens', logedg_BV);
addLineToFig('GASflu2', [1 0 1 ], BVsurfF, 'BVsurf_Num_ens', logedg_BV);
addLineToFig('EPSmin', [1 1 0], BVsurfF, 'BVsurf_Num_ens', logedg_BV);
addLineToFig('GASmin', [.5 .5 0], BVsurfF, 'BVsurf_Num_ens', logedg_BV);
addLineToFig('canuto', [.5 .5 .5]', BVsurfF, 'BVsurf_Num_ens', logedg_BV);
the subroutine addLineToFig consists in:
function addLineToFig(name, ccol, fighandle, variab, x)%, flippo, depthi)
cd(['E:\SIMULATIONS\',name,'\COMPARED\ensamble']);
load([name,'_ensamble'], variab);
[n_BVn, xout_BVn] = histc(histazza(log10(sqrt(-BVsurf_Num_ens))), x); %%new data
figure(fighandle)
[LEGHbv,OBJHbv,OUTHbv,OUTMbv] = legend;
P=plot(x,n_BVn/sum(n_BVn),'color',ccol,'linewidth',2); %%plot new data
legend([OUTHbv;P],OUTMbv{:},name) %%update legend
end
Basically, I create a plot of the red area and then add data with addLineToFig and correctly obtain:
The problem arises when I try to duplicate the figure:
h1=gcf;
h2=figure;
objects=allchild(h1);
copyobj(get(h1,'children'),h2);
set(gca,'yscale','log')
As you can see the translucency of the red distribution is not duplicated and the legend has some problems.
The problem appears to be the last line where I set the yscale to log. If I comment it the code works fine. Does anyone know a workaround?
Minimal code
i=linspace(0,5,100);
edges_eps=10.^(-i);
logedg_BV= log10(fliplr(edges_eps));
a = 1e-5;
b = 1e-2;
r = (b-a).*rand(1000,1) + a;
[n_BV,xout_BV] = histc(histazza(log10(r)), logedg_BV);
x = logedg_BV;
%model data
r2 = (b-a).*rand(1000,1) + a;
[n_BVn,xout_BVn] = histc(histazza(log10(r2)), logedg_BV);
BVsurfF = figure;hold on
h=area(x,n_BV/sum(n_BV),'facecolor',[1 0 0]); %%red area where the problem gonna be
legend('SCAMP')
xlabel('$$ N~[1/s]$$','Interpreter','latex','fontsize',18)
set(gca,'fontsize',14,'ygrid','on')
alpha(.5) %%translucency of the red area
%%add new data
r3 = (b-a).*randn(1000,1) + a;
[n_BVn,xout_BVn] = histc(histazza(log10(r3)), logedg_BV);
figure(BVsurfF)
[LEGHbv,OBJHbv,OUTHbv,OUTMbv] = legend;
P=plot(x,n_BVn/sum(n_BVn),'color','k','linewidth',2);
legend([OUTHbv;P],OUTMbv{:},'data2')
%%add new data
r4 = (b-a).*rand(1000,1) + a;
[n_BVn,xout_BVn] = histc(histazza(log10(r4)), logedg_BV);
figure(BVsurfF)
[LEGHbv,OBJHbv,OUTHbv,OUTMbv] = legend;
P=plot(x,n_BVn/sum(n_BVn),'color','y','linewidth',2);
legend([OUTHbv;P],OUTMbv{:},'data3')
h1=gcf;
h2=figure;
objects=allchild(h1);
copyobj(get(h1,'children'),h2);
I solved the problem by setting the renderer of the original figure to 'OpenGL':
set(BVsurfF,'renderer','OpenGL')
I want to compare two image chunks such that if they are exactly the same, the result must be 1, and if they match 60 percent, the answer must be 0.6.
In Matlab, I can do this using corr2 command, but in python I couldn't find a way. I have tried numpy.corrcoef but it returns a matrix and scipy.signal.correlate2d returns the same.
This is what I have tried:
import numpy as np
import matplotlib.pyplot as plt
from skimage.filter import threshold_otsu
import matplotlib.cm as cm
import Image
import scipy
from PIL import Image as im
fname = 'testi.jpg'
image = Image.open(fname).convert("L")
arr = np.asarray(image)
global_thresh = threshold_otsu(arr)
global_otsu = arr >= global_thresh
global_otsu = np.invert(global_otsu).astype(int)
a1 = global_otsu[80:150,1350:1350+160]
fname1 = 'testi2.jpg'
image1 = Image.open(fname1).convert("L")
arr1 = np.asarray(image1)
global_thresh1 = threshold_otsu(arr1)
global_otsu1 = arr1 >= global_thresh1
global_otsu1 = np.invert(global_otsu1).astype(int)
a2 = global_otsu1[80:150,1350:1350+160]
co = scipy.signal.correlate2d(a1,a2)
plt.gray()
plt.subplot(121)
plt.imshow(a1)
plt.subplot(122)
plt.imshow(a2)
plt.show()
and the result is:
[[0 0 0 ..., 0 0 0]
[0 0 0 ..., 0 0 0]
[0 0 0 ..., 0 0 0]
...,
[0 0 0 ..., 0 0 0]
[0 0 0 ..., 0 0 0]
[0 0 0 ..., 0 0 0]]
These are the images that I want to compare:
Since you want to compare pixel by pixel you can perform correlation on the flattened images, :
cm = np.corrcoef(a1.flat, a2.flat)
cmcontains the symmetric correlation matrix where the off-diagonal element is the correlation coefficient. You get it by
r = cm[0, 1]
Edit:
There is a problem with using correlation for comparing images. If any of them is completely flat (all pixel the same value) the correlation is undefined.
If the images are binary you could instead simply calculate the percantage of equal pixels:
agreement = np.sum(a == b) / a.size
I have this MATLAB code:
d=[1 0 1 1 0]; % Data sequence
b=2*d-1; % Convert unipolar to bipolar
T=1; % Bit duration
Eb=T/2; % This will result in unit amplitude waveforms
fc=3/T; % Carrier frequency
t=linspace(0,5,1000); % discrete time sequence between 0 and 5*T (1000 samples)
N=length(t); % Number of samples
Nsb=N/length(d); % Number of samples per bit
dd=repmat(d',1,Nsb); % replicate each bit Nsb times
bb=repmat(b',1,Nsb); dw=dd'; % Transpose the rows and columns
dw=dw(:)';
% Convert dw to a column vector (colum by column) and convert to a row vector
bw=bb';
bw=bw(:)'; % Data sequence samples
w=sqrt(2*Eb/T)*cos(2*pi*fc*t); % carrier waveform
bpsk_w=bw.*w; % modulated waveform
% plotting commands follow
subplot(4,1,1);
plot(t,dw); axis([0 5 -1.5 1.5])
subplot(4,1,2);
plot(t,bw); axis([0 5 -1.5 1.5])
subplot(4,1,3);
plot(t,w); axis([0 5 -1.5 1.5])
subplot(4,1,4);
plot(t,bpsk_w,'.'); axis([0 5 -1.5 1.5])
xlabel('time')
Which gives me the graphs shown below:
Below is my converted Python Code using Numpy / Scipy
import numpy as np
import scipy
import matplotlib.pylab as plt
plt.clf()
plt.close('all')
d = np.array(np.hstack((1, 0, 1, 1, 0)))
b = 2*d-1.
T = 1
Eb = T/2
fc = 3/T
t = np.linspace(0, 5, 1000)
N = t.shape
Nsb = np.divide(N, d.shape)
dd = np.tile(d.conj().T, Nsb)
bb = np.tile(b.conj().T, Nsb)
dw = dd.conj().T
dw = dw.flatten(0).conj()
bw = bb.conj().T
bw = bw.flatten(0).conj()
w = np.dot(np.sqrt(np.divide(2*Eb, T)), np.cos(np.dot(np.dot(2*np.pi, fc), t)))
bpsk_w = bw*w
plt.subplot(4, 1, 1)
plt.plot(t, dw)
plt.axis(np.array(np.hstack((0, 5, -1.5, 1.5))))
plt.subplot(4, 1, 2)
plt.plot(t, bw)
plt.axis(np.array(np.hstack((0, 5, -1.5, 1.5))))
plt.subplot(4, 1, 3)
plt.plot(t, w)
plt.axis(np.array(np.hstack((0, 5, -1.5, 1.5))))
plt.subplot(4, 1, 4)
plt.plot(t, bpsk_w, '.')
plt.axis(np.array(np.hstack((0, 5, -1.5, 1.5))))
plt.xlabel('time')
plt.show()
But I neither get an error nor the proper output:
Please let me know where is my error in migrating this code?
=====UPDATE======
When I change the Python code to use the following lines, I get some better output:
..............
b = 2.*d-1.
T = 1.
Eb = T/2.
fc = 3./T
...............
w = np.dot(np.sqrt(np.divide(2.*Eb, T)), np.cos(np.dot(np.dot(2.*np.pi, fc), t)))
.............
Your problem stems from using np.tile rather than np.repeat.
To give a simple example of the difference between both:
>>> a = np.arange(3)
>>> a
array([0, 1, 2])
>>> np.repeat(a, 4)
array([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2])
>>> np.tile(a, 4)
array([0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2])
So basically tile takes a "tiling array" and concatenates it, similar to the way you would tile a kitchen floor, whereas repeat repeats each element in the vector a specified number of times before it takes the next element of that vector.
Now, using that knowledge you could rewrite the matlab sample and wind up with the following:
from __future__ import division
import numpy as np
import scipy
import matplotlib.pylab as plt
unipolar_arr = np.array([1, 0, 1, 1, 0])
bipolar = 2*unipolar_arr - 1
bit_duration = 1
amplitude_scaling_factor = bit_duration/2 # This will result in unit amplitude waveforms
freq = 3/bit_duration # carrier frequency
n_samples = 1000
time = np.linspace(0, 5, n_samples)
samples_per_bit = n_samples/unipolar_arr.size # no need for np.divide. Also, use size rather than shape if you want something similar to Matlab's "length"
# 1. Use repeat rather than tile (read the docs)
# 2. No need for conjugate transpose
dd = np.repeat(unipolar_arr, samples_per_bit) # replicate each bit Nsb times
bb = np.repeat(bipolar, samples_per_bit) # Transpose the rows and columns
dw = dd
# no idea why this is here
#dw = dw.flatten(0).conj()
bw = bb # one again, no need for conjugate transpose
# no idea why this is here
#bw = bw.flatten(0).conj()
waveform = np.sqrt(2*amplitude_scaling_factor/bit_duration) * np.cos(2*np.pi * freq * time) # no need for np.dot to perform scalar-scalar multiplication or scalar-array multiplication
bpsk_w = bw*waveform
f, ax = plt.subplots(4,1, sharex=True, sharey=True, squeeze=True)
ax[0].plot(time, dw)
ax[1].plot(time, bw)
ax[2].plot(time, waveform)
ax[3].plot(time, bpsk_w, '.')
ax[0].axis([0, 5, -1.5, 1.5])
ax[0].set_xlabel('time')
plt.show()
I've added more comments to show what is not needed at all (so much clutter, was the code you showed us somehow produced by a conversion program?) and taken the liberty to change most of your 1-2 character variable names into something more readable, that's just one of my pet peeves.
Also, in Python2.x, integer division is the default, so 5/2 will evaluate as 2, rather than 2.5. In Python3.x, this was changed for the better and by using the line from __future__ import division you can get that behaviour in Python2.x as well.
Given a 2x3 matrix x and a 4x2 matrix y, I'd like to use each row of y to index into x. If the value in x is not equal to -1 I'd like to remove that row from y. Here's an example that does what I'd like, except I'd like to do it in a fast, simple way without a loop.
x = [1, 2, 3; -1, 2, -1];
y = [1, 1; 1, 3; 2, 1; 2, 3];
for i=size(y,1):-1:1
if x(y(i,1), y(i,2)) ~= -1
y(i,:) = [];
end
end
This results in:
y =
2 1
2 3
A raw approach to what sub2ind follows (as used by this pretty nice-looking solution posted by Luis) inherently would be this -
y = y(x((y(:,2)-1)*size(x,1)+y(:,1))==-1,:)
Benchmarking
Benchmarking Code
N = 5000;
num_runs = 10000;
x = round(rand(N,N).*2)-1;
y = zeros(N,2);
y(:,1) = randi(size(x,1),N,1);
y(:,2) = randi(size(x,2),N,1);
disp('----------------- With sub2ind ')
tic
for k = 1:num_runs
y1 = y(x(sub2ind(size(x), y(:,1), y(:,2)))==-1,:);
end
toc,clear y1
disp('----------- With raw version of sub2ind ')
tic
for k = 1:num_runs
y2 = y(x((y(:,2)-1)*size(x,1)+y(:,1))==-1,:);
end
toc
Results
----------------- With sub2ind
Elapsed time is 4.095730 seconds.
----------- With raw version of sub2ind
Elapsed time is 2.405532 seconds.
This can be easily vectorized as follows (see sub2ind):
y = y(x(sub2ind(size(x), y(:,1), y(:,2)))==-1,:);
>> x = [1, 2, 3; -1, 2, -1];
>>y = [1, 1;
1, 2;
1, 3;
2, 1;
2, 2;
2, 3];
>>row_idx = reshape((x == -1)',1,6);
>>y = y(row_idx,:);
I think you didn't include all the index of x in y. I included all of them in y. Have a look..
Generalized version:
>> x = [1, 2, 3; -1, 2, -1];
>>y = [1, 1;
1, 2;
1, 3;
2, 1;
2, 2;
2, 3];
>>row_idx = reshape((x == -1)',1,size(x,1)*size(x,2));
>>y = y(row_idx,:);