I have a text file A with N rows and 8 columns (example below):
1 2 0.02 0.28 0.009 0.04 0.03 0.16
2 4 0.04 0.21 0.4 0.04 0.03 0.13
if my variables x=2 and y=4 (I get during the program run -values change with every run.)
I know the index of the row I need to get:
rIndex = find((A{1}==x)&(A{2}==y));
here rindex = 2.
I need to access row at rIndex and do operation with columns 3 to 8 in that row. i.e. i get values from another file based on x and y and divide those values by column values at rIndex row.
x_No = 5, y_No = 6;
B = horzcat((x_No-y_No)./A{3}(1),(x_No-y_No))./A{4}(1),(x_No-y_No)./A{5}(1),(x_No-y_No)./A{6}(1),(x_No-y_No)./A{7}(1),(x_No-y_No)./A{8}(1));
What B does right now is operates on all rows from A. I need only at particular rIndex.
Its still not really clear what happens, but I think you can try something like this:
for r = 1:numel(rIndex)
t=rIndex(r);
result(r,:) = horzcat((x_No-y_No)./A{3}(t),(x_No-y_No))./A{4}(t),(x_No-y_No)./A{5}(t),(x_No-y_No)./A{6}(t),(x_No-y_No)./A{7}(t),(x_No-y_No)./A{8}(t));
end
I may be doing too much actually, but hopefully this helps.
Related
I am currently learning for myself the concept of neural networks and I am working with the very good pdf from
http://neuralnetworksanddeeplearning.com/chap1.html
There are also few exercises I did, but there is one exercise I really dont understand, at least one step
Task:
There is a way of determining the bitwise representation of a digit by adding an extra layer to the three-layer network above. The extra layer converts the output from the previous layer into a binary representation, as illustrated in the figure below. Find a set of weights and biases for the new output layer. Assume that the first 3 layers of neurons are such that the correct output in the third layer (i.e., the old output layer) has activation at least 0.99, and incorrect outputs have activation less than 0.01.
I found also the solution, as can be seen on the second image
I understand why the matrix has to have this shape, but I really struggle to understand the step, where the user calculates
0.99 + 3*0.01
4*0.01
I really don't understand these two steps. I would be very happy if someone can help me understand this calculation
Thank you very much for help
Output of previous layer is 10x1(x). Weight matrix is 4x10. New output layer will be 4x1. There are two assumption first:
x is 1 only at one row. xT= [1 0 0 0 0 0 0 0 0 0]. If you multiple this vector with matrix W your output will be yT=[0 0 0 0], because there is only 1 in x. After multiplication by W will be this only 1 multiple by 0th column of W which are zeroes.
Second assumption is, what if x is not 1 anymore, instead of one x can be xT=[0.99 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01]. And if you perform multiplication of x with first row of W result is 0.05(I believe here is typo). When xT=[0.01 0.99 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01] after multiplication with first row of W result is 1.03. Because:
0.01*0 + 0.99*1 + 0.01*0 + 0.01*1 + 0.01*0 + 0.01*1 + 0.01*0 + 0.01*1 + 0.01*0 + 0.01*1 = 1.03
So I believe there is a typo, because author probably assume 4 ones at first row of W, which is not true, because there is 5 ones. Because if there was 4 ones at first first row, than really results will be 0.04 for 0.99 at first row of x and 1.02 for 0.99 at second row of x.
I have two sets of data from different instruments that have common X-variables (XThompsons) but various Y-variables (YCounts) due to various experimental conditions. The data resemble the example below:
[Table1]
XThompsons | YCounts (1) | YCounts (2) | YCounts (3) | .... | ....
------------------------------------------------------------------
[Table2]
XThompsons | YCounts (1) | YCounts (2) | YCounts (3) | .... | ....
------------------------------------------------------------------
When I have two sets of data that are like this, I have written a script to take a single Y-column information from Table1 and do some math to all Y-columns in Table2. However, when comparing two table columns if either column has a value of a specific threshold (0.10) I want to delete that value. In the example below I want to delete row 4 and row 6 because either column has a value containing 0.10 or less
XThompsons | Table1.YCounts(1) | Table2.YCounts(2)
--------------------------------------------------
1 1.00 0.50
2 0.22 0.12
3 0.29 0.14
4 0.29 0.09 (delete row)
5 0.11 0.49
6 0.02 0.83 (delete row)
How can I carry this out in Matlab? My current code is below; I convert each table row to an array first. How can I make it so that if Y < 0.10 delete the row?
datax = readtable('table1.xls'); % Instrument 1
datay = readtable('table2.xls'); % Instrument 2
SIDATA = [];
for idx=2:width(datay);
% Read the indexed column of datax (instrument 1) then normalize to 1
x = table2array(datax(:,idx));
x = x ./ max(x);
% Read indexed column of datay (instrument 2) and carry out loop
for idy=2:width(datay);
% Normalize y data to 1
y = table2array(datay(:,idy));
y = y ./ max(y);
% Calculate similarity index (SI) at using the datax index for all collision energies for datay
xynum = sum(sqrt(x) .* sqrt(y));
xyden = sqrt(sum(x) .* sum(y));
SIDATA(idy,idx) = (xynum/xyden);
end
end
Help would be appreciated.
Thanks!
Generally when looping through and pruning values you want to increment from the end of the matrix back to one; this way, if you delete any rows, you don't skip. (If you delete row 2, then advance to row 3, you skip the data formerly in row 3).
To me, the easiest way to do this is that if all your data is in one matrix A, with columns Y1 Y2,
APruned = A((A(:,1) > 0.1) & (A(:,2) > 0.1),:)
This takes the A matrix, finds the rows where Y1 > 0.1, finds the rows where Y2 > 0.1, finds the overlap, and then outputs only the rows in A where both of these are true.
You should read about logical indecies for more on this topic
EDIT: It looks like you could also clean up your earlier code using element-wise operations;
A = [datax./max(datax) datay./max(datay)];
I have two tables of different size. I'm using Matlab, it doesn't matter I just need the logic of doing this
the first table contains 8 elements the second one contains 3
0.04 0.1
0.08 0.2
0.12 0.3
0.16
0.20
0.24
0.28
0.32
I want to compare the two tables, repeating same value in the second table while tab1(i) < tab2(i) in order to get two tables of the same size the result must be like this
0.04 0.1
0.08 0.1
0.12 0.2
0.16 0.2
0.20 0.2
0.24 0.3
0.28 0.3
0.32 0.0
i've tried this
for ii=1:sizex1(1)
for jj=1:ssimagefile
if x2imagefile(jj)<=z1(ii)
tab2(ii)=z1(ii)
fprintf(file, '%f %f \n', [ii tab2]');
jj=jj+1;
else
ii=ii+1;
end
end
Here is a Matlaby way to do it:
%// Add zeros to the end of tab2 so that it is as long as tab1
tab3(numel(tab1)) = 0;
%// use bsxfun to find for each number, which rows it will be replicated into
I = bsxfun(#le, tab1, tab2') & bsxfun(#gt, tab1, [-inf,tab2(1:end-1)']);
%// use find to convert the rows from the step before to indexes of where the numbers lie in tab1
[~,c] = find(I);
%// populate tab3 with the repeated numbers from tab2
tab3(1:numel(c)) = tab2(c);
And a simpler way using for loops:
tab3 = zeros(size(tab1));
for row = 1:numel(tab1)
idx = tab2 > tab1(row);
if any(idx)
tab3(row) = min(tab2(idx));
end
end
You could also take the following vectorized approach in case you prefer to avoid bsxfun:
tab2_sorted = sort(tab2); % Sort tab2
tab3=zeros(size(tab1)); % Initialize the new table
% Fill the new table with the values of tab2
tab3(tab1<=tab2_sorted(3))=tab2_sorted(3);
tab3(tab1<=tab2_sorted(2))=tab2_sorted(2);
tab3(tab1<=tab2_sorted(1))=tab2_sorted(1);
I have a data set with effect estimates for different points in time (for 1 month, 2 months, 6 months, 12 months and 18 months) and its standard errors. Now I want to plot the means for each period and the corresponding CIs around the means.
My sample looks like:
effect horizon se
0.03 1 0.2
0.02 6 0.01
0.01 6 0.3
0.00 1 0.4
0.04 18 0.2
0.02 2 0.05
0.01 2 0.02
... ...
The means of the effects for each horizon lead to 5 data points that I want to plot in a line chart together with the confidence intervals. I tried this:
egen means = mean(effect), by(horizon)
line means horizon
But how can I add the symmetric confidence bands? Such that I get something that looks like this:
Not entirely certain that this makes sense statistically, but here's how I might do this:
gen variance = se^2
collapse (mean) effect (sum) SV = variance (count) noobs = effect, by(horizon)
gen se_mean = sqrt(SV*(1/noobs)^2)
gen LB = effect - 1.96*se_mean
gen UB = effect + 1.96*se_mean
twoway (rline LB UB horizon, lpattern(dash dash)) (line effect horizon, lpattern(solid)), yline(0, lcolor(gray))
Which yields:
To get the SE of the mean effects T̅, I am using the formula
V(T̅) = 1/(n2) Σin V(Ti)
(which assumes the covariances of the effects are all zero). I then take the square root to get the SE of T̅.
I have a huge set of data in Excel file sheet 1. The number of columns is fixed (6) but there are lots of rows.
For every 3 rows, I need to pick out the minimum value of 2nd column and save the whole row into Excel file or sheet 2, how am I going to write the script?
item.xls (sheet 1):
0.3 0.5 0.1 0.8 0.4 0.6
0.2 0.4 0.9 0.1 0.9 0.4
0.2 0.3 0.1 0.01 0.2 0.5
0.3 0.5 0.1 0.8 0.01 0.2
0.2 0.2 0.9 0.1 0.2 0.4
0.2 0.3 0.1 0.01 0.3 0.5
.......
In the first 3 rows, the minimum value of 2nd column is 0.3, then write the whole row into sheet 2 of the Excel file.
Then the next 3 rows, the minimum value of 2nd column is 0.2, then write the whole row into sheet 2 of the Excel file.
The result I would like to get is:
item.xls (sheet2):
0.2 0.3 0.1 0.01 % 1st 3 rows, the minimum value is 0.3 in 2nd column
0.2 0.2 0.9 0.1 % 2nd set of 3 rows, the minimum value is 0.2 in 2nd column
...
I will show how to extract the relevant rows from the data in a vectorized manner. I will leave the part of reading/writing excel files to you (it should be straightforward):
%# sample data
data = rand(3*10,6);
%# for each three rows, find the min of second column, and get the row index
[r c] = size(data);
d = permute(reshape(data',[c 3 r/3]),[2 1 3]); %'# split into third dimension
[~,idx] = min(d(:,2,:)); %# find min of col2
idx = squeeze(idx) + (0:3:(r-1))'; %'# adjust indices
%# extract these rows from the data matrix
result = data(idx,:);
Down, some clarification on your problem would help to find a better solution. Based on my interpretation of your question, the following bit may help. I'm generating random test data here.
% the number of test data rows
N = 12;
% generate some random vectors for testing
test1 = rand(N,1);
test2 = rand(N,1);
test3 = rand(N,1);
test4 = rand(N,1);
% create a temporary matrix to store the minimum of every
% 3 row set
final = zeros(N/3,4);
% loop in increments of 3
j = 1;
for k=1:3:N
tmp = test2(k:k+2);
% find the index of the minimum in this 3 row group of the 2nd col
idx = find(tmp<=min(tmp));
% offset idx to index into the original data properly
idx = idx+k-1;
% assign the "row" to the final variable
final(j,:)=[test1(idx) test2(idx) test3(idx) test4(idx)];
j = j+1;
end
% write the full results out at once
xlswrite('test.xls',final);
Try this out and if it's not quite what you are looking for, post a comment to clarify your question.