I have some data which look like this:
I would like to pre-process the data in a way that I replace all Mostly false with 1, Mostly true with 2 and Definitely true w/ 3. Is there a find and replace command or what is the best way of doing this?
You can use a map object to do the mapping
m = containers.Map( {'Mostly false', 'Mostly true', 'Definitely true'}, ...
{ 1, 2, 3} );
Then for some example data
data = {'Mostly false', 'Mostly false', 'Mostly true', 'Mostly false', 'Definitely true'};
You can perform the conversion with
data = m.values( data );
% >> data = {1, 1, 2, 1, 3}
This assumes there will always be a match in your map.
Alternatively, you could do the operation manually (for the same example data), this will leave non-matches unaltered, and you could use strcmpi for case-insensitivity:
c = {'Mostly false', 'Mostly true', 'Definitely true'
1, 2, 3};
for ii = 1:size(c,2)
% Make the replacement for each column in 'c'
data( strcmp( data, c{1,ii} ) ) = c(2,ii);
end
Related
There is an array with indices [[0, n_0], [1, n_1], ..., [n, n_n]]. For each n_i a function is called. It is necessary to reorder the result from the threads by first component after every thread has terminated. As far as I could find a way to do this, I organized that the index is hard-coded by asking if the index is e.g. 0 and then starting the code separately for the hard-coded index 0. So far this a possible way to do it (even though the code looks as if someone didn't understand what a loop is for).
rest = []
tpl.each do |idx, vn|
if idx == 0
pool.post do
res = funk(vn)
p ['idx 0: ', res]
rest += [[0, res]]
end#pool.post
elsif idx == 1
pool.post do
res = funk(vn)
p ['idx 1: ', res]
rest += [[1, res]]
end#pool.post
end;end
But now there is a strange behaviour:
Index 0 and 1 are calculated accurately, but when the result of 1 is added one line later, the result of the former function is added (again).
["idx 1: ", [4]]
["idx 0: ", [16900]]
rest: [[0, [16900]], [1, [16900], ...]
This is not always the case, so it depends on the order of the appearance of the results.
If e.g. the calculation of index 0 is finished after the calculation of index 1, then idx 1 is missing, or wrong. But other cases of confused results also appear: idx 0 before idx 1, but result of idx 0 is the result of idx 1.
?
It looks like if the threads are not really separated. Can that be enforced, or is there a smarter way of keeping indeces?
One option, I found out, is to synchronize the threads, but that would make the algorithm slower again, so a better solution is:
The results don't get mixed up, if the rest-tuple already has the structure to differentiate the results coming in:
rest = [[], []]
tpl.each do |idx, vn|
if idx == 0
pool.post do
res = funk(vn)
p ['idx 0: ', res]
rest[0] << [0, res]
end#pool.post
elsif idx == 1
pool.post do
res = funk(vn)
p ['idx 1: ', res]
rest[1] << [1, res]
end#pool.post
end;end
I have a simple convolutional neural network, whose output is a single channel 4x4 feature map. During training, the (regression) loss needs to be computed only on a single value among the 16 outputs. The location of this value will be decided after the forward pass. How do I compute the loss from just this one output, while making sure all irrelevant gradients are zero'ed out during back-prop.
Let's say I have the following simple model in torch:
require 'nn'
-- the input
local batch_sz = 2
local x = torch.Tensor(batch_sz, 3, 100, 100):uniform(-1,1)
-- the model
local net = nn.Sequential()
net:add(nn.SpatialConvolution(3, 128, 9, 9, 9, 9, 1, 1))
net:add(nn.SpatialConvolution(128, 1, 3, 3, 3, 3, 1, 1))
net:add(nn.Squeeze(1, 3))
print(net)
-- the loss (don't know how to employ it yet)
local loss = nn.SmoothL1Criterion()
-- forward'ing x through the network would result in a 2x4x4 output
y = net:forward(x)
print(y)
I have looked at nn.SelectTable and it seems like if I convert the output into tabular form I would be able to implement what I want?
This is my current solution. It works by splitting the output into a table, and then using nn.SelectTable():backward() to get the full gradient:
require 'nn'
-- the input
local batch_sz = 2
local x = torch.Tensor(batch_sz, 3, 100, 100):uniform(-1,1)
-- the model
local net = nn.Sequential()
net:add(nn.SpatialConvolution(3, 128, 9, 9, 9, 9, 1, 1))
net:add(nn.SpatialConvolution(128, 1, 3, 3, 3, 3, 1, 1))
net:add(nn.Squeeze(1, 3))
-- convert output into a table format
net:add(nn.View(1, -1)) -- vectorize
net:add(nn.SplitTable(1, 1)) -- split all outputs into table elements
print(net)
-- the loss
local loss = nn.SmoothL1Criterion()
-- forward'ing x through the network would result in a (2)x4x4 output
y = net:forward(x)
print(y)
-- returns the output table's index belonging to specific location
function get_sample_idx(feat_h, feat_w, smpl_idx, feat_r, feat_c)
local idx = (smpl_idx - 1) * feat_h * feat_w
return idx + feat_c + ((feat_r - 1) * feat_w)
end
-- I want to back-propagate the loss of this sample at this feature location
local smpl_idx = 2
local feat_r = 3
local feat_c = 4
-- get the actual index location in the output table (for a 4x4 output feature map)
local out_idx = get_sample_idx(4, 4, smpl_idx, feat_r, feat_c)
-- the (fake) ground-truth
local gt = torch.rand(1)
-- compute loss on the selected feature map location for the selected sample
local err = loss:forward(y[out_idx], gt)
-- compute loss gradient, as if there was only this one location
local dE_dy = loss:backward(y[out_idx], gt)
-- now convert into full loss gradient (zero'ing out irrelevant losses)
local full_dE_dy = nn.SelectTable(out_idx):backward(y, dE_dy)
-- do back-prop through who network
net:backward(x, full_dE_dy)
print("The full dE/dy")
print(table.unpack(full_dE_dy))
I would really appreciate it somebody points out a simpler OR more efficient method.
Is it possible to retrieve object properties that were saved in MATLAB with -v7.3 in Python?
For example, take this class:
classdef TestClass < handle
properties
Name = 'Nobody';
Value = 42;
end
methods
function this = TestClass(name, value)
if nargin > 0
this.Name = name;
end
if nargin > 1
this.Value = value;
end
end
function doSomething(this)
fprintf('My name is â%sâ and my value is â%dâ.\n', this.Name, this.Value);
end
end
end
and create and save a few objects:
a=TestClass('Theo', 17)
b=TestClass
c=[a,b]
d={a,b}
save('testClassF.mat', 'a', 'b', 'c', 'd')
Using hdf5 in Python, I can see that a is represented as
array([3707764736, 2, 1, 1, 1, 1], dtype=uint32)
and b is
array([3707764736, 2, 1, 1, 1, 2], dtype=uint32).
A little more digging shows that '#refs#/e' yields a's Name, and '#refs#/f' yields its Value. But where does the mapping occur?
Hi I have the following looking dataset:
[
{ date:"somedatehere", series1:"series1Value", series2:"series2Value" ..., seriesX:"seriesXValue"},
{ date:"anotherDateHere", series1:"anotherseries1Value", series2:"anotherseries2Value"...,seriesX:"anotherseriesXValue"},...
]
I'd like to loop through this in coffeescript and extract arrays such that I would have an array of dates (comprised of somedatehere, anotherDateHere, etc), series1 values, series2 values, seriesX values, etc.
Preferrably all of these arrays would go in order such that dates[0] === somedatehere and series1[0] === series1Value and series2[0] === series2Value and seriesX[1] === anotherseriesXValue etc.
Is there an easy way to go about doing this in coffeescript?
dates = (obj.date for obj in my_array)
series1 = (obj.series for obj in my_array)
in case you have a lot of series and don't want to manually enumerate them:
types = (k for k, v of my_array[0])
result = {}
result[type] = (obj[type] for obj in my_array) for type in types
Will give you
my_array = [{date: 1, x: 2}, {date: 123, x: 2134}]
result = {
date: [ 1, 123 ],
x: [ 2, 2134 ]
}
So when the index is 0, I want to print it out:
a = [ 1, 2, 3 ]
for i of a
if i == 0
console.log a[i]
But there is no output.
i == 0 is never true...
i returns the index as a string, if you parse them as an integer, it would work
a = [ 1, 2, 3 ]
for i of a
if parseInt(i) == 0
console.log a[i]
It's because i will only be 1, 2 or 3, as you loop over the items in a, not the index numbers.
This works the way you described above:
a = [ 1, 2, 3 ]
for i in [0..a.length]
if i == 0
console.log a[i]
You shouldn't use of to loop over an array, you should use in. From the fine manual:
Comprehensions can also be used to iterate over the keys and values in an object. Use of to signal comprehension over the properties of an object instead of the values in an array.
yearsOld = max: 10, ida: 9, tim: 11
ages = for child, age of yearsOld
"#{child} is #{age}"
So you're trying to iterate over the properties of an array object, not its indexes.
You should use one of these for your loop:
for e, i in a
if(i == 0)
console.log(a[i])
for e, i in a
console.log(e) if(i == 0)
console.log(e) for e, i in a when i == 0
#...
Or, since you have an array and a numeric index, why not just skip the loop and get right to the point:
console.log(a[0])