I am new to Caffe2, and I want to compose an operation like this:
Numpy way
example code
pytoch way
example code
My question is, how to compose Caffe2 operators to make the same operators like above? I have tried some compositions but still I couldn't find the right one. If anyone knows the composition, please help, I will be really appreciate for it.
There is a Gather operator in Caffe2. The main problem with this operator is that you can't set the axis (it's always 0). So, if we run this code:
model = ModelHelper(name="test")
s = np.arange(20).reshape(4, 5)
y = np.asarray([0, 1, 2])
workspace.FeedBlob('s', s.astype(np.float32))
workspace.FeedBlob('y', y.astype(np.int32))
model.net.Gather(['s', 'y'], ['out'])
workspace.RunNetOnce(model.net)
out = workspace.FetchBlob('out')
print(out)
We will get:
[[ 0. 1. 2. 3. 4.]
[ 5. 6. 7. 8. 9.]
[ 10. 11. 12. 13. 14.]]
One solution could be to reshape s to a 1D array and transform y in the same way. First of all, we have to implement an operator to transform y. In this case, we will use a numpy function called ravel_multi_index:
class RavelMultiIndexOp(object):
def forward(self, inputs, outputs):
blob_out = outputs[0]
index = np.ravel_multi_index(inputs[0].data, inputs[1].shape)
blob_out.reshape(index.shape)
blob_out.data[...] = index
Now, we can reimplement our original code:
model = ModelHelper(name="test")
s = np.arange(20).reshape(4, 5)
y = np.asarray([[0, 1, 2],[0, 1, 2]])
workspace.FeedBlob('s', s.astype(np.float32))
workspace.FeedBlob('y', y.astype(np.int32))
model.net.Python(RavelMultiIndexOp().forward)(
['y', 's'], ['y'], name='RavelMultiIndex'
)
model.net.Reshape('s', ['s_reshaped', 's_old'], shape=(-1, 1))
model.net.Gather(['s_reshaped', 'y'], ['out'])
workspace.RunNetOnce(model.net)
out = workspace.FetchBlob('out')
print(out)
Output:
[[ 0.]
[ 6.]
[ 12.]]
You may want to reshape it to (1, -1).
Related
Could someone explain to me the logic behind this hashMap algorithm? I'm getting confused about how the algorithm receives the total sum. I'm starting to learn about algorithms, so it's a little confusing for me. I made comments in my code to pinpoint each line code, but I'm not sure I'm grasping logic correctly. I'm just looking for an easier way to understand how the algorithm works to avoid confusing myself.
//**calculate Two Number Sum
func twoNumberSum(_ array: [Int], _ targetSum: Int) -> [Int] {
//1) initilize our Array to hold Integer Value: Boolean value to store value into hashTable
var numbersHashMap = [Int:Bool]()
//2) create placeHolder called number that iterates through our Array.
for number in array {
//3) variable = y - x
let match = targetSum - number
//4) ??
if let exists = numbersHashMap[match], exists {
//5) match = y / number = x
return [match, number] //
} else {
//6) Store number in HashTable and repeats
numbersHashMap[number] = true
}
}
return []
}
twoNumberSum([3,5,-4, 8, 11, 1, -1, -6], 10)
// x = Number
// y = Unknown *Solve for Y*
Sure, I can walk you through it. So we have a list of numbers, are we are trying to find two numbers that add together to make the specified target. To do this, for each number x, we check if (target - x) is in the list. If it is not, then we add x to the list. If it is, then we return x and (target - x).
Step 4 in your code is the part where we check if (target - x) is in the list. To see why this makes sense, let's walk through an example.
Say we have [2, 3, -1] and our target is 1. In this case, we first consider x = 2 and check our hashmap for (target - x) = (1 - 2) = -1. Since -1 is not in the hashmap, we add 2 to the hashmap. We then consider x = 3 and check for (1 - 3) = -2. Again, -2 is not in the hashmap, so we add it. Now we check x - -1. In this case, when we check (target - x) = (1 - (-1)) = 2, 2 is in the hashmap. Intuitively, we have already "seen" 2, and know that 2 and -1 can be added to get our value.
This is what provides the speed optimization over checking every two numbers in the list.
I've been researching and learning about optimization in general, and ortools in particular, and I need help with understanding what it is I'm doing wrong with this simple problem using ortools.
The problem itself is simple (so simple that ortools should be overkill), but keep in mind this is just for learning the basics:
How to select the smallest (and largest) integer from a list of integers?
Here's the code I have.
# 1. A simple problem:
# Select the smallest number from a list of integers
from __future__ import print_function
from ortools.sat.python import cp_model
# Define data
cost_data = [
5, 4, 3, 6, 9, 12, 5, 9, 12, 14
]
num_hours = len(cost_data)
hours = range(num_hours)
# Create model
model = cp_model.CpModel()
# Create variables
cost_vars = [] # Keep variables for costs
pick_vars = [] # Keep variables for picked items (later we add a constraint for only selecting one pick)
for i in hours:
cost_vars.append(model.NewIntVar(0, 20, ''))
pick_vars.append(model.NewBoolVar(''))
# Create constraints
# Only one pick
model.Add(sum(pick_vars) == 1)
for i in hours:
model.Add(cost_vars[i] == cost_data[i]).OnlyEnforceIf(pick_vars[i])
# Set objective function
model.Minimize(sum(cost_vars)) # This works (returns 3)
# model.Maximize(sum(cost_vars)) # This doesnt work (returns 194 as objective value)
# Solve problem
solver = cp_model.CpSolver()
status = solver.Solve(model)
if status == cp_model.INFEASIBLE:
print("INFEASIBLE")
elif status == cp_model.FEASIBLE:
print("FEASIBLE")
elif status == cp_model.OPTIMAL:
print("OPTIMAL")
print("ObjectiveValue()")
print(solver.ObjectiveValue())
This example works when I use the Minimize function, but if I replace Minimize with Maximize, it somehow returns 194 as the objective value.
What am I doing wrong?
What is happening right now is:
9*20+14 = 194
Because you are telling the solver that cost_var is only equal to your cost if it is picked, else it can be any integer between 0 and 20.
Edit:
the logic that you want is:
model.Add(cost_vars[i] == cost_data[i]).OnlyEnforceIf(pick_vars[i])
model.Add(cost_vars[i] == 0).OnlyEnforceIf(pick_vars[i].Not())
You should also take a look at AddMaxEquality and AddMinEquality
I'm new to Scala and I'm having a mental block on a seemingly easy problem. I'm using the Scala library breeze and need to take an array buffer (mutable) and put the results into a matrix. This... should be simple but? Scala is so insanely type casted breeze seems really picky about what data types it will take when making a DenseVector. This is just some prototype code, but can anyone help me come up with a solution?
Right now I have something like...
//9 elements that need to go into a 3x3 matrix, 1-3 as top row, 4-6 as middle row, etc)
val numbersForMatrix: ArrayBuffer[Double] = (1, 2, 3, 4, 5, 6, 7, 8, 9)
//the empty 3x3 matrix
var M: breeze.linalg.DenseMatrix[Double] = DenseMatrix.zeros(3,3)
In breeze you can do stuff like
M(0,0) = 100 and set the first value to 100 this way,
You can also do stuff like:
M(0, 0 to 2) := DenseVector(1, 2, 3)
which sets the first row to 1, 2, 3
But I cannot get it to do something like...
var dummyList: List[Double] = List(1, 2, 3) //this works
var dummyVec = DenseVector[Double](dummyList) //this works
M(0, 0 to 2) := dummyVec //this does not work
and successfully change the first row to the 1, 2,3.
And that's with a List, not even an ArrayBuffer.
Am willing to change datatypes from ArrayBuffer but just not sure how to approach this at all... could try updating the matrix values one by one but that seems like it would be VERY hacky to code up(?).
Note: I'm a Python programmer who is used to using numpy and just giving it arrays. The breeze documentation doesn't provide enough examples with other datatypes for me to have been able to figure this out yet.
Thanks!
Breeze is, in addition to pickiness over types, pretty picky about vector shape: DenseVectors are column vectors, but you are trying to assign to a subset of a row, which expects a transposed DenseVector:
M(0, 0 to 2) := dummyVec.t
I'm making an app where I have to put a lot of if/else statements. I know you can do as in the title in some other coding language, but I'm not sure if you can do it in Swift.
How do you shorten this:
if x == y || x == z {
//do something
}
To something like this:
if x == y || z {
//do something
}
Perhaps you could consider using an array and checking to see if x is in the array, like in the following example:
let (x, y, z) = (3, 8, 3)
if [y, z].contains(x) {
//True
}
If you're comparing objects (like UIImage), use containsObject instead of contains:
if [x, y, z].containsObject(y) {
//True
}
I conject that there is no sensible language (swift included), that distributes comparison == across logical or ||.
The way you've written it - x == y || x == z - is the most compact form.
Jack Greenhill's answer does indeed go into the right direction. However with more and more values, his method will get very inefficient, since it has to check every element of the array against equality, therefore complexity O(n).
A very underrated data structure, which can do this kind of operation in O(1) should be used instead: The Set. It uses hash values to check quickly whether a value is present or not. You can use it like this:
let x = 3
let values : Set = [1, 3, 6, 1, 7] // {6, 7, 3, 1}
if values.contains(x) {
// ...
}
This takes the same amount of time, whether values contains just one or 1000 elements. An array would be 1000 times slower.
Oftentimes the decision to use an array is made before even considering a set. If your elements don't have any order and can only occur once (which is actually more often the case than you'd think), you probably want a set. A set gives you useful methods, such as union, isSubset, interception and more, for free by just putting your elements in it. The only additional requirement for the element type is to conform to Hashable.
Perhaps this:
switch x {
case y, z : // do then
default : // do else
}
I don't know how to explain this better than by giving you an example.
Suppose I have the following array:
a = magic(6)
And then I take a 'slice' of that like this:
a(:,1)
It will print:
35
3
31
8
30
4
Now I want the first number, so I want to write:
a(:,1)(1)
Instead of:
b = a(:,1)
b(1)
Also, is there a way to do something like this (assignment and comparison, i.e. set b, then evaluate against it):
(b = a(:,1))(1)
Ok, here's an update with a function where it isn't trivial to use a(1, 1)
come_on = sprintf('%i, ', magic(3));
come_on(1:end-2)
8, 3, 4, 1, 5, 9, 6, 7, 2
Also, what if I only want the first 4 numbers on magic(3)?
It would be better to write
sprintf('%i, ', magic(3)(1:4))(1:end-2)
instead of tens of lines, MHO.
You cannot concatenate indexing as foo(1)(2)(3). However, you can index multiple dimensions at once. So in this case, a(1,1) will give you what you want.