Related
I have the following function:
let mtlDevice = MTLCreateSystemDefaultDevice()!
let device = MPSGraphDevice(mtlDevice: mtlDevice)
let inputData: [UInt32] = [0, 45, 0, 0, 45, 81, 0, 54, 0, 0, 54, 81, 1, 45, 0, 1, 45, 81, 1, 54, 0, 1, 54, 81, 7, 63, 567, 7, 63, 648, 7, 72, 567, 7, 72, 648, 8, 63, 567, 8, 63, 648, 8, 72, 567, 8, 72, 648, 1, 9, 81, 1, 9, 162, 1, 18, 81, 1, 18, 162, 2, 9, 81, 2, 9, 162, 2, 18, 81, 2, 18, 162]
let inputTensor = MPSGraphTensorData(device: device, data: Data(bytes: inputData, count: inputData.count * 4),
shape: [3, 8, 3], dataType: .uInt32)
let graph = MPSGraph()
let inputPlaceholder = graph.placeholder(shape: [3, 8, 3], dataType: .uInt32, name: nil)
let output = graph.reductionSum(with: inputPlaceholder, axis: 2, name: nil)
let outputTensor = graph.run(feeds: [inputPlaceholder: inputTensor], targetTensors: [output], targetOperations: nil)[output]!
var outputData = [UInt32].init(repeating: 0, count: 3 * 8 * 1)
outputTensor.mpsndarray().readBytes(&outputData, strideBytes: nil)
print("output: \(outputTensor.shape) \(outputData)")
The intention is to reduce inputData, interpreted as a tensor of shape [3, 8, 3] along the third axis. That is, to obtain a tensor of shape [3, 8, 1] where each element is the sum of the 3 elements previously present in axis 2.
When I run the above code, I receive the following output:
output: [3, 8, 1] [0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
This is surprising to me because it is obviously not the sum I was expecting.
Performing a similar operation in Tensorflow gives me the expected output:
data = tf.reshape(
[0, 45, 0, 0, 45, 81, 0, 54, 0, 0, 54, 81, 1, 45, 0, 1, 45, 81, 1, 54, 0, 1, 54, 81, 7, 63, 567, 7, 63, 648, 7, 72, 567, 7, 72, 648, 8, 63, 567, 8, 63, 648, 8, 72, 567, 8, 72, 648, 1, 9, 81, 1, 9, 162, 1, 18, 81, 1, 18, 162, 2, 9, 81, 2, 9, 162, 2, 18, 81, 2, 18, 162],
shape=[3, 8, 3])
tf.reduce_sum(data, axis=2)
Gives:
[[ 45, 126, 54, 135, 46, 127, 55, 136],
[637, 718, 646, 727, 638, 719, 647, 728],
[ 91, 172, 100, 181, 92, 173, 101, 182]]
as expected.
How can I obtain the same behaviour from reductionSum in MPSGraph? What explains the behaviour that I am observing from MPSGraph?
I am building a student attendance system using a fingerprint sensor. However, all off-the-shelf fingerprint sensors only support a maximum of around 100 fingerprints on the device flash itself.
In some of the fingerprint sensors I have used, I am able to get the finegrprint "template" off the device (for ex in the: R305 and GT-511C3)
Is there any way for me (open source) to match the minutiae "templates" from these sensors in the cloud or even on the pi itself.
Or even a way to upload previous matches into the fingerprint sensor flash dynamically (say, in sets of 60 fingerprints, as I would require only 60 per class)
I can provide code to read the sensor, but it is mostly based of Adafruit Library on github.
Here is the sample output in hex that someone online received from the sensor:
EF, 1, FF, FF, FF, FF, 2, 0, 82, 3, 1, 5F, 25, 0, 0, FE,
3E, FC, 2, F8, 2, F0, 0, F0, 0, F0, 0, E0, 0, E0, 0, E0,
0, E0, 0, E0, 0, E0, 0, E0, 0, E0, 0, E0, 0, E0, 0, E0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 64, 11, E4, FE, 6C, A0, A6, BE, 25, 23, 67, DE, 3C, AA, D1,
3E, 71, AF, 67, 9E, 3B, 3F, E7, 9E, 3D, B, 50, 7F, 58, 34, 51,
7F, 27, BB, A6, FF, 24, 41, D0, 5F, 59, 8, 63, 94, 5C, B, A5,
9C, 33, 39, 67, BC, 73, 13, E4, 75, 74, 10, 4C, 7A, 2B, 1B, CF,
32, 38, B1, D1, 3A, 36, BC, 11, 1A, 2F, BB, EF, 1, FF, FF, FF,
FF, 2, 0, 82, 33, 8E, 27, 7B, 6C, C, 4E, 18, 35, 11, 10, 72,
32, 12, E6, F2, 38, 1E, 8E, F8, 3C, 35, 51, 78, 28, 1E, 66, D3,
33, A2, E6, 99, 40, 35, E7, F9, 6E, 8D, E2, F6, 2C, 34, D1, 70,
34, 16, E6, 34, 3B, 96, CF, 8E, 40, 25, 26, B4, 32, 2F, A7, D4,
36, 24, 8F, F2, 28, 36, 91, CD, 26, 2E, EB, 50, 26, 31, 14, B1,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Any help is appreciated.
I know only one open-source solution for this, it's the SDK provided by NIST for fingerprint recognition: NBIS.
You could downloaded here.
Yes you can do it, but there is some problem.
It is easy to save fingerprint template to a file or database in hex format.
Even you can match 2 templates.
But the problem is how to download templates from database or file back to R305.
I have two RDD's, for example:
firstmapRDD - (0-14,List(0, 4, 19, 19079, 42697, 444, 42748))
secondmapRdd-(0-14,List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94))
I want to find the intersection.
I tried, var interResult = firstmapRDD.intersection(secondmapRdd), which shows no result in output file.
I also tried , cogrouping based on keys, mapRDD.cogroup(secondMapRDD).filter(x=>), but I don't know how to find the intersection between both the values, is it x=>x._1.intersect(x._2), Can someone help me with the syntax?
Even this throws a compile time error, mapRDD.cogroup(secondMapRDD).filter(x=>x._1.intersect(x._2))
var mapRDD = sc.parallelize(map.toList)
var secondMapRDD = sc.parallelize(secondMap.toList)
var interResult = mapRDD.intersection(secondMapRDD)
It may be because of ArrayBuffer[List[]] values, because of which the intersection is not working. Is there any hack to remove it?
I tried doing this
var interResult = mapRDD.cogroup(secondMapRDD).filter{case (_, (l,r)) => l.nonEmpty && r.nonEmpty }. map{case (k,(l,r)) => (k, l.toList.intersect(r.toList))}
Still getting an empty list!
Since you are looking intersect on values, you need to join both RDDs, get all the matched values, then do the intersect on values.
sample code:
val firstMap = Map(1 -> List(1,2,3,4,5))
val secondMap = Map(1 -> List(1,2,5))
val firstKeyRDD = sparkContext.parallelize(firstMap.toList, 2)
val secondKeyRDD = sparkContext.parallelize(secondMap.toList, 2)
val joinedRDD = firstKeyRDD.join(secondKeyRDD)
val finalResult = joinedRDD.map(tuple => {
val matchedLists = tuple._2
val intersectValues = matchedLists._1.intersect(matchedLists._2)
(tuple._1, intersectValues)
})
finalResult.foreach(println)
The output will be
(1,List(1, 2, 5))
I am using elixir-mongo and trying to stream the results of a query. Here's the code...
def archive_stream(z) do
Stream.resource(
fn ->
{jobs, datetime} = z
lt = datetime_to_bson_utc datetime
c = jobs |> Mongo.Collection.find( %{updated_at: %{"$lt": lt}}) |> Mongo.Find.exec
{:cont, c.response.buffer, c}
end,
fn(z) ->
{j, {cont, therest, c}} = next(z)
case cont do
:cont -> {j, {cont, therest, c}}
:halt -> {:halt, {cont, therest, c}}
end
end,
fn(:halt, resp) -> resp end
)
end
All of the sub-bits seem to work (like the query), but when I try to get at the stream, I fail...
Mdb.archive_stream({jobs, {{2013,11,1},{0,0,0}}})|>Enum.take(2)
I get...
(BadArityError) #Function<2.49475906/2 in Mdb.archive_stream/1> with arity 2 called with 1 argument ({:cont, <<90, 44, 0, 0, 7, 95, 105, 100, 0, 82, 110, 129, 221, 102, 160, 249, 201, 109, 0, 137, 233, 4, 95, 115, 108, 117, 103, 115, 0, 51, 0, 0, 0, 2, 48, 0, 39, 0, 0, 0, 109, 97, 110, 97, 103, 101, 114, 45, ...>>, %Mongo.Cursor{batchSize: 0, collection: %Mongo.Collection{db: %Mongo.Db{auth: {"admin", "edd5404c4f906060b125688e26ffb281"}, mongo: %Mongo.Server{host: 'db-stage.member0.mongolayer.com', id_prefix: 57602, mode: :passive, opts: %{}, port: 27017, socket: #Port<0.27099>, timeout: 6000}, name: "db-stage", opts: %{mode: :passive, timeout: 6000}}, name: "jobs", opts: %{}}, exhausted: false, response: %Mongo.Response{buffer: <<188, 14, 0, 0, 7, 95, 105, 100, 0, 82, 110, 129, 221, 102, 160, 249, 201, 109, 0, 137, 242, 4, 95, 115, 108, 117, 103, 115, 0, 45, 0, 0, 0, 2, 48, 0, 33, 0, 0, 0, 114, 101, 116, 97, ...>>, cursorID: 3958284337526732701, decoder: &Mongo.Response.bson_decode/1, nbdoc: 101, requestID: 67280413, startingFrom: 0}}})
(elixir) lib/stream.ex:1020: Stream.do_resource/5
(elixir) lib/enum.ex:1731: Enum.take/2
I'm stumped. Any ideas?
thanks for the help
Dang! Rookie Error.
:halt -> {:halt, {cont, therest, c}} should be _ -> {:halt, z} and fn(:halt, resp) -> resp end should be fn(resp) -> resp end
I've been d**king around with everything but the after function for a day and a half.
A little more explanation for fellow rookies...
the last option in the next_fun() should probably _ inorder to catch other "bad behavior" and not just {:halt}
the after_fn() is only expecting 1 arg and in the above code that would be the z tuple in the last option of the next_fun(). It is not expecting to see :halt and z, just z.
Would like to have REAL experts input.
Thanks in advance for the help.
I have two sets of parallel vectors:
x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 55];
x_count = [7721, 6475, 3890, 2138, 1152, 784, 674, 492, 424, 365, 309, 302, 232, 250, 220, 208, 190, 162, 144, 134, 97, 93, 89, 97, 92, 85, 77, 87, 64, 75, 72, 82, 61, 48, 46, 44, 35, 20, 28, 20, 21, 10, 6, 8, 4, 4, 4, 3, 1, 1];
y = [1, 2, 3, 4, 5, 6, 7, 8, 9, 55];
y_count = [88, 40, 24, 12, 8, 5, 1, 1, 1, 100];
where x, y are the categories, and x_count, y_count are the frequency of each categories. x and y can be of unequal lengths, and need not contain the same categories.
I want to create a side-by-side bar/histogram plot, where the x-axis is the categories, placed side-by-side like this: side by side multiply histogram in matlab. The frequency counts go along the y-axis.
I've tried googling around, but still stuck on this. If someone could help, that would be great. The solution in side by side multiply histogram in matlab works only if x and y have the same length, but mine's not.
You can try this:
% create unique bins
bins = unique([x y]);
% create vectors with zeros same size as bins
xBins = zeros(size(bins));
yBins = zeros(size(bins));
% fill in counts in the respective spots
xBins(ismember(x, bins)) = x_count;
yBins(ismember(y, bins)) = y_count;
bar(bins, [xBins' yBins']);