------ Context ------
i am completely new to swift and metal(kit?).
Problem is: i need to realize a parallel prefixSum(to be specific: the Blelloch scan) calculation in the next 2 weeks.
Due to me not having a NVIDIA GPU but an M1 Macbook, i chose metal, so please be patient with me..
After watching this video (the code is in the video-description), I now try to modify the code so that the prefixSum is calculated instead of just adding two arrays at index i.
------ Question ------
In the first step,I only want to pass an integer myVal and just add up the three values.
Unfortunately, the output for the first 3 values looks like this(note the last digit, which has the expected value but without the additional myVal):
3 + 5 + 10 = 42949672978
2 + 6 + 10 = 42949672978
8 + 3 + 10 = 42949672981
instead of this:
3 + 5 + 10 = 18
2 + 6 + 10 = 18
8 + 3 + 10 = 21
The main.swift code is in this pastebin. The compute.metal can be found here
Related
Disclaimer: I am very new to the Q language so please excuse my silly question.
I have a function that currently is taking on 2 parameters (date;sym).It runs fine for 1 sym and 1 day. however, I need to perform this on multiple syms and dates which will take forever.
How do I create a loop that run the function on every sym, and on every date?
In python, it is straighforward as :
for date in datelist:
for sym in symlist:
func(date,sym)
How can I do something similar to this in Q? and how can I dynamically change the output table names and append them to 1 single table?
Currently, I am using the following:
output: raze .[function] peach paralist
where paralist is a list of parameter pairs: ((2020.06.01;ABC);(2020.06.01;XYZ)) but imho this is nowhere near efficient.
What would be the best way to achieve this in Q?
I'll generalize everything, if you have a given function foo which will operate on an atom dt with a vector s
q)foo:{[dt;s] dt +\: s}
q)dt:10?10
q)s:100?10
q)dt
8 1 9 5 4 6 6 1 8 5
q)s
4 9 2 7 0 1 9 2 1 8 8 1 7 2 4 5 4 2 7 8 5 6 4 1 3 3 7 8 2 1 4 2 8 0 5 8 5 2 8..
q)foo[;s] each dt
12 17 10 15 8 9 17 10 9 16 16 9 15 10 12 13 12 10 15 16 13 14 12 9 11 11 ..
5 10 3 8 1 2 10 3 2 9 9 2 8 3 5 6 5 3 8 9 6 7 5 2 4 4 ..
13 18 11 16 9 10 18 11 10 17 17 10 16 11 13 14 13 11 16 17 14 15 13 10 12 12 ..
9 14 7 12 5 6 14 7 6 13 13 6 12 7 9 10 9 7 12 13 10 11 9 6 8 8 ..
The solution is to project the symList over the function in question, then use each (or peach) for the date variable.
If your function requires an atomic date and sym, then you can just create a new function to implement this
q)bar:{[x;y] foo[x;] each y};
datelist:`date$10?10
symlist:10?`IBM`MSFT`GOOG
function:{0N!(x;y)}
{.[function;x]} peach datelist cross symlist
cross will return all combinations of sym and date
Is this what you need?
Try to use two "double" '
raze function'[datelist]'[symlist]
peach or each won't work here. They are not operators, but anonymous functions with two parameters: each is k){x'y}. That is why function each list1 each list2 statement is invalid, but function'[list1]'[list2] works.
From reading your response to another answer you are looking to save the results with unique names yes? Take a look at this solution using set to save and get to retrieve.
q)t:flip enlist each `colA`colB!(100;`name)
q)t
colA colB
---------
100 name
q)f:{[date;sym]tblName:`$string[date],string sym;tblName set update date:date,sym:sym from t}
q)newTbls:f'[.z.d+til 3;`AAA`BBB`CCC]
q)newTbls
`2020.09.02AAA`2020.09.03BBB`2020.09.04CCC
q)get each newTbls
+`colA`colB`date`sym!(,100;,`name;,2020.09.02;,`AAA)
+`colA`colB`date`sym!(,100;,`name;,2020.09.03;,`BBB)
+`colA`colB`date`sym!(,100;,`name;,2020.09.04;,`CCC)
q)get first newTbls
colA colB date sym
------------------------
100 name 2020.09.02 AAA
Does this meet your needs?
This could be a stab in the dark, but why not create a hdb rather than all these variables output20191005ABC, output20191006ABC .. etc and given you want to append them to 1 table.
Below I have outlined how to create a date partitioned hdb called outputHDB which has one table outputTbl. I created the hdb by running a function by date and sym and then upserting those rows to disk.
C:\Users\Matthew Moore>mkdir outputHDB
C:\Users\Matthew Moore>cd outputHDB
// can change the outputHDB as desired
// start q
h:hopen `::6789; // as a demo I connected to another hdb process and extracted some data per sym / date over IPC
hdbLoc:hsym `$"C:/Users/Matthew Moore/outputHDB";
{[d;sl]
{[d;s]
//output:yourFunc[date;sym];
// my func as a demo, I'm grabbing rows where price = max price by date and by sym from another hdb using the handle h
output:{[d;s]
h({[d;s] select from trades where date = d, sym = s, price = max price};d;s)
}[d;s];
// HDB Part
path:` sv (hdbLoc;`$string d;`outputTbl;`);
// change `outputTbl to desired table name
// dynamically creates the save location and then upserts one syms data directly to disk
// e.g. `:C:/Users/Matthew Moore/outputHDB/2014.04.21/outputTbl/
// extra / at the end saves the table as splayed i.e. each column is it's own file within the outputTbl directory
path upsert .Q.en[`:.;output];
// .Q.en enumerates syms in a table which is required when saving a table splayed
}[d;] each sl;
// applies the parted attribute to the sym column on disk, this speeds up querying for on disk data
#[` sv (hdbLoc;`$string d;`outputTbl;`);`sym;`p#];
}[;`AAPL`CSCO`DELL`GOOG`IBM`MSFT`NOK`ORCL`YHOO] each dateList:2014.04.21 2014.04.22 2014.04.23 2014.04.24 2014.04.25;
Now that the hdb has been created, you can load it from disk and query with qSQL
q)\l .
q)select from outputTbl where date = 2014.04.24, sym = `GOOG
date sym time src price size
------------------------------------------------------------
2014.04.24 GOOG 2014.04.24D13:53:59.182000000 O 46.43 2453
I have been investigating various collision resolution techniques for hashtables implemented via open addressing. However, all the collision resolution methods I have investigated so far (linear probing, quadratic probing, double hashing) have the pitfall that there exists a probing sequence that produces a cycle whose length is less than the size of the table. This becomes problematic when you're trying to insert an element with the open addressing scheme because there are free buckets to insert an entry but they might not be reachable if they're not part of the cycle.
For example, if we're using linear probing on a table of size 12 with the linear function: H(k, i) = (h(k) + 4*i) mod 12 then a cycle would occur if a particular key hashes to 8 and all the slots 0, 4, and 8 are already filled:
H(k, 0) = 8 + 0 mod 12 = 8
H(k, 1) = 8 + 4 mod 12 = 0
H(k, 2) = 8 + 8 mod 12 = 4
H(k, 3) = 8 + 12 mod 12 = 8
H(k, 4) = 8 + 16 mod 12 = 0
H(k, 5) = 8 + 20 mod 12 = 4
H(k, 6) = 8 + 24 mod 12 = 8
...
Similar cycles can also be found with quadratic and double hashing if the probing sequence is bad, so my question then is how are cycles handled? Or do we always pick hash functions/special table sizes which do not permit cycles which are too short?
I have got a nx3 adjacency matrix that contains nodes in the first two dimension and the correspondant weight in the third dimension. I want to filter the matrix for specific thresholds (for nodes indexing). For example, I want to keep the adjacency matrix for nodes smaller than 10.000, 20.000, etc. Which is the most efficient way to do so in matlab? I tried to do the following, find the index which correspond to nodes:
counter = 1;
for i=1: size(graph4, 1)
if (graph4(i,1) >30000) | (graph4(i,2) >30000)
bucket(counter) = i;
counter=counter+1;
end
end
Suppose the adjacency matrix is A as given below:
A =
8 1 6
3 5 7
4 9 2
11 4 9
6 8 10
7 12 5
17 10 15
12 14 16
13 18 11
If you want both column 1 and column 2 to be less than a value, you can do:
value = 10;
T = A(A(:,1) < value & A(:,2) < value, :)
T =
8 1 6
3 5 7
4 9 2
6 8 10
The following line seems to give the same results as your sample code (but it doesn't seem like it fits your description.
value = 10000;
bucket = find((A(:,1)>value) | A(:,2)>value)
I guess you made a mistake and want to increment the counter above the bucket-line and initialize it as counter = 0 before the loop? As it is now, it will be one more than the number of elements in the bucket-list.
1+2+4+⋯+2k=1−2^(k+1)/(1−2)
Here, 2^k=N. You get
1+2+4+⋯+N=1−2N−1
2+4+8⋯+N=2N−2
As shown above the array is doubled every 2^k. But I still don't understand all the steps for my working out doesn't match above. Can someone give me a step by step working out and explanation?
Begin with
1 + 2 + 4 + 8 + 16 + ... + 2k = 2k+1 - 1
If 2k = N, then
1 + 2 + 4 + 8 + 16 + ... + 2k = 2k+1 - 1 = 2 · 2k - 1 = 2N - 1
Hope this helps!
Example: I have a scale between 1 and 7. When I get a value like 8, I want it to be wrapped on that scale so it's converted to 1. More examples:
1 results in 1
5 results in 5
7 results in 7
8 results in 1
9 results in 2
10 results in 3
11 results in 4
12 results in 5
13 results in 6
14 results in 7
15 results in 1
16 results in 2
and so on.
Is there a method or useful C-function to do that? Something tells me I just need a modulo. It's 42°C in my room. My brain is like soap.
int b = ((a-1) % 7) + 1;
Check using Excel, of all things!
Yes it's HOT today.. arrgh!
Try ((number - 1) % 7) + 1.