Implementation of FIFO pnl in kdb/q - kdb

Consider the table below:
Id
Verb
Qty
Price
1
Buy
6
10.0
2
Sell
5
11.0
3
Buy
4
10.0
4
Sell
3
12.0
5
Sell
8
9.0
6
Buy
7
8.0
I would like to compute the PnL in a FIFO way. For example for Id=1, PnL is -6*(10.0) +5*(11.0) + 1*(12.0) = +$7.00. For Id=5, this case is a bit different: our position is +2, and we will firstly fill this position(which will not take account into the PnL of Id=5), then we sell the remaining 6 assets. At Id=6, the -6 position is fulfilled and we get the PnL of Id=5 which is +6*(9.0)-6*(8.0)=+$6.00. Hence this table with PnL is what I want to have :
Id
Verb
Qty
Price
PnL
1
Buy
6
10.0
7.0
2
Sell
5
11.0
0.0
3
Buy
4
10.0
2.0
4
Sell
3
12.0
0.0
5
Sell
8
9.0
6.0
6
Buy
7
8.0
0.0(with 1 asset remaining)
I have read this post and KDB: pnl in FIFO manner and https://code.kx.com/q4m3/1_Q_Shock_and_Awe/#114-example-fifo-allocation. But in their approach, they don't care about the order between buy orders and sell orders, which is not my case.
My idea is to firstly produce the FIFO allocation matrix where the dimension is the trades number:
Id
1
2
3
4
5
6
1
6
0
0
0
0
0
2
1
0
0
0
0
0
3
1
0
4
0
0
0
4
0
0
2
0
0
0
5
0
0
0
0
-6
0
6
0
0
0
0
0
1
Then I compute the diff(price). The inner product of each column and diff(price) is PnL of each trade.
I am having trouble to implement this allocation matrix. Or any advice on solving this problem more directly?

Here's one approach. It's more convoluted than I'd like but it covers a lot of the intermediary steps and generates a type of allocation matrix as you suggested. There are likely edge-cases and tweaks needed but this should give you some ideas at least.
t:([]id:1+til 6;side:`b`s`b`s`s`b;qty:6 5 4 3 8 7;px:10 11 10 12 9 8f);
t:update pos:sums delta from update delta:qty*(1;-1)side=`s from t;
f:{signum[x]*x,{#[(-). z;x;:;abs[y]-sum z 1]}[y;x y]{(x;deltas y&sums x)}[abs where[signum[x]<>signum x y]#x;abs x y]};
t:update fifo:deltas[id!delta;f\[id!delta;id]] from t;
q)update pnl:sum each(id!px)*/:fifo from t
id side qty px delta pos fifo pnl
-----------------------------------------------------
1 b 6 10 6 6 1 2 3 4 5 6!-6 5 0 1 0 0 7
2 s 5 11 -5 1 1 2 3 4 5 6!0 0 0 0 0 0 0
3 b 4 10 4 5 1 2 3 4 5 6!0 0 -4 2 2 0 2
4 s 3 12 -3 2 1 2 3 4 5 6!0 0 0 0 0 0 0
5 s 8 9 -8 -6 1 2 3 4 5 6!0 0 0 0 6 -6 6
6 b 7 8 7 1 1 2 3 4 5 6!0 0 0 0 0 0 0

Related

Gnuplot histogram with Matlab style

I want to plot a 3D histogram with Gnuplot using a style commonly used in Matlab.
The sequence of steps I follow are:
set palette defined ( 0 '#000090',\
1 '#000fff',\
2 '#0090ff',\
3 '#0fffee',\
4 '#90ff70',\
5 '#ffee00',\
6 '#ff7000',\
7 '#ee0000',\
8 '#7f0000')
set pm3d at s
set view map
splot 'test.dat' u 1:2:3
The data to make the plot is provided below. The resulting plot looks like this:
As you may see in the data below, most of the entries are zero which makes the plot very blue. In Matlab one can make this type of histograms and zero values have a white color. Just the points which are not zero have a color as in the palette above. I would like to have those points in white because that would emphasize the actual sampled region.
I wonder if we can do that in Gnuplot. I tried omitting the zeroes in the data file but it resulted in a pointy-corners plot.
Additionally, I modified the palette defining the zero value explicitly as here:
set palette defined ( 0 '#ffffff',\
1 '#000090',\
2 '#000fff',\
3 '#0090ff',\
4 '#0fffee',\
5 '#90ff70',\
6 '#ffee00',\
7 '#ff7000',\
8 '#ee0000',\
9 '#7f0000')
However, the borders of the sampled area look color violet:
Thanks.
1 1 1
1 2 0
1 3 0
1 4 0
1 5 0
1 6 0
1 7 0
1 8 0
2 1 0
2 2 0
2 3 1
2 4 2
2 5 3
2 6 0
2 7 0
2 8 0
3 1 0
3 2 0
3 3 2
3 4 10
3 5 15
3 6 2
3 7 0
3 8 0
4 1 0
4 2 0
4 3 0
4 4 5
4 5 2
4 6 1
4 7 0
4 8 0
5 1 0
5 2 0
5 3 0
5 4 3
5 5 2
5 6 0
5 7 0
5 8 0
6 1 0
6 2 0
6 3 0
6 4 2
6 5 0
6 6 0
6 7 1
6 8 0
7 1 0
7 2 0
7 3 0
7 4 0
7 5 0
7 6 0
7 7 0
7 8 0
8 1 0
8 2 0
8 3 0
8 4 0
8 5 0
8 6 0
8 7 0
8 8 0
Plotting with pm3d averages the data points. If you want to plot exactly the data points as matrix, you must plot with image. To have certain values in white, define them as undefined with 1/0:
set palette defined ( 0 '#000090',\
1 '#000fff',\
2 '#0090ff',\
3 '#0fffee',\
4 '#90ff70',\
5 '#ffee00',\
6 '#ff7000',\
7 '#ee0000',\
8 '#7f0000')
plot 'test.dat' u 1:2:($3 == 0 ? 1/0 : $3) with image notitle
Update
I just saw the answer above using image (I'll upvote that one), and that is probably the best way to accomplish what you asked for. In any case, I made a few tests with the data file and wanted to share just for completeness:
set term png
set out "tmp.png"
set palette defined ( 0 '#ffffff',\
1 '#000090',\
2 '#000fff',\
3 '#0090ff',\
4 '#0fffee',\
5 '#90ff70',\
6 '#ffee00',\
7 '#ff7000',\
8 '#ee0000',\
9 '#7f0000')
set xrange[0:8]
set yrange[0:8]
set pm3d explicit at s
set view map
set multiplot layout 2,2
splot 'test.dat' u 1:2:3 not w pm3d
splot 'test.dat' u 1:2:3 not w p pt 5 ps 5 pal
splot 'test.dat' u 1:2:($3==0 ? 1/0:$3) not w pm3d
Resulting in this:

Every possible sum combination in a vector

Assuming I'm having a vectors of numbers A, for example: A=[1 3 5 3 9 6](A's length >= 2) and an Integer X=6. Need to find how many pairs (A[i],A[j]) where i<j exist in the vector which answer this condition: A[i]+A[j]=X. The number of pairs is printed.
Not allowed to use for/while. Allowed only ceil,floor,mod,repmat,reshape,size,length,transpose,sort,isempty,all,any,find ,sum,max,min.
With repmat, length and sum -
integer1 = 6; %// One of the paramters
A_ind = 1:length(A) %// Get the indices array
%// Expand A_ind into rows and A_ind' into columns, to form a meshgrid structure
A_ind_mat1 = repmat(A_ind,[length(A) 1])
A_ind_mat2 = repmat(A_ind',[1 length(A)]) %//'
%// Expand A into rows and A' into columns, to form a meshgrid structure
A_mat1 = repmat(A,[length(A) 1])
A_mat2 = repmat(A',[1 length(A)]) %//'
%// Form the binary matrix of -> (A[i],A[j]) where i<j
cond1 = A_ind_mat1 < A_ind_mat2
%// Use the binary matrix as a logical mask to select elements from the two
%// matrices and see which element pairs satisfy -> A[i]+A[j]=X and get a
%// count of those pairs with SUM
pairs_count = sum((A_mat1(cond1) + A_mat2(cond1))==integer1)
Outputs from code run to make it clearer -
A =
1 3 5 3 9 6
A_ind =
1 2 3 4 5 6
A_ind_mat1 =
1 2 3 4 5 6
1 2 3 4 5 6
1 2 3 4 5 6
1 2 3 4 5 6
1 2 3 4 5 6
1 2 3 4 5 6
A_ind_mat2 =
1 1 1 1 1 1
2 2 2 2 2 2
3 3 3 3 3 3
4 4 4 4 4 4
5 5 5 5 5 5
6 6 6 6 6 6
A_mat1 =
1 3 5 3 9 6
1 3 5 3 9 6
1 3 5 3 9 6
1 3 5 3 9 6
1 3 5 3 9 6
1 3 5 3 9 6
A_mat2 =
1 1 1 1 1 1
3 3 3 3 3 3
5 5 5 5 5 5
3 3 3 3 3 3
9 9 9 9 9 9
6 6 6 6 6 6
cond1 =
0 0 0 0 0 0
1 0 0 0 0 0
1 1 0 0 0 0
1 1 1 0 0 0
1 1 1 1 0 0
1 1 1 1 1 0
pairs_count =
2
A bit more explanation -
Taking few more steps to clarify why pairs_count must be 2 here -
Set all values in A_mat1 and A_mat2 to be zeros that do not satisfy the less than criteria
>> A_mat1(~cond1)=0
A_mat1 =
0 0 0 0 0 0
1 0 0 0 0 0
1 3 0 0 0 0
1 3 5 0 0 0
1 3 5 3 0 0
1 3 5 3 9 0
>> A_mat2(~cond1)=0
A_mat2 =
0 0 0 0 0 0
3 0 0 0 0 0
5 5 0 0 0 0
3 3 3 0 0 0
9 9 9 9 0 0
6 6 6 6 6 0
Now, add A_mat1 and A_mat2 and see how many 6's you got -
>> A_mat1 + A_mat2
ans =
0 0 0 0 0 0
4 0 0 0 0 0
6 8 0 0 0 0
4 6 8 0 0 0
10 12 14 12 0 0
7 9 11 9 15 0
As you can see there are two 6's and thus our result is verified.

Loop through all combinations of 8 in 15 MATLAB

I have this matrix and want to make all combinations of column composed square matrixes (8x8) composed from this data.
4 2 4 3 2 3 3 2 8 4 9 7 6 6 6
2 0 4 1 0 3 0 8 5 0 9 3 7 7 1
2 1 2 1 1 3 1 4 5 2 4 2 6 6 3
0 0 2 2 1 2 3 9 1 1 4 4 4 4 6
4 0 1 0 4 2 3 1 8 1 3 0 5 5 7
3 1 4 0 0 1 0 2 6 2 9 1 2 2 0
1 2 1 4 0 3 4 1 3 4 3 9 7 7 9
2 0 0 4 0 0 3 1 5 0 1 9 1 1 7
Even after reeding Matlab Loop of all combinations
I'm not really sure how to do all the matrix combinations and include the counter from the for loop in the name of the combination obtained in the itteration.
I called your matrix A.
p=nchoosek(1:15,8);
gives all the combinations of 8 numbers taken from 1 to 15. These represent the columns of the matrix A that you want.
There are now 3 ways to proceed. Firstly, using a for loop:
M=zeros(8,8,size(p,1));
for i=1:size(p,1)
M(:,:,i)=A(:,p(i,:));
end
which puts each 8x8 matrix into a larger 3D array. You would get out individual matrices by doing M(:,:,54), for example.
You can also create a cell array:
N=arrayfun(#(k) A(:,p(k,:)),1:size(p,1),'UniformOutput',false);
and get individual matrices by doing N{54}.
Finally, you could not precompute each matrix, and just pull out the appropriate columns when you need them. This may be the most efficient method if you don't reuse the matrices:
O=A(:,p(54,:));

Matlab: Unique values between two matrices

I have 2 matrices:
T3(:,:,1) =
0 0 0 0 1 0 0 0 0
0 0 0 0 2 0 0 0 0
0 0 0 0 3 0 0 0 0
0 1 0 1 4 2 0 4 0
0 3 0 2 6 3 0 5 0
2 4 2 5 7 5 4 6 5
4 5 5 7 8 8 5 7 6
5 6 6 8 9 9 8 9 8
T3(:,:,2) =
2 1 1 1 1 1 1 1 1
3 3 2 2 2 2 2 2 2
4 4 4 3 3 3 3 3 3
5 5 5 5 4 4 4 4 4
6 6 6 6 6 5 5 5 5
7 7 7 7 7 7 6 6 6
8 8 8 8 8 8 8 7 7
9 9 9 9 9 9 9 9 8
How do I make values present in T3(:,:,1) turn to zero in T3(:,:,2)?
e.g. in the first column of T3(:,:,1) the values are 2,4,5. I'd like the first column of T3(:,:,2) to have the the values 2,4,5 as zero.
T3(:,:,2) =
0 0 1 0 0 1 1 1 1
3 0 0 0 0 0 2 2 2
0 0 4 3 0 0 3 3 3
0 0 0 0 0 4 0 0 4
6 0 0 6 0 0 0 0 0
7 7 7 0 0 7 6 0 0
8 8 8 0 0 0 0 0 7
9 9 9 9 0 0 9 0 0
I wonder if there is a way to do this using setdiff or unique.
for y=1:H-1
for z=1:H-1
for h=1:H
for d=1:D-1
if T3(y,h,d+1) == T3(z,h,d)
T3(y,h,d+1)=0;
end
end
end
end
end
I can do it as a loop where H=number of columns (9) and D= number of dimensions (2). There must be a better way :)?
Many thanks guys.

matlab efficient copying of matrix

I have matrix (a) with (1:10),<10 x 1> double. I would like to copy the values and rearrange them column wise into another matrix var. (b). See example below. Also, what method would be most efficient at this task?
matrix a matrix b
1 1
2 2 2
3 3 3 3
4 4 4 4 4
5 5 5 5 5 5
6 6 6 6 6 6 6
7 7 7 7 7 7 7 7
8 8 8 8 8 8 8 8 8
9 9 9 9 9 9 9 9 9 9
10 10 10 10 10 10 10 10 10 10 10
update:
Hi once again Amro. How about if I wanted to define which values to copy. See below example:
matrix a matrix b
column: 1 2 3 4 5 6 7
1 1
2 2 2
3 3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10 10
Try:
>> a = (1:10)'
a =
1
2
3
4
5
6
7
8
9
10
>> b = tril(repmat(a,1,10))
b =
1 0 0 0 0 0 0 0 0 0
2 2 0 0 0 0 0 0 0 0
3 3 3 0 0 0 0 0 0 0
4 4 4 4 0 0 0 0 0 0
5 5 5 5 5 0 0 0 0 0
6 6 6 6 6 6 0 0 0 0
7 7 7 7 7 7 7 0 0 0
8 8 8 8 8 8 8 8 0 0
9 9 9 9 9 9 9 9 9 0
10 10 10 10 10 10 10 10 10 10
I think in the second matrix you specified you made an error. I'm assuming you wanted to do something like this:
b =
1 0 0 0 0 0
2 2 0 0 0 0
0 3 3 0 0 0
0 0 4 4 0 0
0 0 0 5 5 0
0 0 0 0 6 6
this is simple to do:
%define vector of arbitrary length
a=1:6;
%generate b with shifted diagonal matrices
b=diag(a)+diag(a(2:end),-1);
the second argument of diag just shifts the resulting diagonal.