Retrieving all entries 10 days prior to days in table, which matches where clause - kdb

I have a random table of data - with dates and numbers:
Date Open Volume abschange
2016.12.08D00:00:00.000000000 11035.76 1.74835e+008 1.30177
2016.12.09D00:00:00.000000000 11170.18 1.0383e+008 0.2994598
2016.12.12D00:00:00.000000000 11198.42 8.98117e+007 0.07331357
...
2016.12.30D00:00:00.000000000 11443.31 4.18109e+007 0.3298871
2017.01.02D00:00:00.000000000 11426.38 4.74561e+007 1.504853
So from this table, i would like to be able to create a list, which holds all the entries from 10 days prior to the days, which has abschange>1.
I thought it would be easiest to start with a focus on those dates:
Date abschange
---------------------------------------
2016.12.08D00:00:00.000000000 1.30177
2017.01.02D00:00:00.000000000 1.504853
2017.01.25D00:00:00.000000000 1.099709
2017.01.31D00:00:00.000000000 1.344625
2017.02.06D00:00:00.000000000 1.016427
2017.02.21D00:00:00.000000000 1.265196
Then create a flat list:
mynewdates: raze tablewithDateAndabschange each
which gives me:
2016.12.08D........ 2017.01.02D......
Then i get stuck, when i want to add 10 prior dates for each entry in this list.
Could i actually get my wanted result in one line of code, based on the first table or should i follow the path i am on ?
For both - if possible, what would the possible solution to this be ?

If my understanding is correct your requirement is:
for each date for which abschange>1, get last 10 dates before this date from table.
Below query will create that map. It is based on following assumptions:
Date column is unique and ordered(ascending) as it appears from your example.
If above condition is not true then it will require a minor change in below query to work with duplicates and unordered list.
Table is not keyed.
q) (tbl[`Date]a)!b#'where#'not null b:tbl[`Date] -1+(a:where 1<tbl`abschange)-\:til 10
UPDATE: Based on discussion in comment section.
Just add the second step to check if dates list for first result is empty. In that case generate last 10 dates from that date.
Finally it generates the table where each row contains prior dates(max 10) for each date with abschange>1.
q)d:b#'where#'not null b:tbl[`Date] -1+(a:where 1<tbl`abschange)-\:til 10
q)d[0]:$[0=count d 0;(t[`Date]a 0)-1+til 10;d 0]
q)([]dates:d)

Related

Google Sheets - Expanding dataset according to months between 2 dates

I would like to be able to expand a dataset to include one row per "person" per month, filtered by Start and End Date.
Consider the example below:
This is the desired outcome
Inputs are a start and end date, which could be anywhere between the start of a given financial year (01/04/20XX) and the end of that financial year (31/03/20XY).
The output rows contain each month comprised between these 2 dates for which there is a matching entry in the input (see example output)
I have an example sheet here.
Any help would be appreciated, thanks
Try:
=ARRAYFORMULA(SPLIT(UNIQUE(QUERY(FLATTEN(IF(DAYS(C2:C4,B2:B4)>=SEQUENCE(1,1000,0),A2:A4&","&TEXT(B2:B4+SEQUENCE(1,1000,0),"YYYY-MMM")&","&D2:D4,"")),"where Col1 is not null")),","))
Result:
This works however the only limitation is you have to manually edit the formula to specifically define the End range. It crashes when I set it to use the whole column using the formula below:
=ARRAYFORMULA(SPLIT(UNIQUE(QUERY(FLATTEN(IF(DAYS(C2:C,B2:B)>=SEQUENCE(1,1000,0),A2:A&","&TEXT(B2:B+SEQUENCE(1,1000,0),"YYYY-MMM")&","&D2:D,"")),"where Col1 is not null")),","))
I am assuming this is because of the Query function which runs a Google Visualization API Query Language query across data making it very long and heavy processing.
Explanation:
To explain how this works using the formula from this link: Google sheets - List of dates between a list of start and end dates to get the dates between, but I changed the IF to return all combined 4 columns. Converted it to date format using TEXT (YYYY-MMM). I then used the UNIQUE to remove duplicates so it will only show the months. Then make use of SPLIT to split across 3 columns.
References:
Get dates between 2 dates
TEXT
UNIQUE
QUERY
SPLIT
Hope this helps!

Finding rows with matching id's and the using the date fields in each row to find the time difference in days between them

I have rows of data where up to 3 id's match. I need to try and find the number of days between each of the rows with matching id's using the date field. I am looking to find the number of days between each row in date sequence. So between the first and second row in date sequence, or second and third row but not the first and third row.
So a simplified version of my rows can be seen in the attached image.
image showing 2 ID's with multiple entries. John Smith (highlight yellow) who has 3 entries, two on the same day and one later and Bill Wright who has just two entries on separate days.
The results I would be looking for would ideally be shown in column 4 of the image.
Is there any way I can do this in a formula or pivot table? (basically avoiding VBA) Any suggestions hugely appreciated! Thanks.
This array formula
=IFERROR(B2-INDEX($B$1:$B$9,MAX((A2=$A$1:A1)*ROW($A$1:A1))),0)
will work as long as your data are sorted by date (which your screenshot indicates they are):

Creating a calculated field in Tableau using 3 columns

Date1 Date2 Line Item Total
May10,2009 May9,2009 10 40
May9,2009 May10,2009 20 10
May9,2009 May8,2009 20 30
May8,2009 May11,2009 30 0
This is my Table in Tableau. I want to create a calculated field for last column. If Date2 value has a match in Date1 Column, the new value is equal to the sum of all values corresponding to that that in Line Item column. Is this possible in tableau using calculation or in some way? example: 'May9,2009' in Date2 COLUMN has two occurrences in Date1 Column. hence the two values in Line Item column (20+20) is added and new calculated field shows 40.
Someone please help.
Maybe you can try doing a Join with the same table. Creating new rows matching Date2 with Date1 from a duplicated of the same table. Something like this (Hoja and Hoja12 are the duplicated sheets, my excel is in spanish):
And then calculate de SUM of each row aggregated by Date2. Then you can get a table like this one:
*For [Line Item] itself I had to use average because the data exists in multiple rows due to the join made, but in everyone is the same value. If you want a cleaner way of that maybe you can try using LODs.
Hope this helps.
(Perhaps another approach for the whole problem could be using the LODs, but that means a better comprehension of that kind of expressions and this method was the first one I thought about)

Google Sheet: Pick max date out of row

Assume: in a google sheet the attendance to courses is logged with crosses.
Goal is to get the last attendance date of a person and add 6 months to it. ( course needs to be followed every six months ).
I've tried to retrieve the date of the last course in one column and add the +6months immediately and tried to have the +6 months in the second column, but I didn't succeed in both attempts.
See this link for example google sheet: https://docs.google.com/spreadsheets/d/10DG2I4VgTlOHJ5TG0qH4pJ5hJ9zsYuJCgRzEtuiTE60/edit?usp=sharing
=lookup("x",$B4:$F4,B$2:F$2)
Enter this formula in cell G4 and copy down the rows.
This formula looks for "x" (the attendance indicator) in cells in row 4 (columns B to F) - it's looking for the last instance. Then it returns the cell value from row 2 (the course date) of the relevant column.
=EDATE(G4, 6)
Enter this formula in cell H4, and copy down the rows.
This is pretty straightforward. It takes the date obtained from the lookup formula and adds 6 months.
Note that the lookup formula references column F. That's because as you create more courses you'll want the formula to automatically include them. So in cell F1, type something like "Don't use this column" - and then don't use that column!
Now when you add a new course, insert a new column to the left of the column headed "Don't use this column". This will ensure that the range in the lookup formula will expand to include the new column/course, and your latest attendance date will be updated automatically.
BTW, there's no guarantee that the date of the next course is a working day.

Tableau - YTD, MTD as columns in text table

I'm trying to find a way to create a table like the one above, but instead of having columns of months I would like to have columns of MTD, QTD, YTD for all selected measure values. I created calculated fields on the [Date] field but when I try to use that it still splits the data in to separate columns of months and quarters... I'm using two data sources and they are linked on the [Date] field. If I try to put two of my calculated fields in the columns bar it just combines them like 'MTD/YTD'. How can I get them to display as separate columns?
I know this is strange because there will be overlapping data (everything in the MTD will also be in the QTD and the YTD).
I found a way to do this. I created a calculated field on the date field and assigned 'MTD' to all records from the last month. Then I assigned 'QTD' to all records from the start of the quarter to the day before the start date of the 'MTD' records. Then I assigned 'YTD' to all records from the start of the year to the day before the start of the 'QTD' records. (this way there are no overlapping records).
After that I just did a quick table calc to make each of them a 'running total' table (across).
I created parameters for the dates so the can easily be updated.