How to convert respectively four column to date formatting at MATLAB? - matlab

Excel file(104976x10) includes large data.
A column: Time (unit year)
B column: Year
C column: Day of the year
D column: Hour
E column: Minute
and others including values
I would like to convert column which begins with B column until E column to date format like 'dd/mm/yyyy HH:MM'.
Example for the data:
1998,41655251 1998 152 1 0 12,5 12,0 11,8 11,9 12,0
I would like to do date instead of 2-th, 3-th, 4-th and 5-th columns.
1998,41655251 01/06/1998 01:00 12,5 12,0 11,8 11,9 12,0
or
1998,41655251 01/06/1998 01:00 1998 152 1 0 12,5 12,0 11,8 11,9 12,0

Welcome to SO.
Matlab has two types of date-format:
datetime, introduced in 2014b.
datenum, introcuced in long ago (before 2006b), it is basically a double precision value giving the number of days from January 0, 0000.
I think the best way is to use datetime, and give it the year, month, day, hour and minute values like this:
t=datetime(1998,0,152,1,0,0)
t= '01-May-1998 01:00:00'
As you can see the days automatically overflow into the months. But I end up 1st of may, not 1st of june like in your example.
to change the format:
t.Format='dd/MM/yyyy hh:mm'
t= '01/05/1998 01:00'
to convert it to a string, you can simply use string(t)
This is an example that combines the above functions to read an xlsx file and writes a new one with the updated column.
data=xlsread('test.xlsx');
S = size(data);
t = datetime(data(:,2),0,data(:,3),data(:,4),data(:,5),0);
t.Format='dd/MM/yyyy HH:mm';
data2=num2cell(data(:,1));
data2(:,2)=cellstr(string(t));
data2(:,3:S(2)-3)=num2cell(data(:,6:end));
xlswrite('test2.xlsx',data2);

Related

How to plot a graph for specific day of every month for whole Year MATLAB

I have two variables called x and y. Each has 24*365 values. 24 presents the number of hours in a day and 365 presents the number of days in a year. I am plotting the values of 12th hour and 25th day by the following command:
plot(x(12:12,25), y(12:12,25))
Now I want to do the same for every 25th day for a whole year. Like 25th of Jan, 25th of Feb, 25th of March. I am not bothered about values of hours but I don't know how to create its logic as every month has different number of days.
You can generate the day of year number by getting the datenum values for the 25th of each month and subtracting the datenum of the 1st Jan that year.
dayIdx = datenum(2022,1:12,25) - datenum(2022,1,1) + 1;
Then just use this as your column index
plot(x(12,dayIdx), y(12,dayIdx))
The choice of 2022 above is arbitrary, as long as you pick a non-leapyear to get the 365-day year correct.
The datetime data type is awesome for this type of work.
%Make a vector of datetimes
ts = ( datetime(2001,1,1,0,0,0):hours(1):datetime(2001,12,31,023,0,0) )';
%Find the datetimes which are on the 25th day of the month, and the 12th
%hour of the day
mask = (day(ts) == 25) & (hour(ts) == 12) ;
%Confirm
ts(mask)
The result is below.
(You likely want to use the mask variable itself for your task. Sometimes you want to use find on the logical statement to get a list of indexes instead.)
ans =
12×1 datetime array
25-Jan-2001 12:00:00
25-Feb-2001 12:00:00
25-Mar-2001 12:00:00
25-Apr-2001 12:00:00
25-May-2001 12:00:00
25-Jun-2001 12:00:00
25-Jul-2001 12:00:00
25-Aug-2001 12:00:00
25-Sep-2001 12:00:00
25-Oct-2001 12:00:00
25-Nov-2001 12:00:00
25-Dec-2001 12:00:00

Subtracting 1 ISO 8601 year from a date in BigQuery

I'm trying to manipulate a date value to go back in time exactly 1 ISO-8601 year.
The following does not work, but best describes what I want to accomplish:
date_add(date '2018-01-03', interval -1 isoyear)
I tried string conversion as an intermediate step, but that doesn't work either:
select parse_date('%G%V%u',safe_cast(safe_cast(format_date('%G%V%u',date '2018-01-03') as int64)-1000 as string))
The error provided for the last one is "Failed to parse input string "2017013"". I don't understand why, this should always resolve to a unique date value.
Is there another way in which I can subtract an ISO year from a date?
This gives the corresponding day of the previous ISO year by subtracting the appropriate number of weeks from the date. I based the calculation on the description of weeks per year from the Wikipedia page:
CREATE TEMP FUNCTION IsLongYear(d DATE) AS (
-- Year starting on Thursday
EXTRACT(DAYOFWEEK FROM DATE_TRUNC(d, YEAR)) = 5 OR
-- Leap year starting on Wednesday
(EXTRACT(DAY FROM DATE_ADD(DATE(EXTRACT(YEAR FROM d), 2, 28), INTERVAL 1 DAY)) = 29
AND EXTRACT(DAYOFWEEK FROM DATE_TRUNC(d, YEAR)) = 4)
);
CREATE TEMP FUNCTION PreviousIsoYear(d DATE) AS (
DATE_SUB(d, INTERVAL IF(IsLongYear(d), 53, 52) WEEK)
);
SELECT PreviousIsoYear('2018-01-03');
This returns 2017-01-04, which is the third day of the 2017 ISO year. 2018-01-03 is the third day of the 2018 ISO year.

RR MILLENNIUM equivalent in Postgres

Is there a built in function in PostgreSQL 9.5 version to calculate the appropriate century/millenium?
When I use birth_date::TIMESTAMP from a table, sometimes it prefix 19 and sometimes it prefix 20. Below example
Input:
28JUN80
25APR48
Output:
"1980-06-28 00:00:00"
"2048-04-25 00:00:00"
I also have records in the table with birth_date holding values like "07APR1963" which gets computed appropriately as "1963-04-07 00:00:00".
I need use CASE statement when the length is 7 characters, then prefix with 19 millennium and when its 9 characters, just load it as it is.
https://en.wikipedia.org/wiki/Unix_time Unix epoch is
beginning (00:00:00 1 January 1970)
So if you don't specify the century, but just last YY it will be 20th century from 00:00:00 1 January and 21st century before YY equal 70. If you want it to guess the 20th century either append year as you do, or specify CC, eg:
t=> select
to_timestamp('1JAN70', 'ddmonYY')
, to_timestamp('31DEC69', 'ddmonyy')
, to_timestamp('31DEC69 20', 'ddmonyy cc');
to_timestamp | to_timestamp | to_timestamp
------------------------+------------------------+------------------------
1970-01-01 00:00:00+00 | 2069-12-31 00:00:00+00 | 1969-12-31 00:00:00+00
(1 row)
https://www.postgresql.org/docs/current/static/functions-formatting.html
In conversions from string to timestamp or date, the CC (century)
field is ignored if there is a YYY, YYYY or Y,YYY field. If CC is used
with YY or Y then the year is computed as the year in the specified
century. If the century is specified but the year is not, the first
year of the century is assumed.
update
So in your case you should do smth like:
vao=# create table arasu (member_birth_date character(9)); insert into arasu values ('28JUN80'),('25APR48');
CREATE TABLE
INSERT 0 2
vao=# select to_timestamp(member_birth_date||' 20', 'ddmonYY cc') from arasu;
to_timestamp
------------------------
1980-06-28 00:00:00+03
1948-04-25 00:00:00+03
(2 rows)

Merging average of time series corresponding to time span in a different data set

I have two datasets, one with contracts and one with market prices. The gist of what I am trying to accomplish is to find the average value of a time series that corresponds to a period of time in a cross-sectional data set. Please see below.
Example Dataset 1:
Beginning Ending Price
1/1/2014 5/15/2014 $19.50
3/2/2012 10/9/2015 $20.31
...
1/1/2012 1/8/2012 $19.00
In the example above there are several contracts, the first spanning from January 2014 to May 2014, the second from March 2012 to October 2015. Each one has a single price. The second dataset has weekly market prices.
Example Dataset 2:
Date Price
1/1/2012 $18
1/8/2012 $17.50
....
1/15/2015 $21.00
I would like to find the average "market price" (i.e. the average of the price in dataset 2) between the beginning and ending period for each contract on dataset 1. So, for the third contract from 1/1/2012 to 1/8/2012, from the second dataset the output would be (18+17.50)/2 = 17.75. Then merge this value back to the original dataset.
I work with Stata, but can also work with R or Excel.
Also, if you have a better suggestion for a title I would really appreciate it!
You can cross the contracts cross section data with the time series, which forms every pairwise combination, drop the prices from outside the date range, and calculate the mean like this:
/* Fake Data */
tempfile ts ccs
clear
input str9 d p_daily
"1/1/2012" 18
"1/8/2012" 17.50
"1/15/2015" 21.00
end
gen date = date(d,"MDY")
format date %td
drop d
rename date d
save `ts'
clear
input id str8 bd str9 ed p_contract
1 "1/1/2014" "5/15/2014" 19.50
2 "3/2/2012" "10/9/2015" 20.31
3 "1/1/2012" "1/8/2012" 19.00
end
foreach var of varlist bd ed {
gen date = date(`var',"MDY")
format date %td
drop `var'
rename date `var'
}
save `ccs'
/* Calculate Mean Prices and Merge Contracts Back In */
cross using `ts'
sort id d
keep if d >= bd & d <=ed
collapse (mean) mean_p = p_daily, by(id bd ed p_contract)
merge 1:1 id using `ccs', nogen
sort id
This gets you something like this:
id p_contract bd ed mean_p
1 19.5 01jan2014 15may2014 .
2 20.31 02mar2012 09oct2015 21
3 19 01jan2012 08jan2012 17.75

Qlikview - Data between dates; filter out data past or future data depending on selected date

I've seen threads where the document has Start Date and End Date "widgets" where users type in their dates, however, I'm looking for a dynamic solution, for example on the table below, when I select a date, say "1/1/2004", I only want to see active players (this would exclude Michael Jordan only).
Jersey# Name RookieYr RetirementYr Average PPG
23 Michael Jordan 1/1/1984 1/1/2003 24
33 Scotty Pippen 1/1/1987 1/1/2008 15
1 Derrick Rose 1/1/2008 1/1/9999 16
25 Vince Carter 1/1/1998 1/1/9999 18
The most flexible way is to IntervalMatch the RookieYr * RetireYr dates into a table of all dates. See http://qlikviewcookbook.com/recipes/download-info/count-days-in-a-transaction-using-intervalmatch/ for a complete example.
Here's the interval match for your data. You'll can obviously create your calendar however you want.
STATS:
load * inline [
Jersey#, Name, RookieYr, RetirementYr, Average, PPG
23, Michael Jordan, 1/1/1984, 1/1/2003, 24
33, Scotty Pippen, 1/1/1987, 1/1/2008, 15
1, Derrick Rose, 1/1/2008, 1/1/9999, 16
25, Vince Carter, 1/1/1998, 1/1/9999, 18
];
let zDateMin=37000;
let zDateMax=40000;
DATES:
LOAD
Date($(zDateMin) + IterNo() - 1) as [DATE],
year( Date($(zDateMin) + IterNo() - 1)) as YEAR,
month( Date($(zDateMin) + IterNo() - 1)) as MONTH
AUTOGENERATE 1
WHILE $(zDateMin)+IterNo()-1<= $(zDateMax);
INTERVAL:
IntervalMatch (DATE) load RookieYr, RetirementYr resident STATS;
left join (DATES) load * resident INTERVAL; drop table INTERVAL;
There's not much to it you need to load 2 tables one with the start and end dates and one with the calendar dates then you interval match the date field to the start and end field and from there it will work the last join is just to tidy up a bit.
The result of all of that is this ctrl-t. Don't worry about the Syn key it is required to maintain the interval matching.
Then you can have something like this.
Derrick Rose is also excluded since he had not started by 1/1/2004