Convert E, d M y HH:mm:ss x string date format to yyyy-mm-dd with databricks - pyspark

everything is in the title. I tried this without success so far:
date_format(to_date(col("data")["createdAt"], 'E, d M y HH:mm:ss x'), 'yyyy-mm-dd')
all values returned are null
This is an example of input Fri, 12 Jun 20 07:49:17 +0000

to_date does not accept weekdays (E), so you need to remove that using split first. Also you need to fix the formats (note capitalization):
date_format(to_date(split(col("data")["createdAt"], ', ')[1], 'd MMM yy HH:mm:ss Z'), 'yyyy-MM-dd')

Related

datetime timezone order google sheets yyyy-mm-dd hh:mm:ss GMT to dd-mm-yyyy hh:mm:ss GMT

I have a Google sheet that needs timestamp in one column as is '2022-09-11 00:13:50 GMT' and in another column I need to convert to '11-09-2022 00:13:50 GMT'
I have tried:
copying cells into new column and reformatting but it isn't working
=datetime(A1,"dd/mm/yyyy hh:mm:ss")
=TEXT(A1,"dd/mm/yyyy hh:mm:ss")
=date(A1,"dd/mm/yyyy hh:mm:ss")
=Timestamp(A2,"dd/mm/yyyy hh:mm:ss")
Note the format column A is in is how it imports from another fixed sheet
I probably just too tired and missing the most obvious thing but I can't figure it out
https://docs.google.com/spreadsheets/d/1alTL1z1ZXIr8rjF6hFHmJuD81OFK7TmqFdnC8ILIPwI/edit?usp=sharing
try:
=ARRAYFORMULA(REGEXREPLACE(A1:A, "(\d{4})-(\d+)-(\d+) (.+)", "$3/$2/$1 $4"))
or:
=ARRAYFORMULA(IF(A1:A="",,REGEXREPLACE(A1:A,
"(\d{4})-(\d+)-(\d+) (.+) GMT", "$3/$2/$1 $4")*1))
Put this formula in cell D1:
=arrayformula( iferror( 1 / value( regexreplace(A1:A, " GMT", "") ) ^ -1 ) )
Then format column D through Format > Number > Custom date and time.
Step 01
Use this formula
=ArrayFormula(IF(A1:A="",,DATEVALUE(REGEXEXTRACT(A1:A, "(.+?) "))+
TIMEVALUE(REGEXEXTRACT(A1:A, " (.+?) "))))
Step 02 - Format
Result

How to use formatDateTime in Azure Data Factory?

I would like to format the pipeline trigger time in this format:
10-Mar-2021 08:31:59 AM
Here is the code I am using:
#formatDateTime(pipeline().TriggerTime, 'DD-MON-YYYY HH:MI:SS AM/PM')
However the date is coming out in this format:
'DD-3ON-YYYY 08:3I:SS A3/P3'
How can I resolve this?
The formatDateTime function uses the custom date format strings which you can see listed here. A detailed breakdown
dd - the day of the month from 01 to 31. Note this is lower case and this format gives a leading 0. Using d for no leading 0
MMM - the abbreviated name of the month, eg JAN, FEB, MAR. Note this is upper case
yyyy - the year as a four-digit number. Note this is lower case
hh - the hour using a 12-hour clock from 01 to 12. Note this is lower case. As you are using AM/PM it makes sense to have the 12-hour clock rather than 24 (ie 23:00 PM would not make sense). Note this is lower case
mm - the minute from 00 to 50. Note this is lower case
ss - the second from 00 to 50. Note this is lower case
tt - the AM/PM designator. Note this is lower case
So using this information, your code should be more like this:
#formatDateTime(pipeline().TriggerTime, 'dd-MMM-yyyy hh:mm:ss tt')
I have test the expression using a Set Variable activity and got the following results:
EEE : Day ( Mon )
MMM : Month in words ( Dec )
MM : Day in Count ( 324 )
mm : Month ( 12 )
dd : Date ( 3 )
HH : Hours ( 12 )
mm : Minutes ( 50 )
ss : Seconds ( 34 )
yyyy: Year ( 2020 ) //both yyyy and YYYY are same
YYYY: Year ( 2020 )
zzz : GMT+05:30
aa : ( AM / PM )

Is there a way to make custom date formats in Julia?

I would like to be able to convert
"Tue Sep 01 00:00:26 +00:00 2020"
into a date type in Julia using the built in Date function.
I only need the year, month, and date.
This is tricky because you have a time zone here hence you need to use TimeZones.jl
using TimeZones, Dates
df = Dates.DateFormat("e u d H:M:S z y");
d = ZonedDateTime("Tue Sep 01 00:00:26 +00:00 2020", df)
Let se what we got:
julia> d = ZonedDateTime("Tue Sep 01 00:00:26 +00:00 2020", df)
2020-09-01T00:00:26+00:00
julia> Date(d)
2020-09-01
For more try typying ?DateFormat in the console - you will see the docs.
Code Matches Comment
–––––––– ––––––––– ––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
y 1996, 96 Returns year of 1996, 0096
Y 1996, 96 Returns year of 1996, 0096. Equivalent to y
m 1, 01 Matches 1 or 2-digit months
u Jan Matches abbreviated months according to the locale keyword
U January Matches full month names according to the locale keyword
d 1, 01 Matches 1 or 2-digit days
H 00 Matches hours (24-hour clock)
I 00 For outputting hours with 12-hour clock
M 00 Matches minutes
S 00 Matches seconds
s .500 Matches milliseconds
e Mon, Tues Matches abbreviated days of the week
E Monday Matches full name days of the week
p AM Matches AM/PM (case-insensitive)
yyyymmdd 19960101 Matches fixed-width year, month, and day
To parse dates you need the Dates.jl standard library. To parse this particular format though, I think you additionally need the TimeZones.jl package:
using Dates
using TimeZones # gives the `z` for the format below
fmt = dateformat"e u d H:M:S z y" # the format of your string
d = Date("Tue Sep 01 00:00:26 +00:00 2020", fmt)
Then you can simply look at d's values for example with:
julia> d
2020-09-01
julia> year(d)
2020
julia> month(d)
9
julia> day(d)
1

Change date time to specific format

I've a Date Time string as 27th October 2020 07:30 PM.
And I want it in format yyyy-mm-dd 07:30 PM.
Is there a way to convert in flutter?
If you import the intl package from pub.dev you can format the date any way you like.
https://pub.dev/packages/intl
You can see here in the docs where all the available premade datetime formatting constructors are. https://pub.dev/documentation/intl/latest/intl/DateFormat-class.html
ICU Name Skeleton
-------- --------
DAY d
ABBR_WEEKDAY E
WEEKDAY EEEE
ABBR_STANDALONE_MONTH LLL
STANDALONE_MONTH LLLL
NUM_MONTH M
NUM_MONTH_DAY Md
NUM_MONTH_WEEKDAY_DAY MEd
ABBR_MONTH MMM
ABBR_MONTH_DAY MMMd
ABBR_MONTH_WEEKDAY_DAY MMMEd
MONTH MMMM
MONTH_DAY MMMMd
MONTH_WEEKDAY_DAY MMMMEEEEd
ABBR_QUARTER QQQ
QUARTER QQQQ
YEAR y
YEAR_NUM_MONTH yM
YEAR_NUM_MONTH_DAY yMd
YEAR_NUM_MONTH_WEEKDAY_DAY yMEd
YEAR_ABBR_MONTH yMMM
YEAR_ABBR_MONTH_DAY yMMMd
YEAR_ABBR_MONTH_WEEKDAY_DAY yMMMEd
YEAR_MONTH yMMMM
YEAR_MONTH_DAY yMMMMd
YEAR_MONTH_WEEKDAY_DAY yMMMMEEEEd
YEAR_ABBR_QUARTER yQQQ
YEAR_QUARTER yQQQQ
HOUR24 H
HOUR24_MINUTE Hm
HOUR24_MINUTE_SECOND Hms
HOUR j
HOUR_MINUTE jm
HOUR_MINUTE_SECOND jms
HOUR_MINUTE_GENERIC_TZ jmv
HOUR_MINUTE_TZ jmz
HOUR_GENERIC_TZ jv
HOUR_TZ jz
MINUTE m
MINUTE_SECOND ms
SECOND s
You can also make your own formatting however you like. For example, I have my formatting like so:
DateFormat.yMMMd().format(transactions[index].date),
Do you have a DateTime type or a String type ?
DateTime date = DateTime.now();
print(date.toString())
Cause I get what you want with that code.
console :
2020-10-23 15:36:11.846645

Formatting the result of a query in google sheets UTC to local date

I built a formula to retrieve data from another sheet the following formula works, however, T is coming back as a UTC date time string and I would like the Date part only and in the format dd mmm yyyy e.g. 01 Jan 2018
UTC format being returned is 2018-11-14 10:06:32.416206+00:00
=iferror(
QUERY(DATA!A:AA, "
SELECT A,B, D, T, N
WHERE( AA CONTAINS '" & lower(B9) & "' AND L = TRUE)
Order by T Asc"))
I have tried adding
format T ‘dd-mmm-yyyy' but I guess I'm doing it wrong!
=iferror(
QUERY(DATA!A:AA, "
SELECT A,B, D, T, N
WHERE( AA CONTAINS '" & lower(B9) & "' AND L = TRUE)
Order by T Asc format T ‘dd-mmm-yyyy'"))
Could someone point me in the right direction please?
I suggest you extend your query by one column and in AB apply, copied down:
=left(T2,10)+mid(T2,12,15)-right(T2,5)
then use your first attempt with T replaced by AB and the dates in the result formatted:
dd mmm yyyy