Hello i have varchar column with datetime.
its utc
|2022-02-09T13:27:17.165116Z
|2022-02-09T13:27:27.3850848Z
|2022-02-09T13:28:18.5537284Z
on select query parse as by standart but return 'null' for all rows:
PARSE_TIMESTAMP(entrydate, 'yyyy-MM-ddTHH:mm:ss.SSSZZZZ')
code:
JsonSerializer.Serialize(DateTime.UtcNow);
This is because ksqldb only supports up to millisecond precision see https://www.confluent.io/blog/ksqldb-2-0-introduces-date-and-time-data-types/
There is a github issue tracking support for finer time resolution like microseconds here https://github.com/confluentinc/ksql/issues/8243
The problem is your DateFormat, yyyy-MM-ddTHH:mm:ss.SSSZZZZ which expects a precision of milliseconds but, from your input, it's microseconds.
To cover all the scenarios (as you are serializing it with JsonSerializer.Serialize, and it can happen that you don't even have anything after the seconds, or, milliseconds is zero: 2022-02-09T13:28:18Z) you would need to use the following format:
'yyyy-MM-dd''T''HH:mm:ss[.SSSSSSS][.SSSSSS][.SSSSS][.SSSS][.SSS][.SS][.S]''Z'''
This will cover all the use cases with/without the milliseconds/microseconds precision
More about the formatting:
https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html
So we use the following formatting options to specify that the precision is optional:
[ optional section start
] optional section end
and
S fraction-of-second fraction 978
note that you also have to 'escape' the letters:
https://docs.ksqldb.io/en/latest/developer-guide/ksqldb-reference/create-stream/#timestamp_format
Possible improvement (give it a try):
PARSE_TIMESTAMP(entrydate, 'yyyy-MM-dd''T''HH:mm:ss[.SSSSSSS][.SSSSSS][.SSSSS][.SSSS][.SSS][.SS][.S]X', 'UTC')
(you have to specify timezone so that it can parse the X to a correct timestamp including the zone)
from the Java DateTimeFormatter docs:
Pattern letter 'X' (upper case) will output 'Z' when the offset to be output would be zero, whereas pattern letter 'x' (lower case) will output '+00', '+0000', or '+00:00'.
Related
I'm using ffprobe to get the sexagesimal time and after trimming the last three (unnecessary) digits I get the following format:
#examples
0:05:51.15
11:03:15.24
Is there a way to add these 2 so that the result is 11:09:06.39 ?
You can cast the strings to type [timespan], which allows you to perform arithmetic on the resulting objects, and to apply custom formatting on output:
PS> ([timespan] ' 0:05:51.15' + [timespan] '11:03:15.24').ToString('h\:mm\:ss\.ff')
11:09:06.39
Note: If there's a chance that the resulting time span exceeds 24 hours, more work is needed.[1]
Note how \-escaping must be used to specify the output separators as literals.
In this case, the simpler .ToString('g') would have yielded the same output, but only in cultures that use . as the decimal separator, because the standard g format specifier is culture-sensitive.
See the [timespan]::ToString() documentation as well as the documentation on standard and custom time-span format specifiers.
By contrast, PowerShell uses the invariant culture when interpreting the input format cast to [timespan], where . is the decimal separator; similarly, using a [timespan] instance in expandable strings yields a culture-invariant representation; e.g.:
[timespan] '11:03:15.24' always works, irrespective of the current culture, because the invariant culture expects . as the decimal separator.
"$([timespan] '1' - 1)" always yields 23:59:59.9999999, irrespective of the current culture[2].
As No Refunds No Returns notes, if you're dealing with differently formatted input, possibly from a different culture, you can use [timespan]::Parse() / [timespan]::ParseExact() / [timespan]::TryParseExact()
Parsing the standard formats of a given culture:
[timespan]::Parse(' 0:05:51,15', 'fr-FR') # OK
Note the , as the decimal separator.
If you omit the culture argument (or pass $null), the current culture is applied. Note how that differs from using a [timespan] cast, which is always culture-invariant (and assumes . as the decimal separator).
Parsing with a custom format:
[timespan]::ParseExact(' 0:05:51,15'.Trim(), 'h\:mm\:ss\,ff', $null) # OK
Note that using such a literal custom format is never culture-sensitive, because all the separators must be specified as - escaped - literals (e.g., \:), so $null is passed as the culture argument (IFormatProvider).
Conversely, passing a specific culture only makes sense with the culture-sensitive standard format specifiers, g and G.
Parsing with a culture-aware custom format:
If you don't know what culture will be in effect at runtime, but you want to respect that culture's decimal separator in combination with a custom format, you need to dynamically embed the current culture's decimal separator in your custom format string:
$tsText = ' 0:05:51.15'
[timespan]::ParseExact($tsText.Trim(),
('h\:mm\:ss\{0}ff' -f [cultureinfo]::CurrentCulture.NumberFormat.NumberDecimalSeparator),
$null
) # OK in cultures that use "." as the decimal separator
[1] h and hh only ever reflect the hours not included in full days in the input time span. To reflect the number of days too, prepend something like d\. - there is no format specifier that allows you express the total number of hours across multiple days, but you can use general-purpose string formatting to achieve that - do note, however, that you'll also need custom parsing code in order to convert the resulting string back to a [timespan] instance:
$ts = [timespan] ' 1:05:51.15' + [timespan] '23:03:15.24'
'{0}:{1:mm\:ss\.ff}' -f [Math]::Floor($ts.TotalHours), $ts
[2] At the .NET level, calling .ToString() on objects typically yields a culture-sensitive representation (if the type supports it), but with [timespan] the output happens to be culture-invariant. By contrast, PowerShell explicitly uses the invariant culture behind the scenes in casts and string interpolation, so that "$var" and $var.ToString() may not yield the same representation - see this answer for details.
With java.time , I'm trying to format the time as the following "2018-03-15T23:47:15+01:00" .
With this formatter I'm close to the result in Scala.
val formatter: DateTimeFormatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ssZ")
ZonedDateTime.now() // 2018-03-14T19:25:23.397+01:00
ZonedDateTime.now().format(formatter) // => 2018-03-14 19:25:23+0100
But I cannot insert the extra character "T" between the day and hour.
What does this "T" mean BTW ?
How to format as "2018-03-15T23:47:15+01:00" ?
Notes:
In case you wonder why LocalDateTime cannot be formatted
Format LocalDateTime with Timezone in Java8
Try this
val ZONED_DATE_TIME_ISO8601_FORMATTER3 = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSSxxx")
ZonedDateTime.now().format(ZONED_DATE_TIME_ISO8601_FORMATTER3)
// 2018-03-14T19:35:54.321+01:00
See here
Offset X and x: This formats the offset based on the number of pattern letters. One letter outputs just the hour, such as '+01', unless the minute is non-zero in which case the minute is also output, such as '+0130'. Two letters outputs the hour and minute, without a colon, such as '+0130'. Three letters outputs the hour and minute, with a colon, such as '+01:30'. Four letters outputs the hour and minute and optional second, without a colon, such as '+013015'. Five letters outputs the hour and minute and optional second, with a colon, such as '+01:30:15'. Six or more letters throws IllegalArgumentException. Pattern letter 'X' (upper case) will output 'Z' when the offset to be output would be zero, whereas pattern letter 'x' (lower case) will output '+00', '+0000', or '+00:00'.
Converting the ZonedDateTime to OffsetDateTime - as suggested in the other answers - works, but if you want to use a DateTimeFormatter, there's a built-in constant that does the job:
ZonedDateTime.now().format(DateTimeFormatter.ISO_OFFSET_DATE_TIME)
But it's important to note some differences between all the approaches. Suppose that the ZonedDateTime contains a date/time equivalent to 2018-03-15T23:47+01:00 (the seconds and milliseconds are zero).
All the approaches covered in the answers will give you different results.
toString() omits seconds and milliseconds when they are zero. So this code:
ZonedDateTime zdt = // 2018-03-15T23:47+01:00
zdt.toOffsetDateTime().toString()
prints:
2018-03-15T23:47+01:00
only hour and minute, because seconds and milliseconds are zero
The built-in formatter will omit only the milliseconds if it's zero, but it'll print the seconds, regardless of the value. So this:
zdt.format(DateTimeFormatter.ISO_OFFSET_DATE_TIME)
prints:
2018-03-15T23:47:00+01:00
seconds printed, even if it's zero; milliseconds ommited
And the formatter that uses an explicit pattern will always print all the fields specified, regardless of their values. So this:
zdt.format(DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSSxxx"))
prints:
2018-03-15T23:47:00.000+01:00
seconds and milliseconds are printed, regardless of their values
You'll also find a difference in values such as 2018-03-15T23:47:10.120+01:00 (note the 120 milliseconds). toString() and ofPattern will give you:
2018-03-15T23:47:10.120+01:00
While the built-in DateTimeFormatter.ISO_OFFSET_DATE_TIME will print only the first 2 digits:
2018-03-15T23:47:10.12+01:00
Just be aware of these details when choosing which approach to use.
As your question already shows, you may just rely on ZonedDateTime.toString() for getting a string like 2018-03-14T19:25:23.397+01:00. BTW, that string is in ISO 8601 format, the international standard. Only two minor modifications may be needed:
If you don’t want the fraction of second — well, I don’t see what harm it does, it agrees with ISO 8601, so whoever receives your ISO 8601 string should be happy to have it. But if you don’t want it, you may apply myZonedDateTime.truncatedTo(ChronoUnit.SECONDS) to get rid of it.
ZonedDateTime.toString() often appends a zone name, for example 2018-03-14T19:25:23+01:00[Europe/Paris], which is not part of the ISO 8601 standard. To avoid that, convert to OffsetDateTime before using its toString method: myZonedDateTime.toOffsetDateTime().toString() (or myZonedDateTime.truncatedTo(ChronoUnit.SECONDS).toOffsetDateTime().toString()).
Building your own formatter through a format pattern string is very flexible when this is what you need. However, very often we can get through with less (and then should do for the easier maintainability of our code): toString methods or built-in formatters including both the ISO ones and the localized ones that we can get from DateTimeFormatter.ofLocalizedPattern().
What does this "T" mean BTW ?
The T is part of the ISO 8601 format. It separates the date part from the time-of-day part. You may think of it as T for time since it denotes the start of the time part. If there is only a date (2018-04-25) or only a time-of-day (21:45:00), the T is not used, but when we have both, the T is required. You may think that the format might have been specified without the T, and you are probably right. When it comes to the format for periods/durations it is indispensable, however, and also needed when there are no days: P3M means a period of 3 months, while PT3M means 3 minutes.
Link: Read more in the Wikipedia article on ISO 8601.
I'm trying to achieve something like this in a rundeck 2.6 job:
touch /foo/bar/${DATE:MM/dd/yyyy}-baz
but it doesn't work properly and the date is not interpreted at all. Is there a proper way to do this?
You can use this bash script :
#!/bin/bash
touch /foo/bar/`date "+%m/%d/%Y"`-baz
The backquotes act as command substitution and replace the output of the date command in the touch command.
According to the date man page :
An operand with a leading plus (`+') sign signals a user-defined format string which
specifies the format in which to display the date and time. The format string may contain any of the conversion specifications described in the strftime(3) manual page, as
well as any arbitrary text.
The date format string use the following conversion specifier character :
%m The month as a decimal number (range 01 to 12). (Calculated
from tm_mon.)
%d The day of the month as a decimal number (range 01 to 31).
(Calculated from tm_mday.)
%Y The year as a decimal number including the century.
(Calculated from tm_year)
You can also define an option that uses that date format specifier.
Set the default value of the option to use the specifier. Eg:
<option name="date" value="${DATE:MM/dd/yyyy}-baz" />
Inside your step reference the ${option.date}.
How can one produce an ISO date string "yyyy-MM-dd" from a Q date type? I looked at concatenating the various parts but am not even able to get the day/month, e.g. d:2015.12.01;d.month prints 2015.12, i.e. more than just the month.
If you plan to do it on a large scale (i.e. a large vector/list of dates or a column in a table) and you're sure your dates are always well-formed, then you could use a dot-amend:
q)update .[;(::;4 7);:;"-"]string date from ([] date:2#.z.D)
date
------------
"2016-01-04"
"2016-01-04"
This way you wouldn't have to apply to "each" entry of the vector/list, it works on the vector/list itself.
q)"-" sv "." vs string[2015.12.01]
"2015-12-01"
vs vector from string, splits by "." above;
sv string to vector, join by "-" above.
Remember a string is just a char array, so you can grab each part as you require with indexing. But the above is useful as the resulting vector of vs gives a 3-length vector that you manipulate any way you like
I believe the shortest (and cleanest) option for ISO8601 UTC timestamp available since at least kdb v3.4 would be to use .h.iso8601 builtin
i.e.
q).h.iso8601 .z.p
"2020-11-09T15:42:19.292301000"
Or, if you just need milliseconds similar to what JS toISOString() does, use:
q).isotime:{(23#.h.iso8601 x),"Z"}
q).isotime[.z.p]
"2020-11-09T16:02:02.601Z"
q).isotime[2015.12.01]
"2015-12-01T00:00:00.000Z"
Note .z.p is important, as .h.iso8601 .z.P would silently give you local time without timezone (+0100 etc) so it would still be interpreted as UTC by compliant ISO8601 parser :(
Check-out this GitHub library for datetime formatting. It supports the excel way of formatting date and time. It might not be the right fit for formatting a large number of objects.
q).dtf.format["yyyy-mm-dd"; 2018.06.08T01:02:03.456]
"2018-06-08"
time formatting :
q).dtf.format["yyyy-mmmm-dd hh:uu AM/PM"; 2018.01.08T01:02:03.456]
"2018-January-08 01:02 AM"
I am using something like this:
q)ymd:{[x;s](4#d),s,(2#-5#d),s,-2#d:string[x]}
q)ymd[.z.D;"-"]
"2016-01-25"
q)ymd[.z.D;"/"]
"2016/01/25"
q)ymd[.z.D;""]
"20160125"
Or for tables:
q)t:([]a:5#1;5#.z.d)
q)update s:ymd[;"-"] each d from t
a d s
-------------------------
1 2016.01.26 "2016-01-26"
1 2016.01.26 "2016-01-26"
1 2016.01.26 "2016-01-26"
1 2016.01.26 "2016-01-26"
1 2016.01.26 "2016-01-26"
Please change the separator like - or / in the update statement.
update s:{ssr[string x;".";y]}'[d;"-"] from ([]a:5#1;5?.z.d)
a d s
-------------------------
1 2010.12.31 "2010-12-31"
1 2012.08.24 "2012-08-24"
1 2004.12.05 "2004-12-05"
1 2000.10.02 "2000-10-02"
1 2006.09.10 "2006-09-10"
perl code:
#!/bin/env perl
use DateTime;
print DateTime->now;
OUTPUT:
2013-01-28T06:02:33
what's mean of 'T' letter in the output string ?
ISO8601 and RFC3339 both use "T" to join the date and the time, and DateTime's default stringifier apparently adopted that convention. If you want another format, you can use one of the DateTime::Format::* modules or ->strftime.
my $now = DateTime->now( time_zone => 'local' );
say $now->strftime('%Y-%m-%d %H:%M:%S');
It stands for "Time". You can read more at:
http://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations
It's the separator between date and time in ISO 8601. It's always a T in this format.
The character [T] shall be used as time designator to indicate the start of the representation of the time of day
component in these expressions. The hyphen [-] and the colon [:] shall be used, in accordance with 4.4.4, as
separators within the date and time of day expressions, respectively, when required.
NOTE By mutual agreement of the partners in information interchange, the character [T] may be omitted in
applications where there is no risk of confusing a date and time of day representation with others defined in this
International Standard.
--Data elements and interchange
formats — Information interchange —
Representation of dates and times, ISO 8601:2004(E)