I have timestamps in some log files in this format: 12-Jun-2020 11:56:23.272 and I am trying parse these in elasticsearch using this mapping definition: "time_stamp": {"type": "timestamp", "format": "dd-MMM-yyyy HH:mM:ss.SSS" }
I have tried various variations like d-MMM-y H:m:s.S
but always get "reason":"Invalid format: \"12-Jun-2020 11:56:23.272\" is malformed
I have been using ES manual and joda docs but find the joda specs leave a lot of room for interpretation. In particular should it be dd or d for the days. I found a tutorial here which has more examples and spells out that for moths Mar needs MMM and March MMMM
In the mappings have tried "type" of both date and timestamp and that does not appear to make any difference.
Related
Is there a way to change the date format in AnyLogic, without constructing something with the implemented Time functions?
I want my Date to be displayed in such a way
"E dd.MM.yyyy HH:mm"
With the timefunctions I have problems, to get the date into the right format
Not sure what that E stands for, but if it's the timezone, you can get it with this:
ZoneId.of("Europe/Oslo").getDisplayName(TextStyle.SHORT_STANDALONE, Locale.ENGLISH)
I used Europe/Oslo there, but you can find the zone ids that you need here:
https://docs.oracle.com/middleware/12211/wcs/tag-ref/MISC/TimeZones.html
to use this you need to import libraries in Main/Advanced Java/Imports section
import java.time.ZoneId;
import java.time.format.TextStyle;
And then to get the required code with that format:
"E "+String.format("%02d", getDayOfMonth())+"."
+String.format("%02d",getMonth()+1)+"."
+String.format("%04d",getYear())+" "
+String.format("%02d",getHourOfDay())+":"
+String.format("%02d",getMinute())
You can replace the "E " with what I explained before
Or you can use whatever is currently your timezone... AnyLogic uses your computer timezone in order to know what timezone you are in:
Calendar cal=Calendar.getInstance();
cal.setTime(date());
String timezoneCode=cal.getTimeZone().toZoneId().getDisplayName(TextStyle.SHORT_STANDALONE, Locale.ENGLISH);
I read from a very old post here on stackoverflow that joda is a possible solution to convert Unix timestamp.
import org.joda.time._
new DateTime(1511544070).toString("yyyy-MM-dd")
I got 1970-01-18 for this case, however, this is wrong because the date should be
according to this online converter: 11/24/2017 # 5:21pm (UTC)
It is possible the online converter is correct because the sample unix timestamp 1511544070 is from a dataset that date range is November 25 to December 03, 2017, the dataset is from China time which is 8 hours ahead of UTC, meaning 11/24/2017 # 5:21pm (UTC) is actually 11/25/2017 # 1:21am (Beijing Time)
Where can I get a working library or is there a working library that can get the same result like the online converter?
You can do that using java.time:
import java.time.{ LocalDateTime, ZoneOffset }
import java.time.format.DateTimeFormatter
LocalDateTime.ofEpochSecond(1511544070, 0, ZoneOffset.UTC)
.format(DateTimeFormatter.ofPattern("yyyy-MM-dd # h:mm a"))
Looking at the documentation for joda-time we see that a DateTime can take a Long specifying the milliseconds since 1 Jan 1970. However, you seem to be providing a value in seconds. Joda-time is actually calculating it correctly, since since 1511544070/(1000*3600*24) equals 17.49 days, i.e. 1970-01-18.
To get the expected result multiply with 1000:
new DateTime(1511544070*1000).toString("yyyy-MM-dd")
To get the time in another timezone, add withZone() as follows (for Shanghai/Beijing):
new DateTime(1511544070*1000).toString("yyyy-MM-dd")
.withZone(DateTimeZone.forID("Asia/Shanghai"))
Have a string object with a specific format of date.
Need to check if that dateStr is after the current time on local machine.
Having trouble with conversions and LocalDateTime
String dateStr = "Oct 27 2017 02:29:00 GMT+0000";
public static final String DATE_FORMAT = "MMM dd yyyy HH:mm:ss zzzZ";
I know something is fishy in the below code with the usage of LocalDateTime
public static boolean isFutureDate(String dateStr){
DateTimeFormatter formatter = DateTimeFormatter.ofPattern(DATE_FORMAT);
LocalDateTime dateTime = LocalDateTime.parse(dateStr, formatter);
return(dateTime.isAfter(LocalDateTime.now()));
}
Trouble is with timezones and date conversions.
Please help find the right way of checking if a dateStr is after the current local date this in Java 8?
Local… types have no time zone
You are using the wrong type for your data.
The Local… types including LocalDateTime purposely have no concept of time zone or offset-from UTC. As such they not represent a moment on the time line, only rough idea of a range of possible moments. Use LocalDateTime only when the time zone is unknown or irrelevant; never use it for an actual moment in history.
Use OffsetDateDate for values with an offset-from-UTC, a number of hours and minutes.
Use ZonedDateTime for values with an assigned time zone. A time zone such as Asia/Kolkata or America/Montreal is a particular region’s history of past, present, and future changes to its offset-from-UTC. Anomalies such as Daylight Saving Time (DST) mean a change to the offset.
If you know all your inputs are in GMT/UTC, use OffsetDateTime. If the inputs may use time zones, parse as ZonedDateTime objects.
This input data format is terrible. If you have any control, use standard ISO 8601 formats instead when exchanging date-time values as text.
All this has been covered many times already on Stack Exchange. Please search more thoroughly before posting. And search Stack Overflow to learn more. I kept my Answer here brief, as this is a duplicate.
When parsing to a LocalDateTime, you're ignoring the offset (+0000), and I'm not sure if that's what you really want.
In this case, the +0000 offset means the date/time is October 27th 2017 at 02:29 AM in UTC. When you parse to a LocalDateTime, you're ignoring the offset (so it represents only "October 27th 2017 at 02:29 AM", not attached to any timezone) and comparing to your local date/time (or the current date/time in the JVM's default timezone).
If you want to make a comparison that also considers the offset, you can parse it to OffsetDateTime and convert to Instant to compare it with the actual UTC instant, regardless of the timezone.
Also, the month name is in English (I'm assuming it's English, but you can change this accordingly), so you must a java.util.Locale in the formatter (if you don't set a locale, it'll use the JVM default, and it's not guaranteed to always be English):
// parse to OffsetDateTime (use the same formatter)
String dateStr = "Oct 27 2017 02:29:00 GMT+0000";
DateTimeFormatter fmt = DateTimeFormatter.ofPattern("MMM dd yyyy HH:mm:ss zzzZ", Locale.US);
OffsetDateTime odt = OffsetDateTime.parse(dateStr, fmt);
// compare Instant's
System.out.println(odt.toInstant().isAfter(Instant.now()));
Although it works for you now, keep in mind that the default locale can be changed without notice, even at runtime. If your input has locale-sensitive date (such as month names), it's better to specify it as above.
I have a ISO8601 string (e.g. date="2015-07-10T04:31:25") I need to convert this to the format:
July 7, 2015, 4:31:25 PM (EDT)
Even though I can write a template and use substring() to transform the string in the date time format. However I am not sure how to achieve the AM/PM and time zone information?
Working code templates would be highly appreciated. Thanks!
I am not sure how to achieve the AM/PM ...
It can be calculated from the hour component as:
substring('AMPM', 1 + 2*(number($hour) > 11), 2)
Of course, in the given input, where $hour would be "04", the correct result is "AM", not "PM".
... and time zone information?
Your input does not contain any time zone information, so unless you want to hard-code EDT as a string, there is no way to get it.
I'm having some problems converting a string to a date object in google apps script.
My dates are in the following format, from a 3rd party API:
2013-01-17T17:34:50.507
I am attempting to convert this to a Date object:
return Date(stringDate);
And this is being returned:
Thu Jan 01 01:00:00 GMT+01:00 1970
Can someone tell me what i'm doing wrong, and how to resolve this issue ?
With moment.js, it is as easy as this to parse any of ISO 8601 format.
var date = Moment.moment("2013-01-17T17:34:50.507").toDate();
You can use moment.js to parse your arbitrary date string as well.
To use moment.js in GAS, you just need to add it in the script editor.
Open your script in GAS script editor and go to "Resources" then "Libraries...", then put this project key MHMchiX6c1bwSqGM1PZiW_PxhMjh3Sh48 and click "Add". Choose the version from the dropdown, then click "Save". Now, you are ready to use moment.js in GAS.
moment.js can be used to parse date string, create formatted date string, and many other date manipulation. Thanks to the author!
You can find the moment.js documentation here.
It doesn't appear that the Date object knows how to handle that date. The date is in ISO 8601 format. Javascript can handle Dates if they are given timezone information.
You will have to do some testing, but if those dates given to you are in UTC time, then just add a Z to the end of the date string before you call new Date().
Edit: The Apps Script Date object can't seem to handle a timezone other than UTC when parsing a Date. I opened an issue for it.
It doesn't work in GScript, at least for me at the time I'm writing it.
This post serves a working alternative: How do I format this date string so that google scripts recognizes it?
As of now new Date() seems to work:
var dT = new Date("2013-01-17T17:34:50.507");
console.info("dT: %s or %d", dT, dT.getTime());
returns dT: Thu Jan 17 17:34:50 GMT+01:00 2013 or 1.358440490507E12 in Google Apps Script