I am trying to convert a long utc value into "yyyy-MM-dd HH:mm:ss" formatted pattern. I am expecting my data to be converted on 24 hours range scale and in GMT. My code passes all the test cases, I push the data into database using the jar that is newly built with this code -
dbRecord("order_dt_utc") = if (orderTs.isDefined) Some(new DateTime(orderTs.get, DateTimeZone.UTC).toString("yyyy-MM-dd HH:mm:ss")) else None
and now, when I query my database, I find that the data is still converting on 12 hours range. The query -
SELECT order_id, order_dt, order_dt_utc, order_ts_utc, from_unixtime(order_ts_utc/1000) FROM order_items where order_dt >= '2018-08-01' AND order_dt <= '2018-08-02' ORDER BY order_dt_utc LIMIT 1000;
And you can see the the values are not matching in the columns from_unixtime(order_ts_utc/1000) and order_dt_utc -
I am not able to figure the reason for this behaviour.
To convert Time Zone use the function first:
CONVERT_TZ (dateobj, oldtz, newtz)
After that use the date_format function:
date_format(from_unixtime(order_ts_utc), '%Y-%m-%d %H:%i:%s');
to format your time to 00-23 format.
Related
I have a Spark data frame with the column timestamp. I need to create event_hour in unix_timestamp format out of this column. The current issue is that the timestamp is in unix_timestamp format with a granularity of milliseconds while I need the granularity of hours.
Current values for timestamp:
1653192037
1653192026
1653192025
1653192024
1653192023
1653192022
Expected values:
1653192000
1653195600
1653199200
1653202800
How can I achieve that using Spark functions?
I've already tried to convert it to timestamp and then format it but I got null as the result:
inputDf
.withColumn("event_hour", unix_timestamp(date_format($"timestamp".cast(TimestampType), "MM-dd-yyyy HH")))
A (not very explicit but) efficient way would be to use modulus operation with 3600 (as 3600 seconds = 1 hour):
timestamp_hour = timestamp_second - (timestamp_second % 3600)
This assumes you are manipulating data as numeric.
You can use DateUtils API,
import org.apache.commons.lang3.time.DateUtils;
Long epochTimestamp_hour = DateUtils.truncate(Timestamp_column, Calendar.HOUR)).getTime();
create new column of type timestamp
use that column to truncate timestamp to epochTimestamp_hour
I have a timestamp pulled from MongoDB
example: 2007-01-01 01:00:00
I need it to be a simple date: 2007-01-01
Been looking at: Convert UNIX epoch to Date object
Having a hard time formatting
Using sources:
Convert column in data.frame to date
I had to assess if the class that I was trying to convert was a class that as.Date was able to take.
Turns out it was a data.frame class
Used the following to convert it:
dat_dump %>%
mutate( date = as.Date(date, format = "%Y-%m-%d"))
I have a set of data stored in neo4j in milliseconds.
I'm trying to get the data then change it to date format and compare it to today's date in where clause to get Media posts for today only.
I have tried with these
MATCH (media:Media)
RETURN date(datetime({millisecond:media.dateCreated}))
and it returns
Neo.ClientError.Statement.ArgumentError: year must be specified
next, I've tried
MATCH (media:Media)
RETURN apoc.date.field(media.dateCreated)
and it returns
Neo.ClientError.Statement.SyntaxError: Unknown function
'apoc.date.field' (line 2, column 28 (offset: 45)) "MATCH
(media:Media) RETURN apoc.date.field(media.dateCreated)" ^
I have tried multiple ways that more or less return the same kind of error
I expect the data to be shown in date format instead of milliseconds.
You can use epochMillis to create the date from milliseconds.
MATCH (media:Media)
RETURN date(datetime({epochMillis:media.dateCreated}))
This returns the date in the format as shown in the following screenshot:
This query:
RETURN datetime({epochMillis: 1475292465000});
returns the datetime that corresponds to the epoch timestamp 1475292465000:
╒════════════════════════════════════════╕
│"datetime({epochMillis: 1475292465000})"│
╞════════════════════════════════════════╡
│"2016-10-01T03:27:45Z" │
└────────────────────────────────────────┘
If you actually want a date, you can use this query:
RETURN date(datetime({epochMillis: 1475292465000}));
I creating an application, for that I need to find data by month using JPA and java.time.LocalDate. So, is it possible to retrieve data by month from mysql?
Thanks in advance for help.
First find start and end date of month and use between method of JPA to find data of current month.
LocalDate start = LocalDate.ofEpochDay(System.currentTimeMillis() / (24 * 60 * 60 * 1000) ).withDayOfMonth(1);
LocalDate end = LocalDate.ofEpochDay(System.currentTimeMillis() / (24 * 60 * 60 * 1000) ).plusMonths(1).withDayOfMonth(1).minusDays(1);
In Repository
List<Object> findByCreatedateGreaterThanAndCreatedateLessThan(LocalDate start,LocalDate end);
Its better to use the between keyword, it makes things allot shorter.
List<Object> findByCreatedateBetween(LocalDate start,LocalDate end);
Also if you want to use the LocalDate or LocalDateTime objects with Spring Data you should use the converter class Jsr310JpaConverters or else the documents will be stored as Blobs instead of Dates (which is bad for portability of the database). Please see this tutorial on how to implement the Converter.
https://www.mkyong.com/spring-boot/spring-boot-spring-data-jpa-java-8-date-and-time-jsr310/
tl;dr
YearMonth.now( ZoneId.of( "Pacific/Auckland" ) ) // Get current month for particular time zone.
.atDayOfMonth( 1 ) // Get the first date of that month.
.plusMonths( 1 ) // Get first of next month for Half-Open query.
Details
Assuming your column in MySQL is of DATE type…
LocalDate
The LocalDate class represents a date-only value without time-of-day and without time zone.
Time zone
A time zone is crucial in determining a date. For any given moment, the date varies around the globe by zone. For example, a few minutes after midnight in Paris France is a new day while still “yesterday” in Montréal Québec.
Specify a proper time zone name in the format of continent/region, such as America/Montreal, Africa/Casablanca, or Pacific/Auckland. Never use the 3-4 letter abbreviation such as EST or IST as they are not true time zones, not standardized, and not even unique(!).
ZoneId z = ZoneId.of( "America/Montreal" );
LocalDate today = LocalDate.now( z );
YearMonth
The YearMonth class represents an entire month. Getting the current month requires a time zone as discussed above. Around the beginning/ending of the month, the current moment could be “next” month in Auckland New Zealand while still “previous” month in Kolkata India.
YearMonth currentMonth = YearMonth.now( z ) ;
Get the first date of the month.
LocalDate start = currentMonth.atDayOfMonth( 1 ) ;
Half-Open
Generally best to use the Half-Open [) approach to defining a span of time, where the beginning is inclusive while the ending is exclusive. So defining a month means starting with the first date of the month and running up to, but not including, the first date of the following month.
LocalDate stop = start.plusMonths( 1 ) ;
Query
Do not use the BETWEEN command in SQL as it is fully closed [], both beginning and ending being inclusive. Half-Open uses >= & < logic.
SELECT when FROM tbl
WHERE when >= start
AND when < stop
;
it's also useful
#Query("from PogWorkTime p where p.codePto = :codePto and month(p.dateApply) = :month and year(p.dateApply) = :year")
Iterable<PtoExceptWorkTime> findByCodePtoAndDateApply_MonthAndDateApply_Year(#Param("codePto") String codePto,#Param("month") int month, #Param("year") int year);
I have a column in a table in which we are storing date in DATETIME format. (DD-MON-RRRR HH24:MI:SS) - Database Oracle 11g.
Data Type of a column is DATE, and storing date in 01-01-2012 01:00 PM (i.e. jan 1, 2012) format.
entity
#NotNull
#Column(name = "dateColumnName")
#Temporal(TemporalType.TIMESTAMP)
private Date sampleDate;
I am fetching all data by passing date
SAMPLE_QUERY = "select * from TableA tab where tab.dateWithTime = :sampleDate order by tab.dateWithTime ASC "
singleDate is "Tue Jan 24 00:00:00 IST 2012" , fasttime :
1327343400000
The problem is I am passing only date in the query, though Date through which records are being fetched is in DATE TIME format i.e 01-01-2012 01:00 PM.
How can i change my query so that it fetches all the records in ascending order of DateTime.
If you want to fetch all times for that day, then change your query to be more like
SELECT ... WHERE dateField >= :lowerParam AND dateField < :upperParam
Oracle has no DATE TIME datatype. The DATE datatype contains both a date and a time component, down to the second. TIMESTAMPS get a bit more complicated.
If your dateWithTime column is indeed a DATE datatype, the ORDER BY dateWithTime ASC clause should order your results in ascending order.
You may not be displaying the time component of your date. You can convert a date to a string in that format with TO_CHAR( dateWithTime, 'dd/mm/yyyy hh24/mm/ss' ) or whatever format you want.
Edit:
Oh, do you mean you want to find the cases where the date component of the DATE matches, but you don't care about the time component? That can be handled in the where clause with something like:
WHERE TRUNC( tab.dateWithTime ) = TRUNC( :sampledate )
TRUNC by default truncates a date to the beginning of the day.