I have defined a date field in the Solr, I'm using DIH to populate value from DB to Solr. InsertTs value in solr always storing either 4:00:00 or 5:00:00 but the date part is stored properly.
Solr Value: 2013-11-07T05:00:00Z or 2015-05-13T04:00:00Z
DB Value: 07-11-13 02:29:53.00 PM or 07-11-13 12:00:00.00 AM
Schema.xml: INSERTTS is defined as type "date"
DIH: name="INSERTTS" column="INSERTTS"
DIH Query:
SELECT TO_DATE(TO_CHAR(INSERTTS, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') AS INSERTTS FROM EMPLOYEE
InsertTs is defined as TimeStamp in the db.
Solr is Running on Tomcat server in Linux machine. Linux machine is in EDT timezone.
DB is Oracle 11g and in UTC timezone.
Issue was with JDBC driver it was not fetching time part from the date field.
Related
Fetch "current datetime" from Postgres Database instead of PDI server/client time using Table Input step.
When I use "Table Input" step to get the Postgres Timestamp, I am getting PDI server/client time.
How to get Postgres DB timestamp instead PDI Server/Client time.
, TO_CHAR(CURRENT_TIMESTAMP, 'YYYY/MM/DD HH24:MI:SS') p_to_date1
, TO_CHAR(LOCALTIMESTAMP, 'YYYY/MM/DD HH24:MI:SS') p_to_date2
, TO_CHAR(CAST(CURRENT_TIMESTAMP AS TIMESTAMP), 'YYYY/MM/DD HH24:MI:SS') p_to_date3
p_to_date1 --> 2023/02/03 11:34:54
p_to_date2 --> 2023/02/03 11:34:54
p_to_date3 --> 2023/02/03 11:34:54
My Unix Spark Server timezone is CDT but when I'm running Beam SQL CURRENT_TIMESTAMP as below it is always coming as UTC. I tried locally also but it is always displaying UTC. I want this to be CDT same as server zone in CURRENT_TIMESTAMP function. Can you please advise fix for this?
PCollection<Row> cnct = testApps
.apply(SqlTransform.query("SELECT current_timestamp as ts "
+ "FROM PCOLLECTION"));
I am running a query in PrestoDB through a MongoDB connector. The query fetches data from a single collection in MongoDB. The query is something like:
SELECT studentId, classId, sum(date_diff('DAY', entryTime, (CASE WHEN (exitTime <= TIMESTAMP '2018-04-15 23:59:59 UTC') THEN exitTime ELSE TIMESTAMP '2018-04-15 23:59:59 UTC' END))) as timeSpent
FROM mongodb.school.student WHERE entryTime BETWEEN TIMESTAMP '2017-10-30 00:00:00 UTC' AND TIMESTAMP '2018-05-15 23:59:59 UTC' AND contains(classId, '1234') AND subject = 'Maths'
GROUP BY classId, studentId
ORDER BY timeSpent DESC;
I have about 8 million records in the collection and this query takes about 45 seconds to execute.
My PrestoDB is set up on a single Ubuntu instance acting as coordinator and worker with total RAM of 8GB. The jvm.config file looks like:
-server
-Xmx8G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOverheadLimit
-XX:+AggressiveOpts
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:+ExitOnOutOfMemoryError
The config.properties file has the following config:
coordinator=true
node-scheduler.include-coordinator=true
http-server.http.port=8080
query.max-memory=5GB
query.max-memory-per-node=1GB
discovery-server.enabled=true
discovery.uri=http://localhost:8080
The -Xmx8G was earlier -Xmx4G but I changed it to -Xmx8G to try but performance was almost the same. Am I:
Using instance with too low RAM (8GB)?
Should I try running PrestoDB as a cluster? What configuration is expected if there are to be about 60 million records in that collection with this query?
Or is it something with my current configuration itself?
Please run EXPLAIN ANALYZE for your query in Presto and show us the output.
It should be clear which part of the query takes most of the time.
I want to setup my postgreSQL server to 'Europe/Berlin' but having an error:
SET time zone 'Europe/Berlin';
ERROR: invalid value for parameter "TimeZone": "Europe/Berlin"
But the real issue is with DdbSchema, when I want to connect to my DB i've got the error
FATAL: invalid value for parameter "TimeZone": "Europe/Berlin"
DbSchema works when I connect to my local db but not with my NAS (Synology) DB.
Any idea ?
Found a way to solve the problem:
You have to start java with the proper time zone.
In my case, my server is GMT, so i had to add the args -Duser.timezone=GMT
For DbSchema, edit the file DbSchema.bat or DbSchema.sh
Find the declaration of SWING_JVM_ARGS
Add the argument -Duser.timezone=GMT a the end of the line
Start DbSchema with this script DbSchema.bat or DbSchema.sh
I think your solution is only a workaround for the actual problem concerning the zoneinfo on the synology diskstation.
I got exactly the same error when trying to connect to the postgres database on my diskstation. The query select * from pg_timezone_names; gives you all timezone names postgresql is aware of.
There are 87 entries all starting with "Timezone":
name | abbrev | utc_offset | is_dst
------------------------+--------+------------+--------
Timezone/Kuwait | AST | 03:00:00 | f
Timezone/Nairobi | EAT | 03:00:00 | f
...
The configured postgres timezonesets contain much more entries, so there must be another source that postgres is building this view of at startup. I discovered that there is a compile-option --with-system-tzdata=DIRECTORY that tells postgres to obtain its values from system zoneinfo.
I looked in /usr/share/zoneinfo and found one subdirectory called Timezone with exactly 87 entries. And there obviously was no subdirectory called Europe (with a timezone file called Berlin). I did not quickly find a solution for the diskstation to update the tzdata automatically or manually by unpacking tzdata2016a.tar.gz and making (make not found...). As a quickfix I copied the Berlin timezone file from another linux system and the problem was solved, so that I now can connect via java/jdbc using the correct timezone "Europe/Berlin"!
I am using SQL Developer as a client for an Oracle 11G RAC. The database server is set to Pacific Daylight Timezone (PDT) and when I query sysdate in sqlplus, it always shows the PDT time. But in SQL Developer it displays time as GMT -4.
The system date on the system where SQL Developer is running is also set to PDT, even if I changed from Central timezone. I tried to add this parameter to the SQL Developer configuration files:
AddVMOption -Duser.timezone=GMT-7
But I continue to see the following results:
From SQL Developer:
select to_char(current_date,'DD-MON-YY HH:MI:SS'), to_char(sysdate,'DD-MON-YY HH:MI:SS'), sessiontimezone from dual;
CURRENT_DATE SYSDATE SESSIONTIMEZONE
09-AUG-13 12:57:11 10-AUG-13 03:57:11 -07:00
From sqlplus:
SQL> select to_char(current_date,'DD-MON-YY HH:MI:SS'), to_char(sysdate,'DD-MON-YY HH:MI:SS'), sessiontimezone from dual;
TO_CHAR(CURRENT_DATE,'DD-MO TO_CHAR(SYSDATE,'DD-MON-YYH
SESSIONTIMEZONE
09-AUG-13 12:55:11 09-AUG-13 12:55:11 -07:00
Anyone knows how to have the same output as generated by sqlplus?
I have to schedule jobs in a production environment and I guess it is better to use sysdate instead of the current_date.