Note: I'm running Postgres 11.7 and Python 2.7.17.
It appears that time.sleep() has 60 sec limit in plpythonu function. It works as expected up to 60 seconds. But if passed a value greater than 60 then it stops at 60 seconds.
CREATE OR REPLACE FUNCTION test_sleep(interval_str INTERVAL)
RETURNS INTERVAL
AS $$
try:
import time
import datetime
start_time = time.time()
t = datetime.datetime.strptime(interval_str,"%H:%M:%S")
tdelta = datetime.timedelta(hours=t.hour, minutes=t.minute, seconds=t.second)
interval_secs = tdelta.total_seconds()
print 'interval_secs=%s' % repr(interval_secs)
time.sleep(interval_secs)
elapsed_secs = time.time() - start_time
return datetime.timedelta(seconds=elapsed_secs)
except:
import traceback
print traceback.format_exc()
raise
$$ LANGUAGE PLPYTHONU;
Here's a test on command line using psql. The first test runs as expected. I told it to sleep for two seconds and it dis. The second test only sleeps for 60 seconds even though I requested a sleep for 120 seconds.
$ echo "select now(), test_sleep('2 sec'); select now();" | psql
now | test_sleep
------------------------------+-----------------
2021-05-07 11:10:27.63041-04 | 00:00:02.005745
(1 row)
now
-------------------------------
2021-05-07 11:10:29.652542-04
(1 row)
$ echo "select now(), test_sleep('2 min'); select now();" | psql
now | test_sleep
-------------------------------+-----------------
2021-05-07 11:10:36.056578-04 | 00:00:59.977787
(1 row)
now
-------------------------------
2021-05-07 11:11:36.050637-04
(1 row)
$
Here's output from the Postgres log file.
interval_secs=2.0
interval_secs=120.0
This is not unexpected (although I can't reliably reproduce it).
https://realpython.com/python-sleep/
Note: In Python 3.5, the core developers changed the behavior of
time.sleep() slightly. The new Python sleep() system call will last at
least the number of seconds you’ve specified, even if the sleep is
interrupted by a signal. This does not apply if the signal itself
raises an exception, however.
So in python2, time.sleep can return early if interrupted by a signal. If you don't like that, have it loop to sleep again for the remainder of the time.
Related
I am writing a script so that to filter jobs that are failed in past 24hrs and also the failed job is not completed or not in running state again.
rm -rf jobsfailed.txt jobscompleted.txt jobsnotcompleted.txt ## Remove the files which are created
export PGPASSWORD="$PG_PASS"
failed_jobs=`psql -t -U bacula bacula << EOD
SELECT jobid,job,name,jobbytes,realendtime,jobstatus,level FROM job WHERE jobstatus IN ('f','A','E') AND realendtime > NOW() - INTERVAL '24 HOURS';
\q
EOD` ### Collect all the jobs which are in the defined states
echo "$failed_jobs" >> jobsfailed.txt ### redirect the values to a file
sortedlist=$(awk -F'|' '{print $3}' jobsfailed.txt | sort | uniq) ## sort the values based on the jobname
for i in $sortedlist
do
retVal=$?
jobs_notcompleted=`psql -t -U bacula bacula << EOD1
SELECT jobid,job,name,jobbytes,realendtime,jobstatus,level FROM job WHERE name LIKE '$i' AND jobstatus IN ('T','R') AND starttime > NOW() - INTERVAL '24 HOURS' ORDER BY jobid DESC LIMIT 1;
\q
EOD1` ### If the job is in above defined states(T or R) then jobs completed successfully. Any other state apart from above then job not completed
if [[ $retVal -eq 0 ]]; then
echo "$jobs_notcompleted" >> jobscompleted.txt
else
echo "$jobs_notcompleted" >> jobsnotcompleted.txt
fi
exit $retVal
done
But i am not getting desired output. Since if no state is getting matched, then it is producing (0 rows) output. Please let me know, is there any other way if 0 rows are matched then that $jobs_notcompleted value should redirect to the jobsnotcompleted.txt file. jobscompleted.txt file is getting created and working as expected.
For some reason, I feel that you need a "NOT" between "jobstatus" and "in" for your second query, because the ${retVal} would, in my mind, always return 0 unless there was a DB error.
If I am wrong on that point, then the "retVal=$?" needs to be moved to after the 2nd query.
I'm using Oracle 18c.
I'm trying to determine elapsed time, but I get an error when I subtract two date variables in PL SQL.
The following code works fine:
DECLARE
l_zero_date date;
l_current_date date;
l_elapsed_time date;
BEGIN
Execute Immediate 'ALTER SESSION set nls_timestamp_format = "DD-MM-YYYY HH24:MI:SS"';
l_zero_date := to_date('01-01-1900 00:00:00', 'DD-MM-YYYY HH24:MI:SS');
dbms_output.put_line('The value of l_zero_date is: ' || l_zero_date);
Select ls.duration Into l_current_date From LIT_STATS ls Where ls.prim_key = 1002;
dbms_output.put_line('The value of l_curr_date is: ' || l_current_date);
-- dbms_output.put_line('The elapsed time is: ' || l_current_date - l_zero_date);
END;
This produces the results:
The value of l_zero_date is: 1900-01-01 00:00:00
The value of l_curr_date is: 1900-01-01 00:35:22
However, If I un-comment the last dbms_output line I get the error:
Error report -
ORA-06502: PL/SQL: numeric or value error: character to number conversion error
ORA-06512: at line 14
06502. 00000 - "PL/SQL: numeric or value error%s"
*Cause: An arithmetic, numeric, string, conversion, or constraint error
occurred. For example, this error occurs if an attempt is made to
assign the value NULL to a variable declared NOT NULL, or if an
attempt is made to assign an integer larger than 99 to a variable
declared NUMBER(2).
*Action: Change the data, how it is manipulated, or how it is declared so
that values do not violate constraints.
I don't understand why I get the error on subtraction involving two fields declared as DATE. For example, the following code works fine:
declare
a date;
b date;
begin
a := sysdate;
dbms_lock.sleep(10); -- sleep about 10 seconds give or take
b := sysdate;
dbms_output.put_line( b-a || ' of a day has elapsed' );
dbms_output.put_line( (b-a)*24 || ' of an hour has elapsed' );
dbms_output.put_line( (b-a)*24*60 || ' of a minute has elapsed' );
dbms_output.put_line( (b-a)*24*60*60 || ' seconds has elapsed' );
end;
Why does the line dbms_output.put_line('The elapsed time is: ' || l_current_date - l_zero_date); produce an error?
Thanks for looking at this.
As I mentioned in the comments, this is an order of operations issue. Take the following example:
SELECT 'TEST'||SYSDATE-SYSDATE FROM DUAL
When this runs, I get the following error: ORA-00932: inconsistent datatypes: expected CHAR got DATE
But when I wrap the dates in ( and ):
SELECT 'TEST'||(SYSDATE-SYSDATE) FROM DUAL
The result is TEST0.
It is order of operations, the code moves left to right unless there are parentheses informing it to do the date subtraction first.
Here is a DBFiddle showing the queries being run (LINK)
I have an AWS lambda function that refreshes a Postgres view. The data appears to refresh for the current connection, but does not persist for future connections.
kwargs = { 'async': 0 }
conn = extensions.connection(dsn, **kwargs)
cur.execute("select max(updated_at), count(*) from my_view; ")
last_updated = cur.fetchone()
print("LambdaRefreshView: Starting to refresh, last updated {}".format(last_updated))
cur.execute("REFRESH MATERIALIZED VIEW my_view;")
cur.execute("select max(updated_at), count(*) from my_view; ")
last_updated = cur.fetchone()
print("LambdaRefreshView: Finished refreshing, last updated {}".format(last_updated))
outputs:
LambdaRefreshView: Starting to refresh, last updated (2021-09-24 21:31:46, 299906)
LambdaRefreshView: Finished refreshing, last updated (2021-09-29 15:37:37, 302511)
but then I run it again, and the first select query still shows old data. The view doesn't "stay" refreshed with the latest data:
LambdaRefreshView: Starting to refresh, last updated (2021-09-24 21:31:46, 299906)
LambdaRefreshView: Finished refreshing, last updated (2021-09-29 15:37:37, 302514)
I've different variations of the refresh command with/without CONCURRENTLY and WITH DATA, but it has no effect.
Any ideas? Python 3.6.14, psycopg2 2.9.1, PostgreSQL 9.6.22
If I run REFRESH MATERIALIZED VIEW CONCURRENTLY my_view; from psql, with the same DSN/user/password, it refreshes properly! And re-running from Lambda shows the updated data:
psql:
postgres=> select max(updated_at), count(*) from my_view;
max | count
-------------------------------+--------
2021-09-24 21:31:46.262262+00 | 299906
(1 row)
postgres=> REFRESH MATERIALIZED VIEW CONCURRENTLY my_view;
REFRESH MATERIALIZED VIEW
postgres=> select max(updated_at), count(*) from my_view;
max | count
-------------------------------+--------
2021-09-29 15:40:53.123746+00 | 302515
(1 row)
And the Lambda function sees this new, fresh, data:
LambdaRefreshView: Starting to refresh, last updated (2021-09-29 15:40:53, 302515)
LambdaRefreshView: Finished refreshing, last updated (2021-09-29 15:40:53, 302515)
Okay I've confirmed this works explicitly with PHP.
$ php --version
PHP 5.6.16 (cli) (built: Dec 30 2015 15:09:50) (DEBUG)
<pdo version>
pdo_pgsql
PDO Driver for PostgreSQL enabled
PostgreSQL(libpq) Version 9.4.0
Module version 1.0.2
Revision $Id: fe003f8ab9041c47e97784d215c2488c4bda724d $
I would like to recreate the following SQL in PHP using PDO:
UPDATE relationships SET status = 4 WHERE created > NOW() - interval '2 seconds';
This script is working:
<?php
$db = new PDO('pgsql:dbname=db;host=localhost;user=stevetauber');
$stmt = $db->prepare("UPDATE relationships SET status = 4 WHERE created > NOW() - interval '?'");
$stmt->execute(array("2 seconds"));
Here it is with named placeholders:
<?php
$db = new PDO('pgsql:dbname=db;host=localhost;user=stevetauber');
$stmt = $db->prepare("UPDATE relationships SET status = 4 WHERE created > NOW() - interval ':blah'");
$stmt->execute(array(":blah" => "2 seconds"));
Which gives this error:
Warning: PDOStatement::execute(): SQLSTATE[HY093]: Invalid parameter number: :blah in ... line 5
Now according to PHP documentation,
Example #6 Invalid use of placeholder:
<?php
$stmt = $dbh->prepare("SELECT * FROM REGISTRY where name LIKE '%?%'");
$stmt->execute(array($_GET['name']));
// placeholder must be used in the place of the whole value
$stmt = $dbh->prepare("SELECT * FROM REGISTRY where name LIKE ?");
$stmt->execute(array("%$_GET[name]%"));
?>
Here is the updated code:
<?php
$db = new PDO('pgsql:dbname=db;host=localhost;user=stevetauber');
$stmt = $db->prepare("UPDATE relationships SET status = 4 WHERE created > NOW() - :blah");
$stmt->execute(array(":blah" => "interval '2 seconds'"));
Which yields these DB errors (no script errors):
ERROR: operator does not exist: timestamp with time zone > interval at character 51
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
STATEMENT: UPDATE relationships SET status = 4 WHERE created > NOW() - $1
PDO is doing something weird here though because:
# select NOW() - interval '2 seconds' as a , pg_typeof(NOW() - interval '2 seconds') as b;
a | b
-------------------------------+--------------------------
2015-12-30 18:02:20.956453+00 | timestamp with time zone
(1 row)
So how do I use named placeholders with PostgreSQL and interval?
Placeholders are for pure values, not for values decorated with units (or with anything else).
To express interval '2 seconds' in a placeholder, there are two options:
in the query, write :secs * interval '1 second'
and bind :secs to a number in php
or write: cast(:mystring as interval), and bind :mystring to the string '2 seconds'. It will be interpreted dynamically through the explicit cast.
When experimenting with the psql command line client to compare with the PDO driver, use the PREPARE and EXECUTE SQL statements with postgres native $N placeholders, as opposed to having the parameters values already written literally in the query. This will match what the PHP driver is essentially doing when PDO::ATTR_EMULATE_PREPARES is set to false.
In the last part of you question, when trying this in psql (your query, just simplified to not need a table):
select now() > now() - interval '2 seconds';
it does work and returns 't' (true).
But if you tried that:
prepare p as select now() > now() - $1;
if would fail with
ERROR: operator does not exist: timestamp with time zone > interval
which is the same error as with PDO's prepare/execute.
On the other hand, this does work:
=> prepare p as select now() > now() - interval '1 second'*$1;
PREPARE
=> execute p(2);
?column?
----------
t
I would like to get query execution time in iSQL.
For instance :
SELECT * FROM students;
How do i get query execution time ?
Use SET STATS:
SQL> SET STATS;
SQL> SELECT * FROM RDB$DATABASE;
... query output removed ....
Current memory = 34490656
Delta memory = 105360
Max memory = 34612544
Elapsed time= 0.59 sec
Buffers = 2048
Reads = 17
Writes 0
Fetches = 270
SQL>