I am trying to build View PostgreSQL table in Liquibase for my JHipster application. Therefore, I have tried createView and sqlFile methods.
My query has date_trunc() function as following:
CREATE OR REPLACE VIEW periodical_statistics AS
SELECT tran.id, date_trunc('day', tran.pub_date) as, ...
FROM transaction tran
...(LEFT JOINS - not relative)...
WHERE ...(condtions - not relative)...
When I run my JHispster app as Dev Mode in maven(./mvnw). It is working ver fine.
But when I run it as Prod Mode in maven(./mvnw -Pprod package). It gives following error.
20180817101122-1::liquibase-docs failed. Error: Function "DATE_TRUNC" not found;
SQL statement: CREATE OR REPLACE VIEW periodical_statistics AS ....
is there are any way to solve this error problem?
After adding dbms='postgresql' to the changeset. Liquibase recognized the 'date_trunc' function. It is as following:
<changeSet author="sanatbek" id="20180904094713" dbms="postgresql">
<createView replaceIfExists="true"
schemaName="public"
viewName="periodical_statistics">
SELECT
tran.id
date_trunc('day', tran.pub_date) as truncated_date,
....
...(LEFT JOINS - not relative)...
WHERE ...(condtions - not relative)...
</createView>
</changeSet>
Related
I am working with a script made for SQLite but I need to adapt it to a PostgreSQL database and this line is always returning an error:
PRAGMA table_info(table)
How can I get an equivalent result on postgres >
Found it !
SELECT *
FROM information_schema.columns
WHERE table_name = 'table_name'
Using rails-5.0.7.1 (according to bundle show rails)
I wrote a migration which adds the "uuid-ossp" extension, and the SQL gets executed, and the extension shows up when I type \dx in the psql console. However, the functions that this extension provides (such as uuid_generate_v4) do NOT show up when I type \df, and so any attempt to use the functions that should be added fails.
When I take the SQL from the ActiveRecord migration and copy+paste it into the psql console directly, everything works as expected - extension is added, and functions are available.
Here is my migration code:
class EnableUuidOssp < ActiveRecord::Migration[5.0]
def up
enable_extension "uuid-ossp"
end
def down
disable_extension "uuid-ossp"
end
end
Here is the output:
$ bundle exec rake db:migrate
== 20190308113821 EnableUuidOssp: migrating ==============================
-- enable_extension("uuid-ossp")
-> 0.0075s
== 20190308113821 EnableUuidOssp: migrated (0.0076s) =====================
^ this all appears to run successfully, but no functions are enabled. Which means future SQL that includes statements such as ... SET uuid = uuid_generate_v4() ... fail with the this error HINT: No function matches the given name and argument types. You might need to add explicit type casts.
What does work
Going directly into psql and typing:
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
^ This installs the extension and makes the functions avaiable.
And yet, what doesn't work
Okay, so if I take the above SQL and rewrite my migration this way:
...
def up
execute <<~SQL
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
SQL
end
...
^ this migration will run without error, yet it will still not make the functions available.
So, the same copy+paste SQL that works in psql doesn't work via the ActiveRecord execute method, which really puzzles me. I'm not sure what piece I'm missing that's causing this to fail.
I assume that the schema where the extension is installed is not on your search_path.
You can see that schema with
\dx "uuid-ossp"
Try to qualify the functions with the schema, like in public.uuid_generate_v4().
I am very new to Oracle so please base with me if this is covered else where.
I have a MS SQL box running Jobs calling batch files running scripts in SQLPLUS to ETL to an Oracle 10G database.
I have an intermittent issue with a script that is causing the ETL to fail which at the minute without error logging is something of an unknown. The current solution highlights the load failure based on rowcounts for before and after the script has finsihed.
I'd like to be able to insert any errors encoutered whilst running the offending script into an error log table on the same database receiving the data loads.
There's nothing too technical about the script, at a high level is performs the following steps all via SQL code and no procedural calls.
Updates a table with Date and current row counts
Pulls data from a remote source into a staging table
Merges the Staging table into an intermediate staging table
Performs some transformational actions
Merges the intermediate staging table into the final Fact table
Updates a table with new row counts
Please advise whether it is possible to pass error messages, codes, line number etc etc via SQLPLUS into a Database table? and if so the easiest method to achieve this.
A first few lines of the script are shown below to give a flavour
/*set echo off*/
set heading off
set feedback off
set sqlblanklines on
/* ID 1 BATCH START TIME */
INSERT INTO CITSDMI.CITSD_TIMETABLE_ORDERLINE TGT
(TGT.BATCH_START_TIME)
(SELECT SYSDATE FROM DUAL);
COMMIT;
insert into CITSDMI.CITSD_TIMETABLE_ALL_LOADS
(LOAD_NAME, LOAD_CRITICALITY,LOAD_TYPE,BATCH_START_TIME)
values
('ORDERLINE','HIGH','SMART',(SELECT SYSDATE FROM DUAL));
commit;
/* Clear the Staging Tables */
TRUNCATE TABLE STAGE_SMART_ORDERLINE;
Commit;
TRUNCATE TABLE TRANSF_FACT_ORDERLINE;
Commit;
and so it goes on with the rest of the steps.
Any assistant will be greatly appreciated.
Whilst not fully understanding your requirement, a couple of pointers.
The WHENEVER command will help you control what sqlplus should do when an error occurs, e.g.
WHENEVER SQLERROR EXIT FAILURE ROLLBACK
WHENEVER OSERROR EXIT FAILURE ROLLBACK
INSERT ...
INSERT ...
This will cause sqlplus to exit with error status 1 if any of the following statements fail.
You can also have WHENEVER SQLERROR CONTINUE ...
Since the WHENEVER ... EXIT FAILURE/SUCCESS controls the exit status, the calling script/program will know if it worked failed.
Logging
use SPOOL to spool the out to a file.
Logging to table.
Best way is to wrap your statements into PLSQL anonymous blocks and use exception hanlders to log errors.
So, putting the above together, using a UNIX shell as the invoker:
sqlplus -S /nolog <<EOF
WHENEVER SQLERROR EXIT FAILURE ROLLBACK
CONNECT ${USRPWD}
SPOOL ${SPLFILE}
BEGIN
INSERT INTO the_table ( c1, c1 ) VALUES ( '${V1}', '${V2}' );
EXCEPTION
WHEN OTHERS THEN
INSERT INTO the_error_tab ( name, errno, errm ) VALUES ( 'the_script', SQLCODE, SQLERRM );
COMMIT;
END;
/
SPOOL OFF
QUIT
EOF
if [ ${?} -eq 0 ]
then
echo "Success!"
else
echo "Oh dear!! See ${SPLFILE}"
fi
The following works in PostgreSQL 8.4:
insert into credentials values('demo', pgp_sym_encrypt('password', 'longpassword'));
When I try it in version 9.1 I get this:
ERROR: function pgp_sym_encrypt(unknown, unknown) does not exist LINE
1: insert into credentials values('demo', pgp_sym_encrypt('pass...
^ HINT: No function matches the given name and argument types. You might need to add
explicit type casts.
*** Error ***
ERROR: function pgp_sym_encrypt(unknown, unknown) does not exist SQL
state: 42883 Hint: No function matches the given name and argument
types. You might need to add explicit type casts. Character: 40
If I try some explicit casts like this
insert into credentials values('demo', pgp_sym_encrypt(cast('password' as text), cast('longpassword' as text)))
I get a slightly different error message:
ERROR: function pgp_sym_encrypt(text, text) does not exist
I have pgcrypto installed. Does anyone have pgp_sym_encrypt() working in PostgreSQL 9.1?
On explanation could be that the module was installed into a schema that is not in your search path - or to the wrong database.
Diagnose your problem with this query and report back the output:
SELECT n.nspname, p.proname, pg_catalog.pg_get_function_arguments(p.oid) as params
FROM pg_catalog.pg_proc p
JOIN pg_catalog.pg_namespace n ON n.oid = p.pronamespace
WHERE p.proname ~~* '%pgp_sym_encrypt%'
AND pg_catalog.pg_function_is_visible(p.oid);
Finds functions in all schemas in your database. Similar to the psql meta-command
\df *pgp_sym_encrypt*
Make sure you install the extension on the desired schema.
sudo -i -u postgres
psql $database
CREATE EXTENSION pgcrypto;
OK, problem solved.
I was creating the pgcrypto extension as the first operation in the script. Then I dropped and added the VGDB database. That's why pgcrypto was there immediately after creating it, but didn't exist when running the sql later in the script or when I opened pgadmin.
This script is meant for setting up new databases and if I had tried it on a new database the create extension would have failed right away.
My bad. Thanks for the help, Erwin.
Just mention de schema where is installed pgcrypto like this:
#ColumnTransformer(forColumn = "TEST",
read = "public.pgp_sym_decrypt(TEST, 'password')",
write = "public.pgp_sym_encrypt(?, 'password')")
#Column(name = "TEST", columnDefinition = "bytea", nullable = false)
private String test;
I ran my (python) script again and the CREATE EXTENSION ran without error. The script also executes this command
psql -d VGDB -U postgres -c "select * from pg_available_extensions order by name"
which includes the following in the result set:
pgcrypto | 1.0 | 1.0 | cryptographic functions
So psql believes that it has installed pgcrypto.
Later in the same script when I execute
psql -d VGDB -U postgres -f sql/Create.Credentials.table.sql
where sql/Create.Credentials.table.sql includes this
insert into credentials values('demo', pgp_sym_encrypt('password', 'longpassword'));
I get this
psql:sql/Create.Credentials.table.sql:31: ERROR: function pgp_sym_encrypt(unknown, unknown) does not exist
LINE 1: insert into credentials values('demo', pgp_sym_encrypt('pass...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
When I open pgadmin it does not show pgcrypto in either the VGDB or postgres databases even though the query above called by psql shows that pgcrypto is installed.
Could there be an issue with needing to commit after using psql to execute the "create extension ..." command? None of my other DDL or SQL statements require a commit when they get executed with psql.
It's starting to look like psql is just flakey. Is there another way to call "create extension pgcrypto" - e.g. with Python's database support classes - or does that have to be run through psql?
I am running PostgreSQL 8.4.4 with Ubuntu 10.04.
I am trying to generate uuid but can't find a way to do it.
I do have the uuid-ossp.sql in /usr/share/postgresql/8.4/contrib/uuid-ossp.sql
When I try this is what I get :
postgres=# SELECT uuid_generate_v1();
ERROR: function uuid_generate_v1() does not exist
LINE 1: SELECT uuid_generate_v1();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
Any idea ?
The stuff in contrib aren't run automatically. You have to run it yourself to install the functions. I don't know about the 8.4 version, but in the 8.3 version it appears to only install it per-database, so open up the database you're using in psql and issue the command \i /usr/share/postgresql/8.4/contrib/uuid-ossp.sql
I saw this in my PostgreSQL travels. It requires the pgcrypto contrib module.
CREATE OR REPLACE FUNCTION generate_uuid() RETURNS UUID AS
$$
SELECT ENCODE(GEN_RANDOM_BYTES(16), 'hex')::UUID
$$ LANGUAGE SQL IMMUTABLE;