I use liquibase and I want to create enum in my xml file (PostgreSQL).
As an example below changelog file (only a piece of the file) that creates new table:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-2.0.xsd
http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
<changeSet id="r3.3.0_table_creation_research_document "
author="anonim">
<createTable tableName="research_document">
<column name="id" type="bigint">
<constraints primaryKey="true"
primaryKeyName="research_document_pkey" nullable="false" />
</column>
I can't find any info/examples in Internet!
I don't think liquibase supports enum for postgres natively in the xml format. However, since it is possible in postgres, you could always use liquibase's formatted sql instead of xml:
--liquibase formatted sql
--changeset ronak:1
CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
CREATE TABLE person (
name text,
current_mood mood
);
--rollback drop table person;
I am using Liquibase version 4.0.0 to deploy DB migration scripts in PostgreSQL.
I use a master changelog file and it looks like the following.
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
<includeAll path="4.460.7" relativeToChangelogFile="true"/>
</databaseChangeLog>
My project structure looks like the following.
I use a docker container to run liquibase update in my Jenkinsfile as the following.
docker run --rm -v /home/jenkins/workspace/mate_DB_Migration_Scripts_master:/liquibase/changelog liquibase/liquibase --url="jdbc:postgresql://host:5432/postgres?currentSchema=schema1" --changeLogFile=../liquibase/changelog/changelog.xml --username=postgres --password=some_password update
docker run --rm -v /home/jenkins/workspace/mate_DB_Migration_Scripts_master:/liquibase/changelog liquibase/liquibase --url="jdbc:postgresql://host:5432/postgres?currentSchema=schema2" --changeLogFile=../liquibase/changelog/changelog.xml --username=postgres --password=some_password update
It runs the update just fine. The issue is I can see 2 rows in the databasechangelog file as the following for the same changeset.
Does anyone know why this happens? Please let me know if you want any other information to resolve this.
It looks like there is an issue with the Liquibase Docker image.
More specifically with the combination of v4.0.0 and the "includeAll" tag.
As a workaround you can try the "include" tag instead, but you will have to include every single file.
like this:
<include file="path/to/<filename>.sql" relativeToChangelogFile="true"/>
You can also try the sqlFile tag like this:
<changeSet author="SteveZ" id="external-sql+rollback-script-example" context="QA" labels="Jira1000">
<sqlFile dbms="mysql" splitStatements="true" endDelimiter="//" stripComments="true" path="objects/function/myFunction.sql"/>
<rollback>
<sqlFile dbms="mysql" splitStatements="true" endDelimiter="//" stripComments="true" path="objects/function/myFunction_rollback.sql"/>
</rollback>
</changeSet>
I'm trying to upload a lot of data from a .sql file using the COPY command for Postgresql.
I have those data in a file.sql in the following format :
COPY my_table(id, name, status) FROM stdin;
1 peter active
1 steve active
1 maria active
\.
And my changeset like this:
<changeSet id="sqlFile-example" author="me" >
<sqlFile encoding="UTF-8"
path="file.sql"
relativeToChangelogFile="true"
endDelimiter=";"
splitStatements="false"
/>
</changeSet>
And get this error:
[ERROR] Failed to execute goal
org.liquibase:liquibase-maven-plugin:3.6.3:update (default-cli) on
project lincoln-soft: Error setting up or running Liquibase: Migration
failed for change set
src/main/resources/db/liquibase/db-changelog.xml::sqlFile-example::me
[ERROR] Reason: liquibase.exception.DatabaseException: ERROR: unexpected message type 0x50 during COPY from stdin
[ERROR] Where:
COPY my_table, line 1 [Failed SQL: COPY my_table(id, name, status)
FROM stdin;
[ERROR] 1 peter active
[ERROR] 1 steve active
[ERROR] 1 maria active
[ERROR] \.]
I there a way to upload those data by liquibase?
Finally got a solution, as #a_horse_with_no_name and #Laurenz Albe
mentioned, can't use COPY FROM STDIN directly in JDBC, so I used pg_dump to generate insert statements like this:
pg_dump --table=public.my_table --data-only --column-inserts my_databse > /tmp/my_table_data.sql
It gives me a file my_table_data.sql with the inserts statements like this:
INSERT INTO public.my_table (id, name, status) VALUES (1,peter,active);
INSERT INTO public.my_table (id, name, status) VALUES (1,peter,active);
INSERT INTO public.my_table (id, name, status) VALUES (1,peter,active);
And then I use this liquibase Chageset to upload the sql file:
<changeSet id="sqlFile-example" author="me" >
<sqlFile encoding="UTF-8"
path="my_table_data.sql"
relativeToChangelogFile="true"
splitStatements="true"
stripComments="true"
/>
</changeSet>
It works for me
As Laurenz already mentioned: you can't use COPY FROM STDIN directly in JDBC (you can use the CopyManager API to implement that manually, but Liquibase doesn't support that and I also don't know of any plugin that would do that)
I would suggest you use Liquibase's built-in ability to load CSV (text) files. Put your input data in CSV file, e.g. my_table_data.txt with a header line for the columns:
id,name,status
1,peter,active
1,steve,active
1,maria,active
Then use <loadData> instead of running a SQL script:
<changeSet id="sqlFile-example" author="me" >
<loadData tableName="my_table"
file="my_table_data.txt"
separator=","
encoding="UTF-8">
</changeSet>
Mixing the COPY statement and the data in the same file only works in psql scripts.
Moreover, COPY FROM STDIN is not supported by the JDBC driver at all.
You should use INSERT statements in your script.
My scenario is to export data from the real DB2 and then load it back for integration tests purpose. I am using liquibase (v3.5.3) to manage it.
I have found that TIMESTAMP values are changed during this cycle. When timestamp value is exported I can see it in changelog file as "2018-06-28 22:47:38.816343". After that when I load it back into DB2 it becomes "2018-06-28 23:01:14.343".
The reason is that the "816343" part is treated not as a part of a second but rather as milliseconds amount and that amount is added to the result timestamp.
In the tests a business decision criteria is made by comparing of those timestamps. I need them to be equal.
Any thoughts and proposals will be appreciated.
There are steps to reproduce:
1. Create a file "01_test_data_to_setup.xml" with content
<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.5.xsd">
<changeSet author="OK" id="1">
<createTable tableName="TT">
<column name="TS_LOADED" type="TIMESTAMP"/>
<column name="TS_GENERATED" type="TIMESTAMP" defaultValueComputed="CURRENT_TIMESTAMP">
<constraints nullable="false"/>
</column>
</createTable>
</changeSet>
<changeSet author="OK" id="2">
<insert tableName="TT">
<column name="TS_LOADED"/>
</insert>
</changeSet>
</databaseChangeLog>
2. Execute above changelog file with liquibase update command
liquibase.bat --driver=com.ibm.db2.jcc.DB2Driver --logLevel=info --classpath=~jdbc driver path here~ --changeLogFile=01_test_data_to_setup.xml --url=jdbc:db2:~jdbc url here~ --defaultSchemaName=~schema name here~ --username=~user name here~ --password=~password here~ update
3. Export data from DB
liquibase.bat --driver=com.ibm.db2.jcc.DB2Driver --logLevel=info --classpath=~jdbc driver path here~ --changeLogFile=db2_exported_test_data.xml --url=jdbc:db2:~jdbc url here~ --defaultSchemaName=~schema name here~ --username=~user name here~ --password=~password here~ --diffTypes="data" generateChangeLog --includeObjects="table:TT"
As result you will have "db2_exported_test_data.xml" file
<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.5.xsd">
<changeSet author="OK (generated)" id="1530226079041-1">
<insert tableName="TT">
<column name="TS_LOADED"/>
<column name="TS_GENERATED" valueDate=~your generated timestamp value here. in my case it's "2018-06-28 22:47:38.816343"/>
</insert>
</changeSet>
</databaseChangeLog>
4. A file to load exported timestamp back to DB "02_test_data_to_load.xml"
<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.5.xsd">
<changeSet author="OK" id="3">
<insert tableName="TT">
<column name="TS_LOADED" valueDate=~value in TS_GENERATED from previous step, i.e."2018-06-28 22:47:38.816343"~/>
</insert>
</changeSet>
</databaseChangeLog>
Liquibase command
liquibase.bat --driver=com.ibm.db2.jcc.DB2Driver --logLevel=info --classpath=~jdbc driver path here~ --changeLogFile=02_test_data_to_load.xml --url=jdbc:db2:~jdbc url here~ --defaultSchemaName=~schema name here~ --username=~user name here~ --password=~password here~ update
5. Check exported and loaded timestamp in DB or export TT table data one more time.
TS_LOADED timestamp value in second row will be different from TS_GENERATED value in first row
I would guess this is a bug in Liquibase similar to this one
https://liquibase.jira.com/browse/CORE-1958
Or your datatype is TIMESTAMP(3) on the target table
I am using liquibase to create triggers for postgressql database. Below is the syntax which I use in liquibase
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.1.xsd">
<changeSet id="1" author="yc14ik1">
<createProcedure catalogName="cat" dbms="postgresql"
encoding="utf8" procedureName="UPDATE_LAST_ROW_CHG_TS()" relativeToChangelogFile="true" schemaName="sub">
CREATE OR REPLACE FUNCTION UPDATE_LAST_ROW_CHG_TS() RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
NEW.LAST_ROW_CHG_TS = NOW();
RETURN NEW;
END;
$$;
</createProcedure>
<sql></sql>
</changeSet>
When this procedure is executed in Jenkins Job, I get this error
Unexpected error running Liquibase: ERROR: language "plpgsql" does not exist
Hint: Use CREATE LANGUAGE to load the language into the database. [Failed SQL: CREATE OR REPLACE FUNCTION UPDATE_LAST_ROW_CHG_TS() RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
NEW.LAST_ROW_CHG_TS = NOW();
RETURN NEW;
END;
$$;]
Build step 'Execute shell' marked build as failure
Notifying upstream projects of job completion
Warning: you have no plugins providing access control for builds, so falling back to legacy behavior of permitting any downstream builds to be triggered
Can some one help me how can I fix this problem ?
I set up mine differently and it works just fine.
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.1.xsd
http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
<changeSet author="l" id="UPDATE_LAST_ROW_CHG_TS" runOnChange="true">
<createProcedure catalogName="UPDATE_LAST_ROW_CHG_TS"
dbms="postgresql"
encoding="utf8"
path="../files/UPDATE_LAST_ROW_CHG_TS.sql"
procedureName="UPDATE_LAST_ROW_CHG_TS"
relativeToChangelogFile="true"
schemaName="public"></createProcedure>
</changeSet>
</databaseChangeLog>
from there you put all the sql inside a .sql file referenced in the path attribute.
I dont know that your way wont work but it seems odd to me that you have empty tags towards the bottom. Maybe you meant for those to wrap your sql function?
Anyways I know the example above will work if liquibase is configured and ran properly.