How to specify schema name within xml databaseChangeLog? E.g. I have created a new schema with the following changelog, after which I want to select newly created schema to populate changes.
<databaseChangeLog>
<changeSet author="quickstart-1" id="quickstart-1">
<sql>
CREATE SCHEMA IF NOT EXISTS schema_name AUTHORIZATION schema_name_user;
</sql>
</changeSet>
</databaseChangeLog>
Related
Problem: liquibase can't find table without setting schema in SQL script.
How to say liquibase use default schema in SQL changelog?
Before sql changelog, for adding check constraint, I create all table, without setting schema. Schema was set in application.properties and all table was created correctly in $RM_DB_SCHEMA.
RM_DB_SCHEMA: MANAGER
RM_DB_URL: "jdbc:h2:file:~/rmdb;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE;AUTO_SERVER=TRUE;INIT=CREATE SCHEMA IF NOT EXISTS ${RM_DB_SCHEMA}"
RM_DB_USER: sa
RM_DB_PASSWORD: admin
RM_LB_USER: ${RM_DB_USER}
RM_LB_PASSWORD: ${RM_DB_PASSWORD}
spring:
datasource:
hikari:
schema: ${RM_DB_SCHEMA}
username: ${RM_DB_USER}
password: ${RM_DB_PASSWORD}
jdbc-url: ${RM_DB_URL}
liquibase:
change-log: "classpath:db/manager-changelog.xml"
default-schema: ${RM_DB_SCHEMA}
user: ${RM_LB_USER}
password: ${RM_LB_PASSWORD}
jpa:
database: postgresql
Caused by: liquibase.exception.LiquibaseException: liquibase.exception.MigrationFailedException: Migration failed for change set changelog.xml::d::d:
Reason: liquibase.exception.DatabaseException: Таблица "STATUS" не найдена
Table "STATUS" not found; SQL statement:
ALTER TABLE TEST ADD CONSTRAINT STATUS_ID CHECK (exists (SELECT 1 FROM STATUS s WHERE STATUS_ID = s.id)) [42102-200] [Failed SQL: (42102) ALTER TABLE TEST ADD CONSTRAINT STATUS_ID CHECK (exists (SELECT 1 FROM STATUS s WHERE STATUS_ID = s.id))]
I found another solution.
The problem was in local developing with h2. (it always init as public schema). I'm just adding SET SCHEMA after creating it.
in test properties:
jdbc-url: 'jdbc:h2:file:~/rmdb;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE;AUTO_SERVER=TRUE;INIT=CREATE SCHEMA IF NOT EXISTS ${application.database.schema}\;SET SCHEMA ${application.database.schema}'
I want to create an external table in Redshift Spectrum from CSV files. When I try doing so with dbt, I get a strange error. But when I manually remove some double quotes from the SQL generated by dbt and run it directly, I get no such error.
First I run this in Redshift Query Editor v2 on default database dev in my cluster:
CREATE EXTERNAL SCHEMA example_schema
FROM DATA CATALOG
DATABASE 'example_db'
REGION 'us-east-1'
IAM_ROLE 'iam_role'
CREATE EXTERNAL DATABASE IF NOT EXISTS
;
Database dev now has an external schema named example_schema (and Glue catalog registers example_db).
I then upload example_file.csv to the S3 bucket s3://example_bucket. The file looks like this:
col1,col2
1,a,
2,b,
3,c
Then I run dbt run-operation stage_external_sources in my local dbt project and get this output with an error:
21:03:03 Running with dbt=1.0.1
21:03:03 [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- models.example_project.example_models
21:03:03 1 of 1 START external source example_schema.example_table
21:03:03 1 of 1 (1) drop table if exists "example_db"."example_schema"."example_table" cascade
21:03:04 Encountered an error while running operation: Database Error
cross-database reference to database "example_db" is not supported
I try running the generated SQL in Query Editor:
DROP TABLE IF EXISTS "example_db"."example_schema"."example_table" CASCADE
and get the same error message:
ERROR: cross-database reference to database "example_db" is not supported
But when I run this SQL in Query Editor, it works:
DROP TABLE IF EXISTS "example_db.example_schema.example_table" CASCADE
Note that I just removed some quotes.
What's going on here? Is this a bug in dbt-core, dbt-redshift, or dbt_external_tables--or just a mistake on my part?
To confirm, I can successfully create the external table by running this in Query Editor:
DROP SCHEMA IF EXISTS example_schema
DROP EXTERNAL DATABASE
CASCADE
;
CREATE EXTERNAL SCHEMA example_schema
FROM DATA CATALOG
DATABASE 'example_db'
REGION 'us-east-1'
IAM_ROLE 'iam_role'
CREATE EXTERNAL DATABASE IF NOT EXISTS
;
CREATE EXTERNAL TABLE example_schema.example_table (
col1 SMALLINT,
col2 CHAR(1)
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde'
STORED AS TEXTFILE
LOCATION 's3://example_bucket'
TABLE PROPERTIES ('skip.header.line.count'='1')
;
dbt config files
models/example/schema.yml (modeled after this example:
version: 2
sources:
- name: example_source
database: dev
schema: example_schema
loader: S3
tables:
- name: example_table
external:
location: 's3://example_bucket'
row_format: >
serde 'org.apache.hadoop.hive.serde2.OpenCSVSerde'
with serdeproperties (
'strip.outer.array'='false'
)
columns:
- name: col1
data_type: smallint
- name: col2
data_type: char(1)
dbt_project.yml:
name: 'example_project'
version: '1.0.0'
config-version: 2
profile: 'example_profile'
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]
target-path: "target"
clean-targets:
- "target"
- "dbt_packages"
models:
example_project:
example:
+materialized: view
packages.yml:
packages:
- package: dbt-labs/dbt_external_tables
version: 0.8.0
While trying to generate database using liquibase changelog in Postgres, I am getting following error:
forIndexName is not allowed on postgresql
Following is my change log which is generating same error:
<changeSet author="chintan.patel" id="CP0001">
<createIndex indexName="PK_USER" tableName="USER" unique="true">
<column name="FIRSTNAME"/>
<column name="MIDDLENAME"/>
<column name="LASTNAME"/>
</createIndex>
<addPrimaryKey columnNames="FIRSTNAME, MIDDLENAME, LASTNAME" constraintName="PK_USER" forIndexName="PK_USER" tableName="USER"/>
</changeSet>
This same change set works fine in Oracle.
Please suggest me, whats wrong here.
I am trying to create app using OpenJPA & Postgres 9.2.xx, Currently facing issue at DB level
1) Created schema say PCM:-
CREATE SCHEMA PCM
2) Tried create table :-
CREATE TABLE PCM.USER_PROFILE (
USER_PROFILE_ID BIGINT NOT NULL,
USER_FNAME VARCHAR(60),
USER_LNAME VARCHAR(60)
};
Got error "pcm" schema does not exists
Then tried creating table :-
CREATE TABLE "PCM.USER_PROFILE" (
USER_PROFILE_ID BIGINT NOT NULL,
USER_FNAME VARCHAR(60),
USER_LNAME VARCHAR(60)
};
Table is created successful,
If I list the schema:-
[postgres#DBMigration ~] $ psql -c "\dn"
List of schemas
Name | Owner
--------+----------
pcm | dbadmin
public | postgres
B) In persistence.xml , I have entered configuration
<property name="openjpa.jdbc.Schema" value="PCM" />
Now I am getting issue in OpenJPA stating schema is not present.
I tried refering here, but no success.
I have tried entering schema name in configuration as '\"PCM\"', "\"PCM\"", '\"pcm\"', "\"pcm\"".
Not sure where am I going wrong.
I need suggestion/help,
1) how or what is proper standard to create schema in Postgres & refer while creating table.
2) Is my entry in persistence.xml correct? Then why its not identifying the schema
Object names in Postgres when not quoted are implicitly converted to lower case.
When you create a table the way you did below with quotation mark on "PCM.USER_PROFILE" then the table is created in default public schema with the name of "PCM.USER_PROFILE".
CREATE TABLE "PCM.USER_PROFILE" (
USER_PROFILE_ID BIGINT NOT NULL,
USER_FNAME VARCHAR(60),
USER_LNAME VARCHAR(60)
);
However, your create statement mentioned in the post is completely valid (with the exception of changing } to ) at the end of command:
CREATE TABLE PCM.USER_PROFILE (
USER_PROFILE_ID BIGINT NOT NULL,
USER_FNAME VARCHAR(60),
USER_LNAME VARCHAR(60)
);
It creates user_profile table under pcm schema succesfully.
The error that I did was created schema outside database environment & root user. When we tried running select * from information_schema.schemata; under both users (root & db user) the schema was not listing.
Hence create schema under a DB by running query
psql -U [dbUser] -d [database] -c "CREATE SCHEMA pcm;"
or
psql -h localhost -U [dbUser] -d [database]
[database]#=> CREATE SCHEMA pcm;
Try running query to test if schema is loaded successfully under database & dbowner user.
[database]#=> select * from information_schema.schemata;
Index:
CREATE INDEX guild_name_lower_ops
ON guilds
USING btree
(lower(name::text) COLLATE pg_catalog."default" varchar_pattern_ops);
Generated changeset:
<changeSet author="Vlad (generated)" id="1450262497286-89">
<createIndex indexName="guild_name_lower_ops" tableName="guilds" />
</changeSet>
And it doesn't pass "status" command check with "columns is empty" error message.
Why it's not exported? Are there any workarounds so I can still use liquibase with my db?
Liquibase does not currently figure this out for Postgres. The workaround is to alter the XML after it is generated so that when creating new databases, the indexes are created properly.