postgres schema not found when create with upper case - postgresql

I am trying to create app using OpenJPA & Postgres 9.2.xx, Currently facing issue at DB level
1) Created schema say PCM:-
CREATE SCHEMA PCM
2) Tried create table :-
CREATE TABLE PCM.USER_PROFILE (
USER_PROFILE_ID BIGINT NOT NULL,
USER_FNAME VARCHAR(60),
USER_LNAME VARCHAR(60)
};
Got error "pcm" schema does not exists
Then tried creating table :-
CREATE TABLE "PCM.USER_PROFILE" (
USER_PROFILE_ID BIGINT NOT NULL,
USER_FNAME VARCHAR(60),
USER_LNAME VARCHAR(60)
};
Table is created successful,
If I list the schema:-
[postgres#DBMigration ~] $ psql -c "\dn"
List of schemas
Name | Owner
--------+----------
pcm | dbadmin
public | postgres
B) In persistence.xml , I have entered configuration
<property name="openjpa.jdbc.Schema" value="PCM" />
Now I am getting issue in OpenJPA stating schema is not present.
I tried refering here, but no success.
I have tried entering schema name in configuration as '\"PCM\"', "\"PCM\"", '\"pcm\"', "\"pcm\"".
Not sure where am I going wrong.
I need suggestion/help,
1) how or what is proper standard to create schema in Postgres & refer while creating table.
2) Is my entry in persistence.xml correct? Then why its not identifying the schema

Object names in Postgres when not quoted are implicitly converted to lower case.
When you create a table the way you did below with quotation mark on "PCM.USER_PROFILE" then the table is created in default public schema with the name of "PCM.USER_PROFILE".
CREATE TABLE "PCM.USER_PROFILE" (
USER_PROFILE_ID BIGINT NOT NULL,
USER_FNAME VARCHAR(60),
USER_LNAME VARCHAR(60)
);
However, your create statement mentioned in the post is completely valid (with the exception of changing } to ) at the end of command:
CREATE TABLE PCM.USER_PROFILE (
USER_PROFILE_ID BIGINT NOT NULL,
USER_FNAME VARCHAR(60),
USER_LNAME VARCHAR(60)
);
It creates user_profile table under pcm schema succesfully.

The error that I did was created schema outside database environment & root user. When we tried running select * from information_schema.schemata; under both users (root & db user) the schema was not listing.
Hence create schema under a DB by running query
psql -U [dbUser] -d [database] -c "CREATE SCHEMA pcm;"
or
psql -h localhost -U [dbUser] -d [database]
[database]#=> CREATE SCHEMA pcm;
Try running query to test if schema is loaded successfully under database & dbowner user.
[database]#=> select * from information_schema.schemata;

Related

liquibase default schema ignored in sql changelog

Problem: liquibase can't find table without setting schema in SQL script.
How to say liquibase use default schema in SQL changelog?
Before sql changelog, for adding check constraint, I create all table, without setting schema. Schema was set in application.properties and all table was created correctly in $RM_DB_SCHEMA.
RM_DB_SCHEMA: MANAGER
RM_DB_URL: "jdbc:h2:file:~/rmdb;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE;AUTO_SERVER=TRUE;INIT=CREATE SCHEMA IF NOT EXISTS ${RM_DB_SCHEMA}"
RM_DB_USER: sa
RM_DB_PASSWORD: admin
RM_LB_USER: ${RM_DB_USER}
RM_LB_PASSWORD: ${RM_DB_PASSWORD}
spring:
datasource:
hikari:
schema: ${RM_DB_SCHEMA}
username: ${RM_DB_USER}
password: ${RM_DB_PASSWORD}
jdbc-url: ${RM_DB_URL}
liquibase:
change-log: "classpath:db/manager-changelog.xml"
default-schema: ${RM_DB_SCHEMA}
user: ${RM_LB_USER}
password: ${RM_LB_PASSWORD}
jpa:
database: postgresql
Caused by: liquibase.exception.LiquibaseException: liquibase.exception.MigrationFailedException: Migration failed for change set changelog.xml::d::d:
Reason: liquibase.exception.DatabaseException: Таблица "STATUS" не найдена
Table "STATUS" not found; SQL statement:
ALTER TABLE TEST ADD CONSTRAINT STATUS_ID CHECK (exists (SELECT 1 FROM STATUS s WHERE STATUS_ID = s.id)) [42102-200] [Failed SQL: (42102) ALTER TABLE TEST ADD CONSTRAINT STATUS_ID CHECK (exists (SELECT 1 FROM STATUS s WHERE STATUS_ID = s.id))]
I found another solution.
The problem was in local developing with h2. (it always init as public schema). I'm just adding SET SCHEMA after creating it.
in test properties:
jdbc-url: 'jdbc:h2:file:~/rmdb;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE;AUTO_SERVER=TRUE;INIT=CREATE SCHEMA IF NOT EXISTS ${application.database.schema}\;SET SCHEMA ${application.database.schema}'

DataJpaTest: Numeric scale default seems to be 0 with spring-boot-starter 2.7.1

I have a DataJpaTest with some schema.sql and data.sql for preparing the postgresql in-memory database. I've just upgraded spring-boot-starter-parent from 2.6.3 to 2.7.1, and now the test fails.
schema:
CREATE TABLE IF NOT EXISTS some_table(
id BIGSERIAL,
name TEXT,
problematic_number NUMERIC NOT NULL
);
data:
INSERT INTO some_table (name, problematic_number) VALUES ('something', 1.4321);
For some reason a test is failing now with:
org.opentest4j.AssertionFailedError:
Expected :1.4321
Actual :1
I also connected to the h2 database and I got really "1" in here instead of "1.4321". Before my spring upgrade, the test was fine.
Did the default scale for numeric maybe change? if I change my schema.sql to NUMERIC(10,4), the test succeeds.

Gcloud crashed (ValueError): Invalid header value

I used the following DDL command to create a table in gcloud spanner database named "messages" in the "guestbook" spanner instance
gcloud spanner databases ddl update messages \
--instance=guestbook --ddl="$(<~/guestbook-service/db/spanner.ddl)"
spanner.ddl contains the following
CREATE TABLE guestbook_message (
id STRING(36) NOT NULL,
name STRING(255) NOT NULL,
image_uri STRING(255),
message STRING(255)
) PRIMARY KEY (id);
But I get the following error.
ERROR: gcloud crashed (ValueError): Invalid header value
b'/usr/bin/../lib/google-cloud-sdk/lib/gcloud.py spanner databases ddl
update messages --instance=guestbook --d dl=CREATE TABLE
guestbook_message (\n id STRING(36) NOT NULL,\n name STRING(255)
NOT NULL,\n image_uri STRING(255),\n message STRING(255)\n)
PRIMARY KEY (id );'
How can I fix this?
This gcloud command is not accepting ddl statements with new line character \n.
It is enough to change spanner.ddl to:
CREATE TABLE guestbook_message (id STRING(36) NOT NULL,name STRING(255) NOT NULL,image_uri STRING(255),message STRING(255)) PRIMARY KEY (id);
If all is in one line, without those \n it works fine.

Confluent Kafka Sink Connector is not loading data to Postgres table

I am trying to load data to Postgres table(s) through Kafka Sink connector but I am getting the following error:
Caused by: org.apache.kafka.connect.errors.ConnectException: Cannot ALTER to add missing field SinkRecordField{schema=Schema{STRING}, name='A_ABBREV', isPrimaryKey=false}, as it is not optional and does not have a default value
The table in Postgres DB already has the field A_ABBREV, but now sure why I am getting missing field error.
Did anyone face the similar kind of issue?
Below is my Sink Connector Configuration:
connector.class=io.confluent.connect.jdbc.JdbcSinkConnector
table.name.format=AGENCY
connection.password=passcode
topics=AGENCIES
tasks.max=1
batch.size=10000
fields.whitelist=A_ID, A_NAME, A_ABBREV
connection.user=pmmdevuser
name=partner5-jdbcSinkConnector
connection.url=jdbc:postgresql://aws-db.sdfdgfdrwwisc.us-east- 1.rds.amazonaws.com:3306/pmmdevdb?currentSchema=ams
insert.mode=upsert
pk.mode=record_value
pk.fields=A_ID
auto.create=false
I am using Liquibase scripts to create tables and below is the create query from the postgres DB which got created through Liquibase scripts:
"CREATE TABLE gds.agency
(
a_id integer NOT NULL,
a_name character varying(100) COLLATE pg_catalog."default" NOT NULL,
a_abbrev character varying(8) COLLATE pg_catalog."default" NOT NULL,
source character varying(255) COLLATE pg_catalog."default" NOT NULL DEFAULT 'AMS'::character varying,
CONSTRAINT pk_agency PRIMARY KEY (a_id),
CONSTRAINT a_abbrev_uk1 UNIQUE (a_abbrev)
)"
From my experience, this means that that the field definition for the sink does not match the field definition for the source table/database. Make sure the field definitions match. Inspect the individual record the sink connector is trying to write to your target db. You should be able to see this insert statement in debug mode in the stack trace. take that query and run it manually to get a clearer idea of the error from the database.

Spring Boot 2 - H2 Database - #SpringBootTest - Failing on org.h2.jdbc.JdbcSQLException: Table already exists

Unable to test Spring Boot & H2 with a script for creation of table using schema.sql.
So, what’s happening is that I have the following properties set:
spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.initialization-mode=always
spring.datasource.username=sa
spring.datasource.password=
spring.datasource.platform=h2
spring.datasource.url=jdbc:h2:mem:city;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.jpa.generate-ddl=false
spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true
and, I expect the tables to be created using the schema.sql. The application works fine when I run gradle bootRun. However, when I run tests using gradle test, my tests for Repository passes, but the one for my Service fails stating that it’s trying to create the table when the table already exists:
Exception raised:
Caused by: org.h2.jdbc.JdbcSQLException: Table "CITY" already exists;
SQL statement:
CREATE TABLE city ( id BIGINT NOT NULL, country VARCHAR(255) NOT NULL, map VARCHAR(255) NOT NULL, name VARCHAR(255) NOT NULL, state VARCHAR(2555) NOT NULL, PRIMARY KEY (id) ) [42101-196]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:179)
at org.h2.message.DbException.get(DbException.java:155)
at org.h2.command.ddl.CreateTable.update(CreateTable.java:117)
at org.h2.command.CommandContainer.update(CommandContainer.java:101)
at org.h2.command.Command.executeUpdate(Command.java:260)
at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:192)
at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:164)
at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:95)
at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java)
at org.springframework.jdbc.datasource.init.ScriptUtils.executeSqlScript(ScriptUtils.java:471)
... 105 more
The code is setup and ready to recreate the scenario. README has all the information ->
https://github.com/tekpartner/learn-spring-boot-data-jpa-h2
If the tests are run individually, they pass. I think the problem is due to schema.sql being executed twice against the same database. It fails the second time as the tables already exist.
As a workaround, you could set spring.datasource.continue-on-error=true in application.properties.
Another option is to add the #AutoConfigureTestDatabase annotation where appropriate so that a unique embedded database is used for each test.
There are 2 other possible solutions you could try:
Add a drop table if exists [tablename] in your schema.sql before you create the table.
Change the statement from CREATE TABLE to CREATE TABLE IF NOT EXISTS