Creating dockerized Postgresql with initial scripts - postgresql

I have a yml file that
2 volumes:¬
6 quartz-pg-master_data:¬
7 ¬
8 networks:¬
9 hostnet:¬
10 external: true¬
11 name: host¬
21 configs:¬
24 quartz-create_quartz_tables-20201019-1.sh:¬
25 file: ./config/quartz/create.sql¬
15 quartz-pg-master:¬
14 image: postgres¬
13 networks:¬
12 - internal¬
11 ports:¬
10 - published: 5432¬
9 target: 5432¬
8 mode: host¬
7 environment:¬
6 PGDATA: /pg_data¬
5 POSTGRES_DB: "quartz"
POSTGRES_USER: "quartz"
POSTGRES_PASSWORD: "password"¬
4 configs:¬
3 - source: quartz-create_quartz_tables-20201019-1.sh¬
2 target: /docker-entrypoint-initdb.d/create.sql¬
1 volumes:¬
0 - quartz-pg-master_data:/pg_data¬
which has create.sql:
CREATE TABLE qrtz_service_job_details
(
SCHED_NAME VARCHAR(120) NOT NULL,
JOB_NAME VARCHAR(200) NOT NULL,
JOB_GROUP VARCHAR(200) NOT NULL,
DESCRIPTION VARCHAR(250) NULL,
JOB_CLASS_NAME VARCHAR(250) NOT NULL,
IS_DURABLE BOOL NOT NULL,
IS_NONCONCURRENT BOOL NOT NULL,
IS_UPDATE_DATA BOOL NOT NULL,
REQUESTS_RECOVERY BOOL NOT NULL,
JOB_DATA BYTEA NULL,
PRIMARY KEY (SCHED_NAME,JOB_NAME,JOB_GROUP)
);
CREATE TABLE qrtz_service_triggers
(
SCHED_NAME VARCHAR(120) NOT NULL,
TRIGGER_NAME VARCHAR(200) NOT NULL,
TRIGGER_GROUP VARCHAR(200) NOT NULL,
JOB_NAME VARCHAR(200) NOT NULL,
JOB_GROUP VARCHAR(200) NOT NULL,
DESCRIPTION VARCHAR(250) NULL,
NEXT_FIRE_TIME BIGINT NULL,
PREV_FIRE_TIME BIGINT NULL,
PRIORITY INTEGER NULL,
TRIGGER_STATE VARCHAR(16) NOT NULL,
TRIGGER_TYPE VARCHAR(8) NOT NULL,
START_TIME BIGINT NOT NULL,
END_TIME BIGINT NULL,
CALENDAR_NAME VARCHAR(200) NULL,
MISFIRE_INSTR SMALLINT NULL,
JOB_DATA BYTEA NULL,
PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP),
FOREIGN KEY (SCHED_NAME,JOB_NAME,JOB_GROUP)
REFERENCES qrtz_service_JOB_DETAILS(SCHED_NAME,JOB_NAME,JOB_GROUP)
);
...
for quartz init.db.
I am successfully initializg postresql with logs:
When I am running
docker exec -ti bash, then
su quartz
and my result is:
root#3fac28fb199d:/# su quartz
su: user quartz does not exist.
However, I have tried with postgres -> su postgres
I can successfully go in but after that when I run psql it gives me this error
postgres#3fac28fb199d:/$ psql
psql: error: could not connect to server: FATAL: role "postgres" does not exist
Seems like my db and users are not created at all. What could be the problem?

In the docker bash container I need to use
psql dbname username so in my case it worked and my tables were created.
psql quartz quartz

Related

postgres creates table having trouble [duplicate]

This question already has answers here:
translate mysql "create table" to postgresql
(1 answer)
Error while creating table in PostgreSQL while migrating from MySQL
(1 answer)
MySQL to PostgreSQL table create conversion - charset and collation
(2 answers)
Closed 6 months ago.
CREATE TABLE if not exists `blue_async_cmd`
(
`id` int(11) NOT NULL AUTO_INCREMENT,
`task_id` varchar(64) NOT NULL,
`type` tinyint(4) NOT NULL DEFAULT '0',
`cmd` varchar(500) DEFAULT NULL,
`createTime` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `blue_async_cmd_index_taskId` (`task_id`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8mb4
[2022-08-29 12:47:50] [42601] ERROR: syntax error at or near "`"
[2022-08-29 12:47:50] 位置:28
Here is an example of a valid postgres SQL statement. Maybe that helps to update yours.
CREATE TABLE accounts (
user_id serial PRIMARY KEY,
username VARCHAR ( 50 ) UNIQUE NOT NULL,
password VARCHAR ( 50 ) NOT NULL,
email VARCHAR ( 255 ) UNIQUE NOT NULL,
created_on TIMESTAMP NOT NULL,
last_login TIMESTAMP
);

Installation failed to create the core tables | Installing Revive Adserver 5.2.0

I am installing revive ad server on Ubuntu 20.4 which is having MySQL Version 8.0.23. The Log I got from file /var/debug.log is
[Last executed query: CREATE TABLE `bv_banners` (`bannerid` MEDIUMINT(9) AUTO_INCREMENT NOT NULL, `campaignid` MEDIUMINT(9) DEFAULT 0 NOT NULL, `contenttype` ENUM('gif','jpeg','png','html','swf','dcr','rpm','mov','txt') DEFAULT 'gif' NOT NULL, `pluginversion` MEDIUMINT(9) DEFAULT 0 NOT NULL, `storagetype` ENUM('sql','web','url','html','network','txt') DEFAULT 'sql' NOT NULL, `filename` VARCHAR(255) DEFAULT '' NOT NULL, `imageurl` VARCHAR(255) DEFAULT '' NOT NULL, `htmltemplate` MEDIUMTEXT NOT NULL, `htmlcache` MEDIUMTEXT NOT NULL, `width` SMALLINT(6) DEFAULT 0 NOT NULL, `height` SMALLINT(6) DEFAULT 0 NOT NULL, `weight` TINYINT(4) DEFAULT 1 NOT NULL, `seq` TINYINT(4) DEFAULT 0 NOT NULL, `target` VARCHAR(16) DEFAULT '' NOT NULL, `url` TEXT NOT NULL, `alt` VARCHAR(255) DEFAULT '' NOT NULL, `statustext` VARCHAR(255) DEFAULT '' NOT NULL, `bannertext` TEXT NOT NULL, `description` VARCHAR(255) DEFAULT '' NOT NULL, `adserver` VARCHAR(255) DEFAULT '' NOT NULL, `block` INT(11) DEFAULT 0 NOT NULL, `capping` INT(11) DEFAULT 0 NOT NULL, `session_capping` INT(11) DEFAULT 0 NOT NULL, `compiledlimitation` TEXT NOT NULL, `acl_plugins` TEXT DEFAULT NULL, `append` TEXT NOT NULL, `bannertype` TINYINT(4) DEFAULT 0 NOT NULL, `alt_filename` VARCHAR(255) DEFAULT '' NOT NULL, `alt_imageurl` VARCHAR(255) DEFAULT '' NOT NULL, `alt_contenttype` ENUM('gif','jpeg','png') DEFAULT 'gif' NOT NULL, `comments` TEXT DEFAULT NULL, `updated` DATETIME NOT NULL, `acls_updated` DATETIME DEFAULT '0000-00-00 00:00:00' NOT NULL, `keyword` VARCHAR(255) DEFAULT '' NOT NULL, `transparent` TINYINT(1) DEFAULT 0 NOT NULL, `parameters` TEXT DEFAULT NULL, `status` INT(11) DEFAULT 0 NOT NULL, `ext_bannertype` VARCHAR(255) DEFAULT NULL, `prepend` TEXT NOT NULL, `iframe_friendly` TINYINT(1) DEFAULT 1 NOT NULL, PRIMARY KEY (bannerid)) ENGINE = INNODB]
[Native message: Invalid default value for 'acls_updated']
I got to know adserver is not officially supporting mysql version 8. But some people are able to run fine. I want to know how.
Here is the refrence I got
https://github.com/revive-adserver/revive-adserver/issues/1048
I want to run this.
Thanks in advance.
set sql_mode='40'' to sql_mode=''
in 5 files;
lib/OA/DB.php
lib/OA/DaL/Delivery/mysql.php
lib/OA/Dal/Delivery/mysqli.php
lib/OA/Upgrade/DB_Upgrade.php
lib/OA/Upgrade/Upgrade.php
and use table type INNODB
This trick works for me. I am calling this as a trick because it is not officially supported.
I got solution from github.
Here is the refrence
https://github.com/revive-adserver/revive-adserver/issues/1048

Error after upgrading Apache Superset - RuntimeError: Working outside of application context

I just upgraded to newest version of Apache Superset.
When I start Apache Superset with the command
gunicorn \
-w 10 \
-k gevent \
--timeout 120 \
-b 0.0.0.0:8088 \
--limit-request-line 0 \
--limit-request-field_size 0 \
--statsd-host localhost:8088 \
superset:app
I get the error "Internal server error"
On the server it says:
respiter = self.wsgi(environ, resp.start_response)
File "/usr/local/superset/lib/python3.7/site-packages/werkzeug/local.py", line 376, in <lambda>
__call__ = lambda x, *a, **kw: x._get_current_object()(*a, **kw)
File "/usr/local/superset/lib/python3.7/site-packages/werkzeug/local.py", line 307, in _get_current_object
return self.__local()
File "/usr/local/superset/lib/python3.7/site-packages/flask/globals.py", line 52, in _find_app
raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
...
I found out that it I changed to "superset.app:create_app()" I could login, like this:
gunicorn \
-w 10 \
-k gevent \
--timeout 120 \
-b 0.0.0.0:8088 \
--limit-request-line 0 \
--limit-request-field_size 0 \
--statsd-host localhost:8088 \
"superset.app:create_app()"
But When I open a dashboard I now get this error:
File "/usr/local/superset/lib/python3.7/site-packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1054, "Unknown column 'slices.schema_perm' in 'field list'")
[SQL: SELECT slices.created_on AS slices_created_on, slices.changed_on AS slices_changed_on, slices.id AS slices_id, slices.slice_name AS slices_slice_name, slices.datasource_id AS slices_datasource_id, slices.datasource_type AS slices_datasource_type, slices.datasource_name AS slices_datasource_name, slices.viz_type AS slices_viz_type, slices.params AS slices_params, slices.description AS slices_description, slices.cache_timeout AS slices_cache_timeout, slices.perm AS slices_perm, slices.schema_perm AS slices_schema_perm, slices.created_by_fk AS slices_created_by_fk, slices.changed_by_fk AS slices_changed_by_fk, tables_1.created_on AS tables_1_created_on, tables_1.changed_on AS tables_1_changed_on, tables_1.id AS tables_1_id, tables_1.description AS tables_1_description, tables_1.default_endpoint AS tables_1_default_endpoint, tables_1.is_featured AS tables_1_is_featured, tables_1.filter_select_enabled AS tables_1_filter_select_enabled, tables_1.offset AS tables_1_offset, tables_1.cache_timeout AS tables_1_cache_timeout, tables_1.params AS tables_1_params, tables_1.perm AS tables_1_perm, tables_1.schema_perm AS tables_1_schema_perm, tables_1.table_name AS tables_1_table_name, tables_1.main_dttm_col AS tables_1_main_dttm_col, tables_1.database_id AS tables_1_database_id, tables_1.fetch_values_predicate AS tables_1_fetch_values_predicate, tables_1.`schema` AS tables_1_schema, tables_1.`sql` AS tables_1_sql, tables_1.is_sqllab_view AS tables_1_is_sqllab_view, tables_1.template_params AS tables_1_template_params, tables_1.created_by_fk AS tables_1_created_by_fk, tables_1.changed_by_fk AS tables_1_changed_by_fk
FROM dashboard_slices, slices LEFT OUTER JOIN `tables` AS tables_1 ON slices.datasource_id = tables_1.id AND slices.datasource_type = %s
WHERE %s = dashboard_slices.dashboard_id AND slices.id = dashboard_slices.slice_id]
[parameters: ('table', 7)]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
I have searched more for a solution. It seems that my MySQL-database is not corrected with the upgrade, when using the db upgrade command:
(superset) [root#host .superset]# superset db upgrade
logging was configured successfully
INFO:superset.utils.logging_configurator:logging was configured successfully
/usr/local/superset/lib/python3.7/site-packages/flask_caching/__init__.py:188: UserWarning: Flask-Cache: CACHE_TYPE is set to null, caching is effectively disabled.
warnings.warn("Flask-Cache: CACHE_TYPE is set to null, "
WARNI [alembic.env] SQLite Database support for metadata databases will be removed in a future version of Superset.
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
I dont see any errors?
When I call init I get this:
(superset) [root#host .superset]# superset init
logging was configured successfully
INFO:superset.utils.logging_configurator:logging was configured successfully
/usr/local/superset/lib/python3.7/site-packages/flask_caching/__init__.py:188: UserWarning: Flask-Cache: CACHE_TYPE is set to null, caching is effectively disabled.
warnings.warn("Flask-Cache: CACHE_TYPE is set to null, "
Syncing role definition
INFO:superset.security.manager:Syncing role definition
Syncing Admin perms
INFO:superset.security.manager:Syncing Admin perms
Syncing Alpha perms
INFO:superset.security.manager:Syncing Alpha perms
Syncing Gamma perms
INFO:superset.security.manager:Syncing Gamma perms
Syncing granter perms
INFO:superset.security.manager:Syncing granter perms
Syncing sql_lab perms
INFO:superset.security.manager:Syncing sql_lab perms
Fetching a set of all perms to lookup which ones are missing
INFO:superset.security.manager:Fetching a set of all perms to lookup which ones are missing
Creating missing datasource permissions.
INFO:superset.security.manager:Creating missing datasource permissions.
Creating missing database permissions.
INFO:superset.security.manager:Creating missing database permissions.
Creating missing metrics permissions
INFO:superset.security.manager:Creating missing metrics permissions
Cleaning faulty perms
INFO:superset.security.manager:Cleaning faulty perms
Is it because superset needs some permission to MySQL to make the changes in tables?
I don't know why the command 'superset db upgrade' didn't update my MYSQL-database, but I then manually created these fields and tables and now it seems to be working:
slices.schema_perm
tables.schema_perm
dbs.encrypted_extra
dbs.allow_cvas
dbs.server_cert
saved_query.extra_json
And these tables with fields
CREATE TABLE `alerts` (
`id` int(11) NOT NULL,
`label` varchar(250) DEFAULT NULL,
`active` tinyint(2) DEFAULT NULL,
`crontab` varchar(250) DEFAULT NULL,
`sql` varchar(250) DEFAULT NULL,
`alert_type` varchar(250) DEFAULT NULL,
`recipients` varchar(250) DEFAULT NULL,
`log_retention` varchar(250) DEFAULT NULL,
`grace_period` varchar(250) DEFAULT NULL,
`slice_id` int(11) DEFAULT NULL,
`dashboard_id` int(11) DEFAULT NULL,
`database_id` int(11) DEFAULT NULL,
`last_eval_dttm` varchar(250) DEFAULT NULL,
`last_state` varchar(250) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
ALTER TABLE `alerts`
ADD PRIMARY KEY (`id`);
CREATE TABLE `tab_state` (
`created_on` datetime DEFAULT NULL,
`changed_on` datetime DEFAULT NULL,
`extra_json` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`id` int(11) NOT NULL,
`user_id` int(11) NOT NULL,
`label` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`active` tinyint(1) DEFAULT NULL,
`database_id` int(11) NOT NULL,
`schema` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`sql` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`query_limit` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`latest_query_id` int(11) NOT NULL,
`autorun` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`template_params` varchar(250) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_by_fk` int(11) NOT NULL,
`changed_by_fk` int(11) NOT NULL,
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
I don't know the field types, so maybe someone can help with the correct types.

What is wrong with this insert query for Aurora postgres 10.7?

I'm stumped as to why I'm getting the following error
operator does not exist: character varying = text[]
when the following query is running in production:-
INSERT INTO message_status
(created_datetime,correlation_id,batch_id,entity_type,entity_id,entity_status,entity_message,effective_date)
VALUES
('2020-02-04 10:24:14.291000000','6dc16864-5820-475e-918e-51b15722c08a','34d9c646-9bc2-4389-9789-9c0482ba743e','Benchmark','ABCDEFGH','VALIDATION_ERRORED','{"type":"NO_RESULTS","properties":{}}','2020-01-12');
Not able to reproduce LOCALLY or in DBEAVER. Is there anything obviously wrong with this query? We recently migrated from AWS RDS MySQL to AWS Aurora Postgres
DDL
CREATE TABLE public.message_status (
id serial NOT NULL,
correlation_id varchar(36) NULL,
entity_type varchar(32) NULL,
entity_id varchar(32) NULL,
entity_status varchar(32) NULL,
entity_message text NULL,
effective_date timestamp NOT NULL,
record_count int4 NULL,
created_datetime timestamp NOT NULL DEFAULT CURRENT_DATE,
batch_id varchar(50) NULL,
CONSTRAINT message_status_pkey PRIMARY KEY (id)
)
WITH (
OIDS=FALSE
) ;
CREATE INDEX ix_message_status ON public.message_status USING btree (effective_date, entity_type, entity_id, created_datetime) ;
CREATE INDEX ix_message_status_effective_date ON public.message_status USING btree (effective_date) ;
Postgres version = 10.7
pg-promise npm library version = 9.2.1

Getting error on Postgresql TABLESPACE and CREATE commands

I am completely new in Postgres commands.
I have the following DB2 commands to create tables and table spaces:
CREATE USER TEMPORARY TABLESPACE MYSPACE MANAGED BY AUTOMATIC STORAGE#
CREATE TABLESPACE SYSTOOLSPACE MANAGED BY AUTOMATIC STORAGE#
CREATE SEQUENCE REVISION AS BIGINT START WITH 1 INCREMENT BY 1 MAXVALUE 4611686018427387903 CYCLE CACHE 1000#
Now i want to run these commands in postgresql , I have tried with
my_db=# CREATE USER TEMPORARY TABLESPACE MYSPACE MANAGED BY AUTOMATIC STORAGE;
ERROR: syntax error at or near "TABLESPACE"
LINE 1: CREATE USER TEMPORARY TABLESPACE MYSPACE MANAGED BY AUTO...
my_db=# CREATE TABLESPACE SYSTOOLSPACE MANAGED BY AUTOMATIC STORAGE;
ERROR: syntax error at or near "MANAGED"
LINE 1: CREATE TABLESPACE SYSTOOLSPACE MANAGED BY AUTOMATIC STORAGE;
I also have this DB2 CREATE TABLE statement:
CREATE TABLE USER (
ID BIGINT GENERATED BY DEFAULT AS IDENTITY (START WITH 1 INCREMENT BY 1 NO MAXVALUE NO CYCLE CACHE 100),
E_VER BIGINT NOT NULL,
NAME VARCHAR(38) NOT NULL UNIQUE,
EMAIL_ADDRESS VARCHAR(255) NOT NULL,
PASSWORD VARCHAR(32) NOT NULL,
SUPER_ADMIN SMALLINT NOT NULL,
MAIN_ADMIN SMALLINT NOT NULL,
SERVER_ADMIN SMALLINT NOT NULL,
GROUP_ADMIN SMALLINT NOT NULL,
CLIENT_ADMIN SMALLINT NOT NULL,
ENABLED SMALLINT NOT NULL,
HIDDEN SMALLINT NOT NULL,
PRIMARY KEY (ID)
)#
and I have tried to convert this to Postgres:
CREATE SEQUENCE USER_seq START WITH 1 INCREMENT BY 1;
CREATE TABLE USER (
ID BIGINT DEFAULT NEXTVAL ('USER_seq'),
E_VER BIGINT NOT NULL,
NAME VARCHAR(38) NOT NULL UNIQUE,
EMAIL_ADDRESS VARCHAR(255) NOT NULL,
PASSWORD VARCHAR(32) NOT NULL,
SUPER_ADMIN SMALLINT NOT NULL,
MAIN_ADMIN SMALLINT NOT NULL,
SERVER_ADMIN SMALLINT NOT NULL,
GROUP_ADMIN SMALLINT NOT NULL,
CLIENT_ADMIN SMALLINT NOT NULL,
ENABLED SMALLINT NOT NULL,
HIDDEN SMALLINT NOT NULL,
PRIMARY KEY (ID)
)#
by http://www.sqlines.com/online this online site. After running this command i am getting error like
my_db=# CREATE SEQUENCE USER_seq START WITH 1 INCREMENT BY 1;
CREATE SEQUENCE
my_db=#
my_db=#
my_db=# CREATE TABLE USER (
my_db(# ID BIGINT DEFAULT NEXTVAL ('USER_seq'),
my_db(# E_VER BIGINT NOT NULL,
my_db(# NAME VARCHAR(38) NOT NULL UNIQUE,
my_db(# EMAIL_ADDRESS VARCHAR(255) NOT NULL,
my_db(# PASSWORD VARCHAR(32) NOT NULL,
my_db(# SUPER_ADMIN SMALLINT NOT NULL,
my_db(# MAIN_ADMIN SMALLINT NOT NULL,
my_db(# SERVER_ADMIN SMALLINT NOT NULL,
my_db(# GROUP_ADMIN SMALLINT NOT NULL,
my_db(# CLIENT_ADMIN SMALLINT NOT NULL,
my_db(# ENABLED SMALLINT NOT NULL,
my_db(# HIDDEN SMALLINT NOT NULL,
my_db(# PRIMARY KEY (ID)
my_db(# );
ERROR: syntax error at or near "USER"
LINE 1: CREATE TABLE USER (
^
Anything wrong this conversion? Any suggestion solve this error?
USER is a reserved word, you need to escape it as CREATE TABLE "USER".
As for your CREATE USER and CREATE TABLESPACE commands, that's just wrong syntax. There's no MANAGED BY in Postgres for example.