Database tables for QuickFIX/J - quickfix

QuickFIX/J includes SQL scripts to create four database tables:
sessions
messages
messages_log
event_log
I cannot find any documentation that describes the purpose of each of these tables.
What are they for, when are they written to, do any of them grow indefinitely, etc...

Some tables are used for the store, others are used for logging (the *_log tables.) A store is required for QuickFIX/J to operate (it tracks session state and supports resends of messages) whereas the log is optional.
sessions
This table tracks active FIX sessions. Sessions have a composite key of the eight values shown in the primary key declaration below.
creation_time is used to determine which session this applies to.
*_seqnum columns track the current sequence numbers for the session, and are used for reliability along with resend requests.
create table sessions (
beginstring char(8) not null,
sendercompid varchar(64) not null,
sendersubid varchar(64) not null,
senderlocid varchar(64) not null,
targetcompid varchar(64) not null,
targetsubid varchar(64) not null,
targetlocid varchar(64) not null,
session_qualifier varchar(64) not null,
creation_time timestamp not null,
incoming_seqnum integer not null,
outgoing_seqnum integer not null,
primary key (beginstring, sendercompid, sendersubid, senderlocid,
targetcompid, targetsubid, targetlocid, session_qualifier)
);
messages
This table provides a persistent store of the FIX messages sent during an active session. If the party being communicated with requires messages to be resent, QuickFIX/J will use this table to determine the content of the messages.
At the start of each session, messages for the session_qualifier are removed. Thus this table will not grow indefinitely and the upper bound of its size depends upon how many messages may be sent during a session.
create table messages (
beginstring char(8) not null,
sendercompid varchar(64) not null,
sendersubid varchar(64) not null,
senderlocid varchar(64) not null,
targetcompid varchar(64) not null,
targetsubid varchar(64) not null,
targetlocid varchar(64) not null,
session_qualifier varchar(64) not null,
msgseqnum integer not null,
message text not null,
primary key (beginstring, sendercompid, sendersubid, senderlocid,
targetcompid, targetsubid, targetlocid, session_qualifier,
msgseqnum)
);
messages_log
QuickFIX/J can log all inbound/outbound messages to a database. The table is write-only from the perspective of the library, so it is up to you if you wish to use this table or not.
Different tables may be specified in config for inbound and outbound message logs. By default, all messages are logged to a single table.
create sequence messages_log_sequence;
create table messages_log (
id integer default nextval('messages_log_sequence'),
time timestamp not null,
beginstring char(8) not null,
sendercompid varchar(64) not null,
sendersubid varchar(64) not null,
senderlocid varchar(64) not null,
targetcompid varchar(64) not null,
targetsubid varchar(64) not null,
targetlocid varchar(64) not null,
session_qualifier varchar(64),
text text not null,
primary key (id)
);
event_log
A log of events is written to this table. Examples include:
Session FIX.4.2:FOO->BAR schedule is daily, 07:00:00-UTC - 21:00:00-UTC
Created session: FIX.4.2:FOO->BAR
Initiated logon request
Received logon
create sequence event_log_sequence;
create table event_log (
id integer default nextval('event_log_sequence'),
time timestamp not null,
beginstring char(8) not null,
sendercompid varchar(64) not null,
sendersubid varchar(64) not null,
senderlocid varchar(64) not null,
targetcompid varchar(64) not null,
targetsubid varchar(64) not null,
targetlocid varchar(64) not null,
session_qualifier varchar(64),
text text not null,
primary key (id)
);
As #DumbCoder points out, the table names may be customised via config.

Related

Postgres - Create or replace table

I have a table that is already created with data and I need only modify the schema to add some constraints .
my created tabled schema
CREATE TABLE public.note (
note_id bigint NOT NULL,
confidential boolean NOT NULL,
follow_up_date date,
notification_date date,
priority integer NOT NULL,
recurring_follow_up_interval interval,
status character varying(255) NOT NULL,
create_date timestamp with time zone,
deleted boolean NOT NULL,
last_modified_date timestamp with time zone,
time_spent integer,
title text,
version bigint NOT NULL,
assigned_to_id bigint,
note_category_id bigint NOT NULL,
created_by_id bigint,
last_modified_by_id bigint
);
what I need to match
create table note
(
note_id bigint default nextval('note_id_seq'::regclass) not null,
confidential boolean not null,
follow_up_date date,
notification_date date,
priority integer default 0 not null,
recurring_follow_up_interval interval,
status varchar(255) not null,
create_date timestamp with time zone,
deleted boolean default false not null,
last_modified_date timestamp with time zone,
time_spent integer,
title text,
version bigint default 0 not null,
assigned_to_id bigint,
note_category_id bigint not null,
created_by_id bigint,
last_modified_by_id bigint,
constraint note_pkey
primary key (note_id),
constraint fk_4nrhbn2j8j2vqqh78vleef9xr
foreign key (created_by_id) references admin_user,
constraint fk_eid7x7jfvjoe1h5tnyouhmqpa
foreign key (assigned_to_id) references admin_user,
constraint fk_oi5l4dg3sg5ep5neagmvp9r7o
foreign key (note_category_id) references note_category,
constraint fk_tk8ncyc0hmdi3gfh67b4jyu3l
foreign key (last_modified_by_id) references admin_user
);
as you can see it missing all defaults + all constraint to the other tables
any way to copy the intended schema to the created one without loosing the data
I am using Postgres 12
UPDATE
I know I could use alter to modify some columns but it will be a long process for me as there are many columns and I got more than 300 tables that have the same case
I manually alter one column to add sequence but I need easier way to do that for all columns
ALTER TABLE ONLY note ALTER COLUMN note_id SET DEFAULT nextval('note_id_seq'::regclass);

How to change the migration and not destroy anything?

I have a user table and I should remove the NOT NULL restrictions from the firstname, username and lastname fields, do I understand correctly that nothing can be changed directly, do I need to add a new sql file and with the command? How can I remove NOT NULL and break nothing? everything breaks down
CREATE TABLE "user" (
id SERIAL UNIQUE PRIMARY KEY,
username VARCHAR(255) NOT NULL,
password VARCHAR(255) NOT NULL,
email VARCHAR(255) NOT NULL,
first_name VARCHAR(255) NOT NULL,
last_name VARCHAR(255) NOT NULL,
last_password_reset TIMESTAMP NOT NULL DEFAULT NOW(),
roles JSONB NOT NULL,
active BOOLEAN NOT NULL DEFAULT TRUE,
date_created TIMESTAMP NOT NULL DEFAULT NOW(),
date_updated TIMESTAMP NOT NULL DEFAULT NOW()
);
ALTER TABLE user ALTER COLUMN username DROP NOT NULL;

postgresql cannot insert data to newly added column

In postgresql I have a table which I need to add a new column. the original table ddl is belowing:
CREATE TABLE survey.survey_response (
id uuid NOT NULL DEFAULT uuid_generate_v4(),
survey_id uuid NOT NULL,
survey_question_id uuid NULL,
user_id varchar(256) NULL,
device_id varchar(256) NULL,
user_country varchar(100) NULL,
client_type varchar(100) NULL,
product_version varchar(100) NULL,
answer text NULL,
response_date timestamptz NOT NULL DEFAULT now(),
survey_category varchar(100) NULL,
tags varchar(250) NULL,
tracking_id uuid NULL,
CONSTRAINT survey_response_pkey PRIMARY KEY (id)
)
WITH (
OIDS=FALSE
) ;
Then I alter the table to add a new column:
alter table survey.survey_response add column system_tags varchar(30) ;
But after that I found my instert statement cannot make change to this new column, for all the original columns it works fine:
INSERT INTO survey.survey_response
(id, survey_id, user_id, tags, system_tags)
VALUES(uuid_generate_v4(), uuid_generate_v4(),'1123','dsfsd', 'dsfsd');
select * from survey.survey_response where user_id = '1123';
The "tags" columns contains inserted value, however, system_tags keeps null.
I tested the above scenario in my local postgreSQL 9.6, any ideas about this strange behavior? Thanks a lot
-----------------update----------
I found this survey.survey_response table has been partitioning based on month, So my inserted record will also be displayed in survey.survey_response_y2017m12. but the new system_tags column is also NULL
CREATE TABLE survey.survey_response_y2017m12 (
id uuid NOT NULL DEFAULT uuid_generate_v4(),
survey_id uuid NOT NULL,
survey_question_id uuid NULL,
user_id varchar(256) NULL,
device_id varchar(256) NULL,
user_country varchar(100) NULL,
client_type varchar(100) NULL,
product_version varchar(100) NULL,
answer text NULL,
response_date timestamptz NOT NULL DEFAULT now(),
survey_category varchar(100) NULL,
tags varchar(250) NULL,
tracking_id uuid NULL,
system_tags varchar(30) NULL,
CONSTRAINT survey_response_y2017m12_response_date_check CHECK (((response_date >= '2017-12-01'::date) AND (response_date < '2018-01-01'::date)))
)
INHERITS (survey.survey_response)
WITH (
OIDS=FALSE
) ;
If I run the same scenario in a non-partition table then the insert works fine.
So do I need any special settings for alter table for partition table?
Old thread but you need to drop and create again the RULE to fix the issue.

Getting error on Postgresql TABLESPACE and CREATE commands

I am completely new in Postgres commands.
I have the following DB2 commands to create tables and table spaces:
CREATE USER TEMPORARY TABLESPACE MYSPACE MANAGED BY AUTOMATIC STORAGE#
CREATE TABLESPACE SYSTOOLSPACE MANAGED BY AUTOMATIC STORAGE#
CREATE SEQUENCE REVISION AS BIGINT START WITH 1 INCREMENT BY 1 MAXVALUE 4611686018427387903 CYCLE CACHE 1000#
Now i want to run these commands in postgresql , I have tried with
my_db=# CREATE USER TEMPORARY TABLESPACE MYSPACE MANAGED BY AUTOMATIC STORAGE;
ERROR: syntax error at or near "TABLESPACE"
LINE 1: CREATE USER TEMPORARY TABLESPACE MYSPACE MANAGED BY AUTO...
my_db=# CREATE TABLESPACE SYSTOOLSPACE MANAGED BY AUTOMATIC STORAGE;
ERROR: syntax error at or near "MANAGED"
LINE 1: CREATE TABLESPACE SYSTOOLSPACE MANAGED BY AUTOMATIC STORAGE;
I also have this DB2 CREATE TABLE statement:
CREATE TABLE USER (
ID BIGINT GENERATED BY DEFAULT AS IDENTITY (START WITH 1 INCREMENT BY 1 NO MAXVALUE NO CYCLE CACHE 100),
E_VER BIGINT NOT NULL,
NAME VARCHAR(38) NOT NULL UNIQUE,
EMAIL_ADDRESS VARCHAR(255) NOT NULL,
PASSWORD VARCHAR(32) NOT NULL,
SUPER_ADMIN SMALLINT NOT NULL,
MAIN_ADMIN SMALLINT NOT NULL,
SERVER_ADMIN SMALLINT NOT NULL,
GROUP_ADMIN SMALLINT NOT NULL,
CLIENT_ADMIN SMALLINT NOT NULL,
ENABLED SMALLINT NOT NULL,
HIDDEN SMALLINT NOT NULL,
PRIMARY KEY (ID)
)#
and I have tried to convert this to Postgres:
CREATE SEQUENCE USER_seq START WITH 1 INCREMENT BY 1;
CREATE TABLE USER (
ID BIGINT DEFAULT NEXTVAL ('USER_seq'),
E_VER BIGINT NOT NULL,
NAME VARCHAR(38) NOT NULL UNIQUE,
EMAIL_ADDRESS VARCHAR(255) NOT NULL,
PASSWORD VARCHAR(32) NOT NULL,
SUPER_ADMIN SMALLINT NOT NULL,
MAIN_ADMIN SMALLINT NOT NULL,
SERVER_ADMIN SMALLINT NOT NULL,
GROUP_ADMIN SMALLINT NOT NULL,
CLIENT_ADMIN SMALLINT NOT NULL,
ENABLED SMALLINT NOT NULL,
HIDDEN SMALLINT NOT NULL,
PRIMARY KEY (ID)
)#
by http://www.sqlines.com/online this online site. After running this command i am getting error like
my_db=# CREATE SEQUENCE USER_seq START WITH 1 INCREMENT BY 1;
CREATE SEQUENCE
my_db=#
my_db=#
my_db=# CREATE TABLE USER (
my_db(# ID BIGINT DEFAULT NEXTVAL ('USER_seq'),
my_db(# E_VER BIGINT NOT NULL,
my_db(# NAME VARCHAR(38) NOT NULL UNIQUE,
my_db(# EMAIL_ADDRESS VARCHAR(255) NOT NULL,
my_db(# PASSWORD VARCHAR(32) NOT NULL,
my_db(# SUPER_ADMIN SMALLINT NOT NULL,
my_db(# MAIN_ADMIN SMALLINT NOT NULL,
my_db(# SERVER_ADMIN SMALLINT NOT NULL,
my_db(# GROUP_ADMIN SMALLINT NOT NULL,
my_db(# CLIENT_ADMIN SMALLINT NOT NULL,
my_db(# ENABLED SMALLINT NOT NULL,
my_db(# HIDDEN SMALLINT NOT NULL,
my_db(# PRIMARY KEY (ID)
my_db(# );
ERROR: syntax error at or near "USER"
LINE 1: CREATE TABLE USER (
^
Anything wrong this conversion? Any suggestion solve this error?
USER is a reserved word, you need to escape it as CREATE TABLE "USER".
As for your CREATE USER and CREATE TABLESPACE commands, that's just wrong syntax. There's no MANAGED BY in Postgres for example.

Will My Table Be Acceptable For PostgreSQL?

I want to create a table in a PostgreSQL database to store user data (Account info, contact info, geographical info). I have typed out the following SQL and I am wondering is it acceptable for PostgreSQL (in terms of best practices, data types, and lengths)?
CREATE TABLE users (
userID SERIAL,
username VARCHAR(255) NOT NULL,
password VARCHAR(60) NOT NULL,
email VARCHAR(255) NOT NULL,
active VARCHAR(255) NOT NULL,
lenderAcct BOOLEAN NOT NULL DEFAULT FALSE,
resetToken VARCHAR(255) DEFAULT NULL,
resetComplete VARCHAR(3) DEFAULT 'No',
CONSTRAINT users_pk PRIMARY KEY (userID),
firstName VARCHAR(20) NOT NULL,
middleName VARCHAR(20),
lastName VARCHAR(20) NOT NULL,
primaryPhone VARCHAR(50) NOT NULL,
primaryPhoneExt VARCHAR(10),
altPhone VARCHAR(50),
altPhoneExt VARCHAR(10),
fax VARCHAR(50),
legalAddress1 VARCHAR(25) NOT NULL,
legalAddress2 VARCHAR(25),
legalCity VARCHAR(25) NOT NULL,
legalState VARCHAR(25) NOT NULL,
legalZip VARCHAR(16) NOT NULL,
legalCountry VARCHAR(25) NOT NULL,
mailAddress1 VARCHAR(25) NOT NULL,
mailAddress2 VARCHAR(25),
mailCity VARCHAR(25) NOT NULL,
mailState VARCHAR(25) NOT NULL,
mailZip VARCHAR(16) NOT NULL,
mailCountry VARCHAR(25) NOT NULL
);
The PostgreSQL side of the table looks ok. You could consider changing the varchar(x) types to text (check out this answer). The PRIMARY KEY can be introduced as userID SERIAL PRIMARY KEY, ... although this has no effect on the table structure.
Best practises cannot be commented on more without knowing your full table structure but those considerations are mostly not PostgreSQL specific anyway.