I am currently trying to convert a postgres query to be compatible with Snowflake and work the same way.
Postgres
CREATE SEQUENCE IF NOT EXISTS public.etl_jobs_delta_loading_id_seq
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;
CREATE TABLE IF NOT EXISTS public.etl_jobs_delta_loading
(
id INTEGER DEFAULT nextval('public.etl_jobs_delta_loading_id_seq'::regclass) NOT NULL,
job_name VARCHAR(500) NOT NULL,
loaded_until TIMESTAMP,
etl_execution_time TIMESTAMP,
execution_status VARCHAR(30)
);
I translated the sequence to Snowflake, but keep getting errors while trying to get the nextVal in snowflake.
Snowflake
CREATE OR REPLACE SEQUENCE etl_jobs_delta_loading_id_seq
START = 1
INCREMENT = 1
CREATE TABLE IF NOT EXISTS public.etl_jobs_delta_loading
(
id INTEGER DEFAULT nextval('public.etl_jobs_delta_loading_id_seq'::regclass) NOT NULL, -- statement that needs to be converted
job_name VARCHAR(500) NOT NULL,
loaded_until TIMESTAMP,
etl_execution_time TIMESTAMP,
execution_status VARCHAR(30)
);
I have tried various approaches on creating the etl_jobs_delta_loading table but no luck till now. Any ideas on how to implement this in snowflake?
The correct syntax to get value from sequence is <seq_name>.NEXTVAL:
CREATE OR REPLACE SEQUENCE PUBLIC.etl_jobs_delta_loading_id_seq
START = 1
INCREMENT = 1;
CREATE TABLE IF NOT EXISTS public.etl_jobs_delta_loading
(
id INTEGER DEFAULT public.etl_jobs_delta_loading_id_seq.NEXTVAL NOT NULL,
job_name VARCHAR(500) NOT NULL,
loaded_until TIMESTAMP,
etl_execution_time TIMESTAMP,
execution_status VARCHAR(30)
);
Related: Sequences as Expressions
Alternatively using IDENTITY/AUTOINCREMENT property:
CREATE OR REPLACE TABLE public.etl_jobs_delta_loading
(
id INTEGER NOT NULL IDENTITY(1,1),
job_name VARCHAR(500) NOT NULL,
loaded_until TIMESTAMP,
etl_execution_time TIMESTAMP,
execution_status VARCHAR(30)
);
Related
I have a table that is already created with data and I need only modify the schema to add some constraints .
my created tabled schema
CREATE TABLE public.note (
note_id bigint NOT NULL,
confidential boolean NOT NULL,
follow_up_date date,
notification_date date,
priority integer NOT NULL,
recurring_follow_up_interval interval,
status character varying(255) NOT NULL,
create_date timestamp with time zone,
deleted boolean NOT NULL,
last_modified_date timestamp with time zone,
time_spent integer,
title text,
version bigint NOT NULL,
assigned_to_id bigint,
note_category_id bigint NOT NULL,
created_by_id bigint,
last_modified_by_id bigint
);
what I need to match
create table note
(
note_id bigint default nextval('note_id_seq'::regclass) not null,
confidential boolean not null,
follow_up_date date,
notification_date date,
priority integer default 0 not null,
recurring_follow_up_interval interval,
status varchar(255) not null,
create_date timestamp with time zone,
deleted boolean default false not null,
last_modified_date timestamp with time zone,
time_spent integer,
title text,
version bigint default 0 not null,
assigned_to_id bigint,
note_category_id bigint not null,
created_by_id bigint,
last_modified_by_id bigint,
constraint note_pkey
primary key (note_id),
constraint fk_4nrhbn2j8j2vqqh78vleef9xr
foreign key (created_by_id) references admin_user,
constraint fk_eid7x7jfvjoe1h5tnyouhmqpa
foreign key (assigned_to_id) references admin_user,
constraint fk_oi5l4dg3sg5ep5neagmvp9r7o
foreign key (note_category_id) references note_category,
constraint fk_tk8ncyc0hmdi3gfh67b4jyu3l
foreign key (last_modified_by_id) references admin_user
);
as you can see it missing all defaults + all constraint to the other tables
any way to copy the intended schema to the created one without loosing the data
I am using Postgres 12
UPDATE
I know I could use alter to modify some columns but it will be a long process for me as there are many columns and I got more than 300 tables that have the same case
I manually alter one column to add sequence but I need easier way to do that for all columns
ALTER TABLE ONLY note ALTER COLUMN note_id SET DEFAULT nextval('note_id_seq'::regclass);
In postgresql I have a table which I need to add a new column. the original table ddl is belowing:
CREATE TABLE survey.survey_response (
id uuid NOT NULL DEFAULT uuid_generate_v4(),
survey_id uuid NOT NULL,
survey_question_id uuid NULL,
user_id varchar(256) NULL,
device_id varchar(256) NULL,
user_country varchar(100) NULL,
client_type varchar(100) NULL,
product_version varchar(100) NULL,
answer text NULL,
response_date timestamptz NOT NULL DEFAULT now(),
survey_category varchar(100) NULL,
tags varchar(250) NULL,
tracking_id uuid NULL,
CONSTRAINT survey_response_pkey PRIMARY KEY (id)
)
WITH (
OIDS=FALSE
) ;
Then I alter the table to add a new column:
alter table survey.survey_response add column system_tags varchar(30) ;
But after that I found my instert statement cannot make change to this new column, for all the original columns it works fine:
INSERT INTO survey.survey_response
(id, survey_id, user_id, tags, system_tags)
VALUES(uuid_generate_v4(), uuid_generate_v4(),'1123','dsfsd', 'dsfsd');
select * from survey.survey_response where user_id = '1123';
The "tags" columns contains inserted value, however, system_tags keeps null.
I tested the above scenario in my local postgreSQL 9.6, any ideas about this strange behavior? Thanks a lot
-----------------update----------
I found this survey.survey_response table has been partitioning based on month, So my inserted record will also be displayed in survey.survey_response_y2017m12. but the new system_tags column is also NULL
CREATE TABLE survey.survey_response_y2017m12 (
id uuid NOT NULL DEFAULT uuid_generate_v4(),
survey_id uuid NOT NULL,
survey_question_id uuid NULL,
user_id varchar(256) NULL,
device_id varchar(256) NULL,
user_country varchar(100) NULL,
client_type varchar(100) NULL,
product_version varchar(100) NULL,
answer text NULL,
response_date timestamptz NOT NULL DEFAULT now(),
survey_category varchar(100) NULL,
tags varchar(250) NULL,
tracking_id uuid NULL,
system_tags varchar(30) NULL,
CONSTRAINT survey_response_y2017m12_response_date_check CHECK (((response_date >= '2017-12-01'::date) AND (response_date < '2018-01-01'::date)))
)
INHERITS (survey.survey_response)
WITH (
OIDS=FALSE
) ;
If I run the same scenario in a non-partition table then the insert works fine.
So do I need any special settings for alter table for partition table?
Old thread but you need to drop and create again the RULE to fix the issue.
I have a database about weather that updates every second.
It contains temperature and wind speed.
This is my database:
CREATE TABLE `new_table`.`test` (
`id` INT(10) NOT NULL,
`date` DATETIME() NOT NULL,
`temperature` VARCHAR(25) NOT NULL,
`wind_speed` INT(10) NOT NULL,
`humidity` FLOAT NOT NULL,
PRIMARY KEY (`id`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8
COLLATE = utf8_bin;
I need to find the average temperature every hour.
This is my code:
Select SELECT AVG( temperature ), date
FROM new_table
GROUP BY HOUR ( date )
My coding is working but the problem is that I want to move the value and date of the average to another table.
This is the table:
CREATE TABLE `new_table.`table1` (
`idsea_state` INT(10) NOT NULL,
`dateavg` DATETIME() NOT NULL,
`avg_temperature` VARCHAR(25) NOT NULL,
PRIMARY KEY (`idsea_state`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8
COLLATE = utf8_bin;
Is it possible? Can you give me the coding?
In order to insert new rows into a database based on data you have obtained from another table, you can do this by setting up an INSERT query targeting the destination table, then run a sub-query which will pull the data from the source table and then the result set returned from the sub-query will be used to provide the VALUES used for the INSERT command
Here is the basic structure, note that the VALUES keyword is not used:
INSERT INTO `table1`
(`dateavg`, `avg_temperature`)
SELECT `date` , avg(`temperature`)
FROM `test`;
Its also important to note that the position of the columns returned by result set will be sequentially matched to its respective position in the INSERT fields of the outer query
e.g. if you had a query
INSERT INTO table1 (`foo`, `bar`, `baz`)
SELECT (`a`, `y`, `g`) FROM table2
a would be inserted into foo
y would go into bar
g would go into baz
due to their respective positions
I have made a working demo - http://www.sqlfiddle.com/#!9/ff740/4
I made the below changes to simplify the example and just demonstrate the concept involved.
Here is the DDL changes I made to your original code
CREATE TABLE `test` (
`id` INT(10) NOT NULL AUTO_INCREMENT,
`date` DATETIME NOT NULL,
`temperature` FLOAT NOT NULL,
`wind_speed` INT(10),
`humidity` FLOAT ,
PRIMARY KEY (`id`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8
COLLATE = utf8_bin;
CREATE TABLE `table1` (
`idsea_state` INT(10) NOT NULL AUTO_INCREMENT,
`dateavg` VARCHAR(55),
`avg_temperature` VARCHAR(25),
PRIMARY KEY (`idsea_state`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8
COLLATE = utf8_bin;
INSERT INTO `test`
(`date`, `temperature`) VALUES
('2013-05-03', 7.5),
('2013-06-12', 17.5),
('2013-10-12', 37.5);
INSERT INTO `table1`
(`dateavg`, `avg_temperature`)
SELECT `date` , avg(`temperature`)
FROM `test`;
Using Postgres, what I would like to achieve is to be able to have many different instrument types, with corresponding [TYPE].instrument tables, which all have a unique ID in the table, but also reference a unique ID in the instrument.master table. I have the following:
create schema instrument
CREATE TABLE instrument.type (
id smallserial NOT NULL,
name text not null,
code text not null,
CONSTRAINT pk_instrument_type PRIMARY KEY (id)
);
ALTER TABLE instrument.type ADD CONSTRAINT unq_instrument_type_code UNIQUE(code);
ALTER TABLE instrument.type ADD CONSTRAINT unq_instrument_type_name UNIQUE(name);
insert into instrument.type (name, code) values ('futures', 'f');
CREATE TABLE instrument.master (
id serial NOT NULL,
type smallint not null references instrument.type (id),
timestamp timestamp with time zone not null,
CONSTRAINT pk_instrument_master PRIMARY KEY (id)
);
CREATE TABLE futures.definition (
id smallserial NOT NULL,
code text not null,
CONSTRAINT pk_futures_definition PRIMARY KEY (id)
);
ALTER TABLE futures.definition ADD CONSTRAINT unq_futures_definition_code UNIQUE(code);
insert into futures.definition (code) values ('ED');
CREATE TABLE futures.instrument (
id smallserial NOT NULL,
master serial not null references instrument.master (id),
definition smallint not null references futures.definition (id),
month smallint not null,
year smallint not null,
CONSTRAINT pk_futures_instrument PRIMARY KEY (id),
check (month >= 1),
check (month <= 12),
check (year >= 1900)
);
ALTER TABLE futures.instrument ADD CONSTRAINT unq_futures_instrument UNIQUE(definition, month, year);
CREATE OR REPLACE FUNCTION trigger_master_futures()
RETURNS trigger AS
$BODY$
BEGIN
insert into instrument.master (type, timestamp)
select id, current_timestamp from instrument.type where code = 'f';
NEW.master := currval('instrument.master_id_seq');
RETURN NEW;
END;
$BODY$
LANGUAGE 'plpgsql' VOLATILE;
create trigger trg_futures_instrument before insert on futures.instrument
for each row
execute procedure trigger_master_futures();
I then test with:
insert into futures.instrument (definition, month, year)
select id, 3, 2015 from futures.definition where code = 'ED';
Everything works almost as I would like it to. The only issue is that somehow, instrument.master.id ends up being one more than futures.instrument.master. I am not sure what I need to do to achieve the behavior I want, which is that whenever an entry is inserted into futures.instrument, an entry should be inserted into instrument.master, and the id entry of the latter should be inserted into the master entry of the former. I actually think it should have failed since the foreign key relationship is violated somehow.
As it turns out, everything was correct. The issue was that in futures.instrument, the type of the master column is serial, and it should have been int.
I try to create tables from file on my schema, but i receive error message. Users table created on public schema was successfully, but other table i created on test1 schema was wrong. I don't know why, help me please.
Thank for advance.
CREATE SCHEMA test1
--Create sequence
CREATE SEQUENCE test1.table_id_seq START 1;
--Create function to auto generate ID
CREATE OR REPLACE FUNCTION test1.next_id(OUT result bigint) AS $$
DECLARE
our_epoch bigint := 1314220021721;
seq_id bigint;
now_millis bigint;
shard_id int := 1;
BEGIN
SELECT nextval('test1.table_id_seq') % 1024 INTO seq_id;
SELECT FLOOR(EXTRACT(EPOCH FROM clock_timestamp()) * 1000) INTO now_millis;
result := (now_millis - our_epoch) << 23;
result := result | (shard_id << 10);
result := result | (seq_id);
END;
$$ LANGUAGE PLPGSQL;
--Talbe ----users----
CREATE TABLE users
(
id bigserial NOT NULL PRIMARY KEY,
username varchar(30) NOT NULL UNIQUE,
password varchar NOT NULL,
first_name varchar(10),
last_name varchar(10),
profile_picture varchar,
create_time timestamp (0) without time zone
);
--Table ----photos----
CREATE TABLE test.photos
(
id bigint NOT NULL PRIMARY KEY DEFAULT test1.next_id(),
caption text,
low_resolution varchar,
hight_resolution varchar,
thumnail varchar,
user_id bigint NOT NULL REFERENCES users(id),
create_time timestamp (0) without time zone
--Table ----comments----
CREATE TABLE test1.comments
(
id bigint NOT NULL PRIMARY KEY DEFAULT test1.next_id(),
create_time timestamp (0) without time zone,
text text,
user_id bigint REFERENCES users(id),
photo_id bigint REFERENCES test1.photos(id)
);
--Table ----likes----
CREATE TABLE test1.likes
(
photo_id bigint REFERENCES test1.photos(id),
user_id bigint REFERENCES users(id),
);
--Table ----follows----
CREATE TABLE test1.follows
(
user_id bigint NOT NULL REFERENCES users(id),
target_id bigint NOT NULL REFERENCES users(id),
);
CREATE TABLE teset1.feeds
(
user_id bigint NOT NULL REFERENCES users(id),
photo_id bigint NOT NULL REFERENCES test1.photos(id),
create_time timestamp (0) without time zone,
);
Well, it would have been helpful for you to have shown us what errors you were getting and what you tried to do to fix them. But here are some obvious problems with the SQL you posted:
Missing semicolon after "CREATE SCHEMA test1 "
Missing closing paren and semicolon at the end of CREATE TABLE test.photos ...
Dump neglects to create the "test" schema where the "photos" table wants to go.
Several foreign key references of test1.photos.id, but "photos" was created in the "test" schema, not "test1".
Extraneous trailing commas after the last column definitions for several tables
Typo: "teset1.feeds", should be "test.feeds"
TL;DR Whoever created this dump file wasn't paying very close attention.
After fixing all the problems above, I managed to load your SQL.