I have a database.gdb running with Firebird 3.0.
This database has two tables:
Table1 and
Table2.
Every day I add records to these tables and when I have finished my work I need to export the two tables to another newer database.
I need a procedure which dumps the two tables into a script so to import data in the newer database using the script.
I am only able to create a script of tables which have always the same number of records (no records added every day).
This script should include:
CREATE TABLE
Export all records procedure of the two tables
I do not need code but just a hint. I will study how to write code by myself.
I have created a handmade script.
CREATE TABLE TABYEARS (
ID INTEGER NOT NULL,
YEARS INTEGER,
/* Keys */
PRIMARY KEY (ID)
);
CREATE TABLE TABCODE (
ID INTEGER NOT NULL,
NAME VARCHAR(50),
CODE VARCHAR(50),
/* Keys */
PRIMARY KEY (ID)
);
COMMIT;
INSERT INTO TABYEARS (ID, YEARS) VALUES (1, 2021);
INSERT INTO TABYEARS (ID, YEARS) VALUES (2, 2022);
INSERT INTO TABCODE (ID, NAME, CODE) VALUES (1, 'Robert', '10');
INSERT INTO TABCODE (ID, NAME, CODE) VALUES (2, 'Paul', '87');
COMMIT;
I do not add records very often to these tables. The first one has just one record every year.
How to create (not manually) a script like this but regarding
two tables in which every day I add 50 records?
I can use FlameRobin or IBExpert or similar.
Related
I have a time-series location data table containing the following columns (time, first_name, last_name, loc_lat, loc_long) with the first three columns as the primary key. The table has more than 1M rows.
I notice that first_name and last_name duplicate quite often. There are only 100 combinations in 1M rows. Therefore, to save disk space, I am thinking about creating a separate people table with columns (id, first_name, last_name) where (first_name, last_name) is a unique constraint, in order to simplify the time-series location table to be (time, person_id, loc_lat, loc_long) where person_id is a foreign key for the people table.
I want to first create a new table from my existing 1M row table to test if there is indeed meaningful disk space save with this change. I feel like this task is quite doable but cannot find a concrete way to do so yet. Any suggestions?
That's a basic step of database normalization.
If you can afford to do so, it will be faster to write a new table exchanging full names for IDs, than altering the schema of the existing table and update all rows. Basically:
BEGIN; -- wrap in single transaction (optional, but safer)
CREATE TABLE people (
people_id integer GENERATED ALWAYS AS IDENTITY PRIMARY KEY
, first_name text NOT NULL
, last_name text NOT NULL
, CONSTRAINT full_name_uni UNIQUE (first_name, last_name)
);
INSERT INTO people (first_name, last_name)
SELECT DISTINCT first_name, last_name
FROM tbl
ORDER BY 1, 2; -- optional
ALTER TABLE tbl RENAME TO tbl_old; -- free up org. table name
CREATE TABLE tbl AS
SELECT t.time, p.people_id, t.loc_lat, t.loc_long
FROM tbl_old t
JOIN people p USING (first_name, last_name);
-- ORDER BY ??
ALTER TABLE tbl ADD CONSTRAINT people_id_fk FOREIGN KEY (people_id) REFERENCES people(people_id);
-- make sure the new table is complete. indexes? constraints?
-- Finally:
DROP TABLE tbl_old;
COMMIT;
Related:
Best way to populate a new column in a large table?
Add new column without table lock?
Updating database rows without locking the table in PostgreSQL 9.2
DISTINCT is simple. But for only 100 distinct full names - and with the right index support! - there are more sophisticated, (much) faster ways. See:
Optimize GROUP BY query to retrieve latest row per user
I am using postgres 9.5. As part of application initialization I make some inserts in database at application startup with random ids. Something like insert into student values(1,'abc') , insert into student values(10,'xyz'). Then I have some rest APIs developed which insert new rows programatically. Is there any way we can tell postgres to skip already taken ids?
It tried to take up already used ids. I noticed it does not have the sequence updated accounting for the initial inserts
Here is how I create the table
CREATE TABLE student(
id SERIAL PRIMARY KEY,
name VARCHAR(64) NOT NULL UNIQUE
);
It sounds like you might be better served with UUIDs as your primary key values, if your data is distributed.
You can advance the sequence that is populating the id column to the highest value:
insert into student (id, name)
values
(1, 'abc'),
(2, 'xyz');
select setval(pg_get_serial_sequence('student', 'id'), (select max(id) from student));
I've the following table:
DROP SEQUENCE IF EXISTS CATEGORY_SEQ CASCADE;
CREATE SEQUENCE CATEGORY_SEQ START 1;
DROP TABLE IF EXISTS CATEGORY CASCADE;
CREATE TABLE CATEGORY (
ID BIGINT NOT NULL DEFAULT nextval('CATEGORY_SEQ'),
NAME CHARACTER VARYING(255) NOT NULL,
PARENT_ID BIGINT
);
ALTER TABLE CATEGORY
ADD CONSTRAINT CATEGORY_PK PRIMARY KEY (ID);
ALTER TABLE CATEGORY
ADD CONSTRAINT CATEGORY_SELF_FK FOREIGN KEY (PARENT_ID) REFERENCES CATEGORY (ID);
Now I need to insert the data. So I start with parent:
INSERT INTO CATEGORY (NAME) VALUES ('PARENT_1');
And now I need the ID of the just inserted parent to add children to it:
INSERT INTO CATEGORY (NAME, PARENT_ID) VALUES ('CHILDREN_1_1', <what_goes_here>);
INSERT INTO CATEGORY (NAME, PARENT_ID) VALUES ('CHILDREN_1_2', <what_goes_here>);
How can I get and store the ID of the parent to later use it in the subsequent inserts?
You can use a data modifying CTE with the returning clause:
with parent_cat (parent_id) as (
INSERT INTO CATEGORY (NAME) VALUES ('PARENT_1')
returning id
)
INSERT INTO CATEGORY (NAME, PARENT_ID)
VALUES
('CHILDREN_1_1', (select parent_id from parent_cat) ),
('CHILDREN_1_2', (select parent_id from parent_cat) );
The answer is to use RETURNING along with WITH
WITH inserted AS (
INSERT INTO CATEGORY (NAME) VALUES ('PARENT_1')
RETURNING id
) INSERT INTO CATEGORY (NAME, PARENT_ID) VALUES
('CHILD_1_1', (SELECT inserted.id FROM inserted)),
('CHILD_2_1', (SELECT inserted.id FROM inserted));
( tl;dr : goto option 3: INSERT with RETURNING )
Recall that in postgresql there is no "id" concept for tables, just sequences (which are typically but not necessarily used as default values for surrogate primary keys, with the SERIAL pseudo-type).
If you are interested in getting the id of a newly inserted row, there are several ways:
Option 1: CURRVAL(<sequence name>);.
For example:
INSERT INTO persons (lastname,firstname) VALUES ('Smith', 'John');
SELECT currval('persons_id_seq');
The name of the sequence must be known, it's really arbitrary; in this example we assume that the table persons has an id column created with the SERIAL pseudo-type. To avoid relying on this and to feel more clean, you can use instead pg_get_serial_sequence:
INSERT INTO persons (lastname,firstname) VALUES ('Smith', 'John');
SELECT currval(pg_get_serial_sequence('persons','id'));
Caveat: currval() only works after an INSERT (which has executed nextval() ), in the same session.
Option 2: LASTVAL();
This is similar to the previous, only that you don't need to specify the sequence number: it looks for the most recent modified sequence (always inside your session, same caveat as above).
Both CURRVAL and LASTVAL are totally concurrent safe. The behaviour of sequence in PG is designed so that different session will not interfere, so there is no risk of race conditions (if another session inserts another row between my INSERT and my SELECT, I still get my correct value).
However they do have a subtle potential problem. If the database has some TRIGGER (or RULE) that, on insertion into persons table, makes some extra insertions in other tables... then LASTVAL will probably give us the wrong value. The problem can even happen with CURRVAL, if the extra insertions are done intto the same persons table (this is much less usual, but the risk still exists).
Option 3: INSERT with RETURNING
INSERT INTO persons (lastname,firstname) VALUES ('Smith', 'John') RETURNING id;
This is the most clean, efficient and safe way to get the id. It doesn't have any of the risks of the previous.
Drawbacks? Almost none: you might need to modify the way you call your INSERT statement (in the worst case, perhaps your API or DB layer does not expect an INSERT to return a value); it's not standard SQL (who cares); it's available since Postgresql 8.2 (Dec 2006...)
Conclusion: If you can, go for option 3. Elsewhere, prefer 1.
Note: all these methods are useless if you intend to get the last globally inserted id (not necessarily in your session). For this, you must resort to select max(id) from table (of course, this will not read uncommitted inserts from other transactions).
Hello I want to delete all data in my postgresql tables, but not the table itself.
How could I do this?
Use the TRUNCATE TABLE command.
The content of the table/tables in PostgreSQL database can be deleted in several ways.
Deleting table content using sql:
Deleting content of one table:
TRUNCATE table_name;
DELETE FROM table_name;
Deleting content of all named tables:
TRUNCATE table_a, table_b, …, table_z;
Deleting content of named tables and tables that reference to them (I will explain it in more details later in this answer):
TRUNCATE table_a, table_b CASCADE;
Deleting table content using pgAdmin:
Deleting content of one table:
Right click on the table -> Truncate
Deleting content of table and tables that reference to it:
Right click on the table -> Truncate Cascaded
Difference between delete and truncate:
From the documentation:
DELETE deletes rows that satisfy the WHERE clause from the specified
table. If the WHERE clause is absent, the effect is to delete all rows
in the table.
http://www.postgresql.org/docs/9.3/static/sql-delete.html
TRUNCATE is a PostgreSQL extension that provides a faster mechanism to
remove all rows from a table. TRUNCATE quickly removes all rows from a
set of tables. It has the same effect as an unqualified DELETE on each
table, but since it does not actually scan the tables it is faster.
Furthermore, it reclaims disk space immediately, rather than requiring
a subsequent VACUUM operation. This is most useful on large tables.
http://www.postgresql.org/docs/9.1/static/sql-truncate.html
Working with table that is referenced from other table:
When you have database that has more than one table the tables have probably relationship.
As an example there are three tables:
create table customers (
customer_id int not null,
name varchar(20),
surname varchar(30),
constraint pk_customer primary key (customer_id)
);
create table orders (
order_id int not null,
number int not null,
customer_id int not null,
constraint pk_order primary key (order_id),
constraint fk_customer foreign key (customer_id) references customers(customer_id)
);
create table loyalty_cards (
card_id int not null,
card_number varchar(10) not null,
customer_id int not null,
constraint pk_card primary key (card_id),
constraint fk_customer foreign key (customer_id) references customers(customer_id)
);
And some prepared data for these tables:
insert into customers values (1, 'John', 'Smith');
insert into orders values
(10, 1000, 1),
(11, 1009, 1),
(12, 1010, 1);
insert into loyalty_cards values (100, 'A123456789', 1);
Table orders references table customers and table loyalty_cards references table customers. When you try to TRUNCATE / DELETE FROM the table that is referenced by other table/s (the other table/s has foreign key constraint to the named table) you get an error. To delete content from all three tables you have to name all these tables (the order is not important)
TRUNCATE customers, loyalty_cards, orders;
or just the table that is referenced with CASCADE key word (you can name more tables than just one)
TRUNCATE customers CASCADE;
The same applies for pgAdmin. Right click on customers table and choose Truncate Cascaded.
For small tables DELETE is often faster and needs less aggressive locking (important for concurrent load):
DELETE FROM tbl;
With no WHERE condition.
For medium or bigger tables, go with TRUNCATE, like #Greg posted:
TRUNCATE tbl;
Hard to pin down the line between "small" and "big", as that depends on many variables. You'll have to test in your installation.
I found a very easy and fast way for everyone who might use a tool like DBeaver:
You just need to select all the tables that you want to truncate (SHIFT + click or CTRL + click) then right click
And if you have foreign keys, select also CASCADE option on Settings panel. Start and that's all it takes!
I have some problems with SQL-code.
Sequences and tables creation, some data inserting in:
CREATE SEQUENCE tmp_id_places START 1;
CREATE SEQUENCE tmp_id_books START 1;
CREATE TABLE tmp_places (
id int PRIMARY KEY DEFAULT nextval('tmp_id_places'),
name text
);
CREATE TABLE tmp_cities (population int) INHERITS (tmp_places);
CREATE TABLE tmp_rivers (lenght int) INHERITS (tmp_places);
INSERT INTO tmp_cities (name, population) VALUES
('Moscow', 15),
('St. Petersburg', 9);
INSERT INTO tmp_rivers (name, lenght) VALUES
('Volga', 115),
('Angara', 319);
CREATE TABLE tmp_books (
id int PRIMARY KEY DEFAULT nextval('tmp_id_books'),
id_place int REFERENCES tmp_places(id),
title text
);
Вut this code make an error:
INSERT INTO tmp_books (title, id_place) VALUES
('Some book about Moscow', 1),
('Another book about Angara', 4);
Table tmp_books contain information about places. But I can't insert data in it, because there aren't any data in master-table tmp_places (all data in child-tables).
So can this been resolved anyway?
Take a closer look at this section in the PostrgeSQL docs. If you will insert data into the child table, then data will be found only in the child table. On the other hand, inserting into the master table makes new rows visible in all the child tables also. So you have to always work on the master tables at first hand.
I've been working with inheritance a while ago and also faced the same problem.
I ended up with the following:
INSERT a new entry into the tmp_places;
UPDATE extra fields, say, in tmp_cities with their respective values.
Back in 7.4 times I had to create a set of functions for such activities.
Now it is possible to use the RETURNING clause of INSERT statement and CTEs with UPDATE (also on SQL Fiddle):
WITH theid AS (
INSERT INTO tmp_places (name) VALUES ('Moscow') RETURNING id
)
UPDATE tmp_cities tc SET population = 15
FROM theid
WHERE tc.id = theid.id;
You should also be careful with constraints, as not all of them are inherited.
Denis,
Inheritance does not propagate in INSERT and COPY statements in Postgres.
In PostgreSQL you shouldn't create a foreign key to parent table because, as you just find out, this table acts almost as a view instead of a table (since actual data are in their respective children). For now only be solved by triggers.
You can see an example of this "type of triggers".