I'm new to table partition in Postgres and I was trying something which didn't seem to work. Here's a simplified version of the problem:
cREATE table if not exists Location1
PARTITION OF "Location"
FOR VALUES FROM
(extract(epoch from now()::timestamp))
TO
(extract(epoch from now()::timestamp)) ;
insert into "Location" ("BuyerId", "Location")
values (416177285, point(-100, 100));
It fails with:
ERROR: syntax error at or near "extract"
LINE 2: FOR VALUES FROM (extract(epoch from now()::timestamp))
If I try to create the partition using literal values, it works fine.
I'm using Postgres 10 on linux.
After a lot of trial and error, I realized that Postgres 10 doesn't really have a full partitioning system out of the box. For a usable partitioned database, the following postgres modules are necessary to be installed:
pg_partman https://github.com/pgpartman/pg_partman (literally a gift from God)
pg_jobmon https://github.com/omniti-labs/pg_jobmon (to monitor what pg_partman is doing)
The above advice is for those new to the scene, to help you avoid a lot of headaches.
I assumed that automatic creation of partitions done by postgres-core was a no-brainer, but evidently the postgres devs know something I don't?
If any relational database from the 80's is asked to insert a row, it succeeds after passing the constraint checks.
The latest version of Postgres 2018 is asked to insert a row: "Sorry, I don't know where to put it."
It's a good first effort, but hopefully Postgres 11 will have a proper partitioning system OOTB...
Related
Provider is on OEL 7 postgres 12.4 and Subscriber in on RDS 13.2
pglogical.replicate_ddl_command works fine as long as there are no quotes between start and end of the command.
for example, below works fine
select pglogical.replicate_ddl_command('create table public.foo ( like public.orders including all)','{default}'::text[]);
Event triggers are setup to add this newly created table to default replication_set.
Next, we need to attach table foo as a partition to table orders, and that's where quotes in FOR VALUES become a problem.
select pglogical.replicate_ddl_command('Alter table public.orders attach partition public.foo for values from ('2021-05-01') TO ('2021-06-01')','{default}'::text[]);
ERROR: syntax error at or near "2021"
LINE 1: ...ers attach partition public.foo for values from ('2021-05-01...
^
Could not find anything in docs related to this.
please help.
while I was able to get away with the error by using double quotes '', as the command argument on the function is text, the ATTACH PARTITION did not replicate. Although that is a separate issue than what I asked in this question, appreciate any feedback and ideas.
Leaving this note for anyone else who runs into my situation. The requirement is you need to add both parent and all its partitions to the same replication set. In my case I had all the partitions added but not the parent.
When are DB2 declared global temporary tables 'cleaned up' and automatically deleted by the system...? This is for DB2 on AS400 v7r3m0, with DBeaver 5.2.5 as the dev client, and MS-Access 2007 for packaged apps for the end-users.
Today I started experimenting with a DGTT, thanks to this answer. So far I'm pleased with the functionality, although I did find our more recent system version has the WITH DATA option, which is an obvious advantage.
Everything is working, but at times I receive this error:
SQL Error [42710]: [SQL0601] NEW_PKG_SHEETS_DATA in QTEMP type *FILE already exists.
The meaning of the error is obvious, but the timing is not. When I started today, I could run the query multiple times, and the error didn't occur. It seemed as if the system was cleaning up and deleting it, which is just what I was looking for. But then the error started and now it's happening with more frequency.
If I make strategic use of DROP TABLE, this resolves the error, unless the table doesn't exist, in which case I get another error. I can also disconnect/reconnect to the server from my SQL dev client, as I would expect, since that would definitely drop the session.
This IBM article about DGTTs speaks much of sessions, but not many specifics. And this article is possibly the longest command syntax I've yet encountered in the IBM documentation. I got through it, but it didn't answer the question of what decided when a DGTT is deleted.
So I would like to ask:
What are the boundaries of a session..?
I'm thinking this is probably defined by the environment in my SQL client..?
I guess the best/safest thing to do is use DROP TABLE as needed..?
Does any one have any tips, tricks, or pointers they could share..?
Below is the SQL that I'm developing. For brevity, I've excluded chunks of the WITH-AS and SELECT statements:
DROP TABLE SESSION.NEW_PKG_SHEETS ;
DECLARE GLOBAL TEMPORARY TABLE SESSION.NEW_PKG_SHEETS_DATA
AS ( WITH FIRSTDAY AS (SELECT (YEAR(CURDATE() - 4 MONTHS) * 10000) +
(MONTH(CURDATE() - 4 MONTHS) * 100) AS DATEISO
FROM SYSIBM.SYSDUMMY1
-- <VARIETY OF ADDITIONAL CTE CLAUSES>
-- <SELECT STATEMENT BELOW IS A BIT LONGER>
SELECT DAACCT AS DAACCT,
DAIDAT AS DAIDAT,
DAINV# AS DAINV,
CAST(DAITEM AS NUMERIC(6)) AS DAPACK,
CAST(0 AS NUMERIC(14)) AS UPCNUM,
DAQTY AS DAQTY
FROM DAILYTRANS
AND DAIDAT >= (SELECT DATEISO+000 FROM FIRSTDAY) -- 1ST DAY FOUR MONTHS AGO
AND DAIDAT <= (SELECT DATEISO+399 FROM FIRSTDAY) -- LAST DAY OF LAST MONTH
) WITH DATA ;
DROP TABLE SESSION.NEW_PKG_SHEETS ;
The DGTT will only get cleaned automatically up by Db2 when the connection ends successfully (connect reset or equivalent according to whatever interface to Db2 is being used ).
For both Db2 for i and Db2-LUW, consider using the WITH REPLACE clause for the DECLARE GLOBAL TEMPORARY TABLE statement. That will ensure you don't need to explicitly drop the DGTT if the session remains open but the code needs the table to be replaced at next execution whether or not the DGTT already exists.
Using that WITH REPLACE clause means you do not need to worry about issuing a DROP statement for the DGTT, unless you really want to issue a drop.
Sometimes sessions may get re-used, or a close/disconnect might not happen or might not complete, or more likely a workstation performs a retry, and in those cases the WITH REPLACE can be essential for easily avoiding runtime errors.
Note that Db2 for Z/OS (at v12) does not offer the WITH REPLACE clause for DGTT, but has instead an optional syntax on commit drop table (but this is not documented for Db2-for-i and Db2-LUW).
I am trying to do what should be a pretty straightforward insert statement in a postgres database. It is not working, but it's also not erroring out, so I don't know how to troubleshoot.
This is the statement:
INSERT INTO my_table (col1, col2) select col1,col2 FROM my_table_temp;
There are around 200m entries in the temp table, and 50m entries in my_table. The temp table has no index or constraints, but both columns in my_table have btree indexes, and col1 has a foreign key constraint.
I ran the first query for about 20 days. Last time I tried a similar insert of around 50m, it took 3 days, so I expected it to take a while, but not a month. Moreover, my_table isn't getting longer. Queried 1 day apart, the following produces the same exact number.
select count(*) from my_table;
So it isn't inserting at all. But it also didn't error out. And looking at system resource usage, it doesn't seem to be doing much of anything at all, the process isn't drawing resources.
Looking at other running queries, nothing else that I have permissions to view is touching either table, and I'm the only one who uses them.
I'm not sure how to troubleshoot since there's no error. It's just not doing anything. Any thoughts about things that might be going wrong, or things to check, would be very helpful.
For the sake of anyone stumbling onto this question in the future:
After a lengthy discussion (see linked discussion from the comments above), the issue turned out to be related to psycopg2 buffering the query in memory.
Another useful note: inserting into a table with indices is slow, so it can help to remove them before bulk loads, and then add them again after.
in my case it was date format issue. i commented date attribute before interting to DB and it worked.
In my case it was a TRIGGER on the same table I was updating and it failed without errors.
Deactivated the trigger and the update worked flawlessly.
I'm using Enterprise Postgres 9.5 with Oracle Compatibility. I have a problem with the EXECUTE IMMEDIATE command.
Say I have a table with few columns and one of them can accept NULLs. If I do
EXECUTE IMMEDIATE 'select null_col from '||table_name||' where col1=10' into x;
It sends the value to x, if null_col returns any.
When I given the condition col1=19, where 19 is not present in the table, then I get the error like this.
query returned no rows
and my execution stops. So how can I handle that. Oracle doesn't given any error for such statements, whereas EDB does. Please help.
I didn't find any EDB tags, so please tag if you think this is inappropriate question here. Thanks for understanding.
Recently I got could not read block error by displaying the following message:
pg_tblspc/16010/PG_9.3_201306121/16301/689225.365
After this error, I am trying the below query by assuming few of the numbers as oid, but my query result is empty rows.
select oid,relname from pg_class where oid=16010 or oid=16301;
Now my question is, what are the numbers on that pg_tablspc? I have gone through the link and I believe I might have missed the main point from there too!
Update: much more detailed write-up at http://blog.2ndquadrant.com/postgresql-filename-to-table/
The following info doesn't consider relfilenode changes due to vacuum full etc.
In:
pg_tblspc/16010/PG_9.3_201306121/16301/689225.365
we have:
pg_tblspc: Indicates that it's a relation in a tablespace other than the default or global tablespaces
16010: the tablespace oid from pg_tablespace.oid,
PG_9.3_201306121: A version-specific, catversion-specific string to allow different Pg versions to co-exist in a tablespace,
16301: the database oid from pg_database.oid
689225: the relation oid from pg_class.oid
365: The segment number. PostgreSQL splits big tables up into extents (segments) of 1GB each.
There may also be a fork number, but there isn't one in this path.
It took a fair bit of source code digging for me to be sure about this. The macro you want is relpathbackend in src/include/common/relpath.h, for anyone else looking, and it calls GetRelationPath in src/common/relpath.c.