I have a postgres table set up with a list partition defined on a column like so:
create table table1 (
val varchar(10),
t_type varchar(10)
) partition by list(t_type);
create table table1_xvals partition of table1 for values in ('xval1','xval2');
create table table1_yvals partition of table1 for values in ('yval1','yval2');
As I insert data into table1, and I can see the size of the table and individual partitions increasing, however when I try to select data from any of those tables, nothing shows up (select * from ). Is there anything wrong with how I'm creating the tables or selecting data?
That is normal. You are loading the data in a single database transaction, and the effects of a database transaction only become visible when the transaction is completed (committed). If you are loading the data with a single INSERT or COPY statement, the transaction will end as soon as the statement is done.
Related
I have two KSQL Tables each having the same key. I am running the following query on them
CREATE TABLE TEMP1 AS SELECT
b.MFG_DATE,
b.rowtime as bd_rowtime,
s.rowtime as sd_rowtime,
b.EXPIRY_DATE as EXP_DATE,
b.BATCH_NO as BATCH_NO,
s.rowkey as SD_ID
FROM GR_SD4 s
INNER JOIN GR_BD4 b ON b.rowkey = s.rowkey;
PARTITION BY s.rowkey;
The resulting table does not get populated with data but when I run the select query separately it populates the data. I am confused on what could be the reason for the table not being populated with data.
The issue may be related to the PARTITION BY clause in your query. Since you are joining two tables, the resulting table will have a composite primary key (rowkey, s.rowkey). The PARTITION BY clause should be updated to reflect this, i.e. PARTITION BY rowkey, s.rowkey. This should ensure that the data is correctly partitioned and can be inserted into the table.
My stored procedure includes the following code:
CREATE TEMPORARY TABLE #lala
(
idx int IDENTITY(1,1),
tablename nvarchar(128)
);
INSERT INTO #lala(tablename)
SELECT LEFT(tablename, LEN(tablename) - 3)
FROM SVV_EXTERNAL_TABLES
WHERE schemaname = 'spectrum'
AND tablename LIKE '%_v2';
I'm then calling it like this:
BEGIN;
CALL myschema.make_union_views('spectrum_views','spectrum','mycursor');
FETCH ALL FROM mycursor;
COMMIT;
At first it was running succesfully.
Then it began falling over, and debugging, I listed the contents of '#lala
I am confused as to how this has come about - that the [idx] column is not sequential ?
Hope that someone can shed some light on what might be happening?
This is by design. Redshift is a cluster and as such communications between parts of the cluster are expensive. Redshift ensures the uniqueness of identity columns but NOT sequentiality. Per the CREATE TABLE documentation (https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_TABLE_NEW.html):
When you load the table using an INSERT INTO [tablename] SELECT * FROM
or COPY statement, the data is loaded in parallel and distributed to
the node slices. To be sure that the identity values are unique,
Amazon Redshift skips a number of values when creating the identity
values.
is it ok to rename a parent table from which partitioned child tables are based ?
For example if I have
CREATE TABLE demos_qa (
demo_id int,
demo_date VARCHAR,
demo_text TEXT)
PARTITION BY RANGE (trxn_post_dt);
CREATE TABLE demos_2022 PARTITION OF demos_qa
FOR VALUES FROM ('2022-01-01') TO ('2023-01-01');
CREATE TABLE demos_2021 PARTITION OF demos_qa
FOR VALUES FROM ('2021-01-01') TO ('2022-01-01');
CREATE TABLE demos_2020 PARTITION OF demos_qa
FOR VALUES FROM ('2020-01-01') TO ('2021-01-01');
and I bulk load the partition tables.
I now want to rename the parent table from demo_qa to demo. Can I do that and will things work as normal without having to change the definitions of the partition tables ?
Yes, this will work.
The partitions won't be renamed though, in case you expected that.
I have a table that is partitioned by hash on a column.
CREATE TABLE TEST(
ACCOUNT_NUMBER VARCHAR(20)
)
PARTITION BY HASH (ACCOUNT_NUMBER)
PARTITIONS 16
Now, I want to update the account_number column itself in the table because of certain requirements.
As this column is partitioned, I'm not able to issue an update command on the table like
Update test set account_number = new_value
as it results to the below error:
Error is: ORA-14402: UPDATING PARTITION WOULD CAUSE PARTITION CHANGE.
Row movement is set to disable for the table.
The one way I know is to enable row movement but I also want to explore other options.
Could you please advise me on how to solve this?
I am trying to ALTER a table to use partitions LIST in postgres 11. I have been trying for hours but i keep getting errors.
I have a huge table, clients, with ( client_id, customer_id, value).
I have already created a new empty table, clients, by renaming the old table to clients_old and then created the new table with: CREATE TABLE clients( like clients_old including all).
And from here I am stuck when trying to add the LIST partition.
I have tried to:
ALTER TABLE Clients attach PARTITION BY LIST (client_id) --> fail;
ALTER TABLE Clients attach PARTITION LIST (client_id) --> fail;
ALTER TABLE Clients ADD PARTITION LIST (client_id) --> fail;
What syntax should I use to alter the table to use partitions?
Quote from the manual
It is not possible to turn a regular table into a partitioned table or vice versa
So, you can not change an existing non-partitioned table to a partitioned table.
You need to create a new table (with a different name) that is partitioned, create all necessary partitions and then copy the data from the old table to the new, partitioned table.
Something like:
create table clients_partitioned
(
.... all columns ...
)
PARTITION BY LIST (client_id);
Then create the partitions:
create table clients_1
partition of clients_partioned
for values in (1,2,3);
create table clients_1
partition of clients_partioned
for values in (4,5,6);
Then copy the data:
insert into clients_partitioned
select *
from clients;
Once that is done, you can drop the old table and rename the new table:
drop table clients;
alter table clients_partitioned rename to clients;
Don't forget to re-create your foreign keys and indexes.
I had to add for tag in order to add the partition:
create table clients_1
partition of clients_partioned
for values in (4,5,6);
because without for was a syntax error.