Altering a Postgres integer column to type boolean - postgresql

I've recently been optimizing some of our Postgres tables by converting more complex data types to simpler ones where possible. In every case except one so far, this has been fairly straightforward, for instance:
ALTER TABLE products ALTER COLUMN price TYPE integer USING price::integer;
For converting text into custom enumerated data types, this has also been simple enough. I just wrote a PLPGSQL function that would convert text to the enum, then converted the column like so:
ALTER TABLE products ALTER COLUMN color TYPE color_enum USING text_to_color_enum(color);
This syntax fails though, in cases where I have to convert an integer to a boolean. These all fail:
ALTER TABLE products ALTER return_policy TYPE boolean USING return_policy > 0;
ALTER TABLE products ALTER return_policy TYPE boolean USING bool(return_policy);
ALTER TABLE products ALTER COLUMN return_policy TYPE boolean USING bool(return_policy);
ALTER TABLE products ALTER COLUMN return_policy TYPE boolean USING CASE WHEN return_policy <> 0 THEN TRUE ELSE FALSE END;
The error message is always the same:
ERROR: operator does not exist: boolean = integer
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
There are no null values in that column. All values are either zero or positive. SELECT pg_typeof(return_policy) FROM products LIMIT 1; returns integer. Creating a custom cast from integer to boolean fails, because apparently one already exists. The same thing happen in Postgres 9.4 and 9.5. What am I doing wrong here?

Verify if the column is a constraint, if yes you need to remove the constraint before change the type.
ALTER TABLE products ALTER COLUMN price DROP DEFAULT;
ALTER TABLE products ALTER price TYPE bool USING CASE WHEN price = 0 THEN FALSE ELSE TRUE END;
ALTER TABLE products ALTER COLUMN price SET DEFAULT FALSE;

One of my partial indexes had a condition WHERE return_policy = 30. (This number is meant to be the number of days the return policy is, but since we're giving everything either no return policy or a 30-day return policy, it doesn't make sense for it to be an int anymore.) Dropping the index allowed my original SQL code to run correctly.

Related

Convert column domain as smallint to boolean in firebird

I'm using Firebird 4.0 and I would convert a column from smallint 0|1 to boolean.
So I have this kind of domain:
CREATE DOMAIN D_BOOL
AS SMALLINT
DEFAULT 0
NOT NULL
CHECK (VALUE IN (0,1))
;
This domain is used in my test table:
CREATE TABLE TBOOL
(
ID INTEGER,
INTVAL D_BOOL
);
How can I convert the column INTVAL to BOOLEAN?
I tried this query but I got an error:
alter table tbool
alter column INTVAL TYPE BOOLEAN,
alter column INTVAL SET DEFAULT FALSE
Error:
Error: *** IBPP::SQLException ***
Context: Statement::Execute( alter table tbool
alter column INTVAL TYPE BOOLEAN,
alter column INTVAL SET DEFAULT FALSE )
Message: isc_dsql_execute2 failed
SQL Message : -607
This operation is not defined for system tables.
Engine Code : 335544351
Engine Message :
unsuccessful metadata update
ALTER TABLE TBOOL failed
MODIFY RDB$RELATION_FIELDS failed
Unfortunately, this is an incompatible column change, because there is no conversion defined from SMALLINT to BOOLEAN. Altering the type of a column only works for a limited combination of types (and there is no combination that allows modification to or from BOOLEAN).
The only real option is to add a new column, populate it based on the value of the old column, drop the old column and rename the new column. This can have a huge impact if this column is used in triggers, procedures and/or views.
Your options are basically:
Keep existing columns as-is, and only use BOOLEAN moving forward for new columns
Do a very invasive change to change all your columns.
If you have a lot of columns that need to change, this is likely easier to do by creating a new database from scratch and pumping the data over, than by changing the database in-place.
The background of this limitation is that Firebird doesn't actually modify existing values when changing the type of a column. Instead, it will convert values on the fly when reading rows created with an older "format version" (inserts and updates will write the new "format version").
This makes for fast DDL, but all conversions must be known to succeed. This basically means only "widening" conversions between similar types are allowed (e.g. longer (VAR)CHAR, longer integer types, etc).

Postgresql 9.4 Alter Column Text to Boolean with Values

I have a table that looks like this:
CREATE TABLE "TestResults" (
"Id" text PRIMARY KEY,
"Name" text NOT NULL UNIQUE,
"Result" text CHECK ("Comment" IN ('Pass', 'Fail')),
"CreatedBy" text NOT NULL,
"CreatedOn" timestamp with time zone NOT NULL,
);
We are using PostgreSQL 9.4
The user is currently able to select either Pass or Fail from a drop down menu, and we have been storing those strings in the database in the Result column. We would like to change that to a boolean value.
How can I change the Result column from text to boolean while keeping the values the users have already entered?
Thank you.
Any time you want to change a column's type but there is no default conversion from the old values to the new ones, you want to use a USING clause to specify how to convert the old values to the new ones:
The optional USING clause specifies how to compute the new column value from the old; if omitted, the default conversion is the same as an assignment cast from old data type to new. A USING clause must be provided if there is no implicit or assignment cast from old to new type.
So first get rid of the CHECK constraint, you can find the name by doing a \d "TestResults" from psql but it is probably "TestResults_Result_check":
alter table "TestResults" drop constraint "TestResults_Result_check";
Then an ALTER COLUMN (with a USING clause) to change the type:
alter table "TestResults"
alter column "Result"
set data type boolean
using case
when "Result" = 'Pass' then true
when "Result" = 'Fail' then false
else null
end;

PostgreSQL ADD COLUMN DEFAULT NULL locks and performance

I have a table in my PostgreSQL 9.6 database with 3 million rows. This table already has a null bitmap (it has 2 other DEFAULT NULL fields). I want to add a new boolean nullable column to this table. I stuck with the difference between these two statements:
ALTER TABLE my_table ADD COLUMN my_column BOOLEAN;
ALTER TABLE my_table ADD COLUMN my_column BOOLEAN DEFAULT NULL;
I think that these statements have no difference, but:
I can't find any proof of it in documentation. Documentation tells that providing DEFAULT value for the new column makes PostgreSQL to rewrite all the tuples, but I don't think that it's true for this case, cause default value is NULL.
I ran some tests on copy of this table, and the first statement (without DEFAULT NULL) took a little bit more time than the second. I can't understand why.
My questions are:
Will PostgreSQL use the same lock type (ACCESS EXCLUSIVE) for those two statements?
Will PostgreSQL rewrite all tuples to add NULL value to every of them in case that I use DEFAULT NULL?
Are there any difference between those two statements?
There's a issue in the response of Vao Tsun in point 2.
If you use ALTER TABLE my_table ADD COLUMN my_column BOOLEAN; it won't rewrite all the tuples, it will be just a change in the metadata.
But if you use ALTER TABLE my_table ADD COLUMN my_column BOOLEAN DEFAULT NULL, it will rewrite all the tuples, and it will last for ever on long tables.
The documentation itself tells this.
When a column is added with ADD COLUMN, all existing rows in the table are initialized with the column's default value (NULL if no DEFAULT clause is specified). If there is no DEFAULT clause, this is merely a metadata change and does not require any immediate update of the table's data; the added NULL values are supplied on readout, instead.
This tell us that if there is a DEFAULT clause, even if it is NULL, it will rewrite all the tuples.
This is due to a performance issue on the updates clause. If you need to make an update over a no rewrited tuple, it will need to move the tuple to another disk space, consuming more time.
I tested this by my own on Postgresql 9.6, when i had to add a column, on a table that had 300+ million tuples. Without the DEFAULT NULL it lasted 11 ms, and with the DEFAULT NULL it lasted more than 30 minutes.
https://www.postgresql.org/docs/current/static/sql-altertable.html
Yes - same ACCESS EXCLUSIVE, no exceptions for DEFAULT NULL or no DEFAULT mentionned (statistics, "options", constraints, cluster would require less strict I think, but not add column)
Note that the lock level required may differ for each subform. An
ACCESS EXCLUSIVE lock is held unless explicitly noted. When multiple
subcommands are listed, the lock held will be the strictest one
required from any subcommand.
No - it will rather append NULL to result on select
When a column is added with ADD COLUMN, all existing rows in the table
are initialized with the column's default value (NULL if no DEFAULT
clause is specified). If there is no DEFAULT clause, this is merely a
metadata change and does not require any immediate update of the
table's data; the added NULL values are supplied on readout, instead.
No - no difference AFAIK. Just metadata change in both cases (as I believe it is one case expressed with different semantics)
Edit - Demo:
db=# create table so(i int);
CREATE TABLE
Time: 9.498 ms
db=# insert into so select generate_series(1,10*1000*1000);
INSERT 0 10000000
Time: 13899.190 ms
db=# alter table so add column nd BOOLEAN;
ALTER TABLE
Time: 1025.178 ms
db=# alter table so add column dn BOOLEAN default null;
ALTER TABLE
Time: 13.849 ms
db=# alter table so add column dnn BOOLEAN default true;
ALTER TABLE
Time: 14988.450 ms
db=# select version();
version
----------------------------------------------------------------------------------------------------------------
PostgreSQL 9.6.1 on x86_64-apple-darwin15.6.0, compiled by Apple LLVM version 8.0.0 (clang-800.0.42.1), 64-bit
(1 row)
lastly to avoid speculations it is data type specific:
db=# alter table so add column t text;
ALTER TABLE
Time: 25.831 ms
db=# alter table so add column tn text default null;
ALTER TABLE
Time: 13.798 ms
db=# alter table so add column tnn text default 'null';
ALTER TABLE
Time: 15440.318 ms

How to convert a currency column of a table into a numeric column in postgresql

We need to make our schema support multiple currencies. So, using currency field is not a option. So, I am trying to convert currency column into numeric(12,2). I tried the following approaches:
ALTER TABLE lead ALTER COLUMN deal_size TYPE NUMERIC(12, 2);
ALTER TABLE lead ALTER COLUMN deal_size TYPE NUMERIC(12, 2) using deal_size::money::numeric(12,2);
each time I get the following error:
ERROR: numeric field overflow
DETAIL: A field with precision 12, scale 2 must round to an absolute value less than 10^10.
I verified that none of the values for this column in the table is more than $1,000,000
I test the following in my PostgreSQL and works perfectly well. What version of PostgreSQL are you running?
create temp table lead (id serial not null primary key, deal_size money);
insert into lead (deal_size) select (random()*100000000)::numeric(14,4) from generate_series(1,10000) a;
ALTER TABLE lead ALTER COLUMN deal_size TYPE NUMERIC(12, 2);
You don't have values greater than a million. Have you tested for large negative values?

How to change column datatype from character to numeric in PostgreSQL 8.4

I am using following query:
ALTER TABLE presales ALTER COLUMN code TYPE numeric(10,0);
to change the datatype of a column from character(20) to numeric(10,0) but I am getting the error:
column "code" cannot be cast to type numeric
You can try using USING:
The optional USING clause specifies how to compute the new column value from the old; if omitted, the default conversion is the same as an assignment cast from old data type to new. A USING clause must be provided if there is no implicit or assignment cast from old to new type.
So this might work (depending on your data):
alter table presales alter column code type numeric(10,0) using code::numeric;
-- Or if you prefer standard casting...
alter table presales alter column code type numeric(10,0) using cast(code as numeric);
This will fail if you have anything in code that cannot be cast to numeric; if the USING fails, you'll have to clean up the non-numeric data by hand before changing the column type.
If your VARCHAR column contains empty strings (which are not the same as NULL for PostgreSQL as you might recall) you will have to use something in the line of the following to set a default:
ALTER TABLE presales ALTER COLUMN code TYPE NUMERIC(10,0)
USING COALESCE(NULLIF(code, '')::NUMERIC, 0);
(found with the help of this answer)
Step 1: Add new column with integer or numeric as per your requirement
Step 2: Populate data from varchar column to numeric column
Step 3: drop varchar column
Step 4: change new numeric column name as per old varchar column