We had a table with a column with a Clob(length 1048576) that would store search text that helps with searching. When I transferred it from DB2 to Postgres in our migration, I found that it wasn't working as well. So I going to try text or varchar, but I was finding it would take much longer for the long text entries to be added to the table to the point my local wildfly window would timeout when trying to run.
What is the equilavelent of a datatpye that accepts text that I should be using in postgres to replace a Clob that was length 1048576 in DB2? It might be that I was using the right datatypes but didn't have the right corresponding size.
Use text. That is the only reasonable data type for long character strings.
Related
I am approaching the 10 GB limit that Express has on the primary database file.
The main problem appears to be some fixed length char(500) columns that are never near that length.
I have two tables with about 2 million rows between them. These two tables add up to about 8 GB of data with the remainder being spread over another 20 tables or so. These two tables each have 2 char(500) columns.
I am testing a way to convert these columns to varchar(500) and recover the trailing spaces.
I tried this:
Alter Table Test_MAILBACKUP_RECIPIENTS
Alter Column SMTP_address varchar(500)
GO
Alter Table Test_MAILBACKUP_RECIPIENTS
Alter Column EXDN_address varchar(500)
This quickly changed the column type but obviously didn’t recover the space.
The only way I can see to do this successfully is to:
Create a new table in tempdb with the varchar(500) columns,
Copy the information into the temp table trimming off the trailing spaces,
Drop the real table,
Recreate the real table with the new varchar(500) columns,
Copy the information back.
I’m open to other ideas here as I’ll have to take my application offline while this process completes?
Another thing I’m curious about is the primary key identity column.
This table has a Primary Key field set as an identity.
I know I have to use Set Identity_Insert on to allow the records to be inserted into the table and turn it off when I’m finished.
How will recreating a table affect new records being inserted into the table after I’m finished. Or is this just “Microsoft Magic” and I don’t need to worry about it?
The problem with you initial approach was that you converted the columns to varchar but didn't trim the existing whitespace (which is maintained after the conversion), after changing the data type of the columns to you should do:
update Test_MAILBACKUP_RECIPIENTS set
SMTP_address=rtrim(SMTP_address), EXDN_address=rtrim(EXDN_address)
This will eliminate all trailing spaces from you table, but note that the actual disk size will be the same, as SQL Server don't shrink automatically database files, it just mark that space as unused and available for other data.
You can use this script from another question to see the actual space used by data in the DB files:
Get size of all tables in database
Usually shrinking a database is not recommended but when there is a lot of difference between used space and disk size you can do it with dbcc shrinkdatabase:
dbcc shrinkdatabase (YourDatabase, 10) -- leaving 10% of free space for new data
OK I did a SQL backup, disabled the application and tried my script anyway.
I was shocked that it ran in under 2 minutes on my slow old server.
I re-enabled my application and it still works. (Yay)
Looking at the reported size of the table now it went from 1.4GB to 126Mb! So at least that has bought me some time.
(I have circled the Data size in KB)
Before
After
My next problem is the MailBackup table which also has two char(500) columns.
It is shown as 6.7GB.
I can't use the same approach as this table contains a FileStream column which has around 190gb of data and tempdb does not support FleStream as far as I know.
Looks like this might be worth a new question.
Does it possible using some db2 trick to alter a column from type char to type varchar trimming the white space on the right?
I know that is possible to alter the column type from char to varchar (to extends its size)
but db2 leaves the left white space on the right, so I need to issue an update statement after the alter table statement to trim the white space on the right.
But we have table also with 400 million of records and the update statement has an important cost in terms of time.
I ask this question also after I read the db2 documentation of alter table statement: seems that does exists nothing that allows me to change the type and trim right the values at the same time.
As far as I know it is not possible to do what you ask for. Have you considered export and load replace?
As you said, you can do the UPDATE after the ALTER. You have the spaces now. In a sense, you are simply making a small improvement to the data. It may take a while, but how much does it matter if that process chugs away for a while in the background?
Doing an export+load would have a significant downtime for the table.
If you use the alter to varchar and update approach you could minimize the log and lock amount by performing the update within a compound sql or stored procedure which commits after X rows updated.
This would still run quite long but locks won't be held that long.
I'm trying to define a bit field of 10 bytes. In SQL Server I'd use varbinary(10). I know that bytea replaces varbinary (MAX) for images, but didn't find any documentation on limiting the number of bits in it.
Is there a way to do that?
you want to look at bit(n), not bytea
http://www.postgresql.org/docs/current/static/datatype-bit.html
I am trying to change length of a column from character varying(40) to character varying(100).
Following the method described in this question Increasing the size of character varying type in postgres without data loss
ALTER TABLE info_table ALTER COLUMN docs TYPE character varying(100);
Tried with this command but returning syntax error
ERROR: syntax error at or near "TYPE" at character 52
Is there any change needed in this command? Using PostgreSQL version 7.4.30 (upgrade to 9.2 in process :) ).
I tried this same command in test db which is now upgraded with version 9.2. It is working fine there.
Changing the column type on the fly was not possible in the ancient version 7.4. Check the old manual. You had to add another column, update it with the (possibly transformed) values and then drop the old one, rename the new one - preferably in a single transaction. With side effects on views or other depending objects ...
To avoid this kind of problem altogether I suggest to use plain text or varchar (without length modifier) for character data. Details in this related question.
Remove the word TYPE, that syntax wasn't recognized 10 years ago, but you should be fine without it.
I have to transfer data from old database to new database where table name and column name is different.
Can it be done with DOS command or any other solution?
One is POSTGRESQL and old is MYSQL.
My concern is table name and column names are different, column number is same.
Thank you
I do not know the postgresql part but for sql server you can use sqlcmd.exe to export data as text format with or w/o column names
Please check
http://msdn.microsoft.com/en-us/library/ms162773.aspx