When a DB2 database is restored and recovered from backups
is the restored database a physical copy - ie: identical block for block with the source database (as it was at time of backup) - and of identical size of source database?
or
is the restored database a logical copy - where datafile blocks are reorganized and coalesced (so the most of unused, fragmented free space in datafiles has been removed - causing restored database to often be smaller in storage footprint?
It is a page-for-page physical copy, but only of the used extents of pages in each table space. You can not change the logical contents of the data during a restore but you could alter the layout of the physical persistent storage.
There are also some changes you can cause during the restore process which can affect the persistently stored state of the system, such as a redirected restore altering the table space definitions or storage groups, replacing the DB history file, changing the encryption method in use, or upgrading the DB to a new release level.
It's a page-for-page physical copy. (This is why you can't, for example, backup on a little-endian system and restore onto a big-endian system.)
Related
Is it possible to dis-connect and re-connect a POSTGRES tablespace and all the associated objects within that tablespace?
I have a Postgres database with two tablespaces, one on a high-speed SSD drive (I've named this FASTSPACE) , and the other on a slower, traditional magnetic HDD (named SLOWSPACE). The slower tablespace is reserved for large volumes of historic data which is rarely accessed.
Is it possible to temporarily disconnect SLOWSPACE, with the intention of reconnecting it at a later date? the DROP TABLESPACE documentation can only be used once all database objects within it have been dropped.
I'm aware that I can backup all the tables in SLOWSPACE, then delete them, and then DROP the tablespace, however this will take time (there are several Terabytes of data). If I then need the archived data again I'll have create a new version of the SLOWSPACE tablespace from blank, then re-create all the objects from the backups. Again, this will take time.
Is there any way of temporarily disconnecting SLOWSPACE from the database - whilst still leaving the rest of the database up and running?
Update - happy to accept Franks Heikens two letter answer - 'no'
We are using pgbackrest to backup our database to Amazon S3. We do full backups once a week and an incremental backup every other day.
Size of our database is around 1TB, a full backup is around 600GB and an incremental backup is also around 400GB!
We found out that even read access (pure select statements) on the database has the effect that the underlying data files (in /usr/local/pgsql/data/base/xxxxxx) change. This results in large incremental backups and also in very large storage (costs) on Amazon S3.
Usually the files with low index names (e.g. 391089.1) change on read access.
On an update, we see changes in one or more files - the index could correlate to the age of the row in the table.
Some more facts:
Postgres version 13.1
Database is running in docker container (docker version 20.10.0)
OS is CentOS 7
We see the phenomenon on multiple servers.
Can someone explain, why postgresql changes data files on pure read access?
We tested on a pure database without any other resources accessing the database.
This is normal. Some cases I can think of right away are:
a SELECT or other SQL statement setting a hint bit
This is a shortcut for subsequent statements that access the data, so they don't have t consult the commit log any more.
a SELECT ... FOR UPDATE writing a row lock
autovacuum removing dead row versions
These are leftovers from DELETE or UPDATE.
autovacuum freezing old visible row versions
This is necessary to prevent data corruption if the transaction ID counter wraps around.
The only way to fairly reliably prevent PostgreSQL from modifying a table in the future is:
never perform an INSERT, UPDATE or DELETE on it
run VACUUM (FREEZE) on the table and make sure that there are no concurrent transactions
I have a mobile/web project, using pg9.3 as database, and linux as server.
The data won't be huge, but as time goes on, the data increase.
For long term considering, I want to know about:
Questions:
1. Is it necessary for me to create tablespace for my database, or just use the default one?
2. If I create new tablespace, what is the proper location on linux to create the folder, and why?
3. If I don't create it now, and wait until I have to, till then, will it be easy for me to migrate db with data to new tablespace?
Just use the default tablespace, do not create new tablespaces. Tablespaces are only useful if you have multiple physical disks, so you can define which data is stored on which physical disk. The directory where your data is located is not that important for the workings of postgres, so if you only have one disk it is useless to use tablespaces
Should your data grow beyond the capacity of 1 disk, you will have to perform a full data migration anyway to move it to another physical disk, so you can configure tablespaces at that time
The idea behind defining which data is located on which disk (with tablespaces) is that you can do things like putting a big table which is hardly used on a slow disk, and putting this very intensively used table on a separated faster disk. But I assume you're not there yet, so don't over complicate things
Installed postgres 9.1 in both the machine.
Initially the DB size is 7052 MB then i used the following command for copy to another server.
pg_dump -C dbname | bzip2 | ssh remoteuser#remotehost "bunzip2 | psql dbname"
After successfully copies, In destination machine i check size it shows 6653 MB.
then i checked for table count its same.
Has there been data loss? Is there missing data?
Note:
Two machines have same hardware and software configuration.
i used:
SELECT pg_size_pretty(pg_database_size('dbname'));
One of the PostgreSQL's most sophisticated features is so called Multi-Version Concurrency Control (MVCC), a standard technique for avoiding conflicts between reads and writes of the same object in database. MVCC guarantees that each transaction sees a consistent view of the database by reading non-current data for objects modified by concurrent transactions. Thanks to MVCC, PostgreSQL has great scalability, a robust hot backup tool and many other nice features comparable to the most advanced commercial databases.
Unfortunately, there is one downside to MVCC, the databases tend to grow over time and sometimes it can be a problem. In recent versions of PostgreSQL there is a separate server process called the autovacuum daemon (pg_autovacuum), whose purpose is to keep the database size reasonable. It does that by trying to recover reusable chunks of the database files. Still, there are many scenarios that will force the database to grow, even if the amount of the useful data in it doesn't really change. That happens typically if you have lots of UPDATE and/or DELETE statements in the applications that are using the database.
When you do a COPY, you recover extraneous space and so your copied DB appears smaller.
That looks normal. Databases are often smaller after restore, because a newly created b-tree index is more compact than one that's been progressively built by inserts. Additionally, UPDATEs and DELETEs leave empty space in the tables.
So you have nothing to worry about. You'll find that if you diff an SQL dump from the old DB and a dump taken from the just-restored DB, they'll be the same except for comments.
I'm using SOL server 2012 as database server.
Right now mdf file size is around 10 GB.
When ever I'm doing any transaction into this database sql server troughs bellow error
Error Number : 1105 Error Message :Could not allocate space for object dbo.tblsdr . PK_tblsdr_3213E83F0AD2A005 in database hwbsssdr because the PRIMARY filegroup is full.
Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the file-group.
There is almost 400 GB of free space is available on my disc.
Can any one tell me what is the issue and how can i solve that.
Since you are using Express edition of SQL Server 2012, there is a limitations of 10GB per database, so this is your problem.
By the way, problem doesn't need necessarily to be with disk that is running out.
Also, if you have database with set up MAXSIZE to specific value, and if database reach that value every next transaction will report error from your question.
So, if you are sure that you have enough disk space for next transactions, check MAXSIZE property of your database executing next code:
use master;
go
exec sp_helpdb [YourDatabase]
If you want to change MAXSIZE database property, you can do this with next code:
alter database [YourDatabase]
modify file
(
name = 'YourDatabaseFile',
maxsize = X MB
)
Explanation
The specified filegroup has run out of free space.
Action
To gain more space, you can free disk space on any disk drive containing a file in the full filegroup, allowing files in the group to grow. Or you can gain space using a data file with the specified database.
Freeing disk space
You can free disk space on your local drive or on another disk drive. To free disk space on another drive:
Move the data files in the filegroup with an insufficient amount of free disk space to a different disk drive.
Detach the database by executing sp_detach_db.
Attach the database by executing sp_attach_db, pointing to the moved files.
Using a data file
Another solution is to add a data file to the specified database using the ADD FILE clause of the ALTER DATABASE statement. Or you can enlarge the data file by using the MODIFY FILE clause of the ALTER DATABASE statement, specifying the SIZE and MAXSIZE syntax.