Postgres comparing two databases - postgresql

Question

We running two database servers with around 10 databases and multiple schemas in it (production and test). Every night we restore some of the prod to the test. The backup we do with pg_dump and restore with pg_restore. Now the size of the production and the restored database differnces from a few kb up to 700mb.
Now I want to ensure that the restored data is the same as on the production. I already saw that this is usual to postgres but we have to proof this to our manager. What would be the easiest to proof that the database servers are the same after the restore?

Answer 1

Let me suggest you another approach. You can use pg_restore with single transaction. Quote from de docs:
1
--single-transaction Execute the restore as a single transaction (that is, wrap the emitted commands in BEGIN/COMMIT). This ensures that
either all the commands complete successfully, or no changes are
applied. This option implies --exit-on-error.
In this case you don't need to compare a COUNT on all tables from both sides just because if pg_restore returned 0 (zero) then everything went well.

Answer 2

just solved the provide our solution for the moment.
#!/bin/bash
#
#change to the dir this script is located in (just to be sure)
cd "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# CONFIGURATION
remoteHost=stackoverflow.com
remotePort=5432
remoteUser=postgres
logfile=00comparsion.txt
#######################
cat "" > $logfile
databases=$(psql -U postgres -t -c "SELECT datname FROM pg_database WHERE datistemplate = false")
for database in $databases; do
echo ------------------------------------------------------------------------------- >> $logfile
echo $database >> $logfile
echo -------------------------------------------------------------------------------
echo $database
if [ "$(psql -U $remoteUser -h $remoteHost -p $remotePort -tAc "SELECT datname FROM pg_database WHERE datname='$database'" )" = '' ]
then
echo "Database '$database' does not exist on remote host! I quit now!" >> $logfile
exit 3
fi
schemas=$(psql -U postgres -d $database -t -c "SELECT SCHEMA_NAME FROM information_schema.schemata
WHERE SCHEMA_NAME NOT IN ('pg_toast',
'pg_temp_1',
'pg_toast_temp_1',
'pg_catalog',
'information_schema')")
for schema in $schemas; do
echo --$schema >> $logfile
if [ "$(psql -U $remoteUser -h $remoteHost -p $remotePort -tAc "SELECT SCHEMA_NAME FROM information_schema.schemata
WHERE SCHEMA_NAME='$schema'" )" = '' ]
then
echo "Schema '$schema' does not exist on remote host! I quit now!" >> $logfile
exit 3
fi
tables=$(psql -U postgres -d $database -t -c "SELECT TABLE_NAME FROM information_schema.tables
WHERE table_schema = '$schema'")
for table in $tables; do
if [ "$(psql -U $remoteUser -h $remoteHost -p $remotePort -tAc "SELECT TABLE_NAME FROM information_schema.tables
WHERE table_name = '$table'" )" = '' ]
then
echo "table '$table' does not exist on remote host! I quit now!" >> $logfile
exit 3
fi
orderField=$(psql -U postgres -d $database -t -c "SELECT column_name FROM information_schema.columns
WHERE table_schema = '$schema'
AND table_name = '$table' LIMIT 1")
#psql -U postgres -d $database -t -c "SELECT * FROM $table order by $orderField" > l.txt
#fetch same table data on remote instance
#psql -U $remoteUser -h $remoteHost -p $remotePort -d $database -t -c "SELECT * FROM $table order by $orderField" > r.txt
#if !(cmp --print-chars "l.txt" "r.txt")
lcl=$(psql -U postgres -d $database -t -c "SELECT * FROM $table order by $orderField")
rmt=$(psql -U $remoteUser -h $remoteHost -p $remotePort -d $database -t -c "SELECT * FROM $table order by $orderField")
if !("$lcl"="$rmt")
then
echo ---$table ✘ >> $logfile
exit 3
else
echo ---$table ✔ >> $logfile
fi
done
done
done
rm l.txt r.txt
echo ------------------------------------------------------------------------------- >> $logfile
echo comparsion finished, sir! >> $logfile
exit 0