When cloning a standby, how can I prevent pg_basebackup from copying postgresql.conf and pg_hba.conf from the master to /var/lib/pgsql/9.9/data directory?
Currently I am using this command
[root#xyz..]# pg_basebackup -h {master ipAddr} -D /var/lib/pgsql/9.6/data -U postgres -v -P
according to docs:
The backup will include all files in the data directory and
tablespaces, including the configuration files and any additional
files placed in the directory by third parties. But only regular files
and directories are copied. Symbolic links (other than those used for
tablespaces) and special device files are skipped.
So there is no such option. If you still want to force it, move config files away from data directory (and optionally ln them to data_dir)
This answer is for Postgres 14. pg_basebackup takes backup of the entire data directory. https://www.postgresql.org/docs/14/app-pgbasebackup.html states that the backup utility will skip all directory/file that are symbolic links. So, that could be a workaround to get only desired content into the tar ball.
I had faced similar situations where I wanted to exclude the content of multiple directories like pg_replslot,pg_dynshmem, pg_notify etc. I made the tar ball the usual way: pg_basebackup -D /backup/ -F t -P -v. After the tar ball was made, and before restoring it to another server, I updated the tar manually by excluding content of all the required directories.
Related
Downloaded file is unzipped. I moved it up the folder tree. I have the very latest version of LibreOffice in my Ubuntu. But I can't migrate the data therein to my postgis database. Have been working on this for some time any pointers appreciated. Linux output is below.
shp2pgsql -c -D -I tl_2019_us_cd116.shp gis.congress | psql -U sporter tutorial
Unable to open tl_2019_us_cd116.shp or tl_2019_us_cd116.SHP
tl_2019_us_cd116: dbf file (.dbf) can not be opened.
Per recommendations below I've been working on permissions but am stuck again. I'm logged in as root user on the path that has access to the file and as seen below it states ownership is being changed. But ls -l shows the original owner
root#DESKTOP-VDI7BFT:/mnt/c/Users/sport# chown -v sporter tl_2019_us_cd116
changed ownership of 'tl_2019_us_cd116' from sporter399 to sporter
But I don't think the ownership really changes:
ls -l tl_2019_us_cd116
-rwxrwxrwx 1 sporter399 sporter399 (7 times over for each file within the folder, all showing sporter399 which is what I'm trying to change)
From mongodocs:
Overwrite Files
"Mongodump overwrites output files if they exist in the backup data folder. Before running the mongodump command multiple times, either ensure that you no longer need the files in the output folder (the default is the dump/ folder) or rename the folders or files."
Hey guys,
I want to do a daily backup and sometimes even two backups a day. The Dump-filename gets named by the actual date. If I backup twice a day, the first backup gets overwritten due to same names.
Is there any way to tell mongodump to rename (in e.g. 5.9.2016(1)) the file if it already exists?
You can use the --out option of mongodump to specify the path where to dummp the data.
Create a script that run mongodump and give different name for your path, i.e. using a date:
mongodump --out /data/dump/090516/
Shell script example:
#!/bin/sh
DIR=`date +%m%d%y`
DEST=$DIR
mkdir $DEST
mongodump --out=/data/dump/$DEST
when using gsutil -m rsync -p -d -r
the ownership became root
Any idea how to run gsutil rsync just like rsync -a?
thanks
Peter
gsutil rsync doesn't currently support preserving POSIX file attributes in the cloud.
It's not guaranteed that the uid/gid on the system that uploaded a file is even valid on the system that downloaded the file. So (at least for now), you'll need to manage your file permissions manually.
I'm hosting publicly available static resources in a google storage bucket, and I want to use the gsutil rsync command to sync our local version to the bucket, saving bandwidth and time. Part of our build process is to pre-gzip these resources, but gsutil rsync has no way to set the Content-Encoding header. This means we must run gsutil rsync, then immediately run gsutil setmeta to set headers on all the of gzipped file types. This leaves the bucket in a BAD state until that header is set. Another option is to use gsutil cp, passing the -z option, but this requires us to re-upload the entire directory structure every time, and this includes a LOT of image files and other non-gzipped resources that wastes time and bandwidth.
Is there an atomic way to accomplish the rsync and set proper Content-Encoding headers?
Assuming you're starting with gzipped source files in source-dir you can do:
gsutil -h content-encoding:gzip rsync -r source-dir gs://your-bucket
Note: If you do this and then run rsync in the reverse direction it will decompress and copy all the objects back down:
gsutil rsync -r gs://your-bucket source-dir
which may not be what you want to happen. Basically, the safest way to use rsync is to simply synchronize objects as-is between source and destination, and not try to set content encodings on the objects.
I'm not completely answering the question but I came here as I was wondering the same thing trying to achieve the following:
how to deploy efficiently a static website to google cloud storage
I was able to find an optimized way for deploying my static web site from a local folder to a gs bucket
Split my local folder into 2 folders with the same hierarchy, one containing the content to be gzip (html,css,js...), the other the other files
Gzip each file in my gzip folder (in place)
Call gsutil rsync in for each folder to the same gs destination
Of course, it is only a one way synchronization and deleted local files are not deleted remotely
For the gzip folder the command is
gsutil -m -h Content-Encoding:gzip rsync -c -r src/gzip gs://dst
forcing the content encoding to be gzippped
For the other folder the command is
gsutil -m rsync -c -r src/none gs://dst
the -m option is used for parallel optimization. The -c option is needed to force using checksum validation (Why is gsutil rsync re-downloading all our files?) as I was touching each local file in my build process. the -r option is used for recursivity.
I even wrote a script for it (in dart): http://tekhoow.blogspot.fr/2016/10/deploying-static-website-efficiently-on.html
When I do rsync this is my command:
rsync -a source dest
I am using dest as my web root /var/www/
so some folder which are set to chmod 777 were no longer with 777 permission.
does rsync change folder permission as well?
What is best way to sync two local folders in same server.? Will rsync delete any changes done in destination and use the source files?
The manual page for rsync says this:
-a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)
Among those options is -p, about which it says:
-p, --perms preserve permissions
So, yes, rsync is making the permissions on dest match those on source in this case. If that is not desired, then read the manual page and decide what options are more appropriate to your need than rsync -a, and use those instead. In the simplest case, add the --no-perms flag after -a to disable the permission preservation.