I have written a function that prepares data based on user-input from the web and copies the results down as a csv to a specified directory, from where the package (each request will contain information from 3 different tables/csv files) will be made available for the user to download.
The function itself works if the output directory is in the pg_data directory /var/lib/pgsql/data/adc_downloads, the permissions are:
drwxr-xr-x 2 postgres postgres 4096 Dec 5 15:18 adc_downloads
But ideally I want the output directory to be here: /var/www/vhosts/[vhostname]/httpdocs/adc_downloads. After getting it to work in pg_data, I used chown -R --reference=/var/lib/pgsql/data/adc_downloads /var/www/vhosts/[vhostname]/httpdocs/adc_downloads and chmod with the same parameters to mimic the permissions which appears to have worked:
drwxr-xr-x 2 postgres postgres 4096 Dec 5 14:19 adc_downloads
but I'm still getting a Permission denied error when trying to run the function with the intended output directory.
Something else then must be overriding the permissions I've (tried to?) set, any ideas what this might be? Given the function itself works in one directory but not another I don't think it's an SELinux issue, though I may of course be wrong.
Version Info: Postgres 8.4 CentOS 6.5 built with Plesk.
got this working (thanks to #joop's recommendation) in the intended location /var/www/vhosts/[vhostname]/httpdocs/adc_downloads by adding execute permission for the other users (including the postgres user) for the [vhostname] and httpdocs directories
Related
A) Mounted a Virtualbox directory with the required ownership and permissions:
ls-l /var:
drwx------. 1 postgres postgres 4096 Aug 20 12:43 pgshared
(and /var/pgshared/data directory has been created with exactly same permission)
B) Changed PGDATA environment variable from default to this data directory
C) initdb failed with below error:
fixing permissions on existing directory /var/pgshared/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Europe/Brussels
creating configuration files ... ok
running bootstrap script ... 2021-08-20 12:16:08.088 CEST [2807] LOG: could not link file "pg_wal/xlogtemp.2807" to "pg_wal/000000010000000000000001": Operation not permitted
2021-08-20 12:16:08.090 CEST [2807] FATAL: could not open file "pg_wal/000000010000000000000001": No such file or directory
child process exited with exit code 1
initdb: removing contents of data directory "/var/pgshared/data"
Additional comments:
Using -D option has the same result (regardless of PGDATA environment variable)
The error means that the file system does not support hard links (the link(2) system call fails).
Even if you manage to put a data directory, PostgreSQL will not work. To confirm, run the following a couple of times:
SELECT pg_switch_xlog();
You will have to use a different file system that implements all required system calls.
This is a workaround that I used and I was able to successfully continue with creating databases and connecting to them but I don't have any idea (yet) if I'll face further issues. Additionally, the root cause and a real solution is still unknown:
1- Since the initialization could be done on a local drive, I unmounted the shared drive (umount /var/pgshared) and initialized the database. it was successful and the data folder got created with all expected contents.
2- I moved the data folder to a temporary location and re-mounted the shared folder with the same mount command.
3- I moved back the data folder to the mounted drive.
4- I made simple test with psql commands.
I'm trying to create the first copy of my database. I'm using PostgreSQL and Ubuntu 16+ with Django technology.
I found this documentation to create a copy:
I'm trying to export the entire database to a file so that I can add it to another server. I tried this:
pg_dump app_prod > test_copy
pg_dump --host=localhost --username=app --dbname=app_prod --file=testdb.sql
after selecting ls my directory can see the database. But by running eg WinSCP it is not visible.
How can I take these files, copy them to my Windows system and upload to another Ubuntu server?
I think that it is enough to make them visible in WinSCP. How can I do this?
EDIT:
drwxr-xr-x 3 postgres postgres 4096 Oct 4 08:06 9.5
-rw-rw-r-- 1 postgres postgres 3578964 Jan 18 10:46 test_copy
-rw-rw-r-- 1 postgres postgres 0 Jan 18 10:54 testdb.sql
It seems like this was resolved in the comments: you were looking at the wrong folder in the WinSCP folder explorer.
There are a few items worth noting to bolster the good advice already given:
Your ls -l output indicates that the SQL file is zero bytes in size, so something has gone wrong there. If you manage to transfer it to your local machine, you will find it is empty.
Also, try not to store database dumps in /var/lib/postgresql - this is where your PostgreSQL database keeps live database files on the server, and you don't want to risk changing or deleting anything here. Use /home/maddie instead (change the username as appropriate).
My goal is to import csv file to postgresql database.
my file is located in network shared folder and I do not have no option to make it in a local folder.
My Folder located in :
"smb://file-srv/doc/myfile.csv"
When I run my this PostgreSQL script:
COPY tbl_data
FROM 'smb://file-srv/doc/myfile.csv' DELIMITER ',' CSV;
I would get this error :
ERROR: could not open file "smb://file-srv/doc/myfile.csv" for reading: No such file or directory
SQL state: 58P01
I have no problem to access the file and open it.
I am using PostgreSQL 9.6 under Ubuntu 16.04.
Please Advice how to fix this problem.
Update
When I try to access the file with postgres user I would have same error:
postgres#file-srv:~$$ cat smb://file-srv/doc/myfile.csv
cat: 'smb://file-srv/doc/myfile.csv' : No such file or directory
As I mention when I user mounted folder I created I can access the file.
it is about permission. you have to check read access on file and folders.
also, logging with superuser access may solve your problem.
In short, this is a permissions issue: Your network share is likely locally mounted to your user's UID, while the PostgreSQL server is running as the postgres user.
Second, when you log into your database, there is not an overlap between the database's users and the system's users, even if you have the same username. This means that when you request a file from your network share, the DB user, in this case postgres, does not have the necessary permissions.
To see this, and assuming you have root access on the box in question, you might try to become the postgres user and see that you cannot access the file:
$ sudo su - postgres
$ cat /run/user/.../smb.../yourfile.csv
Permission denied
The fix to your issue will involve -- somehow -- making the file or share accessible to the postgres user. Copying is certainly the quickest way. But that's off the table. You could mount the share (perhaps as read only) as the postgres user. You might do this in fstab.
However, unless this is going to be an automated detail that happens regularly, this seems like heroics. Without more information as to why you can't copy locally, I suggest copying the file locally.
I have a web application querying a Postgresql database (successfully) and I'm looking to move the data folder from location /var/lib/postgres/9.3/main to a customisable location.
Right now I'm prevented from even copying the folder due to permission errors, but I can't assign myself the permissions because that breaks the postgres server.
(I broke the server by running sudo chown <username> -R /var/lib/postgres/9.3/main - which worked as a command but stopped the postgres server from working)
I would simply create a new folder and change the location there, but I'll lose the current instance of my database if that was done.
How can I move the current folder to a new location, so that I can point to it in the .conf file? I need to explicitly move the folder, I can't create a new DB.
You can just copy or move the directory, including all subdirs and files
cp -rp or mv should be enough for this.
Postgres must not be running while you are messing with the files
The base of the data-drectory (PG_DATA) must be owned by postgres and have file mode 0700 . (when not: pg will refuse to start)
[the rest of the files must at least be readable/writeble by postgres]
the new location must also be known to the startup process (in /etc/init.d/ and (possibly) in the postgres.conf file within the data directory. (for the log file location)
Following my previous question I'm now trying to execute a batch file trough NSIS code in order to successfully setup the postgres installation after it is being unzipped. The batch file contains command for initializing the database but it fails because of permission restrictions. I am on a Win7 x64 PC. My user account is the administrator and I start the Setup.exe with Run as adminitrator option. This is the error I get:
C:\Program Files (x86)\Poker Assistant>cd "pgsql\bin"
C:\Program Files (x86)\Poker Assistant\pgsql\bin>initdb -U postgres -A
password
--pwfile "pwd.txt" -E utf8 -D "..\data" The files belonging to this database system will be owned by user "Mandarinite".
This user must also own the server process.
The database cluster will be initialized with locale
"Bulgarian_Bulgaria.1251". initdb: could not find suitable text search
configuration for locale "Bulgarian_ Bulgaria.1251" The default text
search configuration will be set to "simple".
Data page checksums are disabled.
creating directory ../data ... initdb: could not create directory
"../data": Permission denied
EDIT: After tinkering little more with the installer I got to the root of the problem. I cannot in any way execute the following command when the installation is in the Program Files folder:
initdb -U postgres -A password --pwfile "pwd.txt" -E utf8 -D "..\data"
I tried from .bat file. I tried from .cmd file. I tried manually from Command Prompt. I tried start as Administrator. All attempts resulted in the Permission denied error
EDIT2: I did not find any way to fix the problem so I made a workaround. Now I distribute the postgres with its data directory already initialized. Then I only need to create the service and start it.
I just realised what the issue here is.
If you run postgres as Administrator, it uses a special Windows API call to drop permissions (acquire a restricted token), so that it runs without full Administrator rights for security. See PostgreSQL utilities and restricted tokens on windows.
I suspect that what's happening here is that initdb isn't creating the target data directory and setting its permissions before doing that, so it drops permissions and then doesn't have the permissions to create the data directory.
To work around it, simply md ..\data to create the empty directory and then use icacls.exe to grant appropriate permissions before you try to initdb. Or, even better, store it in a more appropriate place like %PROGRAMDATA%\MyApp\pgdata or whatever; application data should not go in %PROGRAMFILES%.