Querying hive schema with HAWQ and PXF - postgresql

I know it's possible to query data managed by Hive with HAWQ, and HCatalog, I have installed Hortonworks ambari that installs all the tools needed. To query the hive i schema followed pivotal instructions where it says to enter PostgreSQL and do the command SET pxf_service_address TO "localhost:51200", but I always get the error unrecognized configuration parameter "pxf_service_address". Does anyone have this problem before?

Your version() output indicates you are connecting to a PostgreSQL instance and this is NOT HAWQ.
The default port for both PostgreSQL and HAWQ is 5432 so on a single node, one has to be changed. I think it is a bug that Hortonworks has decided to change the HAWQ port instead of the PostgreSQL port. So if you want to give some feedback to Hortonworks, tell them to change the PostreSQL port to something other than 5432.
Find hawq_master_address_port in your /usr/local/hawq/etc/hawq-site.xml file. The value will be something other than 5432. You should also look at the hawq_master_address_host value and make sure you are on the Master host. So ssh to your master host to connect via psql.
Now, armed with your HAWQ port and logged in as gpadmin on the Master host, edit your .bashrc file. Add:
export PGPORT=<hawq_master_address_port>
Substitute hawq_master_address_port with the value from hawq-site.xml.
Now source your .bashrc file and then run:
psql template1
I'm also guessing you don't have a gpadmin database so create it.
psql -d template1 -c "create database gpadmin;"
Now you can just run "psql" and connect to HAWQ.

Related

How to open a postgres database created using psql in PgAdmin4?

How do I open a postgres database created using psql (i.e. in the terminal) in PgAdmin4 and vice versa?
I also noticed that the localhost port for psql is 5432 and for PgAdmin4 is 5433.
Is this correct?
Some background is in order. When you install Postgres you create an instance of a server that comes with three databases already created; template0, template1 and postgres. On a given machine you can create more then one Postgres server/instance and have them run simultaneously. In order for that to happen though each server needs to listen on a different port. The default port is the 5432 you mention above. It would seem you also have another server running on port 5433. I'm guessing what you want to know is about connecting to a given server rather then a database in that server. In that case it is important to know that Postgres works on the server/client model where it is the server and in your case psql and pgAdmin are the clients. What this means is that a Postgres server is not tied to a client, it exists on its own. It also means a client can connect to any Postgres server it can reach, assuming it is using the correct credentials. All of the previous means, yes you can connect psql and/or pgAdmin to either server. For psql specify the correct port using -p. For pgAdmin you will need to set up a server using the server dialog Dialog. Then use the appropriate port in the connection tab.

Can't connect to postgres via PSQL or php, only pgAdmin4

I have PostgreSQL 12 installed and using Windows 10. My goal is to connect to postgre via php, because it didn't work I tried with plsql but it didn't work either, i always get (with both plsql and php using PDO):
FATAL: password authentication failed for user "postgres"
Connection via plsql in cmd
However, when trying via pgAdmin4, I can connect and access my databases. I first use my root password then sometime my postgres user one, so I'm sure my password works.
Here is my pg_hba.conf file:
pg_hba.conf file
I have a set password for postgres which was set vua pgAdmin4.
I don't understand why it would work one way and not the other, can anyone help me?
So after trying several things, I found that plsql and php error was due to the wrong port being adressed.
By default 5432 is used, but for some reason my configuration was using 5433.
You can find this information in pgAdmin4 by opening PostgreSQL 12 properties, and then checking in connection tab.
To change the used port in plsql I used:
psql -U postgres -p 5433
Change 5433 by your configured port if you have same issue, you also need to specify the port if using php's PDO.

Distribution, Installation, and Connectivity of PostgreSQL

Have not been able to find an answer to this yet, but if this a duplicate, please mark accordingly!
Trying to understand how PostgreSQL can be distributed to an end-user via silent installation and the respective actions if:
PostgreSQL is already installed on the computer system
How to connect to PostgreSQL as a superuser, if it is already installed
Create a database (relating to #2, since we would not know the credentials of postgresql user when PostgreSQL was initially installed)
For #1, from my research and understanding, there are two methods:
Determining if a postgresql-[version] service is installed (per this QA)
Determining if HKEY_LOCAL_MACHINE\SOFTWARE\PostgreSQL\ registry exists, and if it does, seeing the versions and service names under the \Installations\ and \Services\ subs
However, I am more concerned about the connectivity part. If my application (to be written in C#) is dependent on a database name "MyProgram," how would it be possible to create a database in PostgreSQL and with what credentials?
From my understanding, the way to go would be to attempt to log in as the default superuser, postgres, to the default database, postgres, and create a new user and database from that connection. But, the password for postgres user is set during by the user/program that is initially is installing PostgreSQL.
How to go about this?
Any help is much appreciated!
IMPORTANT NOTE: This may not be the most ethical/proper way of doing this, but it appears to get the job done.
After numerous hours of digging, using the registry key on Windows is the best methodology, it seems, to determine if the version of PostgreSQL you intend to install, if it is already installed.
Assuming that I am running on the Windows system with Administrative rights, in theory, I should be able to change the login credential requirements of the pg_hba.conf file in the data directory (i.e. cluster) that already exists to allow myself to create the database(s) and user(s) I need to, before reverting the credential requirement settings to what they originally were.
So, the answer I have come to the conclusion with is:
Determine if PostgreSQL is already installed or not. Look at the HKEY_LOCAL_MACHINE\SOFTWARE\PostgreSQL\Installation\[version] registry, where [version] is formatted as postgresql-[32 or 64 bit]-[PostgreSQL version], e.g. postgresql-x64-12). If the registry exists, then data should exist for the Data Directory value... obtain that data, and that's where the cluster is located.
Step [2] and on are obviously for when PostgreSQL is already installed.
Make a copy of the pg_hba.conf file in the cluster directory provided by the Data Directory value from step [1].
This will be the file we restore to after we are done. Save file to a temporary directory, such as Desktop.
In the pg_hba.conf file in the cluster directory, change all connection types' methods to trust
Example:
# TYPE DATABASE USER ADDRESS METHOD
host all all 127.0.0.1/32 trust
Restart Postgres' Windows service with the following command: pg_ctl.exe restart -D <cluster directory>
NOTE: pg_ctl.exe is located under the \bin\ folder of Postgres' installation directory.
Example: C:\Program Files\PostgreSQL\12\bin\pg_ctl.exe restart -D "C:\Program Files\PostgreSQL\12\data\"
Connect to the cluster and issue the command to create a superuser role for your needs with the following command: psql -h 127.0.0.1 -p 5432 -d postgres -c "CREATE ROLE <role name> LOGIN SUPERUSER PASSWORD '<password>';
In the above command, I have the cluster running on the local computer (i.e. localhost, IP address 127.0.0.1) on port # 5432 (default), connecting to the default database postgres and issuing the command to create a role with whatever role name provided in place of <role name>, with SUPERUSER rights and the password provided in place of <password>.
Since one HAS to connect to a database, I am connecting to the default one postgres, otherwise template0 and template1 are default databases that could also be utilized.
Connect to the cluster and issue the command to create the needed database for your needs with the following command: psql -h 127.0.0.1 -p 5432 -d postgres -c "CREATE DATABASE <database name>;
Replace the pg_hba.conf file with the original
Restart Postgres' Windows service with the following command: pg_ctl.exe restart -D <cluster directory>

Cannot connect PostgreSQL with psql - wrong UNIX-Domain-Socket

I'm running CentOS 7.5 and cannot setup PostgreSQL.
If I'm logged in as user postgres and type psql postgres I get the following error message:
psql: Could not connect to server: no such file or directory
does the server run locally and accepts connections on Unix-Domain-Socket "/var/run/postgresql/.s.PGSQL.5432"
However, I changed the port to 5543 (did so in etc/systemd/system/postgresql.service by including /lib/systemd/system/postgresql.service as a [Service] and setting Environment=PGPORT=5543). Note that you shouldn't change it directly in /lib/ because that will get overwritten.
So, the server looks for the wrong UNIX-Domain-Socket and does not find one (because it does not exist), but the socket for the correct port 5543 does exist according to sudo netstat -nlp:
5486/postgres /var/run/postgresql/.s.PGSQL.5543
postgresql.service is running according to systemctl status postgresql.service
Any constructive help is appreciated.
Why are you changing it in systemd and not using PostgreSQL's config file? Is that a CentOS thing?
Anyway - you can run the server on any port you like, or run multiple server instances of the same or different versions on a variety of ports. In that case though you need to tell psql what port to use.
You can set an environment variable (PGPORT), specify it with -p on the command-line or in a .psqlrc file. See the manuals for details.
Edit in response to comments:
If you want to set the PGPORT for psql, do it in the user's shell defaults or in /etc/bash... or equivalent. You could of course replace psql with an alias using your custom port or recompile the binary itself if you wanted.
I'm not sure this is really much use from a security perspective. It seems unlikely that someone can run local processes on your machine, has gained access to your postgres user password but isn't smart enough to see what port the server is running on.

Multiple installations of Postgres on Ubuntu

This is the first time I am setting up multiple Postgres postgres servers on Ubuntu 12.04LTS. (I know multiple versions is a bad idea, but need to sanity check new installation before I migrate and kill the old instance)
Original Postgres V9.1 is installed using bundled apt-get install. (Port : 5432)
New Postgres V10 is installed via EnterpriseDB package. (Port : 5433)
I am able to use standard utilities for the V9.1 like psql, pg_dump, etc. However, I am not able to access psql of V10. Instead I am getting this error message (note the incorrect port):
/opt/PostgreSQL/10/bin$ ./psqlpsql.bin: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
I would like to be able to manage both ideally by using psql.
I usually use Python to connect to databases and I am able to create an SQLAlchemy connection to new instance at port 5433. I can query pg_catalog and information_schema as well.
Thanks in advance!
The default port of postgres is 5432.
So If you want to access psql of V10, use the below commands:
export PATH=/opt/PostgreSQL/10/bin:$PATH
psql -p 5433