I would like to access a PostgreSQL database in Erlang. I downloaded the epgsql driver, it was a few directories and files, but I don't understand how to use it.
How can I write an Erlang program and use the epgsql driver to access a PostgreSQL database?
I made a new folder and copied all files from src/ in the driver and pgsql.hrl to my new folder. Then I created a simple test program:
-module(dbtest).
-export([dbquery/0]).
dbquery() ->
{ok,C} = pgsql:connect("localhost", "postgres", "mypassword",
[{database, "mydatabase"}]),
{ok, Cols, Rows} = pgsql:equery(C, "select * from mytable").
Then I started erl and compiled the modules with c(pgsql). and c(dbtest).
But then when I exeute dbtest:dbquery(). I get this error:
** exception error: undefined function pgsql:connect/4
in function dbtest:dbquery/0
Any suggestions on how I can connect to a PostgreSQL database using Erlang?
Rebar is a good tool to use but, I'm finding it's good to know how your project should be structured so you can tell what to do when things go wrong. Try organizing your project like this:
/deps/epqsql/
/src/dbtest.erl
/ebin
Then cd into deps/epqsql and run make to build the library.
Your dbtest.erl file should also explicitly reference the library, add this near the top:
-include_lib("deps/epgsql/include/pgsql.hrl").
You'll probably want to use a Makefile (or rebar) that compiles your code when you make changes but, try this to compile things right now: erlc -I deps/epqsql/ebin -o ebin src/dbtest.erl.
When testing, make sure your load paths are set correctly, try: erl -pz deps/epqsql/ebin/ ebin/. When the erl console loads up, try dbtest:dbquery(). and see what happens!
I don't have Postgresql setup on my machine but, I was able to get more reasonable looking errors with this setup.
I recommend to use ejabber pgsql driver https://svn.process-one.net/ejabberd-modules/pgsql/trunk/
Related
I have created a Python application that utilizes a PostgreSQL database and now I'd like to package it as a stand-alone application. I have worked out how to package everything as a single executable using PyInstaller however it will fail upon launch when attempting to establish a connection to the database if it doesn't exist. To address this my assumption is that I need to also include a mechanism for installing and launching a PostgreSQL database instance if it's not present.
I'm trying to provide a single installation process that creates/configures/launches the database (if not present) and installs the application's executable. I want to create installers for Linux, Windows, and MacOS.
How do I best go about this? I am hoping to find documentation for a solution to this use case -- my Google-Fu appears to be weak and/or the solution is obscure.
Handling external software is not the purpose of Pyinstaller. So you need to handle its dependencies by yourself. So about your question, you need to first find a portable version of PostgreSQL, then in your script, run and configure it manually. For example, there is a good portable version in here.
Just extract the contents of PGRE portable to a directory like PostgreSQL next to your python script (app.py).
Then you can use the below code to launch Database:
import subprocess
import os
import sys
PGRE_EXE = "PostgreSQL/PostgreSQLPortable.exe"
def resource_path(relative_path):
if hasattr(sys, '_MEIPASS'):
return os.path.join(sys._MEIPASS, relative_path)
return os.path.join(os.path.abspath("."), relative_path)
def start_pgre():
pgre_path = os.path.join(resource_path("."), PGRE_EXE)
print(pgre_path)
p = subprocess.Popen(
[pgre_path], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
print(stdout)
print(stderr)
if __name__ == "__main__":
start_pgre()
Then add the whole PostgreSQL as external data with --add-data:
pyinstaller -F --add-data "./PostgreSQL:PostgreSQL" app.py
I'm attempting to import an SQL dump in PgAdmin 4 using the psql client - However the error message returned is - The system cannnot find the file specified.
Here is a screenshot of my psql client -
The file films.sql is currently stored on my desktop, but I suspect the default location that the psql client accesses is not my desktop? Is there anyway to set the location that the client looks in order to resolve this?
The file SQL is viewable here: https://github.com/datacamp/courses-intro-to-sql/tree/master/datasets
I simply want to get the database on my local machine so that I don't need to store queries in an online learning platform. It would be best if this database is available locally to query and practice on.
I've attempted to execute the whole SQL file as a query on the films database but this does not seem to be working either and returns 'Asynchronous query execution/operation underway.
Query returned successfully in 388 msec.' - However it seems to be the case that the Asynchronous query never completes when I refresh the database.
Please can someone help?
Just give the path to your file:
psql -d my_database -f /path/to/the/file.sql
psql -d my_database -f C:/path/to/the/file.sql
Depending on whether you are on a unix/linux machine or Windows.
Oh, and if you aren't familiar with file paths you may want to take a step back and become more familiar with general computer terminology before diving into a RDBMS. Your learning will be much easier if you have a solid foundation to build upon.
I suspect this question might be moot for the asker at this point, but for anyone else stumbling upon it like I did: the interactive connection info prompts are provided by a batch script (in Windows, I'd guess there's an analogous shell script for Unix) called runpsql.bat, which then just passes your inputs as commandline arguments to the psql.exe executable. I was getting this error because I had migrated my Postgres installation and the batch script was calling a nonexistent path for psql.exe, hence The system cannot find the file specified. I edited runpsql.bat to point to the correct location of psql.exe and that resolved the issue. So for OP, I would look into PgAdmin4 and see where it's (presumably) calling runpsql.bat, then make sure that that calls psql.exe with the correct path.
I am using helios Eclipse for debugging my code in postgresql.
My aim is to know how postgresql uses join algorithms during the join query, so I started to debug nodenestloop.c which is in the Executor folder.
I gave break points in that file, But whenever I try to debug that file, the control goes to main.c and never comes back,How do I constraint the control only to that particular file(nodenestloop.c)
Below are the following fields which I gave in Debug configurations of Helios Eclipse.
C/C++ Application - src/backend/postgres and
project - pgsql
I followed the steps given in the following link for running the program.
https://wiki.postgresql.org/wiki/Working_with_Eclipse#
I even uncheked the field "Start on Start up=main" , but When I do that, The step in and Step over buttons are not activated and the following problem has popped up.
Could not save master table to file '/home/ravi/workspace/.metadata/.plugins/org.eclipse.core.resources/.safetable/org.eclipse.core.resources'.
/home/ravi/workspace/.metadata/.plugins/org.eclipse.core.resources/.safetable/org.eclipse.core.resources (Permission denied)
So I started eclipse using sudo, but this time the following error has come in the console of eclipse.
"root" execution of the PostgreSQL server is not permitted.
The server must be started under an unprivileged user ID to prevent
possible system security compromise. See the documentation for
more information on how to properly start the server.
Could any one help me with this.
Thank you
Problem 1: User ID mismatch
Reading between the lines, it sounds like you're trying to debug a PostgreSQL instance that's running as the postgres user, or a different user ID to your own anyway. Hence your attempt to use sudo.
That's painful, especially when using an IDE like Eclipse. With plain gdb you can just sudo the gdb command to the desired uid, e.g. sudo -u postgres -p 12345 to attach to pid 12345 running as user postgres. This will not work with Eclipse. In fact, running it with sudo has probably left your workspace with some messed up file permissions; run:
sudo chown -R ravi /home/ravi/workspace/
to fix file ownership.
If you want to debug processes under other user IDs with Eclipse, you'll need to figure out how to make Eclipse run gdb with sudo. Do not just run all of Eclipse with sudo.
Problem 2: Trying to run PostgreSQL under the control of Eclipse
This:
"root" execution of the PostgreSQL server is not permitted. The server must be started under an unprivileged user ID to prevent possible system security compromise. See the documentation for more information on how to properly start the server.
suggests that you're also attempting to let Eclipse start postgres directly. That's very useful if you're trying to debug the postmaster, but since you're talking about the query planner it's clear you want to debug a particular backend. Launching the postmaster under Eclipse is useless for that, you'll be attached to the wrong process.
I think you probably need to read the documentation on PostgreSQL's internals:
Tour of PostgreSQL Internals
PostgreSQL internals through pictures
Documentation chapter - internals
Doing it right
Here's what you need to do - rough outline, since I've only used Eclipse for Java development and do my C development with vim and gdb:
Compile a debug build of PostgreSQL (compiled with ./configure --enable-debug and preferably also CFLAGS="-ggdb -Og -fno-omit-frame-pointer"). Specify a --prefix within your homedir, like --prefix=$HOME/postgres-debug
Put your debug build's bin directory first on your PATH, e.g. export PATH=$HOME/postgres-debug/bin:$PATH
initdb -U postgres -D $HOME/postgres-debug-data a new instance of PostgreSQL from your debug build
Start the new instance with PGPORT=5599 pg_ctl -D $HOME/postgres-debug-data -l $HOME/postgres-debug-data.log -w start
Connect with PGPORT=5599 psql postgres
Do whatever setup you need to do
Get the backend process ID with SELECT pg_backend_pid() in a psql session. Leave that session open; it's the one you'll be debugging.
Attach Eclipse's debugger to that process ID, using the Eclipse project that contains the PostgreSQL extension source code you're debugging. Make sure Eclipse is configured so it can find the PostgreSQL source code you compiled with too (no idea how to do that, see the manual).
Set any desired breakpoints and resume execution
In the psql session, do whatever you need to do to make your extension run and hit the breakpoint
When execution pauses at the breakpoint in Eclipse, debug as desired.
Basic misunderstandings?
Also, in case you're really confused about how all this works: PostgreSQL is a client/server application. If you are attempting to debug a client program that uses libpq or odbc, and expecting a breakpoint to trigger in some PostgreSQL backend extension code, that is not going to happen. The client application communicates with PostgreSQL over a TCP/IP socket. It's a separate program. gdb cannot set breakpoints in the PostgreSQL server when it's connected to the client, because they are separate programs. If you want to debug the server, you have to attach gdb to the server. PostgreSQL uses one process per connection, so you have to attach gdb to the correct server process. Which is why I said to use SELECT pg_backend_pid() above, and attach to the process ID.
See the internals documentation linked above, and:
PostgreSQL site - coding
PostgreSQL wiki - developer resources
Developer FAQ
Attaching gdb to a backend on linux/bsd/unix
I also faced similar issue and resolved it after some struggle
I misunderstood the following point under Debugging with child processes in the wiki (https://wiki.postgresql.org/wiki/Working_with_Eclipse).
5."Start postmaster & one instant of postgresql client (for creating one new postgres)"
The above step should be performed from terminal by starting postgres server and one client.
Hope this helps
Once this is done then debugger in eclipse needs to be started for C/C++ Attach to Application
I get this exception in PostgreSQL:
org.postgresql.util.PSQLException: ERROR: could not access file "$libdir/plpgsql": No such file or directory
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:1721)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1489)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:193)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:337)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:236)
at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:205)
I searched a lot and most solution points to a wrong installation. But this is my test db which has been running without issues for a long time. Also inserts are working. Issue occurs only on select queries.
Apparently, you moved your PostgreSQL lib directory out of place. To confirm this, try the following in psql:
> SET client_encoding TO iso88591;
ERROR: could not access file "$libdir/utf8_and_iso8859_1": No such file or directory
If you get an error message like this, then my theory is correct. You'll need to find out where those files ended up, or you can reinstall PostgreSQL to restore them.
To find out what $libdir is referring to, run the following command:
pg_config --pkglibdir
For me, this produces:
/usr/lib/postgresql
I have the same problem: the other postgres server instance (8.4) was interfering with the 9.1 one; when the 8.4 instance is removed it works.
the other instance can sometimes be removed from the system while still running (e.g. you do a gentoo update and a depclean without stopping and migrating your data). so the error seems particularly mysterious.
the solution is usually going to be doing a slot install/eselect of the old version (in gentoo terms, or simply downgrading on other distros), running its pg_dumpall, and then uninstalling/reinstalling the new version and importing the data.
this worked pretty painlessly for me
In order to take some development work home I have to be able to run a PostgreSQL database.
I don't want to install anything on the machine at home. Everything should run off the usb drive.
What development tools do you carry on your USB drive?
That question covers pretty much everything else, but I have yet to find a guide to getting postgresql portable. It doesn't seem easy if it's even possible.
So how do I get PostgreSQL portable? Is it even possible?
EDIT:
PostgreSQL Portable works. It's very slow on the usb-drive I have, but it works. I can't recommend doing constant development with it but for what I need it's great.
Perhaps if I pick up a full speed external drive I'll try out virtualization. Given the poor performance of just running the database off this drive, a full virtual OS running off of it would be unusable.
Here's how you can do this on your own:
http://www.postgresonline.com/journal/archives/172-Starting-PostgreSQL-in-windows-without-install.html
An alternate route would be to use something like VirtualBox and just install your development environment (database, whatever) on there.
There are 2 projects to try in 2014: http://sourceforge.net/projects/pgsqlportable/ and http://sourceforge.net/projects/postgresqlportable/?source=recommended.
I can't vouch for the second, but I'm using the first and it works right out of the box.
After unzipping using 7-zip (http://www.7-zip.org/download.html):
1) Run "start service without usuario.bat" ( english translation )
2) Then run "pgadmin3.bat"
The only minimal problem for me was that its in spanish. I've been able to change the language to english by following Change language of system and error messages in PostgreSQL. Using google translate the instructions are:
Description
This is a zip to automatically run postgresql 9.1.0.1 for windows. This version already has pgagent and pldebugger. To run must: 1) unzip
the zip 2) run the "start service without usuario.bat" found in the
pgsql directory within the folder you just unzipped. 3) Optional. If
you want to run the agent works postgresql (pgagent) should only run
the "start pgagent.bat" found in the pgsql directory inside the folder
you just unzipped. 4) Optional. To manage and / or develop the bd you
can run the pgadmin3.bat 5 files) Optional. To stop and / or restart
the server correctly use file "service without stopping usuario.bat"
usuario.bat or restart service without depending on the case.
Now option for Linux (file. Tar.gz). Postgresql portable Linux 9.2
Please use the tickets for your answer bugs.
Username: postgres Password: 123
Just a Note : on a new computer , to get pgadminIII working you may need to add a db. The settings are in attached screenshot.
Hope it helps.
I agree with virtualization solution, but maybe you can find useful this link from portable freeware collection, I have used this locally, not from usb though
1.download and extract : zip version
2.inside pgsql folder create data folder(put any name,I used 'data')
3.initalize data folder: c:\pgsql\bin\initdb.exe -D c:\pgsql\data -U postgres -W -E UTF8 -A scram-sha-256
4.to start/stop see next cmd code that I use (press any key inside it to stop)
c:\pgsql\bin\pg_ctl.exe -D c:\pgsql\data -l logfile start
pause
c:\pgsql\bin\pg_ctl.exe -D c:\pgsql\data stop
more info