PostgreSQL C function: libpq doesn't link? - postgresql

I want to write a C function for PostgreSQL. For this function I will need to query some data using libpq, so I started by writing a dummy function to test this part:
#define _XOPEN_SOURCE
#include <libpq-fe.h>
#include <postgres.h>
#include <fmgr.h>
#include <funcapi.h>
#include <executor/executor.h>
#include "test.h"
PG_FUNCTION_INFO_V1(getAnnotation);
Datum getAnnotation(PG_FUNCTION_ARGS) {
// Connection to the database
PGconn *conn = PQsetdbLogin("localhost",
"5432",
"",
"",
"postgres",
"postgres",
"password");
// Databases names
PGresult *res = PQexec (conn, "SELECT user FROM activity LIMIT 1;");
VarChar* i = PQgetvalue(res, 0, 0);
PG_RETURN_VARCHAR_P(i);
}
It is just supposed to return the first column of the first line of one of my tables. Pretty simple right ? Well it doesn't work.
When I try to use it within psql, it says :
ERROR: could not load library "/usr/local/lib/postgresql/test.so": Error relocating /usr/local/lib/postgresql/test.so: PQexec: symbol not found
The weird part is that both PQsetdbLogin and PQexec are in the libpq-fe.h file, but only the second one causes an error. If I comment the PQexec line, then PQsetdbLogin raises an error as well.
Here is the Makefile I use to build the code:
PG_CPPFLAGS = -I$(libpq_srcdir)
LDFLAGS_INTERNAL = -L$(libdir)
SHLIB_LINK_INTERNAL = $(libpq)
SHLIB_PREREQS = submake-libpq
EXTENSION = test
DATA = test--0.1.sql
MODULES = test
# REGRESS = ... # Script for tests
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
As you can see, I linked libpq therem so everything should work... But it doesn't and I don't know why.
I am using PostgreSQL 9.6 in a docker container.

If you want to link with external libraries, you need SHLIB_LINK, but that only works if you use MODULE_big instead of MODULES.
A working Makefile would be
PG_CPPFLAGS = -I$(libpq_srcdir)
SHLIB_LINK = $(libpq)
EXTENSION = test
DATA = test--0.1.sql
MODULE_big = test
OBJS = test.o
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
But there are other problems with your function:
VarChar is a varlena, but PQgetvalue returns a char *.
You'd have to convert the value to a varlena with code similar to this:
VarChar *result;
char *c = PQgetvalue(res, 0, 0);
result = (VarChar *) palloc(strlen(c) + VARHDRSZ);
strncpy(VARDATA(result), c, strlen(c));
SET_VARSIZE(result, strlen(c) + VARHDRSZ);
It is usually the wrong idea to write client code in a server function.
If all you need is to run a query inside the current session, use the Server Programming Interface.
I guess your function is only sample code, but you need to close the connection before the backend terminates, else you have a connection leak.

Related

How can I unit test a specific DML method?

I'm writing some common DML code that contains a fairly complex method, something like:
saved uint32 checksum_ini;
method calculate_checksum(bytes_t data) -> (uint32 sum) {
uint32 result = checksum_ini;
for (int i = 0; i < data.size; ++i) {
result = f(result, data.data[i]);
}
return result;
}
My device calls the function indirectly by reading and writing some registers, which makes it cumbersome to unit test all corner cases of the checksum algorithm.
How can I efficiently write a unit test for my checksum implementation?
One approach is to create a dedicated test module, say test-checksum, containing a test device, say test_checksum_dev, that imports only your common code, and exposes the calculate_checksum method to Python, where it is easy to write tests. This is done in two steps: First, expose the method to C:
dml 1.4;
device test_checksum_dev;
import "checksum-common.dml";
// Make DML method calculate_checksum available as extern C symbol "calculate_checksum"
// The signature will be:
// uint64 calculate_checksum(conf_object_t *obj, bytes_t data)
export calculate_checksum as "calculate_checksum";
The second step is to expose it to Python. Create checksum.h:
#ifndef CHECKSUM_H
#define CHECKSUM_H
#include <simics/base/types.h>
#include <simics/pywrap.h>
extern uint32 calculate_checksum(conf_object_t *obj, bytes_t data);
#endif /* CHECKSUM_H */
(if you also add header %{ #include "checksum.h" %} to the DML file, you will get a hard check that signatures stay consistent).
Now add the header file to IFACE_FILES in your module makefile to create a Python wrapping:
SRC_FILES = test-checksum.dml
IFACE_FILES = checksum.h
include $(MODULE_MAKEFILE)
You can now call the DML method directly from your test:
SIM_load_module('test-checksum')
from simmod.test_checksum.checksum import calculate_checksum
obj = SIM_create_object('test_checksum_dev', 'dev', checksum_ini=0xdeadbeef)
assert calculate_checksum(obj, b'hello world') == (0xda39ba47).to_bytes(4, 'little')

Calling db2ReadLog from external UDF

When trying to get transaction log information on a DB2 LUW database through an external UDF calling db2ReadLog a SQL0487N (attempted to execute on SQL statement) is returned. Using the same shared library as a stored procedure works without error.
The reason it needs to be a UDF is to be able to eventually create a table valued UDF which will return actual entries from the transaction log.
Here is the C source for the UDF to determine the initial LRI:
#include <sqlca.h>
#include <db2ApiDf.h>
#include <sqludf.h>
#include <string.h>
#include <stdio.h>
#include <inttypes.h>
#ifdef __plusplus
extern "C"
#endif
SQL_API_RC SQL_API_FN get_initial_lri(SQLUDF_CHAR *lri, SQLUDF_NULLIND
*lri_null_ind, SQLUDF_TRAIL_ARGS)
{
struct sqlca sqlca;
db2ReadLogStruct read_log_params;
db2ReadLogInfoStruct info;
SQL_API_RC rc;
char state_msg[1024], error_msg[1024];
memset(&sqlca, 0, sizeof sqlca);
memset(&read_log_params, 0, sizeof read_log_params);
memset(&info, 0, sizeof info);
read_log_params.iCallerAction = DB2READLOG_QUERY;
read_log_params.iFilterOption = DB2READLOG_FILTER_OFF;
read_log_params.poReadLogInfo = &info;
rc = db2ReadLog(db2Version1058, &read_log_params, &sqlca);
if (rc < 0) {
memcpy(SQLUDF_STATE, "38TA0", SQLUDF_SQLSTATE_LEN);
strncpy(SQLUDF_MSGTX, "Could not query log for initial LRI", SQLUDF_MSGTEXT_LEN);
goto error;
} else if (sqlca.sqlcode < 0) {
strncpy(SQLUDF_STATE, sqlca.sqlstate, SQLUDF_SQLSTATE_LEN);
SQLUDF_MSGTX[0] = '\0';
rc = sqlaintp(error_msg, sizeof error_msg, 80, &sqlca);
if (rc > 0) {
strncpy(SQLUDF_MSGTX, error_msg, SQLUDF_MSGTEXT_LEN);
}
strncat(SQLUDF_MSGTX, "|", SQLUDF_MSGTEXT_LEN);
rc = sqlogstt(state_msg, sizeof state_msg, 80, sqlca.sqlstate);
if (rc > 0) {
strncat(SQLUDF_MSGTX, state_msg, SQLUDF_MSGTEXT_LEN);
}
goto error;
} else {
snprintf(lri, 101, "%" PRIx64 ":%" PRIx64 ":%" PRIx64, info.nextStartLRI.lriType, info.nextStartLRI.part1, info.nextStartLRI.part2);
}
return 0;
error:
return SQLZ_DISCONNECT_PROC;
}
The following SQL is used for registering the C function as an external UDF:
create or replace function get_initial_lri()
returns varchar(100)
language c
external name 'get_initial_lri_0!get_initial_lri'
parameter style sql
fenced not threadsafe
reads sql data
no external action
no scratchpad
no final call
disallow parallel
no dbinfo
And for registering the same code as a stored procedure the SQL is:
create or replace procedure get_initial_lri_sp(out lri varchar(100))
language c
dynamic result sets 0
reads sql data
not deterministic
external name 'get_initial_lri_0!get_initial_lri'
fenced not threadsafe
no external action
program type sub
no dbinfo
parameter style sql
The C source code was compiled with:
gcc -o get_initial_lri_0 get_initial_lri.c -L ~/sqllib/lib64 -ldb2 -shared -fpic -D_REENTRANT -I ~/sqllib/include
The version of DB2 is v10.5.0.8.
Is it at all possible to call the db2ReadLog API from a user defined function?
The documentation states that "connection level APIs cannot be called from within external functions or external methods". And the db2ReadLog API requires a connection. The documentation link is here.
There is another API called db2ReadLogNoconn, maybe you should try that to compare its behaviour, although it may be subject to a different restriction. It is documented here.
Otherwise, a non-external UDF can call a stored procedure (subject to some restrictions) so you may be able to wrap the stored procedure, maybe you can investigate.

The value of strings doesn't appear in eclipse mars CDT

Why does the value of strings in Eclipse Mars CDT not appear in the expression or variables windows?
It appears {...} but i want to see the value itself under the value tab.
How can i do this?
What is going on here is CDT is showing the information that GDB is providing to it.
For a trivial program, with the debugger stopped on the line with return 0;
#include <string>
using namespace std;
int main() {
string mystring = "my string here";
return 0;
}
this is what I see in the CDT variable view:
which matches what I see in GDB:
(gdb) p mystring
$1 = {static npos = <optimised out>,
_M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>},
_M_p = 0x602028 "my string here"}}
Pretty Printing C++
However, what I suspect you want is the pretty printers for libstdc++ which makes the variables view look like this:
Create a ~/.gdbinit file with the following contents (updating the path for your machine)
python
import sys
sys.path.insert(0, '/usr/share/gcc-4.8/python/')
from libstdcxx.v6.printers import register_libstdcxx_printers
register_libstdcxx_printers (None)
end
Then point your launch configuration at that gdbinit file and start debugging.
GDB wiki entry on the subject:
https://sourceware.org/gdb/wiki/STLSupport
GCC manual entry:
https://gcc.gnu.org/onlinedocs/libstdc++/manual/debug.html

Linking LAPACKE and Eclipse in Ubuntu

I am new to C++ and I am using Eclipse to write a script. My OS is Ubuntu. I need to use the LAPACKE package partially for my code. I however cannot manage to link Eclipse and LAPACKE. I am trying to compile the following sample code:
#include <stdio.h>
#include <lapacke.h>
int main (int argc, const char * argv[])
{
double a[5][3] = {1,1,1,2,3,4,3,5,2,4,2,5,5,4,3};
double b[5][2] = {-10,-3,12,14,14,12,16,16,18,16};
lapack_int info,m,n,lda,ldb,nrhs;
int i,j;
m = 5;
n = 3;
nrhs = 2;
lda = 3;
ldb = 2;
info = LAPACKE_dgels(LAPACK_ROW_MAJOR,'N',m,n,nrhs,*a,lda,*b,ldb);
for(i=0;i<n;i++)
{
for(j=0;j<nrhs;j++)
{
printf("%lf ",b[i][j]);
}
printf("\n");
}
return(info);
}
I am unable to compile the code as my Eclipse throws the error: "Udefined reference to LAPACKE_dgels". I have tried to link Eclipse to LAPACKE, for which I have added the path to LAPACKE header files in the "Paths and Symbols" tab of Eclipse. Can anyone help with what I need to do in order to resolve this issue? I should be missing something ...
I assume you are using gcc compiler. I guess you are missing -llapack flag in the compile arguments. If it doesn't work, try -llapacke. This flag (-l[LibraryName]) tells linker to use external binaries (see: gcc: Difference between -L and -l option AND how to provide complete path to a library).
Check out this question to see how to add compiler flags in Eclipse: How to add compiler options in Eclipse IDE

How to hide console window of subprocess?

I'm trying to write a very simple program to replace an existing executable. It should munge its arguments slightly and exec the original program with the new arguments. It's supposed to be invoked automatically and silently by a third-party library.
It runs fine, but it pops up a console window to show the output of the invoked program. I need that console window to not be there. I do not care about the program's output.
My original attempt was set up as a console application, so I thought I could fix this by writing a new Windows GUI app that did the same thing. But it still pops up the console. I assume that the original command is marked as a console application, and so Windows automatically gives it a console window to run in. I also tried replacing my original call to _exec() with a call to system(), just in case. No help.
Does anyone know how I can make this console window go away?
Here's my code:
int APIENTRY _tWinMain(HINSTANCE hInstance,
HINSTANCE hPrevInstance,
char* lpCmdLine,
int nCmdShow)
{
char *argString, *executable;
// argString and executable are retrieved here
std::vector< std::string > newArgs;
// newArgs gets set up with the intended arguments here
char const ** newArgsP = new char const*[newArgs.size() + 1];
for (unsigned int i = 0; i < newArgs.size(); ++i)
{
newArgsP[i] = newArgs[i].c_str();
}
newArgsP[newArgs.size()] = NULL;
int rv = _execv(executable, newArgsP);
if (rv)
{
return -1;
}
}
Use the CreateProcess function instead of execve. For the dwCreationFlags paramter pass the CREATE_NO_WINDOW flag. You will also need to pass the command line as a string as well.
e.g.
STARTUPINFO startInfo = {0};
PROCESS_INFORMATION procInfo;
TCHAR cmdline[] = _T("\"path\\to\\app.exe\" \"arg1\" \"arg2\"");
startInfo.cb = sizeof(startInfo);
if(CreateProcess(_T("path\\to\\app.exe"), cmdline, NULL, NULL, FALSE, CREATE_NO_WINDOW, NULL, NULL, &startInfo, &procInfo))
{
CloseHandle(procInfo.hProcess);
CloseHandle(procInfo.hThread);
}
Aha, I think I found the answer on MSDN, at least if I'm prepared to use .NET. (I don't think I'm really supposed to, but I'll ignore that for now.)
System::String^ command = gcnew System::String(executable);
System::Diagnostics::Process^ myProcess = gcnew Process;
myProcess->StartInfor->FileName = command;
myProcess->StartInfo->UseShellExecute = false; //1
myProcess->StartInfo->CreateNowindow = true; //2
myProcess->Start();
It's those two lines marked //1 and //2 that are important. Both need to be present.
I really don't understand what's going on here, but it seems to work.
You need to create a non-console application (i.e. a Windows GUI app). If all this app does is some processing of files or whatever, you won't need to have a WinMain, register any windows or have a message loop - just write your code as for a console app. Of course, you won't be able to use printf et al. And when you come to execute it, use the exec() family of functions, not system().