Connection pooling issues with unixODBC, IBM DB2, and pyodbc - db2

tldr; Have a working setup of Python->DB2, using unixODBC, but connection pooling is not working.
I've tried using the driver directly:
conn = pyodbc.connect('DRIVER=/opt/ibm/db2/lib/libdb2.so;Hostname=mydb2.domain.com;PORT=1234;DATABASE=database1;UID=user1;PWD=password1')
By driver name:
conn = pyodbc.connect('DRIVER=DB2;Hostname=mydb2.domain.com;PORT=1234;DATABASE=database1;UID=user1;PWD=password1')
DSN:
conn = pyodbc.connect('DSN=server1UID=user1;PWD=password1')
I've been using this Python script to test:
import sys
import timeit
#import pypyodbc as pyodbc
import pyodbc
#conn = pypyodbc.connect('DSN=server1;UID=user1;PWD=password1')
def query():
try:
conn = pyodbc.connect('DSN=server1;UID=user1;PWD=password1')
cur = conn.cursor()
cur.execute("select current timestamp from sysibm.sysdummy1")
row = cur.fetchone()
print('.', end='', flush=True)
conn.close()
except Exception as ex:
print(ex, end='', flush=True)
t = timeit.Timer(query)
count = 50
duration = t.timeit(count)
print("\n{count} # {duration}/sec".format(count=count, duration=count/duration))
(When logging is enabled, in odbcinst.ini, it shows a connection open/close for every query)
In my dev environment the above gets around ~10 query/sec
If the connection is created outside of the loop, and cached, then ~120 query/sec
If I install the ibm_db_dbi and use connect the results are ~10 query/sec, and using pconnect ~60 query/sec.
I've also tried iODBC, but I can't get it to connect at all, and on Windows, using ODBC, connection pooling appears to be working as expected.
Why unixODBC and not ibm_db_dbi?
connect doesn't pool
pconnect fails catastrophically, and never recovers, after network interruptions. It's also half the speed of a cached connection.
pyodbc seems better maintained than the IBM driver.
Setup:
Debian Jessie
Docker 1.7 container
IBM DB2 ODBC v10.5 library installed
unixODBC 2.3.4
Python 3.5
pyodbc 3.0.10
Configuration:
odbcinst.ini
[ODBC]
Pooling = Yes
Trace = No
TraceFile = /var/log/odbc.log
[DB2]
Description = IBM DB2 Adapter
Driver = /opt/ibm/db2/lib/libdb2.so
Driver64 = /opt/ibm/db2/lib/libdb2o.so
FileUsage = 1
DontDLClose = 1
CPTimeout = 1000
CPTimeToLive = 100
Threading = 2
Pooling = Yes
odbc.ini
[server1]
Description = Server 1
Driver = DB2
db2cli.ini
[server1]
Database = database1
Protocol = TCPIP
Hostname = mydb2.domain.com
ServiceName = 1234
References:
Using IBM DB2 with unixODBC (from unixodbc.org)
Setting up unixODBC with a remote DB2
Setting up the unixODBC driver manager (from IBM)
Similar question on SO that is stale, unanswered

It looks like you have Pooling = Yes in the wrong place, it needs to be under the [ODBC] heading, not the driver heading
http://www.unixodbc.org/doc/conn_pool.html

Related

How to configure a ODBC connection to teiid on rhel8? GSSAPI negotiation fails

I have a default Teiid 12.2 installation on RHEL 8.
Now I'm trying to configure an ODBC connection that would be used by PHP.
This always results in an error "[unixODBC]received invalid response to GSSAPI negotiation: R"
This is my ODBC configuration
[TEIID12]
Driver = PostgreSQL
Trace = No
Description = PostgreSQL Data Source
Servername = servername
Port = 35432
Protocol = 7.4-1
UserName = someusername
Password = xxxx
Database = vdb
ReadOnly = no
ServerType = Postgres
ConnSettings =
UseServerSidePrepare=1
Debug=0
Fetch = 10000
A regular isql command also fails with the same information
isql -v TEIID12 someusername xxxx
Results in:
[08001][unixODBC]received invalid response to GSSAPI negotiation: R
[ISQL]ERROR: Could not SQLConnect
Additional information:
The same configuration used to work on a different Linux Distro (Ubuntu) but on this machine, kerberos was not configured. So I assume that this is influincing some sort of "preference".
The standalone-teiid.xml configuration has <ssl mode="disabled" /> for the odbc transport. And yet the GSS API errors occur.
In combination with the later, is it possible that the postgresql ODBC driver is requiring GSS to be used? Is there a setting in odbc.ini where this can be disabled?
On Ubuntu the driver version was 10.01, on RHEL it is 10.03
A solution that worked was to set the environment variable PGGSSENCMODE=disable in /etc/profile.

trying to connect to an azure SQL database with python / pyodbc and a raspberry pi

with following code I try to connect to an azure database :
server = 'tcp:myserver.database.windows.net'
database = 'DBFREE'
username = 'user'
password = 'password'
conn = pyodbc.connect('DRIVER={SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
but getting following error :
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'SQL Server' : file not found (0) (SQLDriverConnect)")
I used following commands to install pyodbc :
sudo apt-get install python-pyodbc
sudo apt-get install unixodbc-dev
pip3 install pyodbc
thanx in advance
As mentioned in my comment I recently walked this path and could never get the drivers from Microsoft to work on my Rasberry PIs. I speculate it has something to do with the ARM architecture. So even though Rasberry PI OS is based on Debian and Microsoft supplies the drivers, I could never get them to work.
I ended up using FreeTDS
First, install using the following commands:
sudo apt-get install freetds-dev freetds-bin unixodbc-dev tdsodbc
sudo pip install pyodbc sqlalchemy
When that is done, open the following file in a text editor:
/etc/odbcinst.ini
Add the following:
[FreeTDS]
Description=FreeTDS Driver
Driver=/usr/lib/arm-linux-gnueabihf/odbc/libtdsodbc.so
Setup=/usr/lib/arm-linux-gnueabihf/odbc/libtdsS.so
Then in the python connection string use FreeTDS as the driver and add ;TDS_Version=8.0 to the end of the connection string:
server = 'servername.database.windows.net'
database = 'database'
username = 'user'
password = 'password'
driver = 'FreeTDS'
pyodbc.connect('DRIVER='+driver+';SERVER='+server+';PORT=1433;DATABASE='+database+';UID='+username+';PWD='+ password + ';TDS_Version=8.0')
Please ref the pyodbc here: Quickstart: Use Python to query a database in Azure SQL Database or Azure SQL Managed Instance
import pyodbc
server = '<server>.database.windows.net'
database = '<database>'
username = '<username>'
password = '<password>'
driver= '{ODBC Driver 17 for SQL Server}'
with pyodbc.connect('DRIVER='+driver+';SERVER='+server+';PORT=1433;DATABASE='+database+';UID='+username+';PWD='+ password) as conn:
with conn.cursor() as cursor:
cursor.execute("SELECT TOP 20 pc.Name as CategoryName, p.name as ProductName FROM [SalesLT].[ProductCategory] pc JOIN [SalesLT].[Product] p ON pc.productcategoryid = p.productcategoryid")
row = cursor.fetchone()
while row:
print (str(row[0]) + " " + str(row[1]))
row = cursor.fetchone()
You didn't set the ODBC driver version and the server don't have prefix tcp: .

Unavailable database in combo Sphinx + UnixODBC + Firebird

I'm trying to use Sphinx Search on Firebird via UnixODBC. All components separately work fine: Sphinx passes tests with MySQL samples and ODBC connects to firebird's .fdb
But if I try to connect Sphinx to ODBC, I get this indexing error:
ERROR: index 'fb': sql_connect: [unixODBC][ODBC Firebird
Driver]unavailable database (DSN=odbc://:***#:0/).
Here is part of my sphinx.conf:
source src_fb
{
type=odbc
odbc_dsn=Driver=/usr/lib/libOdbcFb.so;Dbq=localhost:/var/lib/firebird/2.5/data/employee.fdb;Uid=SYSDBA;Pwd=
sql_query = \
SELECT id, group_id, UNIX_TIMESTAMP(date_added) AS date_added, title, content \
FROM documents
}
.fdb is the same file ODBC connects to.
Here is the solution:
In sphinx.conf I changed only one string:
odbc_dsn=DSN=employee
So now Sphinx takes all required data from ODBC configuration files.
Database from odbc.ini:
[employee]
Description = Firebird
Driver = Firebird
Dbname = localhost:/var/lib/firebird/2.5/data/employee.fdb
User = SYSDBA
Password =
Role =
CharacterSet =
ReadOnly = No
NoWait = No
And the driver configs from odbcinst.ini:
[Firebird]
Description = Firebird ODBC Driver
Driver = /usr/lib/libOdbcFb.so
Threading = 1
FileUsage = 1
CPTimeout =
CPReuse =

CentOS 6.7 and Windows MSSQL connection - FreeTDS and PHP

I searched for answers related to my question but not found what I'm looking for.
I have server with Linux CentOS 6.7. I need to connect to MS SQL Server 2005 on Windows Small Business Server 2003. In CentOS I installed FreeTDS and I managed to connect to SQL Server from terminal using this command:
# TDSVER=7.0 tsql -H ServerIPAdress -p 1433 -U username -P password
As far as I know this command bypass settings in freetds.conf.
Now I need to connect from PHP script. I'm using PDO extension and I tried this DSN string:
$db = new PDO("dblib:version=7.0;host=ServerIPAdress;dbname=Database;","username","password");
This leads me to first error:
Could not find driver
Ok, I understand that - my PHP 5.3.3 installation doesn't have installed PDO driver dblib.
My question: How can I install pdo_dblib driver in CentOS? Many tutorials and answers suggest to install through this command:
yum install php5-sybase
But that package doesn't exist:
No package php-sybase available.
If I check tsql configuration with command
tsql -C
I get these output:
Version: freetds v0.95
freetds.conf directory: /usr/local/etc
MS db-lib source compatibility: no
Sybase binary compatibility: no
Thread safety: yes
iconv library: yes
TDS version: 5.0
iODBC: no
unixodbc: yes
As I understand I need to configure FreeTDS:
./configure --enable-msdblib --with-pdo-dblib=/usr/local
But how can I configure FreeTDS after instalation? I tried to remove current version but I can't find way to do that. Also I tried to install another version but configuration doesn't change.
Any advice are appreciated.
When I run command # odbcinst -j I can see this output:
unixODBC 2.3.0
DRIVERS ...........: /usr/local/etc/odbcinst.ini
SYSTEM DATA SOURCES: /usr/local/etc/odbc.ini
FILE DATA SOURCES..: /usr/local/etc/ODBCDataSources
USER DATA SOURCES..: /root/.odbc.ini
SQLULEN Size.......: 8
SQLSETPOSIROW Size.: 8
Content of my freetds.conf:
# $Id: freetds.conf,v 1.12 2007/12/25 06:02:36 jklowden Exp $
#
# This file is installed by FreeTDS if no file by the same
# name is found in the installation directory.
#
# For information about the layout of this file and its settings,
# see the freetds.conf manpage "man freetds.conf".
# Global settings are overridden by those in a database
# server specific section
[global]
# TDS protocol version
tds version = 7.0
# Whether to write a TDSDUMP file for diagnostic purposes
# (setting this to /tmp is insecure on a multi-user system)
; dump file = /tmp/freetds.log
; debug flags = 0xffff
# Command and connection timeouts
; timeout = 10
; connect timeout = 10
# If you get out-of-memory errors, it may mean that your client
# is trying to allocate a huge buffer for a TEXT field.
# Try setting 'text size' to a more reasonable limit
text size = 64512
# A typical Sybase server
[egServer50]
host = symachine.domain.com
port = 5000
tds version = 7.0
# A typical Microsoft server
[MSServer]
host = 192.168.1.55
port = 1433
tds version = 7.0
odbcinst.ini:
[FreeTDS]
Description = FreeTDS v7.0
Driver = /usr/local/lib/libtdsodbc.so
UsageCount = 1
odbc.ini:
[MSSQL]
Driver = FreeTDS
Description = MSSQL Server Connection
TDS_Version = 7.0
Trace = No
Server = MSServer
Port = 1433
Database = MyDatabase
Username = DOMAIN\MyUsername
Password = MyPassword
I can make connection with isql command when I specify username and password:
# isql -v MSSQL "DOMAIN\MyUsername" MyPassword
But if I omit username and password isql generate error:
[S1000][unixODBC][FreeTDS][SQL Server]Unable to connect to data source
[01000][unixODBC][FreeTDS][SQL Server]Adaptive Server connection failed
I suspect that my username make problems because I have to use DOMAIN name followed by backslash before username when I connect to SQL Server 2005.
UPDATE:
I succesfully made connection in PHP and now I can run queries. I added this line on top of my script: putenv("FREETDSCONF=/usr/local/etc/freetds.conf")
Why is this necessary?

connecting to DB2 database:[unixODBC][Driver Manager]Driver's SQLAllocHandle on SQL_HANDLE_HENV failed

odbc.ini:
[DEFAULT]
Driver = DB2
[abc]
Driver = DB2
[dsn_test1]
DESCRIPTION = Connection to DB2
Driver = db2
odbcinst.ini:
[DB2]
Description = DB2 Driver
Driver = /home/user/sqllib/lib/libdb2.so
fileusage=1
dontdlclose=1
[ODBC]
Trace=1
TraceFile=/home/user/sqllib/trace.out
db2cli.ini
[abc]
hostname="hostname"
pwd="passwd"
port="port"
PROTOCOL=TCPIP
database="dbname"
uid="uid"
$ ./isql abc
+---------------------------------------+
| Connected! |
| |
| sql-statement |
| help [tablename] |
| quit |
| |
+---------------------------------------+
while connecting from db2 driver,below error is coming:
Connection attempt for data source name "abc":
===============================================================================
ODBC Driver Manager Path: /home/user/sqllib/odbclib/lib/libodbc.so
[FAILED]: [unixODBC][Driver Manager]Driver's SQLAllocHandle on SQL_HANDLE_HENV
failed
Below is the snippet of odbc trace:
[ODBC][23419][1403783774.660159][SQLConnect.c][1380]Error: IM004
[ODBC][23419][1403783774.660223][SQLError.c][434]
Entry:
Connection = 0x81aaac8
SQLState = 0xffff9593
Native = 0xffff9684
Message Text = 0xffff8d93
Buffer Length = 1024
Text Len Ptr = 0xffff95bc
[ODBC][23419][1403783774.660260][SQLError.c][471]
Exit:[SQL_SUCCESS]
SQLState = IM004
Native = 0xffff9684 -> 0
Message Text = [[unixODBC][Driver Manager]Driver's SQLAllocHandle on SQL_HANDLE_HENV failed]
Googled a lot for root-cause,dint helped much,please provide some pointer to solve this.
its a 32 bit linux machine having 32 bit db driver as well.
According to this IBM Support page, an IM004 SQLState on SQLAllocHandle relates to the new security feature.
Cause
The new security features introduced in DB2® Universal Database™ (DB2
UDB) Version 8.2 prevent users from using the database unless they
belong to the Windows® groups DB2ADMNS or DB2USERS.
Answer
Add the userid (the one used to execute the application) to either the
DB2ADMNS or DB2USERS group. Please refer to the link under "Related
Information" (below) for instructions on how to accomplish this.
Alternatively, there are a number of threads (e.g. Huge problems connecting to a DB2 database) which suggest setting the DB2INSTANCE environment variable to match the instance setting in your odbc.ini file for the DSN concerned, e.g.
export DB2INSTANCE=db2inst1
isql -v FS01DB2