I'm trying to use Sphinx Search on Firebird via UnixODBC. All components separately work fine: Sphinx passes tests with MySQL samples and ODBC connects to firebird's .fdb
But if I try to connect Sphinx to ODBC, I get this indexing error:
ERROR: index 'fb': sql_connect: [unixODBC][ODBC Firebird
Driver]unavailable database (DSN=odbc://:***#:0/).
Here is part of my sphinx.conf:
source src_fb
{
type=odbc
odbc_dsn=Driver=/usr/lib/libOdbcFb.so;Dbq=localhost:/var/lib/firebird/2.5/data/employee.fdb;Uid=SYSDBA;Pwd=
sql_query = \
SELECT id, group_id, UNIX_TIMESTAMP(date_added) AS date_added, title, content \
FROM documents
}
.fdb is the same file ODBC connects to.
Here is the solution:
In sphinx.conf I changed only one string:
odbc_dsn=DSN=employee
So now Sphinx takes all required data from ODBC configuration files.
Database from odbc.ini:
[employee]
Description = Firebird
Driver = Firebird
Dbname = localhost:/var/lib/firebird/2.5/data/employee.fdb
User = SYSDBA
Password =
Role =
CharacterSet =
ReadOnly = No
NoWait = No
And the driver configs from odbcinst.ini:
[Firebird]
Description = Firebird ODBC Driver
Driver = /usr/lib/libOdbcFb.so
Threading = 1
FileUsage = 1
CPTimeout =
CPReuse =
Related
I am trying to query an Oracle database from within an Azure Synapse notebook, preferably using Pyodbc but a pyspark solution would also be acceptable. The complexity here comes from, I believe, the low configurability of the spark pool - I believe the code is generally correct.
host = 'my_endpoint.com:[port here as plain numbers, e.g. 1111]/orcl'
database = 'my_db_name'
username = 'my_username'
password = 'my_password'
conn = pyodbc.connect( 'DRIVER={ODBC Driver 17 for SQL Server};'
'SERVER=' + host + ';'
'DATABASE=' + database + ';'
'UID=' + username + ';'
'PWD=' + password + ';')
Approaches I have tried:
Pyodbc - I can use the default driver available ({ODBC Driver 17 for SQL Server}) and I get login timeouts. I have tried both the normal URL of the server and the IP, and with all combinations of port, no port, comma port, colon port, and without the service name "orcl" appended. Code sample is above, but I believe the issue lies with the drivers.
Pyspark .read - With no JDBC driver specified, I get a "No suitable driver" error. I am able to add the OJDBC .jar to the workspace or to my file directory, but I was not able to figure out how to tell spark that it should be used.
cx_oracle - This is not permitted in my workspace.
If the solution requires setting environment variables or using spark-submit, please provide a link that explains how best to do that in Synapse. I would be happy with either a JDBC or an ODBC solution.
By adding the .jar here (ojdbc8-19.15.0.0.1.jar) to the Synapse workspace packages and then adding that package to the Apache spark pool packages, I was able to execute the following code:
host = 'my_host_url'
port = 1521
service_name = 'my_service_name'
jdbcUrl = f'jdbc:oracle:thin:#{host}:{port}:{service_name}'
sql = 'SELECT * FROM my_table'
user = 'my_username'
password = 'my_password'
jdbcDriver = 'oracle.jdbc.driver.OracleDriver'
jdbcDF = spark.read.format('jdbc') \
.option('url', jdbcUrl) \
.option('query', sql) \
.option('user', user) \
.option('password', password) \
.option('driver', jdbcDriver) \
.load()
display(jdbcDF)
I'm trying to connect oracle to postgreSQL via db link. Is there any option to disable AutoCommit in Postgresql ODBC?
odbc.ini looks like
[PG]
Description = PG
Driver = /usr/lib64/psqlodbc.so
ServerName = xxxxx
Username = authenticator
Password = xxxx
Port = 5432
Database = master
[Default]
Driver = /usr/lib64/libodbcpsqlS.so
I tried autocommit=false and autocommit=off but it did not work.
tldr; Have a working setup of Python->DB2, using unixODBC, but connection pooling is not working.
I've tried using the driver directly:
conn = pyodbc.connect('DRIVER=/opt/ibm/db2/lib/libdb2.so;Hostname=mydb2.domain.com;PORT=1234;DATABASE=database1;UID=user1;PWD=password1')
By driver name:
conn = pyodbc.connect('DRIVER=DB2;Hostname=mydb2.domain.com;PORT=1234;DATABASE=database1;UID=user1;PWD=password1')
DSN:
conn = pyodbc.connect('DSN=server1UID=user1;PWD=password1')
I've been using this Python script to test:
import sys
import timeit
#import pypyodbc as pyodbc
import pyodbc
#conn = pypyodbc.connect('DSN=server1;UID=user1;PWD=password1')
def query():
try:
conn = pyodbc.connect('DSN=server1;UID=user1;PWD=password1')
cur = conn.cursor()
cur.execute("select current timestamp from sysibm.sysdummy1")
row = cur.fetchone()
print('.', end='', flush=True)
conn.close()
except Exception as ex:
print(ex, end='', flush=True)
t = timeit.Timer(query)
count = 50
duration = t.timeit(count)
print("\n{count} # {duration}/sec".format(count=count, duration=count/duration))
(When logging is enabled, in odbcinst.ini, it shows a connection open/close for every query)
In my dev environment the above gets around ~10 query/sec
If the connection is created outside of the loop, and cached, then ~120 query/sec
If I install the ibm_db_dbi and use connect the results are ~10 query/sec, and using pconnect ~60 query/sec.
I've also tried iODBC, but I can't get it to connect at all, and on Windows, using ODBC, connection pooling appears to be working as expected.
Why unixODBC and not ibm_db_dbi?
connect doesn't pool
pconnect fails catastrophically, and never recovers, after network interruptions. It's also half the speed of a cached connection.
pyodbc seems better maintained than the IBM driver.
Setup:
Debian Jessie
Docker 1.7 container
IBM DB2 ODBC v10.5 library installed
unixODBC 2.3.4
Python 3.5
pyodbc 3.0.10
Configuration:
odbcinst.ini
[ODBC]
Pooling = Yes
Trace = No
TraceFile = /var/log/odbc.log
[DB2]
Description = IBM DB2 Adapter
Driver = /opt/ibm/db2/lib/libdb2.so
Driver64 = /opt/ibm/db2/lib/libdb2o.so
FileUsage = 1
DontDLClose = 1
CPTimeout = 1000
CPTimeToLive = 100
Threading = 2
Pooling = Yes
odbc.ini
[server1]
Description = Server 1
Driver = DB2
db2cli.ini
[server1]
Database = database1
Protocol = TCPIP
Hostname = mydb2.domain.com
ServiceName = 1234
References:
Using IBM DB2 with unixODBC (from unixodbc.org)
Setting up unixODBC with a remote DB2
Setting up the unixODBC driver manager (from IBM)
Similar question on SO that is stale, unanswered
It looks like you have Pooling = Yes in the wrong place, it needs to be under the [ODBC] heading, not the driver heading
http://www.unixodbc.org/doc/conn_pool.html
I like to use connect engine of mariadb to connect to a firebird database via ODBC on a server which is running on Centos 7.
I have already established a connection to sqlserver. The odbc-test to the firebird-database with isql works as well.
This is my create-statement:
CREATE TABLE con.test_table_apys
ENGINE=CONNECT
TABLE_TYPE=ODBC
TABNAME='wsk_lager_st'
CONNECTION='DSN=apys;UID=SYSDBA;PWD=myownpassword';
After sending the statement to the server I got this error message:
ERROR 2006 (HY000): MySQL server has gone away
This is the content of odbc.ini
[apys]
Description = Firebird
Driver = Firebird
Dbname = apysdbserver/3051:vm_apys_ori205
Role =
CharacterSet = WIN1252
ReadOnly = No
NoWait = No
Any ideas? Thank you.
Now it works. The modifications are:
I have put username and password in odbc.ini
[apys]
Description = Firebird
Driver = Firebird
Dbname = apysdbserver/3051:vm_apys_ori205
Role =
CharacterSet = WIN1252
ReadOnly = No
NoWait = No
User = SYSDBA
Password = myownpassword
removed user and password from create-statement
CREATE TABLE con.test_table_apys
ENGINE=CONNECT
TABLE_TYPE=ODBC
TABNAME='wsk_lager_st'
CONNECTION='DSN=apys';
and defined the columns
CREATE TABLE con.test_table_apys (
ID INT,
some_text VARCHAR(100)
)
ENGINE=CONNECT
TABLE_TYPE=ODBC
TABNAME='wsk_lager_st'
CONNECTION='DSN=apys';
odbc.ini:
[DEFAULT]
Driver = DB2
[abc]
Driver = DB2
[dsn_test1]
DESCRIPTION = Connection to DB2
Driver = db2
odbcinst.ini:
[DB2]
Description = DB2 Driver
Driver = /home/user/sqllib/lib/libdb2.so
fileusage=1
dontdlclose=1
[ODBC]
Trace=1
TraceFile=/home/user/sqllib/trace.out
db2cli.ini
[abc]
hostname="hostname"
pwd="passwd"
port="port"
PROTOCOL=TCPIP
database="dbname"
uid="uid"
$ ./isql abc
+---------------------------------------+
| Connected! |
| |
| sql-statement |
| help [tablename] |
| quit |
| |
+---------------------------------------+
while connecting from db2 driver,below error is coming:
Connection attempt for data source name "abc":
===============================================================================
ODBC Driver Manager Path: /home/user/sqllib/odbclib/lib/libodbc.so
[FAILED]: [unixODBC][Driver Manager]Driver's SQLAllocHandle on SQL_HANDLE_HENV
failed
Below is the snippet of odbc trace:
[ODBC][23419][1403783774.660159][SQLConnect.c][1380]Error: IM004
[ODBC][23419][1403783774.660223][SQLError.c][434]
Entry:
Connection = 0x81aaac8
SQLState = 0xffff9593
Native = 0xffff9684
Message Text = 0xffff8d93
Buffer Length = 1024
Text Len Ptr = 0xffff95bc
[ODBC][23419][1403783774.660260][SQLError.c][471]
Exit:[SQL_SUCCESS]
SQLState = IM004
Native = 0xffff9684 -> 0
Message Text = [[unixODBC][Driver Manager]Driver's SQLAllocHandle on SQL_HANDLE_HENV failed]
Googled a lot for root-cause,dint helped much,please provide some pointer to solve this.
its a 32 bit linux machine having 32 bit db driver as well.
According to this IBM Support page, an IM004 SQLState on SQLAllocHandle relates to the new security feature.
Cause
The new security features introduced in DB2® Universal Database™ (DB2
UDB) Version 8.2 prevent users from using the database unless they
belong to the Windows® groups DB2ADMNS or DB2USERS.
Answer
Add the userid (the one used to execute the application) to either the
DB2ADMNS or DB2USERS group. Please refer to the link under "Related
Information" (below) for instructions on how to accomplish this.
Alternatively, there are a number of threads (e.g. Huge problems connecting to a DB2 database) which suggest setting the DB2INSTANCE environment variable to match the instance setting in your odbc.ini file for the DSN concerned, e.g.
export DB2INSTANCE=db2inst1
isql -v FS01DB2