Play 2.5 framework and SQL Server 2008 R2 connection - sql-server-2008-r2

I'm very noob at Java and Play 2.5 framework but I'd like to connect to a SQL Server 2008 R2 database via VPN.
I downloaded the file jtds-1.3.1-dist.zip, I extracted it and copied into the libexec folder.
I inserted this line in build.sbt file:
val appDependencies = Seq(
"net.sourceforge.jtds" % "jtds" % "1.3.1"
)
This is the application.conf file:
## Database Connection Pool
# https://www.playframework.com/documentation/latest/SettingsJDBC
# ~~~~~
# Play doesn't require a JDBC database to run, but you can easily enable one.
#
# libraryDependencies += jdbc
#
play.db {
# The combination of these two settings results in "db.default" as the
# default JDBC pool:
config = "db"
default = "default"
# Play uses HikariCP as the default connection pool. You can override
# settings by changing the prototype:
prototype {
# Sets a fixed JDBC connection pool size of 50
hikaricp.minimumIdle = 50
hikaricp.maximumPoolSize = 50
}
}
## JDBC Datasource
# https://www.playframework.com/documentation/latest/JavaDatabase
# https://www.playframework.com/documentation/latest/ScalaDatabase
# ~~~~~
# Once JDBC datasource is set up, you can work with several different
# database options:
#
# Slick (Scala preferred option): https://www.playframework.com /documentation/latest/PlaySlick
# JPA (Java preferred option): https://playframework.com/documentation/latest/JavaJPA
# EBean: https://playframework.com/documentation/latest/JavaEbean
# Anorm: https://www.playframework.com/documentation/latest/ScalaAnorm
#
db {
# You can declare as many datasources as you want.
# By convention, the default datasource is named `default`
# https://www.playframework.com/documentation/latest/Developing-with-the-H2-Database
#default.driver = org.h2.Driver
#default.url = "jdbc:h2:mem:play"
#default.username = sa
#default.password = ""
default.url = "jdbc:jtds:sqlserver://xyz.aaa.bbb.global:1433/MyDB;instance=MSSQLSERVER"
default.driver = net.sourceforge.jtds.jdbc.Driver
default.username = username
default.password = "password"
# You can turn on SQL logging for any datasource
# https://www.playframework.com/documentation/latest/Highlights25#Logging-SQL-statements
default.logSql=true
}
Unfortunately, I always get this error:
Cannot connect to database [default]
Could someone help me, please.

At last I used sqljdbc42.jar to the connection.
I didn't add any line to the build.sbt file just the jar into the lib folder and these two lines to the application.conf file:
default.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
default.url="jdbc:sqlserver://xxx.yyy.zzz.global:1433;databaseName=MyDB;user=uname;password=pword;"

Related

Airflow DAGS are running but the tasks are not running/queuing - Error sending Celery task:Timeout

We have Airflow 1.10.3 working with Celery 4.1.1 and Redis as the message broker.
When we bring up the webserver the scheduled DAGs go into running state indefinitely and We cannot see any active tasks in the Flower UI.
In the logs(airflow-start-up-logs) we get the following error :(Error sending Celery task:Timeout)
{"timestamp":"2020-11-11T09:45:58.326682", "hostname":"", "process":"scheduler", "name":"airflow.executors.celery_executor.CeleryExecutor", "level":"ERROR", "message":"Error sending Celery task:Timeout, PID: 16001\nCelery Task ID: ('tutorial', 'print_date', datetime.datetime(2020, 11, 9, 0, 0, tzinfo=<Timezone [UTC]>), 1)\nTraceback (most recent call last):\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/utils\/functional.py\", line 42, in __call__\n return self.__value__\nAttributeError: 'ChannelPromise' object has no attribute '__value__'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/transport\/virtual\/base.py\", line 921, in create_channel\n return self._avail_channels.pop()\nIndexError: pop from empty list\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/airflow\/executors\/celery_executor.py\", line 118, in send_task_to_executor\n result = task.apply_async(args=[command], queue=queue)\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/celery\/app\/task.py\", line 535, in apply_async\n **options\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/celery\/app\/base.py\", line 745, in send_task\n amqp.send_task_message(P, name, message, **options)\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/celery\/app\/amqp.py\", line 552, in send_task_message\n **properties\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/messaging.py\", line 181, in publish\n exchange_name, declare,\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/connection.py\", line 518, in _ensured\n return fun(*args, **kwargs)\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/messaging.py\", line 187, in _publish\n channel = self.channel\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/messaging.py\", line 209, in _get_channel\n channel = self._channel = channel()\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/utils\/functional.py\", line 44, in __call__\n value = self.__value__ = self.__contract__()\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/messaging.py\", line 224, in <lambda>\n channel = ChannelPromise(lambda: connection.default_channel)\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/connection.py\", line 866, in default_channel\n self.ensure_connection(**conn_opts)\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/connection.py\", line 430, in ensure_connection\n callback, timeout=timeout)\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/utils\/functional.py\", line 343, in retry_over_time\n return fun(*args, **kwargs)\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/connection.py\", line 283, in connect\n return self.connection\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/connection.py\", line 837, in connection\n self._connection = self._establish_connection()\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/connection.py\", line 792, in _establish_connection\n conn = self.transport.establish_connection()\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/transport\/virtual\/base.py\", line 941, in establish_connection\n self._avail_channels.append(self.create_channel(self))\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/transport\/virtual\/base.py\", line 923, in create_channel\n channel = self.Channel(connection)\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/kombu\/transport\/redis.py\", line 521, in __init__\n self.client.ping()\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/redis\/client.py\", line 1351, in ping\n return self.execute_command('PING')\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/redis\/client.py\", line 875, in execute_command\n conn = self.connection or pool.get_connection(command_name, **options)\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/redis\/connection.py\", line 1185, in get_connection\n connection.connect()\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/redis\/connection.py\", line 552, in connect\n sock = self._connect()\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/redis\/connection.py\", line 845, in _connect\n sock = super(SSLConnection, self)._connect()\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/redis\/connection.py\", line 579, in _connect\n socket.SOCK_STREAM):\n File \"\/usr\/lib64\/python3.7\/socket.py\", line 748, in getaddrinfo\n for res in _socket.getaddrinfo(host, port, family, type, proto, flags):\n File \"\/usr\/local\/lib\/python3.7\/site-packages\/airflow\/utils\/timeout.py\", line 43, in handle_timeout\n raise AirflowTaskTimeout(self.error_message)\nairflow.exceptions.AirflowTaskTimeout: Timeout, PID: 16001\n\n"}
Config File
[core]
# The home folder for airflow, default is ~/airflow
airflow_home = /home/ec2-user/airflow
# The folder where your airflow pipelines live, most likely a
# subfolder in a code repository
# This path must be absolute
dags_folder = /home/ec2-user/airflow/dags
# The folder where airflow should store its log files
# This path must be absolute
base_log_folder = /var/log/airflow
# Logging level
logging_level = DEBUG
fab_logging_level = WARN
# Logging class
# Specify the class that will specify the logging configuration
# This class has to be on the python classpath
# logging_config_class = my.path.default_local_settings.LOGGING_CONFIG
logging_config_class = log_config.CUSTOM_LOGGING_CONFIG
# Log format
# we need to escape the curly braces by adding an additional curly brace
log_format = [%%(asctime)s] {%%(filename)s:%%(lineno)d} %%(levelname)s - %%(message)s
simple_log_format = %%(asctime)s %%(levelname)s - %%(message)s
# Log filename format
# we need to escape the curly braces by adding an additional curly brace
log_filename_template = {{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ try_number }}.log
log_processor_filename_template = {{ filename }}.log
# Hostname by providing a path to a callable, which will resolve the hostname
hostname_callable = socket:getfqdn
# Default timezone in case supplied date times are naive
# can be utc (default), system, or any IANA timezone string (e.g. Europe/Amsterdam)
default_timezone = utc
# Airflow can store logs remotely in AWS S3 or Google Cloud Storage. Users
# must supply a remote location URL (starting with either 's3://...' or
# 'gs://...') and an Airflow connection id that provides access to the storage
# location.
remote_base_log_folder =
remote_log_conn_id =
# Use server-side encryption for logs stored in S3
encrypt_s3_logs = False
# DEPRECATED option for remote log storage, use remote_base_log_folder instead!
s3_log_folder =
# The executor class that airflow should use. Choices include
# SequentialExecutor, LocalExecutor, CeleryExecutor
executor = CeleryExecutor
broker_url = redis://***************************************:6379/0
# The SqlAlchemy connection string to the metadata database.
# SqlAlchemy supports many different database engine, more information
# their website
sql_alchemy_conn = mysql://***************************************:3306/airflow
# The SqlAlchemy pool size is the maximum number of database connections
# in the pool.
sql_alchemy_pool_size = 5
# The SqlAlchemy pool recycle is the number of seconds a connection
# can be idle in the pool before it is invalidated. This config does
# not apply to sqlite.
sql_alchemy_pool_recycle = 2000
# The amount of parallelism as a setting to the executor. This defines
# the max number of task instances that should run simultaneously
# on this airflow installation
parallelism = 32
# The number of task instances allowed to run concurrently by the scheduler
dag_concurrency = 6
# Are DAGs paused by default at creation
dags_are_paused_at_creation = False
# When not using pools, tasks are run in the "default pool",
# whose size is guided by this config element
non_pooled_task_slot_count = 128
# The maximum number of active DAG runs per DAG
max_active_runs_per_dag = 16
# Whether to load the examples that ship with Airflow. It's good to
# get started, but you probably want to set this to False in a production
# environment
load_examples = False
# Where your Airflow plugins are stored
plugins_folder = /home/ec2-user/airflow/plugins
# Secret key to save connection passwords in the db
fernet_key =
# Whether to disable pickling dags
donot_pickle = False
# How long before timing out a python file import while filling the DagBag
dagbag_import_timeout = 30
# The class to use for running task instances in a subprocess
task_runner = BashTaskRunner
# If set, tasks without a `run_as_user` argument will be run with this user
# Can be used to de-elevate a sudo user running Airflow when executing tasks
default_impersonation =
# What security module to use (for example kerberos):
security =
# Turn unit test mode on (overwrites many configuration options with test
# values at runtime)
unit_test_mode = False
# full path of dag_processor_manager logfile
dag_processor_manager_log_location = /var/log/airflow/dag_processor_manager/dag_processor_manager.log
# Name of handler to read task instance logs.
# Default to use task handler.
task_log_reader = task
# Whether to enable pickling for xcom (note that this is insecure and allows for
# RCE exploits). This will be deprecated in Airflow 2.0 (be forced to False).
enable_xcom_pickling = True
# When a task is killed forcefully, this is the amount of time in seconds that
# it has to cleanup after it is sent a SIGTERM, before it is SIGKILLED
killed_task_cleanup_time = 60
# Whether to override params with dag_run.conf. If you pass some key-value pairs through `airflow backfill -c` or
# `airflow trigger_dag -c`, the key-value pairs will override the existing ones in params.
dag_run_conf_overrides_params = False
[cli]
# In what way should the cli access the API. The LocalClient will use the
# database directly, while the json_client will use the api running on the
# webserver
api_client = airflow.api.client.local_client
##endpoint_url = http://localhost:8080
endpoint_url = 10.136.119.91
[api]
# How to authenticate users of the API
#auth_backend = airflow.api.auth.backend.default
[lineage]
# what lineage backend to use
#backend =
[atlas]
sasl_enabled = False
host =
port = 21000
username =
password =
[operators]
# The default owner assigned to each new operator, unless
# provided explicitly or passed via `default_args`
default_owner = Airflow
default_cpus = 1
default_ram = 512
default_disk = 512
default_gpus = 0
[webserver]
# The base url of your website as airflow cannot guess what domain or
# cname you are using. This is used in automated emails that
# airflow sends to point links to the right web server
#base_url =
# The ip specified when starting the web server
web_server_host = 0.0.0.0
# The port on which to run the web server
web_server_port = 8080
# Paths to the SSL certificate and key for the web server. When both are
# provided SSL will be enabled. This does not change the web server port.
web_server_ssl_cert = /etc/ssl/certs/airflow-selfsigned.crt
web_server_ssl_key = /etc/ssl/private/airflow-selfsigned.key
# Number of seconds the webserver waits before killing gunicorn master that doesn't respond
web_server_master_timeout = 1200
# Number of seconds the gunicorn webserver waits before timing out on a worker
web_server_worker_timeout = 1200
# Number of workers to refresh at a time. When set to 0, worker refresh is
# disabled. When nonzero, airflow periodically refreshes webserver workers by
# bringing up new ones and killing old ones.
worker_refresh_batch_size = 1
# Number of seconds to wait before refreshing a batch of workers.
worker_refresh_interval = 30
# Secret key used to run your flask app
secret_key = temporary_key
# Number of workers to run the Gunicorn web server
workers = 4
# The worker class gunicorn should use. Choices include
# sync (default), eventlet, gevent
worker_class = sync
# Log files for the gunicorn webserver. '-' means log to stderr.
access_logfile = /var/log/airflow/gunicorn-access.log
error_logfile = /var/log/airflow/gunicorn-error.log
# Expose the configuration file in the web server
expose_config = False
# Set to true to turn on authentication:
# http://pythonhosted.org/airflow/security.html#web-authentication
authenticate = True
auth_backend = airflow.contrib.auth.backends.password_auth
# Filter the list of dags by owner name (requires authentication to be enabled)
filter_by_owner = False
# Filtering mode. Choices include user (default) and ldapgroup.
# Ldap group filtering requires using the ldap backend
#
# Note that the ldap server needs the "memberOf" overlay to be set up
# in order to user the ldapgroup mode.
owner_mode = user
# Default DAG orientation. Valid values are:
# LR (Left->Right), TB (Top->Bottom), RL (Right->Left), BT (Bottom->Top)
dag_orientation = LR
# Puts the webserver in demonstration mode; blurs the names of Operators for
# privacy.
demo_mode = False
# The amount of time (in secs) webserver will wait for initial handshake
# while fetching logs from other worker machine
log_fetch_timeout_sec = 5
# By default, the webserver shows paused DAGs. Flip this to hide paused
# DAGs by default
hide_paused_dags_by_default = False
# Consistent page size across all listing views in the UI
page_size = 100
# Use FAB-based webserver with RBAC feature
rbac = True
# Define the color of navigation bar
navbar_color = #007A87
# Default dagrun to show in UI
default_dag_run_display_number = 25
[email]
email_backend = airflow.utils.email.send_email_smtp
[smtp]
# If you want airflow to send emails on retries, failure, and you want to use
# the airflow.utils.email.send_email_smtp function, you have to configure an
# smtp server here
smtp_host = localhost
smtp_starttls = True
smtp_ssl = False
smtp_port = 25
smtp_mail_from =
[celery]
# This section only applies if you are using the CeleryExecutor in
# [core] section above
# The app name that will be used by celery
celery_app_name = airflow.executors.celery_executor
# The concurrency that will be used when starting workers with the
# "airflow worker" command. This defines the number of task instances that
# a worker will take, so size up your workers based on the resources on
# your worker box and the nature of your tasks
worker_concurrency = 16
# When you start an airflow worker, airflow starts a tiny web server
# subprocess to serve the workers local log files to the airflow main
# web server, who then builds pages and sends them to users. This defines
# the port on which the logs are served. It needs to be unused, and open
# visible from the main web server to connect into the workers.
worker_log_server_port = 8793
# The Celery broker URL. Celery supports RabbitMQ, Redis and experimentally
# a sqlalchemy database. Refer to the Celery documentation for more
# information.
broker_url = redis://***************************************:6379/0
celery_result_backend = db+mysql://***************************************:3306/airflow
# Another key Celery setting
result_backend = db+mysql://***************************************:3306/airflow
# Celery Flower is a sweet UI for Celery. Airflow has a shortcut to start
# it `airflow flower`. This defines the IP that Celery Flower runs on
flower_host = 0.0.0.0
# This defines the port that Celery Flower runs on
flower_port = 8443
# Default queue that tasks get assigned to and that worker listen on.
default_queue = default
# Import path for celery configuration options
celery_config_options = airflow.config_templates.default_celery.DEFAULT_CELERY_CONFIG
# In case of using SSL
ssl_active = True
ssl_key = /etc/ssl/private/airflow-selfsigned.key
ssl_cert = /etc/ssl/certs/airflow-selfsigned.crt
ssl_cacert =
[celery_broker_transport_options]
# This section is for specifying options which can be passed to the
# underlying celery broker transport. See:
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-broker_transport_options
# The visibility timeout defines the number of seconds to wait for the worker
# to acknowledge the task before the message is redelivered to another worker.
# Make sure to increase the visibility timeout to match the time of the longest
# ETA you're planning to use.
#
# visibility_timeout is only supported for Redis and SQS celery brokers.
# See:
# http://docs.celeryproject.org/en/master/userguide/configuration.html#std:setting-broker_transport_options
#
#visibility_timeout = 21600
[scheduler]
# Task instances listen for external kill signal (when you clear tasks
# from the CLI or the UI), this defines the frequency at which they should
# listen (in seconds).
job_heartbeat_sec = 5
# The scheduler constantly tries to trigger new tasks (look at the
# scheduler section in the docs for more information). This defines
# how often the scheduler should run (in seconds).
scheduler_heartbeat_sec = 5
# after how much time should the scheduler terminate in seconds
# -1 indicates to run continuously (see also num_runs)
run_duration = -1
# after how much time a new DAGs should be picked up from the filesystem
min_file_process_interval = 0
# How many seconds to wait between file-parsing loops to prevent the logs from being spammed.
min_file_parsing_loop_time = 1
dag_dir_list_interval = 300
# How often should stats be printed to the logs
print_stats_interval = 30
child_process_log_directory = /var/log/airflow/scheduler
# Local task jobs periodically heartbeat to the DB. If the job has
# not heartbeat in this many seconds, the scheduler will mark the
# associated task instance as failed and will re-schedule the task.
scheduler_zombie_task_threshold = 300
# Turn off scheduler catchup by setting this to False.
# Default behavior is unchanged and
# Command Line Backfills still work, but the scheduler
# will not do scheduler catchup if this is False,
# however it can be set on a per DAG basis in the
# DAG definition (catchup)
catchup_by_default = True
# This changes the batch size of queries in the scheduling main loop.
# If this is too high, SQL query performance may be impacted by one
# or more of the following:
# - reversion to full table scan
# - complexity of query predicate
# - excessive locking
#
# Additionally, you may hit the maximum allowable query length for your db.
#
# Set this to 0 for no limit (not advised)
max_tis_per_query = 512
# Statsd (https://github.com/etsy/statsd) integration settings
statsd_on = True
statsd_host = localhost
statsd_port = 8125
statsd_prefix = airflow
# The scheduler can run multiple threads in parallel to schedule dags.
# This defines how many threads will run. However airflow will never
# use more threads than the amount of cpu cores available.
max_threads = 4
authenticate = False
[mesos]
# Mesos master address which MesosExecutor will connect to.
master = localhost:5050
# The framework name which Airflow scheduler will register itself as on mesos
framework_name = Airflow
# Number of cpu cores required for running one task instance using
# 'airflow run <dag_id> <task_id> <execution_date> --local -p <pickle_id>'
# command on a mesos slave
task_cpu = 1
# Memory in MB required for running one task instance using
# 'airflow run <dag_id> <task_id> <execution_date> --local -p <pickle_id>'
# command on a mesos slave
task_memory = 256
# Enable framework checkpointing for mesos
# See http://mesos.apache.org/documentation/latest/slave-recovery/
checkpoint = False
# Failover timeout in milliseconds.
# When checkpointing is enabled and this option is set, Mesos waits
# until the configured timeout for
# the MesosExecutor framework to re-register after a failover. Mesos
# shuts down running tasks if the
# MesosExecutor framework fails to re-register within this timeframe.
# failover_timeout = 604800
# Enable framework authentication for mesos
# See http://mesos.apache.org/documentation/latest/configuration/
authenticate = False
# Mesos credentials, if authentication is enabled
# default_principal = admin
# default_secret = admin
[admin]
# UI to hide sensitive variable fields when set to True
hide_sensitive_variable_fields = True
Could you please help
1-Usually your celery error logs will be available in your scheduler logs, so its better to check it there.
if you are not running it as daemon or background process, you can see it in detail in your terminal that what exactly is the problem.
2-As I gave it(your configs) a quick look, it looks so much like a default config file or a config that is almost match to the install guides online ==> so there must be no big issue with it.
3- Your error is not clear but i believe if you've just shipped your dag and its a sudden error after that there's a high chance you are facing your broker permission errors or any related errors to that as celery will communicate with it. So I will provide a common solution to a common problem hope it help the community, as you are far past the problem as 3 months past :)
rabbitmqctl set_permissions -p /myvhost guest ".*" ".*" ".*"
guest = your user( what you have provided as broker(in this case, RabbitMQ) user)
/myvhost = for you it might be just slash or /
Good luck.

isql and tsql is connecting to sql server but when connecting with perl ODBC it's getting failed

I am trying to run connect to a MSSQL server from a RHEL 5.5 server with FreeTDS and unixODBC.
Using tsql i can connect to the server with
tsql -S mssqltest -U <username> -P <password>
It's getting connected successfully
isql -v mssqltest 'username' 'password' -b -q
Also connects without any problem
But in perl I get a error message as follows
DBI connect('mssqltest',<username>,...) failed: [unixODBC][Driver Manager]Can't open lib '/usr/local/lib/libtdsodbc.so' : file not found (SQL-01000) at test.pl line 14
Can't connect to DBI:ODBC:mssqltest: [unixODBC][Driver Manager]Can't open lib '/usr/local/lib/libtdsodbc.so' : file not found (SQL-01000) at test.pl line 14.
I tried using FreeTDS as ODBC Driver that also gives similar error also I tried using servername instead of server_ip, but the error continues
DBI connect('Driver=FreeTDS;Server=<server_ip>',<username>,...) failed: [unixODBC][Driver Manager]Can't open lib '/usr/local/lib/libtdsodbc.so' : file not found (SQL-01000) at test.pl line 14
Can't connect to DBI:ODBC:Driver=FreeTDS;Server=<server_ip>: [unixODBC][Driver Manager]Can't open lib '/usr/local/lib/libtdsodbc.so' : file not found (SQL-01000) at test.pl line 14.
my perl code
#!/usr/bin/perl -w
use strict;
use DBI;
# Replace datasource_name with the name of your data source.
# Replace database_username and database_password
# with the SQL Server database username and password.
my $data_source = q/DBI:ODBC:mssqltest/;
my $user = q/<username>/;
my $password = q/<password>/;
# Connect to the data source and get a handle for that connection.
my $dbh = DBI->connect($data_source, $user, $password)
or die "Can't connect to $data_source: $DBI::errstr";
# This query generates a result set with one record in it.
my $sql = "SELECT TOP 3 * FROM tablename";
# Prepare the statement.
my $sth = $dbh->prepare($sql)
or die "Can't prepare statement: $DBI::errstr";
# Execute the statement.
$sth->execute();
# Print the column name.
print "$sth->{NAME}->[0]\n";
# Fetch and display the result set value.
while ( my #row = $sth->fetchrow_array ) {
print "#row\n";
}
# Disconnect the database from the database handle.
$dbh->disconnect;
My config files are:
FreTDS/odbc.ini
;
; odbc.ini
;
[ODBC Data Sources]
JDBC = Sybase JDBC Server
[JDBC]
Driver = /usr/local/lib/libtdsodbc.so
Description = Sybase JDBC Server
Trace = No
Servername = JDBC
Database = pubs2
UID = guest
[Default]
Driver = /usr/local/lib/libtdsodbc.so
odbc.ini
[ODBC Data Sources]
TS = FreeTDS
[TS]
Driver = FreeTDS
Description = ODBC to SQLServer via FreeTDS
Trace = No
Servername = sql-server
Database = RKDB
[mssqltest]
Description = MS SQL connection to mssqltest database
Driver = FreeTDS
Database = RKDB
Server = <server_ip>
UserName = <username>
Password = <password>
Trace = Yes
Port = 1754
obcinst.ini
[FreeTDS]
Description=TDS driver (Sybase/MS SQL)
Driver=/usr/local/lib/libtdsodbc.so
UsageCount=2
freetds-dev.0.99.761/freetds.conf
# $Id: freetds.conf,v 1.12 2007-12-25 06:02:36 jklowden Exp $
#
# This file is installed by FreeTDS if no file by the same
# name is found in the installation directory.
#
# For information about the layout of this file and its settings,
# see the freetds.conf manpage "man freetds.conf".
# Global settings are overridden by those in a database
# server specific section
[global]
# TDS protocol version
tds version = auto
# Whether to write a TDSDUMP file for diagnostic purposes
# (setting this to /tmp is insecure on a multi-user system)
; dump file = /tmp/freetds.log
; debug flags = 0xffff
# Command and connection timeouts
; timeout = 10
; connect timeout = 10
# If you get out-of-memory errors, it may mean that your client
# is trying to allocate a huge buffer for a TEXT field.
# Try setting 'text size' to a more reasonable limit
text size = 64512
# A typical Sybase server
[egServer50]
host = symachine.domain.com
port = 5000
tds version = 5.0
# A typical Microsoft server
[egServer70]
host = ntmachine.domain.com
port = 1433
tds version = 7.0
[mssqltest]
host = <server_ip>
port = 1754
tds version = 8.0
/usr/local/etc/freetds.conf
# $Id: freetds.conf,v 1.12 2007-12-25 06:02:36 jklowden Exp $
#
# This file is installed by FreeTDS if no file by the same
# name is found in the installation directory.
#
# For information about the layout of this file and its settings,
# see the freetds.conf manpage "man freetds.conf".
# Global settings are overridden by those in a database
# server specific section
[global]
# TDS protocol version
tds version = auto
# Whether to write a TDSDUMP file for diagnostic purposes
# (setting this to /tmp is insecure on a multi-user system)
; dump file = /tmp/freetds.log
; debug flags = 0xffff
# Command and connection timeouts
; timeout = 10
; connect timeout = 10
# If you get out-of-memory errors, it may mean that your client
# is trying to allocate a huge buffer for a TEXT field.
# Try setting 'text size' to a more reasonable limit
text size = 64512
# A typical Sybase server
[egServer50]
host = symachine.domain.com
port = 5000
tds version = 5.0
# A typical Microsoft server
[sql-server]
host = TH-SSRS-DB
InstanceName = RKSSRSDB
#port = 1754
tds version = 8.0
client charset = UTF-8
[mssqltest]
host = <server_ip>
port = 1754
tds version = 8.0
Please help.

Pick the right config file

I have created two files in my src/main/resources folder
application-dev.conf
which contains
dev {
oracle {
host = "foo"
}
}
and
application-qa.conf
which contains
qa {
oracle {
host = "bar"
}
}
I read this configuration with the following code
val env = args.lift(0).getOrElse("dev")
val parsedConfig = ConfigFactory.parseFile(new File(s"src/main/resources/application-${env}.conf"))
val conf = ConfigFactory.load(parsedConfig)
val config = conf.getConfig(env)
println(config.getString("oracle.host"))
Everything works great at development time and I am able to read the right configuration file based on the environment specified. If I don't specify anything then development is chosen.
However now when I package my jar file as an assembly using sbt assembly and now try to run my application from the command line java -jar ./target/MyApplication.jar
I get an error
Exception in thread "main" com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'dev'
at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:152)
at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:170)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:184)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:189)
My objective is that I should have multiple configuration files and I should be able to choose between them at dev time and also when the application is packaged as a jar.
Based on the suggestion below I modified my code to
val env = args.lift(0).getOrElse("dev")
val parsedConfig = ConfigFactory.parseFile(new File(getClass.getResource(s"/application-${env}.conf").toURI))
val conf = ConfigFactory.load(parsedConfig)
val config = conf.getConfig(env)
println(config.getString("oracle.host"))
works in dev, but when I try to run my assembly it throws an exception
Exception in thread "main" java.lang.IllegalArgumentException: URI is not hierarchical
at java.io.File.<init>(File.java:418)
The config file is not in src/main/resources in your assembly. It usually is at the root of the classpath (unless you have configured the plugin to package it somewhere else).
Try using like below.
ConfigFactory.load(s"application-${env}.conf")
This loads the config file from the classpath.

ORA-12505, TNS:listener does not currently know of SID given in connect descriptor using oracle sql developer

i have gone through the forum to look for a solution for hours on end, i haven't encountered anyone with the same predicament as i have. I have installed Oracle 12c 64 bit on windows 8.1 64 bit system. When i try to make a new connection to the database, using oracle sql developer, run into this error, being a newbie to oracle database, i did try to run lsnrctl start i got
LSNRCTL for 64-bit Windows: Version 12.1.0.2.0 - Production on 18-OCT-2015 19:00
:45
Copyright (c) 1991, 2014, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1539)))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
64-bit Windows Error: 61: Unknown error
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521_1)))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
64-bit Windows Error: 2: No such file or directory
I tried to make sense of it but i coudn't.
Here is my listener.ora
# copyright (c) 1997 by the Oracle Corporation
#
# NAME
# listener.ora
# FUNCTION
# Network Listener startup parameter file example
# NOTES
# This file contains all the parameters for listener.ora,
# and could be used to configure the listener by uncommenting
# and changing values. Multiple listeners can be configured
# in one listener.ora, so listener.ora parameters take the form
# of SID_LIST_<lsnr>, where <lsnr> is the name of the listener
# this parameter refers to. All parameters and values are
# case-insensitive.
# <lsnr>
# This parameter specifies both the name of the listener, and
# it listening address(es). Other parameters for this listener
# us this name in place of <lsnr>. When not specified,
# the name for <lsnr> defaults to "LISTENER", with the default
# address value as shown below.
#
# LISTENER =
# (ADDRESS_LIST=
# (ADDRESS=(PROTOCOL=tcp)(HOST=localhost(PORT=1521))
# (ADDRESS=(PROTOCOL=ipc)(KEY=PNPKEY)))
# SID_LIST_<lsnr>
# List of services the listener knows about and can connect
# clients to. There is no default. See the Net8 Administrator's
# Guide for more information.
#
# SID_LIST_LISTENER=
# (SID_LIST=
# (SID_DESC=
#BEQUEATH CONFIG
# (GLOBAL_DBNAME=salesdb.mycompany)
# (SID_NAME=sid1)
# (ORACLE_HOME=/private/app/oracle/product/8.0.3)
# #PRESPAWN CONFIG
# (PRESPAWN_MAX=20)
# (PRESPAWN_LIST=
# (PRESPAWN_DESC=(PROTOCOL=tcp)(POOL_SIZE=2)(TIMEOUT=1))
# )
# )
# )
# PASSWORDS_<lsnr>
# Specifies a password to authenticate stopping the listener.
# Both encrypted and plain-text values can be set. Encrypted passwords
# can be set and stored using lsnrctl.
# LSNRCTL> change_password
# Will prompt for old and new passwords, and use encryption both
# to match the old password and to set the new one.
# LSNRCTL> set password
# Will prompt for the new password, for authentication with
# the listener. The password must be set before running the next
# command.
# LSNRCTL> save_config
# Will save the changed password to listener.ora. These last two
# steps are not necessary if SAVE_CONFIG_ON_STOP_<lsnr> is ON.
# See below.
#
# Default: NONE
#
# PASSWORDS_LISTENER = 20A22647832FB454 # "foobar"
# SAVE_CONFIG_ON_STOP_<lsnr>
# Tells the listener to save configuration changes to listener.ora when
# it shuts down. Changed parameter values will be written to the file,
# while preserving formatting and comments.
# Default: OFF
# Values: ON/OFF
#
# SAVE_CONFIG_ON_STOP_LISTENER = ON
# USE_PLUG_AND_PLAY_<lsnr>
# Tells the listener to contact an Onames server and register itself
# and its services with Onames.
# Values: ON/OFF
# Default: OFF
#
# USE_PLUG_AND_PLAY_LISTENER = ON
# LOG_FILE_<lsnr>
# Sets the name of the listener's log file. The .log extension
# is added automatically.
# Default=<lsnr>
#
# LOG_FILE_LISTENER = lsnr
# LOG_DIRECTORY_<lsnr>
# Sets the directory for the listener's log file.
# Default: <oracle_home>/network/log
#
# LOG_DIRECTORY_LISTENER = /private/app/oracle/product/8.0.3/network/log
# TRACE_LEVEL_<lsnr>
# Specifies desired tracing level.
# Default: OFF
# Values: OFF/USER/ADMIN/SUPPORT/0-16
#
# TRACE_LEVEL_LISTENER = SUPPORT
# TRACE_FILE_<lsnr>
# Sets the name of the listener's trace file. The .trc extension
# is added automatically.
# Default: <lsnr>
#
# TRACE_FILE_LISTENER = lsnr
# TRACE_DIRECTORY_<lsnr>
# Sets the directory for the listener's trace file.
# Default: <oracle_home>/network/trace
#
# TRACE_DIRECTORY_LISTENER=/private/app/oracle/product/8.0.3/network/trace
# CONNECT_TIMEOUT_<lsnr>
# Sets the number of seconds that the listener waits to get a
# valid database query after it has been started.
# Default: 10
#
# CONNECT_TIMEOUT_LISTENER=10
and my tnsnames.ora
# This file contains the syntax information for
# the entries to be put in any tnsnames.ora file
# The entries in this file are need based.
# There are no defaults for entries in this file
# that Sqlnet/Net3 use that need to be overridden
#
# Typically you could have two tnsnames.ora files
# in the system, one that is set for the entire system
# and is called the system tnsnames.ora file, and a
# second file that is used by each user locally so that
# he can override the definitions dictated by the system
# tnsnames.ora file.
# The entries in tnsnames.ora are an alternative to using
# the names server with the onames adapter.
# They are a collection of aliases for the addresses that
# the listener(s) is(are) listening for a database or
# several databases.
# The following is the general syntax for any entry in
# a tnsnames.ora file. There could be several such entries
# tailored to the user's needs.
<alias>= [ (DESCRIPTION_LIST = # Optional depending on whether u have
# one or more descriptions
# If there is just one description, unnecessary ]
(DESCRIPTION=
[ (SDU=2048) ] # Optional, defaults to 2048
# Can take values between 512 and 32K
[ (ADDRESS_LIST= # Optional depending on whether u have
# one or more addresses
# If there is just one address, unnecessary ]
(ADDRESS=
[ (COMMUNITY=<community_name>) ]
(PROTOCOL=tcp)
(HOST=<hostname>)
(PORT=<portnumber (1521 is a standard port used)>)
)
[ (ADDRESS=
(PROTOCOL=ipc)
(KEY=<ipckey (PNPKEY is a standard key used)>)
)
]
[ (ADDRESS=
[ (COMMUNITY=<community_name>) ]
(PROTOCOL=decnet)
(NODE=<nodename>)
(OBJECT=<objectname>)
)
]
... # More addresses
[ ) ] # Optional depending on whether ADDRESS_LIST is used or not
[ (CONNECT_DATA=
(SID=<oracle_sid>)
[ (GLOBAL_NAME=<global_database_name>) ]
)
]
[ (SOURCE_ROUTE=yes) ]
)
(DESCRIPTION=
[ (SDU=2048) ] # Optional, defaults to 2048
# Can take values between 512 and 32K
[ (ADDRESS_LIST= ] # Optional depending on whether u have more
# than one address or not
# If there is just one address, unnecessary
(ADDRESS
[ (COMMUNITY=<community_name>) ]
(PROTOCOL=tcp)
(HOST=<hostname>)
(PORT=<portnumber (1521 is a standard port used)>)
)
[ (ADDRESS=
(PROTOCOL=ipc)
(KEY=<ipckey (PNPKEY is a standard key used)>)
)
]
... # More addresses
[ ) ] # Optional depending on whether ADDRESS_LIST
# is being used
[ (CONNECT_DATA=
(SID=<oracle_sid>)
[ (GLOBAL_NAME=<global_database_name>) ]
)
]
[ (SOURCE_ROUTE=yes) ]
)
[ (CONNECT_DATA=
(SID=<oracle_sid>)
[ (GLOBAL_NAME=<global_database_name>) ]
)
]
... # More descriptions
[ ) ] # Optional depending on whether DESCRIPTION_LIST is used or not
am sorry for the long post, i just dint want to exclude anything out, is there a config that am missing?

Problems with unixODBC and FreeTDS config

I have been working on this for way too long and can't seem to figure it out. I am sure I have something wrong in my freetds.conf, odbc.ini or odbcinst.ini. I can connect to my mssql 2008 server using tsql, but still can't with isql or of course through php.
I am on CentOS 5.6.
Can anyone offer some assistance?
Thanks!
Shawn
This is in my sqltrace.log:
[ODBC][12249][1347850711.939084][__handles.c][459]
Exit:[SQL_SUCCESS]
Environment = 0x1b5fc6c0
[ODBC][12249][1347850711.939149][SQLAllocHandle.c][375]
Entry:
Handle Type = 2
Input Handle = 0x1b5fc6c0
[ODBC][12249][1347850711.939187][SQLAllocHandle.c][493]
Exit:[SQL_SUCCESS]
Output Handle = 0x1b5fcff0
[ODBC][12249][1347850711.939231][SQLConnect.c][3654]
Entry:
Connection = 0x1b5fcff0
Server Name = [MSSQL_DSN][length = 9 (SQL_NTS)]
User Name = [InetIndyArtsRemote][length = 18 (SQL_NTS)]
Authentication = [**********][length = 10 (SQL_NTS)]
UNICODE Using encoding ASCII 'ISO8859-1' and UNICODE 'UCS-2LE'
DIAG [01000] [FreeTDS][SQL Server]Unexpected EOF from the server
DIAG [01000] [FreeTDS][SQL Server]Adaptive Server connection failed
DIAG [S1000] [FreeTDS][SQL Server]Unable to connect to data source
[ODBC][12249][1347850711.949640][SQLConnect.c][4021]
Exit:[SQL_ERROR]
[ODBC][12249][1347850711.949694][SQLFreeHandle.c][286]
Entry:
Handle Type = 2
Input Handle = 0x1b5fcff0
[ODBC][12249][1347850711.949735][SQLFreeHandle.c][337]
Exit:[SQL_SUCCESS]
[ODBC][12249][1347850711.949773][SQLFreeHandle.c][219]
Entry:
Handle Type = 1
Input Handle = 0x1b5fc6c0
freetds.conf:
# $Id: freetds.conf,v 1.12 2007/12/25 06:02:36 jklowden Exp $
#
# This file is installed by FreeTDS if no file by the same
# name is found in the installation directory.
#
# For information about the layout of this file and its settings,
# see the freetds.conf manpage "man freetds.conf".
# Global settings are overridden by those in a database
# server specific section
[global]
# TDS protocol version
tds version = 8.0
# Whether to write a TDSDUMP file for diagnostic purposes
# (setting this to /tmp is insecure on a multi-user system)
dump file = /tmp/freetds.log
debug flags = 0xffff
dump file append = yes
# Command and connection timeouts
; timeout = 10
; connect timeout = 10
# If you get out-of-memory errors, it may mean that your client
# is trying to allocate a huge buffer for a TEXT field.
# Try setting 'text size' to a more reasonable limit
text size = 64512
[IndyArtsDB]
host = xxx.xx.xxx.xx
port = 1433
tds version = 8.0
client charset = UTF-8
ODBC.INI
[MSSQL_DSN]
Driver=FreeTDS
Description=IndyArts DB on Rackspace
Trace=No
Server=xxx.xx.xxx.xx
Port=1433
Database=DBName
ODCBINST.INI
[ODBC]
DEBUG=1
TraceFile=/home/ftp/sqltrace.log
Trace=Yes
[FreeTDS]
Description=MSSQL Driver
Driver=/usr/local/lib/libtdsodbc.so
UsageCount=1
Looking at your sqltrace.log it looks to me like an authentication error - you get that "Unexpected EOF from the server" message immediately after authenticating...
Is there any chance the remote server is blocking connections from your CentOS server, either completely or on port 1433? Any chance the "client charset = UTF-8" in your freetds.conf is causing the problem?
This is my (working) setup on an Ubuntu 12.04 (Precise Pangolin) machine.
Here is my /etc/odbc.ini file:
[xyz]
Description = XYZ Server
Driver = freetds
Database = MyDB
ServerName = xyz
TDS_Version = 8.0
And my /etc/odbcinst.ini file:
[freetds]
Description = MS SQL database access with Free TDS
Driver = /usr/lib/i386-linux-gnu/odbc/libtdsodbc.so
Setup = /usr/lib/i386-linux-gnu/odbc/libtdsS.so
UsageCount = 1
And finally my /etc/freetds/freetds.conf file:
[global]
# TDS protocol version
; tds version = 4.2
# Whether to write a TDSDUMP file for diagnostic purposes
# (setting this to /tmp is insecure on a multi-user system)
; dump file = /tmp/freetds.log
; debug flags = 0xffff
# Command and connection timeouts
; timeout = 10
; connect timeout = 10
# If you get out-of-memory errors, it may mean that your client
# is trying to allocate a huge buffer for a TEXT field.
# Try setting 'text size' to a more reasonable limit
text size = 64512
# A typical Sybase server
[egServer50]
host = symachine.domain.com
port = 5000
tds version = 5.0
# A typical Microsoft server
[egServer70]
host = ntmachine.domain.com
port = 1433
tds version = 7.0
# The XYZ database
[xyz]
host = XYZ
port = 1433
tds version = 8.0
Looks like the version numbers in FreeTDS have been changed from 8.0 to 7.1 and 9.0 to 7.2.
See http://www.freetds.org/userguide/choosingtdsprotocol.htm