I have develped a Perl script that can format data into CSV format, and want to save the data into a Postgres database table.
I'm following this tutorial as a guideline for how to interface to Postgres. I verified in the PPM that I have the same versions installed as the tutorial: 1.634 version of DBI and the 3.5.3 version of DBD::Pg installed in Perl 5.22.1 in Windows 10 x64. The database is Postgres 12 on Windows Server 2008r2.
The first part of the Perl script reads, parses, and formats one data record as CSV.
This is an example of one CSV data record:
"2020-05-10 20:39:16+0","0.528239011764526","15:39 05/10/2020","0x1c","LOW STATUS","0x85","Normal","73.8","32","29.11","29.31","61.2","29","80","0.7","2.5","22.6","378.64","3009","7","0.00","0.00","0.97","0.97","11.96"
This is stored in $SQLstring before entering the database interface snippet below:
This is the code I have modified from the tutorial, which compiles and runs without errors.
# ------------- Postgres Database Connection --------------
# Connection config
my $dbname = 'MyDatabase';
my $host = '192.168.1.1';
my $port = 5432;
my $username = 'myuser';
my $password = 'mypassword';
# Create DB handle object by connecting
my $dbh = DBI -> connect("dbi:Pg:dbname=$dbname;host=$host;port=$port",
$username,
$password,
{AutoCommit => 0, RaiseError => 1}
) or die $DBI::errstr;
# Trace to a file
$dbh -> trace(1, 'tracelog.txt');
# Copy from STDIN into the table
my $SQL = "COPY AmbientWeather (
datetimestamp ,
CurrTime ,
IndoorID ,
inBattSta ,
Outdoor1ID ,
outBattSta1 ,
inTemp ,
inHumi ,
AbsPress ,
RelPress ,
outTemp ,
outHumi ,
windir ,
avgwind ,
gustspeed ,
dailygust ,
solarrad ,
uv ,
uvi ,
rainofhourly ,
rainofdaily ,
rainofweekly ,
rainofmonthly ,
rainofyearly
)
FROM STDIN WITH DELIMITER ',' CSV HEADER";
my $sth = $dbh->do($SQL);
# putcopy data from saved line in CSV format
$dbh->pg_putcopydata($SQLstring);
$dbh->pg_putcopyend(); # finished with one line
$dbh->commit or die $DBI::errstr;
exit;
This runs without errors, but the database is unchanged. No record is created.
This is the trace log, which does not show any notable errors, but I am not really familiar with the syntax:
DBI::db=HASH(0x3c62268) trace level set to 0x0/1 (DBI # 0x0/0) in DBI 1.634-ithread (pid 19436)
<- DESTROY(DBI::db=HASH(0x3c62268))= ( undef ) [1 items] at Ambient_Parser.pl line 158
DBI::db=HASH(0x3c76ca8) trace level set to 0x0/1 (DBI # 0x0/0) in DBI 1.634-ithread (pid 2108)
<- do('COPY AmbientWeather (
datetimestamp ,
CurrTime ,
IndoorID ,
inBattSta ,
Outdoor1ID ,
outBattSta1 ,
inTemp ,
inHumi ,
AbsPress ,
RelPress ,
outTemp ,
outHumi ,
windir ,
avgwind ,
gustspeed ,
dailygust ,
solarrad ,
uv ,
uvi ,
rainofhourly ,
rainofdaily ,
rainofweekly ,
rainofmonthly ,
rainofyearly
...')= ( -1 ) [1 items] at Ambient_Parser.pl line 177
<- pg_putcopydata('"2020-05-10 20:35:59+0","0.547099113464355","15:35 05/10/2020","0x1c","LOW STATUS","0x85","Normal","73.6","32","29.11","29.31","61.3","24","193","3.8","4.9","22.6","380.54","3082","7","0.00","0.00","0.97","0.97","11.96"
')= ( 1 ) [1 items] at Ambient_Parser.pl line 182
<- pg_putcopyend= ( 1 ) [1 items] at Ambient_Parser.pl line 183
<- commit= ( 1 ) [1 items] at Ambient_Parser.pl line 184
<- DESTROY(DBI::db=HASH(0x3c76ca8))= ( undef ) [1 items] at Ambient_Parser.pl line 193
For reference line 193 is the final exit; in the file.
I must be missing something, but I don't see what it is. Can you point out my error?
Edit: I compared the options in the tutorial's command in my $SQL = "COPY.... with Postgres COPY command documentation. The tutorial adds options CSV HEADER, which are not seen in Postres docs. I'm not sure why those options are used in the tutorial, or why they cause a silent failure. I removed them and now I am getting errors.
Code from above now looks like this:
rainofyearly
)
FROM STDIN WITH DELIMITER ','";
These errors are now being output:
DBD::Pg::db pg_putcopyend failed: ERROR: invalid input syntax for type real: ""0.520319223403931""
CONTEXT: COPY ambientweather, line 1, column fetchelapsed: ""0.520319223403931"" at Ambient_Parser.pl line 186.
DBD::Pg::db pg_putcopyend failed: ERROR: invalid input syntax for type real: ""0.520319223403931""
CONTEXT: COPY ambientweather, line 1, column fetchelapsed: ""0.520319223403931"" at Ambient_Parser.pl line 186.
Issuing rollback() due to DESTROY without explicit disconnect() of DBD::Pg::db handle dbname=MyDatabase;host=192.168.1.1;port=5432 at Ambient_Parser.pl line 186.
I'm investigating why the real value is being seen as double quoted. This is the same CSV format I have used from the PSQL command line with COPY FROM , and reals were accepted as shown above.
You told it the first line would be an ignored header. But you only sent it one line. So there were no data lines sent.
CSV and HEADER are separate options. These are both present (separately) in the docs you link to. You need to keep CSV, otherwise the quoting is not understood.
I have 02 oracles and oracles vcpu servers on which are created respectively a primary BD (db_name = chicago and db_unique_name = chicago) and a standby BD (db_name = chicago and db_unique_name = boston). I created a service in the 02 servers with the utility srvctl:
srvctl add service -d "db_unique_name" -s CHICAGO_HA -l PRIMARY -q TRUE -e SELECT -m BASIC -z 150 -w 10
Then I added an entry in my tnsnames.ora file:
CHICAGO_HA =
(DESCRIPTION_LIST =
(LOAD_BALANCE = off)
(FAILOVER = on)
(DESCRIPTION =
(CONNECT_TIMEOUT = 10) (RETRY_COUNT = 3)
(ADDRESS_LIST = (ADDRESS = (PROTOCOL = tcp) (HOST = 192.168.17.140) (PORT = 1522)))
(CONNECT_DATA = (SERVICE_NAME = chicago_ha))
)
(DESCRIPTION =
(CONNECT_TIMEOUT = 10) (RETRY_COUNT = 3)
(ADDRESS_LIST = (ADDRESS = (PROTOCOL = tcp) (HOST = 192.168.17.138) (PORT = 1522)))
(CONNECT_DATA = (SERVICE_NAME = chicago_ha))
)
)
I have a client on which i have opened a session :
sqlplus sys/*****#chicago_ha as sysdba
on primary host. i can query tables. but after switchover when i can't query tables because my session ends with errors like I am no longer connected.
If I read it correctly, your setup has connect-time failover, but you are trying to use an already open connection to the now-closed instance? You would need something like Application Continuity or Transparent Application Continuity for an existing connection to be moved to the other instance. Check the Oracle white paper Continuous Availability
Application Continuity for the Oracle Database.
SQL*Plus generally isn't a great testing tool to mimic real-life apps that have connection pools and lots of users.
My DB machine has three instance, I had shutdown the 'EDWDBS1' only (not the whole database) in the RMAN interface by using
shutdown immediate
But when I want to reconnect to/startup 'EDWDBS1', it throws ORA-12505 error either in sqldeveloper or sqlplus or RMAN. Alrd tried to check the tnsnames.ora and listener.ora, but I still got no idea about that.
Here is my listener.ora
# copyright (c) 1997 by the Oracle Corporation
#
# NAME
# listener.ora
# FUNCTION
# Network Listener startup parameter file example
# NOTES
# This file contains all the parameters for listener.ora,
# and could be used to configure the listener by uncommenting
# and changing values. Multiple listeners can be configured
# in one listener.ora, so listener.ora parameters take the form
# of SID_LIST_<lsnr>, where <lsnr> is the name of the listener
# this parameter refers to. All parameters and values are
# case-insensitive.
# <lsnr>
# This parameter specifies both the name of the listener, and
# it listening address(es). Other parameters for this listener
# us this name in place of <lsnr>. When not specified,
# the name for <lsnr> defaults to "LISTENER", with the default
# address value as shown below.
#
# LISTENER =
# (ADDRESS_LIST=
# (ADDRESS=(PROTOCOL=tcp)(HOST=localhost)(PORT=1521))
# (ADDRESS=(PROTOCOL=ipc)(KEY=PNPKEY)))
# SID_LIST_<lsnr>
# List of services the listener knows about and can connect
# clients to. There is no default. See the Net8 Administrator's
# Guide for more information.
#
# SID_LIST_LISTENER=
# (SID_LIST=
# (SID_DESC=
# #BEQUEATH CONFIG
# (GLOBAL_DBNAME=salesdb.mycompany)
# (SID_NAME=sid1)
# (ORACLE_HOME=/private/app/oracle/product/8.0.3)
# #PRESPAWN CONFIG
# (PRESPAWN_MAX=20)
# (PRESPAWN_LIST=
# (PRESPAWN_DESC=(PROTOCOL=tcp)(POOL_SIZE=2)(TIMEOUT=1))
# )
# )
# )
# PASSWORDS_<lsnr>
# Specifies a password to authenticate stopping the listener.
# Both encrypted and plain-text values can be set. Encrypted passwords
# can be set and stored using lsnrctl.
# LSNRCTL> change_password
# Will prompt for old and new passwords, and use encryption both
# to match the old password and to set the new one.
# LSNRCTL> set password
# Will prompt for the new password, for authentication with
# the listener. The password must be set before running the next
# command.
# LSNRCTL> save_config
# Will save the changed password to listener.ora. These last two
# steps are not necessary if SAVE_CONFIG_ON_STOP_<lsnr> is ON.
# See below.
#
# Default: NONE
#
# PASSWORDS_LISTENER = 20A22647832FB454 # "foobar"
# SAVE_CONFIG_ON_STOP_<lsnr>
# Tells the listener to save configuration changes to listener.ora when
# it shuts down. Changed parameter values will be written to the file,
# while preserving formatting and comments.
# Default: OFF
# Values: ON/OFF
#
# SAVE_CONFIG_ON_STOP_LISTENER = ON
# USE_PLUG_AND_PLAY_<lsnr>
# Tells the listener to contact an Onames server and register itself
# and its services with Onames.
# Values: ON/OFF
# Default: OFF
#
# USE_PLUG_AND_PLAY_LISTENER = ON
# LOG_FILE_<lsnr>
# Sets the name of the listener's log file. The .log extension
# is added automatically.
# Default=<lsnr>
#
# LOG_FILE_LISTENER = lsnr
# LOG_DIRECTORY_<lsnr>
# Sets the directory for the listener's log file.
# Default: <oracle_home>/network/log
#
# LOG_DIRECTORY_LISTENER = /private/app/oracle/product/8.0.3/network/log
# TRACE_LEVEL_<lsnr>
# Specifies desired tracing level.
# Default: OFF
# Values: OFF/USER/ADMIN/SUPPORT/0-16
#
# TRACE_LEVEL_LISTENER = SUPPORT
# TRACE_FILE_<lsnr>
# Sets the name of the listener's trace file. The .trc extension
# is added automatically.
# Default: <lsnr>
#
# TRACE_FILE_LISTENER = lsnr
# TRACE_DIRECTORY_<lsnr>
# Sets the directory for the listener's trace file.
# Default: <oracle_home>/network/trace
#
# TRACE_DIRECTORY_LISTENER=/private/app/oracle/product/8.0.3/network/trace
# CONNECT_TIMEOUT_<lsnr>
# Sets the number of seconds that the listener waits to get a
# valid database query after it has been started.
# Default: 10
#
# CONNECT_TIMEOUT_LISTENER=10
tnsnames.ora
EDWDBD1 = (DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = HKHPEDWDBSDEV01)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = EDWDBD1)
)
)
EDWDBS1 = (DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = HKHPEDWDBSDEV01)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = EDWDBS1)
)
)
EDWDBU1 = (DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = HKHPEDWDBSDEV01)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = EDWDBU1)
)
)
EDWPDBD1 = (DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = HKHPEDWDBSDEV01)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = EDWPDBD1)
)
)
Result of lsnrctl status
LSNRCTL for Linux: Version 12.2.0.1.0 - Production on 11-FEB-2019 18:43:54
Copyright (c) 1991, 2016, Oracle. All rights reserved.
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 12.2.0.1.0 - Production
Start Date 11-FEB-2019 18:09:09
Uptime 0 days 0 hr. 34 min. 45 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/12.2.0/db_1/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/HKHPEWDBSDEV01/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=HKHPEWDBSDEV01)(PORT=1521)))
Services Summary...
Service "7a114ab24bf177a5e055315d54627e2c" has 1 instance(s).
Instance "edwdbd1", status READY, has 1 handler(s) for this service...
Service "EDWDBU1" has 1 instance(s).
Instance "EDWDBU1", status READY, has 1 handler(s) for this service...
Service "EDWDBU1XDB" has 1 instance(s).
Instance "EDWDBU1", status READY, has 1 handler(s) for this service...
Service "edwdbd1" has 1 instance(s).
Instance "edwdbd1", status READY, has 1 handler(s) for this service...
Service "edwdbd1XDB" has 1 instance(s).
Instance "edwdbd1", status READY, has 1 handler(s) for this service...
Service "edwpdbd1" has 1 instance(s).
Instance "edwdbd1", status READY, has 1 handler(s) for this service...
The command completed successfully
I am guessing you are trying to start your instance remotely. In your listener.ora you don't have a static service configured, so you can't connect remotely (via TCP/IP) to the instance as the listener doesn't know about that instance.
In order to start the instance you should be connected locally (via SSH) and set all necessary environment variables (ORACLE_HOME, ORACLE_SID, PATH) and use OS authentication (usually be member of dba group) and run:
sqlplus / as sysdba
and
startup;
Also you can use password file authentication but this is another story.
I use Freeradius + CoovaChilli + Nginx + Ubuntu.
I nicely configured the Freeradius and everything is working fine.. Except that ChilliSpot-Max-Total-Octets terminate the session after 1 117 000 000 octets used by the Mac address but the Username can log in again.
I was expecting that the Username cannot login until he waits daily reset.
Is it due to the Unique ID session ?
In /etc/freeradius/sql/mysql/counter.php
sqlcounter chillispot_max_bytes {
counter-name = ChilliSpot-Max-Total-Octets
check-name = ChilliSpot-Max-Total-Octets
reply-name = ChilliSpot-Max-Total-Octets
reply-message = "C'est pas bien de trop télécharger !!"
sqlmod-inst = sql
key = User-Name
reset = daily
query = "SELECT SUM(AcctInputOctets) + SUM(AcctOutputOctets) FROM
radac$
}
In /etc/freeradius/site-enabled/default
Authorize
{
...
#
# Counters for Chillispot
#
chillispot_max_bytes
daily
...
}
Second problem similar:
Daily limit : I set up a Session-time end that works perfectly but I would like to have a OFF period daily.
sqlcounter dailycounter {
counter-name = Daily-Session-Time
check-name = Max-Daily-Session
reply-name = Session-Timeout
reply-message = "You've used up more than one hour today"
sqlmod-inst = sql
key = User-Name
reset = daily
# This query ignores calls that started in a previous
# reset period and continue into into this one. But it
# is a little easier on the SQL server
query = "SELECT SUM(acctsessiontime) FROM radacct WHERE \
username = '%{%k}' AND acctstarttime > FROM_UNIXTIME('%b')"
}
Dictionnary :
$INCLUDE /usr/share/freeradius/dictionary
$INCLUDE /usr/share/freeradius/dictionary.chillispot
ATTRIBUTE Max-Daily-Session 30011 integer
ATTRIBUTE chillispot_max_bytes 3010 integer
Any idea ?
I was thinking about creating some User group, but I am not sure how to manage that with Radius.
Thank you
OK I have a lab setup, I have a Freenas server iscsi setup with Chap setup for discovery and mutual chap for targets.
Here are the requirements:
Implement CHAP security
One-way CHAP for discovery
Two-way (Mutual) CHAP for targets
I can connect and discover sucessfully with two esxi servers, windows 7, windows 2003, 2008, and 2012
Centos can see the discovery list, but when trying to connect with :
iscsiadm --mode node --targetname iqn.2015.lab.com:centos --portal 192.168.1.60:3260 --login
the terminal outputs:
no records found
Here is my iscsid.conf, I left the comments in on the Chap section, but removed it for the rest as it is just so large:
iscsid.startup = /etc/rc.d/init.d/iscsid force-start
node.startup = automatic
node.leading_login = No
# *************
# CHAP Settings
# *************
# To enable CHAP authentication set node.session.auth.authmethod
# to CHAP. The default is None.
node.session.auth.authmethod = CHAP
# To set a CHAP username and password for initiator
# authentication by the target(s), uncomment the following lines:
#node.session.auth.username = group7
#node.session.auth.password = passwordpassword
# To set a CHAP username and password for target(s)
# authentication by the initiator, uncomment the following lines:
node.session.auth.username_in = group7
node.session.auth.password_in = passwordpassword
# To enable CHAP authentication for a discovery session to the target
# set discovery.sendtargets.auth.authmethod to CHAP. The default is None.
discovery.sendtargets.auth.authmethod = CHAP
# To set a discovery session CHAP username and password for the initiator
# authentication by the target(s), uncomment the following lines:
discovery.sendtargets.auth.username = group7
discovery.sendtargets.auth.password = passwordpassword
# To set a discovery session CHAP username and password for target(s)
# authentication by the initiator, uncomment the following lines:
#discovery.sendtargets.auth.username_in = group7
#discovery.sendtargets.auth.password_in = passwordpassword
node.session.timeo.replacement_timeout = 120
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 30
node.session.err_timeo.tgt_reset_timeout = 30
node.session.initial_login_retry_max = 8
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.xmit_thread_priority = -20
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
node.conn[0].iscsi.HeaderDigest = None
node.session.nr_sessions = 1
node.session.iscsi.FastAbort = Yes
Any help is appreciated. Thank you.
You want mutual CHAP for session setup, but in your configuration file you have commented-out the lines that define the login from initiator to target:
# To set a CHAP username and password for initiator
# authentication by the target(s), uncomment the following lines:
#node.session.auth.username = group7
#node.session.auth.password = passwordpassword