I am trying to delete the data from postgres using spark but unable to delete same code is working for select statements - postgresql

Class.forName("org.postgresql.Driver")
val conn = Url
val del = s"(delete from db.table where timestamp = '1950-09-08 00:00:00.000')"
val db = DriverManager.getConnection(conn)
println("delete query :" + del)
val pstdel = db.prepareStatement(del)
try {
pstdel.execute()
}
I am getting the below error:
org.postgresql.util.PSQLException: ERROR: syntax error at or near "delete"
The same code is working for select statements. I do have delete permissions.

Try this
val del = s"delete from db.table where timestamp = '1950-09-08 00:00:00.000'"

Related

Connecting Scala with Hive Database using sbt for dependencies using IntelliJ

I am having a very difficult time connecting to hive database using Intellij or basic Command line with scala ( would be happy with java too). I have in the past been able to connect to a MYSQL database by adding it on the library mysql-Connector. but I am unable somehow add a jar file to the project structure where it works.
and to make things abit more difficult. I have installed ubuntu with hive,spark, hadoop and I am connecting to it over the network.
Is there someway I can add a depedency on the sbt file?
Lastly, I know there are similar questions but they do not show in detail how to connect to a hive database from scala
`import java.sql.SQLException;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;
object HiveJdbcClient extends App {
val driverName = "org.apache.hadoop.hive.jdbc.HiveDriver";
Class.forName(driverName);
val con=DriverManager.getConnection("jdbc:hive://http://192.168.43.64:10000/default", "", "");
val stmt = con.createStatement();
val tableName = "testHiveDriverTable";
stmt.executeQuery("drop table " + "wti");
var res = stmt.executeQuery("create table " + tableName + " (key int, value string)");
// select * query
var sql = "select * from " + tableName;
res = stmt.executeQuery(sql);
while (res.next()) {System.out.println(String.valueOf(res.getInt(1)) + "\t" + res.getString(2));
}
// regular hive query
sql = "select count(1) from " + tableName;
res = stmt.executeQuery(sql);
while (res.next()) {
System.out.println(res.getString(1));
}
}`
The driver name is not correct for hive 3.1.2, it should be
org.apache.hive.jdbc.HiveDriver
Cf https://hive.apache.org/javadocs/r3.1.2/api/org/apache/hive/jdbc/HiveDriver.html

Odoo-Creation sequence based on PostgreSQL sequence

I am working with odoo 14 and I want to customize sale.order number generation. So, I want to create new sequence (ir.sequence) based on PostgreSQL database sequence object.
Do you have any idea?
Thank you for your help.
SAAD
from odoo import api, fields, models
import psycopg2
class ventes(models.Model):
_inherit = ['sale.order']
company = fields.Char()
name = fields.Char(string='Order Reference')
#Connection a la base de donnees
def open_conn(self):
try:
connection = psycopg2.connect(user="user",
password="xxxxxxxxxxxxxx",
host="192.168.1.1",
port="5432",
database="ventes")
print("Using Python variable in PostgreSQL select Query")
cursor = connection.cursor()
postgreSQL_select_Query = "select nextval('myOdoo')"
cursor.execute(postgreSQL_select_Query)
row = cursor.fetchone()
return row[0]
except (Exception, psycopg2.Error) as error:
print("Error fetching data from PostgreSQL table", error)
finally:
# closing database connection
if connection:
cursor.close()
connection.close()
print("PostgreSQL connection is closed \n")
#api.model
def create(self, vals):
num = self.open_conn()
vals['name'] = num
result = super(ventes, self).create(vals)
return result

Insert to postgres DB: invalid String format (parsing exception error)

I am creating a log db where I track execution steps and catch where it might or might not fail.
I am having issues inserting the Exceptions thrown to a DB.
I am calling an endpoint first in order then query a django database.
In this case I wanted to catch a mistake in the selection_query and thus being unable to fetch the data from the DB.
I want to log this message.
Example:
Exception catch
try:
select_query = "SELECT columnn from row"
cur.execute(select_query)
selected_vals = cur.fetchall()
except Exception as e:
response = insert_into_logging(message = f"{e}", logging_id = 3)
Error
File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
This is because of the value passed in the message argument.
(message = f"{e}")
It is not insertable in the DB
The value of "e" is:
auto_close_alarm relation "row" does not exist
LINE 1: SELECT columnn from row
^
I tried to pass
str(e)
Value passed: identical output as above
repr(e.args[0])
Output: 'relation "row" does not exist\nLINE 1: SELECT columnn from row\n ^\n'
repr(e)
Value passed: UndefinedTable('relation "row" does not exist\nLINE 1: SELECT columnn from row\n ^\n',)
And multiple other ways but to no avail.
They all result in the same JSONDecodeError.
Basically the key problem is that this is not a valid String format for it to be able to be parsed in the DB.
All I need is a representation of this Exception error for it to be insertable
Flow of data:
insert function
def insert_into_logging(message, logging_id, date_time):
url = f"""url/insert-into-logging/"""
data = {
"message": message,
"logging_id": logging_id,
}
response = r.post(url, data=json.dumps(data, indent=4, sort_keys=True, default=str))
print(response)
return response.json()
Querying DB
def insert_to_logging(request):
from datetime import datetime, timedelta
if request.method != 'POST':
print(request.method)
return JsonResponse({'status':'error', 'message':'Client insertion only accepts post requests.' + str(request.method)})
today = datetime.today()
body = json.loads(request.body)
message = body.get('message')
logging_id = body.get('logging_id')
Logging.objects.create(message = message, logging_id = logging_id, time_stamp = today)
return JsonResponse({'status':'OK', 'message':'Logging successful'})
Django Model
class Logging(models.Model):
objects = GetOrNoneManager()
message = models.CharField(max_length=64, blank=True, null=True)
logging_id = models.IntegerField(blank=True, null=True)
time_stamp = models.DateTimeField(blank=True, null=True)
class Meta:
db_table = 'logging'

DB2DataAdapter.Fill(dataset) throws error "SQL0901N (Reason "CPF4273".) SQLSTATE=58004"

My application is Dot Net based.I am using VS My application uses IBm DB2 at its backend.
My Query is
SELECT A, B FROM SERVER.D WHERE A IN
(SELECT C FROM SERVER.E WHERE X = '01OBPP' AND Y= 'U' AND Z= '1')
A,B,C,X,Y,Z-COLUMN NAME, SERVER- SERVER NAME, D AND E -TABLE NAME
PFB the code-
DB2DataAdapter dbAdapter = null;
DB2Connection dbConn = null;
DB2Command dbCommand;
DataSet dsReturnDataSet ;
dbConn = new DB2Connection(connectionString);
if (dbConn.State == ConnectionState.Closed) dbConn.Open();
dsReturnDataSet.Clear();
dbCommand = GetCommand();
dbCommand.CommandText = queryString;
dbCommand.Connection = dbConn;
dbAdapter = new DB2DataAdapter((DB2Command)dbCommand);
dbAdapter.Fill(dsReturnDataSet);
return dsReturnDataSet;
The GetCommand() method has the following statement
DB2Command dbCommand;
dbCommand = null;
dbCommand = new DB2Command();
dbCommand.CommandType = CommandType.Text;
dbCommand.CommandTimeout = 30;
return dbCommand;
While it hits the line 'dbAdapter.Fill(dsReturnDataSet);' it stucks there for a very long time and after that it throws the error
"ERROR [58004] [IBM][AS] SQL0901N The SQL statement failed because of
a non-severe system error. Subsequent SQL statements can be processed.
(Reason "CPF4273".) SQLSTATE=58004"
Please provide some pointers.
i will be very grateful if any one can give some pointers as to hoW to solve this error.
Check your database log. There should be a db2diag.log file in the db2dump directory under the instance's sqllib directory. Search this file for the above error and it should contain the root cause (that non-severe system error) that it refers to.

How to use Multiple resultsets with POSTGRES JDBC?

I am using JDBC on a PostgreSQL database.
When I query for an entity in a resultset, it returns 5 rows.
Related to that entity is another entity, for which I query while i am using a row in the above resultset.
When I execute this query, the above resultset is closed.
This means that it is allowing only 1 resultset to be active on 1 connection at a time.
Previously the same code was working perfect for Oracle DB server.
Is it that I need to ask the DB admin to configure the server to allow multiple resultsets?
Or to do some change in the code?
Or is it impossible to do it in postgre?
Here is the code for more details:
Connection conn = PTSConnection.getConnection();
Statement stmt = conn.createStatement();
ResultSet lines = stmt.executeQuery("SELECT LINEID,STARTSTOPID,ENDSTOPID FROM LINES"); **//first resultset is active**
while (lines.next()){
int lineId= lines.getInt(1);
Stop ss = StopStorage.getByID(lines.getInt(2));
Stop es = StopStorage.getByID(lines.getInt(3));
ResultSet stops = stmt.executeQuery("SELECT STOPID FROM STOPSINLINES WHERE LINEID=" + lineId); **//first resultset dies**
List<Stop> lineStops = new ArrayList<Stop>();
while(stops.next()){
Stop stop = StopStorage.getByID(stops.getInt(1));
lineStops.add(stop);
}
stops.close();
Line aLine = null;
ResultSet emergencyLine = stmt.executeQuery("SELECT CAUSE, STARTTIME, ENDTIME FROM EMERGENCYLINES WHERE LINEID =" + lineId);
if(emergencyLine.next()){
String cause = emergencyLine.getString(1);
Time startTime = emergencyLine.getTime(2);
Time endTime = emergencyLine.getTime(3);
aLine = new EmergencyLine(ss, es, cause, startTime, endTime, (Stop[]) lineStops.toArray(new Stop[lineStops.size()]));
} else {
aLine = new Line(ss, es, (Stop[]) lineStops.toArray(new Stop[lineStops.size()]));
}
emergencyLine.close();
LineRepository.getInstance().addLine(aLine);
}
lines.close();
The reason is not that you are using two resultsets on the same connection, but you are re-using the same Statement object for a new query. When you run executeQuery() on a Statement instance, any previous result will be closed (I'm surprised that your code did work with Oracle...)
Simply create a new Statement object before executing the second query:
Statement stmt = conn.createStatement();
Statement nestedStmt = conn.createStatement();
ResultSet lines = stmt.executeQuery("SELECT LINEID,STARTSTOPID,ENDSTOPID FROM LINES"); **//first resultset is active**
while (lines.next()){
...
ResultSet stops = nestedStmt.executeQuery("SELECT STOPID FROM STOPSINLINES WHERE LINEID=" + lineId); **//first resultset dies**
List lineStops = new ArrayList();
while(stops.next()){
Stop stop = StopStorage.getByID(stops.getInt(1));
lineStops.add(stop);
}
stops.close();
...
ResultSet emergencyLine = nestedStmt.executeQuery("SELECT CAUSE, STARTTIME, ENDTIME FROM EMERGENCYLINES WHERE LINEID =" + lineId);
if(emergencyLine.next()){
String cause = emergencyLine.getString(1);
....
}
emergencyLine.close();
And don't for get to properly close all Statements and ResultSets !