multiple value inserts to Postgres using Tokio-postgres in Rust - postgresql

I am using the below code to insert to a Postgres DB using tokio-postgres, is there any better option :
let members = &[obj] //obj is a struct
let mut params = Vec::<&(dyn ToSql + Sync)>::new();
let mut i = 1;
let mut qry:String = "insert into tablename(id,userid,usertype) values".to_string();
for column in members{
if(i ==1){
qry = format!("{} (${},${},${})",qry,i,i+1,i+2);
}else{
qry = format!("{}, (${},${},${})",qry,i,i+1,i+2);
}
params.push(&column.id);
params.push(&column.userid);
params.push(&column.usertype);
i = i+3;
}
println!("qry : {}",qry);
let result = p.execute(&qry, &params[..]).await; //p is the pool manager

No:
Inserting multiple values at the same time
Ability to insert multiple rows by specifying multiple rows in VALUES?
You can marginally improve it by using iterators:
use itertools::Itertools; // For tuples() and format_with()
let params: Vec<_> = members
.iter()
.flat_map(|row| [&row.id as &(dyn ToSql + Sync), &row.userid, &row.usertype])
.collect();
let query = format!(
"insert into tablename(id, userid, usertype) values {}",
(0..params.len())
.tuples()
.format_with(", ", |(i, j, k), f| {
f(&format_args!("(${i}, ${j}, ${k})"))
}),
);
However I don't really think that's better.

Related

Rust + psql. Cannot assign to variable, as it is a captured variable in a `Fn` closure

During program execution, the value of spp_pipe_count_setup is set. I need to transfer this value to the database.
I'm having a problem with Rust closures:
cannot assign to data_pipe_count.value_int, as it is a captured variable in a Fn closure
struct SetupDataSQL<'a> {
value_int: i32,
}
fn main() {
let spp_pipe_count_setup = gtk::SpinButton::with_range(1.0, 10.0, 1.0);
let mut data_pipe_count = SetupDataSQL {
value_int: 0
};
spp_pipe_count_setup.connect_value_changed(move |_| {
data_pipe_count.value_int = spp_pipe_count_setup.value() as i32
});
}
And after that i am passing data_pipe_count.value_int to sql database.
client.execute(
"INSERT INTO setup_data (param_Name, value_int) VALUES ($1, $2)",
&[&data_pipe_count.param_name, &data_pipe_count.value_int],
.unwrap();
I get a different error if I use RefCell in struct: the trait bound RefCell<i32>: ToSql is not satisfied
struct SetupDataSQL<'a> {
param_name: &'a str,
value_int: RefCell<i32>
}
client.execute(
"INSERT INTO setup_data (param_Name, value_int, value_float, value_char) VALUES ($1, $2)",
&[&data_pipe_count.param_name, &data_pipe_count.borrow().value_int],
).unwrap();
No matter how hard I try to pass the value of the SpinButton to the DB, it doesn't work.
Error: the trait bound Rc<RefCell<{integer}>>: ToSql is not satisfied
let temp = Rc::new(RefCell::new(1));
let temp = temp clone();
let mut client = (Client::connect("postgresql://user:pass#localhost:5432/postgres", NoTls).unwrap());
client.execute(
"INSERT INTO setup_data ( value_int) VALUES ($1)",
&[ &(*temp.borrow_mut()]),
.unwrap();

How can I avoid string conversions when ingesting timestamps to postgres in Rust?

I'm using the rust-postgres crate to ingest data. This is a working example adding rows successfully:
let name: &str = "hello from rust";
let val: i32 = 123;
let now: DateTime<Utc> = Utc::now();
let timestamp = now.format("%Y-%m-%dT%H:%M:%S%.6f").to_string();
client.execute(
"INSERT INTO trades VALUES(to_timestamp($1, 'yyyy-MM-ddTHH:mm:ss.SSSUUU'),$2,$3)",
&[&timestamp, &name, &val],
)?;
This doesn't look so nice as I have to do this forward and back string conversion, I would like to be able to write something like
let name: &str = "hello from rust";
let val: i32 = 123;
let now: DateTime<Utc> = Utc::now();
client.execute(
"INSERT INTO trades VALUES($1,$2,$3)",
&[&now, &name, &val],
)?;
What's the most performant way of ingesting timestamps in this way?
Edit:
Here's the returned error from the second example above
Error: Error { kind: ToSql(0), cause: Some(WrongType { postgres: Timestamp, rust: "chrono::datetime::DateTime<chrono::offset::utc::Utc>" }) }
And my cargo.toml looks like this (which has the chrono feature enabled for the rust postgres crate):
[dependencies]
chrono = "0.4.19"
postgres={version="0.19.0", features=["with-serde_json-1", "with-bit-vec-0_6", "with-chrono-0_4"]}
I think the problem is a mismatch between your postgres schema and your Rust type: the error seems to say that your postgres type is timestamp, while your rust type is DateTime<Utc>.
If you check the conversion table, DateTime<Utc> converts to a TIMESTAMP WITH TIME ZONE. The only types which convert to TIMESTAMP are NaiveDateTime and PrimitiveDateTime.
As per Masklinn's response, I needed to pass a NaiveDateTime type for this to work, the full example with naive_local looks like:
use postgres::{Client, NoTls, Error};
use chrono::{Utc};
use std::time::SystemTime;
fn main() -> Result<(), Error> {
let mut client = Client::connect("postgresql://admin:quest#localhost:8812/qdb", NoTls)?;
// Basic query
client.batch_execute("CREATE TABLE IF NOT EXISTS trades (ts TIMESTAMP, date DATE, name STRING, value INT) timestamp(ts);")?;
// Parameterized query
let name: &str = "rust example";
let val: i32 = 123;
let utc = Utc::now();
let sys_time = SystemTime::now();
client.execute(
"INSERT INTO trades VALUES($1,$2,$3,$4)",
&[&utc.naive_local(), &sys_time, &name, &val],
)?;
// Prepared statement
let mut txn = client.transaction()?;
let statement = txn.prepare("insert into trades values ($1,$2,$3,$4)")?;
for value in 0..10 {
let utc = Utc::now();
let sys_time = SystemTime::now();
txn.execute(&statement, &[&utc.naive_local(), &sys_time, &name, &value])?;
}
txn.commit()?;
println!("import finished");
Ok(())
}

Main thread messes up sorting

Dispatching the queue messes up the order in the array as noted below. I'm trying to rank the array and then be able to translate it. So far its not working:
let top5 = Array(labels.sorted{ $0.confidence > $1.confidence}.prefix(upTo:5))
for lulu in top5 {
let translator = ROGoogleTranslate()
var params = ROGoogleTranslateParams()
params.source = "en"
params.target = "es"
params.text = "\(String(describing: lulu.label))"
translator.translate(params: params, callback: { (result) in
DispatchQueue.main.async {
self.textUno.text = self.textUno.text! + "\(lulu.label)" + " \(lulu.confidence*100)\n"
self.textDos.text = self.textDos.text! + "\(result)\n"
self.view.addSubview(self.textUno)
self.view.addSubview(self.textDos)
}
})
}
If I try to put the sorting out of DispatchQueue.main.async then the translation won't be lined up with the right word.
How can I fix this so that the array is sorted and the translation matches up ?
Translate the array first before ranking them.
Make it simpler first and make sure it is working and then put all the parts together.
If you really want to do it this way you will need to put them into a temporary array after sorting them and then use that at the end.
This, as you said, will return a jumbled result.
The below example is pretty close, needs a bit of a polish but you should be able to do it from this.
let top5 = Array(labels.sorted{ $0.confidence > $1.confidence}.prefix(upTo:5))
var tmparr : []
var count: Int = 0
for lulu in top5 {
let translator = ROGoogleTranslate()
count = count + 1
var params = ROGoogleTranslateParams()
params.source = "en"
params.target = "es"
params.text = "\(String(describing: lulu.label))"
params.ordernumber = count
translator.translate(params: params, callback: { (result) in
tmparr.append[params]
})
}
DispatchQueue.main.async {
for lulunew in tmparr {
if (lulunew.ordernumber == correctindex){
self.textUno.text = self.textUno.text! + "\(lulu.label)" + " \(lulu.confidence*100)\n"
self.textDos.text = self.textDos.text! + "\(result)\n"
self.view.addSubview(self.textUno)
self.view.addSubview(self.textDos)
}
}
}

F# SQLClient typeProvider reducing duplicate code

I am trying to create a type safe data access layer in F# using FSharp.Data.SqlClient type providers to be callable from C#. I have highly complicated SQL queries that are performance critical but there are also several variations.
Given the following (obviously simplified schema):
CREATE TABLE [dbo].[company] (
[id] INT IDENTITY (1, 1) NOT NULL,
[name] VARCHAR (50) NOT NULL)
I have the following F# code:
module CompanyDAL =
open FSharp.Data // requires FSharp.Data.SqlClient package
[<Literal>]
let conn = "Data Source=(localdb)\ProjectsV13;Initial Catalog=TESTDB;Integrated Security=True;Pooling=False;Connect Timeout=30"
[<Literal>]
let baseQuery = "select id, name from company where 1 = 1 "
[<Literal>]
let filterById = "and id = #id"
[<Literal>]
let filterByName = "and name = #name"
[<Literal>]
let queryFilteredById = baseQuery + filterById
type GetCompanyById = SqlCommandProvider<queryFilteredById, conn>
[<Literal>]
let queryFilterbyName = baseQuery + filterByName
type GetCompanyByName = SqlCommandProvider<queryFilterbyName, conn>
type GetAllCompanies = SqlCommandProvider<baseQuery, conn>
type Company = {
Name : string
id : int
}
let getCompanyById (runtimeConn:string) =
async {
use query = new GetCompanyById(runtimeConn)
let! result = query.AsyncExecute(id = 10)
return result
|> Seq.map (fun x ->
{ Name = x.name
id = x.id })
|> Seq.toArray
} |> Async.StartAsTask
let getCompanyByName (runtimeConn:string) =
async {
use query = new GetCompanyByName(runtimeConn)
let! result = query.AsyncExecute(name = "abc" )
return result
|> Seq.map (fun x ->
{ Name = x.name
id = x.id })
|> Seq.toArray
} |> Async.StartAsTask
let getAllCompanies (runtimeConn:string) =
async {
use query = new GetAllCompanies(runtimeConn)
let! result = query.AsyncExecute()
return result
|> Seq.map (fun x ->
{ Name = x.name
id = x.id })
|> Seq.toArray
} |> Async.StartAsTask
Is there any way I can reduce the number of duplication while maintaining the raw performance of type safe raw queries?
I've tried using
type GetCompany = SqlEnumProvider<baseQuery, conn, provider>
Then I can write this code
GetCompany.Items |> Seq.where(fun (id, name) -> name = "xxx") |> printfn "%A"
So I can avoid duplication and pass a generic predicate fun (id, name) -> to the only GetCompany type.
Similarly,
GetCompany.Items |> Seq.where(fun (id, name) -> id = "1")
would work as well.
Note
Anyway, notice that the above will filter in memory. In fact Dynamic SQL is not permitted with this library, because it relies only on static analysis at compile time. If you need to construct queries dynamically, you can revert to raw ADO.NET or other light ORM.

How to make a select query using a local variable in SQLite on Swift?

In Swift, if you want to make a query from a local variable it is done by 'selectedButton'. So how is it possible in SQLite?
{
var selectedButton = 1;
let dbPath = NSBundle.mainBundle().pathForResource("practise2015", ofType:"sqlite3")
let db = FMDatabase(path: dbPath)
if db.open(){
let categoryLevelID = try! db.executeQuery("SELECT * FROM `LEVEL` WHERE CATEGORY_ID = " + selectedButton, values: nil)
}
}
Can't you just do something like this?
let categoryLevelID = try! db.executeQuery("SELECT * FROM LEVEL WHERE CATEGORY_ID = \(selectedButton.tag)", values: nil)
actuall you can't just pass a UIButton as an argument, you need some String, Int or Double value. Category_id sounds like an Int to me, so I guess you have the correct category_id in your UIButton as it's Tag
But you should do queries by using data binding anyway. Take a look at this tutorial by Ray Wenderlich: https://www.raywenderlich.com/123579/sqlite-tutorial-swift
Or this one: http://www.techotopia.com/index.php/An_Example_SQLite_based_iOS_8_Application_using_Swift_and_FMDB