I have an SQL statement where I would like to pass a list of tuples customerIdSubCustomerPairs so that I can fetch all the rows matching the where clause from the table customers.
This is the SQL for it:
#Language("PostgreSQL")
private const val getExistingRowsSql = """
select customerId, subcustomer
from customer
where (uuid IN (:uuids) AND is_deleted = false)
union
select customerId, subcustomer
from customer
where (customerId, subcustomer) IN (:customerIdSubCustomerPairs) AND is_deleted = false
"""
I have a list of customers from which I am building a list of lists for customerIdSubCustomerPairs.
It looks like this:
val existingCustomers = fetchRows(
ctx, getExistingRowsSql, mapOf(
"uuids" to customers.map { it["uuid"] },
"customerIdSubCustomerPairs" to customers.map { listOf(it["customerId"] as String, it["subCustomer"] as String) }
))
And I have a function where I convert values to Oid types:
fun jdbcConvertValue(session: Session, v: Any?): Any? =
when (v) {
is Collection<*> -> {
when (val type = v.firstOrNull()) {
null -> null
is String -> session.connection.underlying.createArrayOf("text", v.toTypedArray())
is Long -> session.connection.underlying.createArrayOf("bigint", v.toTypedArray())
is Int -> session.connection.underlying.createArrayOf("bigint", v.toTypedArray())
is UUID -> session.connection.underlying.createArrayOf("uuid", v.toTypedArray())
is List<*> -> v.map { jdbcConvertValue(session, it) }.toTypedArray()
else -> throw Exception("You need to map your array type $type ${type.javaClass}")
}
}
else -> v
}
However, when I try to fetch rows like that, I get an error:
org.postgresql.util.PSQLException: Cannot cast an instance of [Ljava.lang.Object; to type Types.ARRAY
I am not sure how to pass values as tuples into a SQL statement?
The fields customerId and subcustomer are both text columns in the DB table.
Related
I'm running a simple query over the SQLite.swift library, selecting id and start_time from my table where the start_time column has a value greater than zero.
let query = table.where(self.columns.start_time > 0).select(self.columns.id, self.columns.start_time)
for i in try db.prepare(query) {
print(i)
let id = try i.get(self.columns.id)
let start_time = try i.get(self.columns.start_time) // This throws error
}
Where self.columns is an instance of this class:
private class Columns {
let id = Expression<Int>("id")
let start_time = Expression<Double>("start_time")
}
I'm getting an error complaining that there was an unexpected null value for start_time
Unexpected null value for column "start_time"
Where the value is indeed non-null. In fact when I print out the row (variable i) this is how it looks like:
Row(columnNames: ["\"id\"": 0, "\"start_time\"": 1], values: [Optional(355487), Optional(1585212120000)])
Obviously, the value is not a null. So what is going on here?
I just had the same issue and came to your answer. Diving into the source, I found that it basically had to do with data type you are requesting. The error is thrown here:
public func get<V: Value>(_ column: Expression<V>) throws -> V {
if let value = try get(Expression<V?>(column)) {
return value
} else {
throw QueryError.unexpectedNullValue(name: column.template)
}
}
So basically you need to request the data in the correct type, it won't be casted for you automatically. In your case, that would be Int instead of Double
My Original Code
func getUserByEmpNum(_ id: Int) -> String {
let nameQuery: String = "SELECT fld_str_firstname, fld_str_lastName FROM userView WHERE fld_int_id = \(id);"
var returnStr = ""
do {
let dbQueue = try DatabaseQueue(path: MYCDatabase.pathToDatabase)
try dbQueue.inTransaction(.none) { (db) -> Database.TransactionCompletion in
let returnStrs = try String.fetchAll(db, sql: nameQuery)
// Placing a breakpoint here, returnStrs only has one element?
return .commit
}
} catch {
print (error)
}
return returnStr
}
My Question
In this code if I do a query like select fld_str_firstname from myOwnUserView where fld_int_id = 2; I get one element in my returnStrs array, which is as expected. Then selecting the two fields, as in nameQuery, I still only ever get one string in the returnStrs array.
Why is this, and how do I fit it to get all the selected columns in the response?
String.fetchAll returns an array of Strings extracted from the leftmost selected column, as documented. One string for each fetched row. Not one string for each selected column.
If you want to grab strings from several columns, use Row.fetchAll, which returns an array of database rows. From those rows, you can extract each column you are interested into:
let rows = try Row.fetchAll(db, sql: "SELECT fld_str_firstname, fld_str_lastName FROM ...")
for row in rows {
let firstName: String = row["fld_str_firstname"]
let lastName: String = row["fld_str_lastName"]
}
See this chapter of the documentation for more information about extracting values from database rows.
Since you are reading the name from a single row identified with its id, you may prefer the fetchOne method, which consumes a single database row (see Fetching Methods):
if let row = try Row.fetchOne(db, sql: "SELECT ... WHERE fld_int_id = ?", arguments: [id]) {
let firstName: String = row["fld_str_firstname"]
let lastName: String = row["fld_str_lastName"]
// Use first and last name
} else {
// ID does not exist in the database: do what is appropriate.
}
I'm trying to insert a row with a column that is an array of a custom type (ingredient). My tables are:
CREATE TYPE ingredient AS (
name text,
quantity text,
unit text
);
CREATE TABLE IF NOT EXISTS recipes (
recipe_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
name text,
ingredients ingredient[],
// ...
);
Using raw sql, I can insert a row by:
INSERT INTO recipes (name, ingredients) VALUES ('some_name', ARRAY[ROW('aa', 'bb', 'cc'), ROW('xx', 'yy', 'zz')]::ingredient[] );
But I'm struggling to do this in go with the pq lib. I've created a pq.Array interface:
type Ingredient struct {
Name string
Quantity string
Unit string
}
type Ingredients []*Ingredient
func (ings *Ingredients) ConvertValue(v interface{}) (driver.Value, error) {
return "something", nil
}
func (ings *Ingredients) Value() (driver.Value, error) {
val := `ARRAY[]`
for i, ing := range ings {
if i != 0 {
val += ","
}
val += fmt.Printf(`ROW('%v','%v','%v')`, ing.Name, ing.Quantity, ing.Unit)
}
val += `::ingredient[]`
return val, nil
}
// and then trying to insert via:
stmt := `INSERT INTO recipes (
name,
ingredients
)
VALUES ($1, $2)
`
_, err := db.Exec(stmt,
"some_name",
&Ingredients{
&Ingredient{"flour", "3", "cups"},
},
)
But pg keeps throwing the error:
Error insertingpq: malformed array literal: "ARRAY[ROW('flour','3','cups')]::ingredient[]"
Am I returning an incorrect driver.Value?
You can either use this approach outlined here: https://github.com/lib/pq/issues/544
type Ingredient struct {
Name string
Quantity string
Unit string
}
func (i *Ingredient) Value() (driver.Value, error) {
return fmt.Sprintf("('%s','%s','%s')", i.Name, i.Quantity, i.Unit), nil
}
stmt := `INSERT INTO recipes (name, ingredients) VALUES ($1, $2::ingredient[])`
db.Exec(stmt, "some_name", pq.Array([]*Ingredient{{"flour", "3", "cups"}}))
Or if you have records in the table and you query it, you will probably see the ingredient array in its literal form, which you can than mimic during insert.
func (ings *Ingredients) Value() (driver.Value, error) {
val := `{`
for i, ing := range ings {
if i != 0 {
val += ","
}
val += fmt.Sprintf(`"('%s','%s','%s')"`, ing.Name, ing.Quantity, ing.Unit)
}
val += `}`
return val, nil
}
// e.g. `{"('flour','3','cups')"}`
stmt := `INSERT INTO recipes (name, ingredients) VALUES ($1, $2::ingredient[])`
// ...
It seems your database design is very complicated and not taking into account the strengths (and weaknesses) of SQL.
May I suggest you split the ingredients into their own table with a reference to the recipe. Then finding out a full recipe is a JOIN operation.
Creating the DB:
CREATE TABLE ingredients (
recipe_id uuid,
name text,
quantity int,
unit text
);
CREATE TABLE recipes (
recipe_id uuid PRIMARY KEY,
name text
);
Inserting a recipe and querying to read it out:
INSERT INTO recipes VALUES (
'5d1cb631-37bd-46cc-a278-4c8558ed8964', 'cake1'
);
INSERT INTO ingredients (recipe_id, name, quantity, unit) VALUES
('5d1cb631-37bd-46cc-a278-4c8558ed8964', 'flour', 3, 'cups'),
('5d1cb631-37bd-46cc-a278-4c8558ed8964', 'water', 1, 'cups')
;
SELECT r.name, i.name, i.quantity, i.unit
FROM ingredients AS i
INNER JOIN recipes AS r
ON r.recipe_id=i.recipe_id;
SQLFiddle link: http://sqlfiddle.com/#!17/262ad/14
I have been trying to use the postgres IN clause in golang, but keep getting errors. This is the query I want to execute.
SELECT id1 FROM my_table WHERE type = (an int) AND id2 = (an int) AND id1 IN (list of UUIDs)
I used this code to construct this query but got the following error.
var params []interface{}
inCondition := ""
params = append(params, type)
params = append(params, id2)
for _, id := range id1 {
params = append(params, id)
if inCondition != "" {
inCondition += ", "
}
inCondition += "?"
}
query := fmt.Sprintf(`SELECT id1 FROM my_table WHERE type = ? AND id2 = ? AND id1 IN (%s)`, inCondition)
rows, err := db.Query(query, params...)
Query I got:
SELECT id1 FROM my_table WHERE type = ? AND id2 = ? AND id1 IN (?, ?, ?)
Params output:
[]interface {}=[0 7545449 d323f8d5-ab97-46a3-a34e-95ceac2f3a6a d323f8d5-ab97-46a3-a34e-95ceac2f3a6b d323f8d5-ab97-46a3-a34e-95ceac2f3a6d]
Error:
pq: syntax error at or near \"AND\""
What am I missing? or, how will I get this to work? id1 is a slice of UUIDs whose length is variable.
Ran into a similar issue. Can't remember where exactly I picked this up but I remember still running into issues when dealing with arrays of type integer vs string. What I had to do was to have a local custom type and return a driver compatible value for it. See sample below.
// Int64Array is a type implementing the sql/driver/value interface
// This is due to the native driver not supporting arrays...
type Int64Array []int64
// Value returns the driver compatible value
func (a Int64Array) Value() (driver.Value, error) {
var strs []string
for _, i := range a {
strs = append(strs, strconv.FormatInt(i, 10))
}
return "{" + strings.Join(strs, ",") + "}", nil
}
Highly recommend checking out sqlx. Wrote a simple orm wrapper called papergres to make my go + postgres life easier :) Give it a try.
Instead of ?, using $1,$2 etc as placeholder worked.
I am trying to create a type safe data access layer in F# using FSharp.Data.SqlClient type providers to be callable from C#. I have highly complicated SQL queries that are performance critical but there are also several variations.
Given the following (obviously simplified schema):
CREATE TABLE [dbo].[company] (
[id] INT IDENTITY (1, 1) NOT NULL,
[name] VARCHAR (50) NOT NULL)
I have the following F# code:
module CompanyDAL =
open FSharp.Data // requires FSharp.Data.SqlClient package
[<Literal>]
let conn = "Data Source=(localdb)\ProjectsV13;Initial Catalog=TESTDB;Integrated Security=True;Pooling=False;Connect Timeout=30"
[<Literal>]
let baseQuery = "select id, name from company where 1 = 1 "
[<Literal>]
let filterById = "and id = #id"
[<Literal>]
let filterByName = "and name = #name"
[<Literal>]
let queryFilteredById = baseQuery + filterById
type GetCompanyById = SqlCommandProvider<queryFilteredById, conn>
[<Literal>]
let queryFilterbyName = baseQuery + filterByName
type GetCompanyByName = SqlCommandProvider<queryFilterbyName, conn>
type GetAllCompanies = SqlCommandProvider<baseQuery, conn>
type Company = {
Name : string
id : int
}
let getCompanyById (runtimeConn:string) =
async {
use query = new GetCompanyById(runtimeConn)
let! result = query.AsyncExecute(id = 10)
return result
|> Seq.map (fun x ->
{ Name = x.name
id = x.id })
|> Seq.toArray
} |> Async.StartAsTask
let getCompanyByName (runtimeConn:string) =
async {
use query = new GetCompanyByName(runtimeConn)
let! result = query.AsyncExecute(name = "abc" )
return result
|> Seq.map (fun x ->
{ Name = x.name
id = x.id })
|> Seq.toArray
} |> Async.StartAsTask
let getAllCompanies (runtimeConn:string) =
async {
use query = new GetAllCompanies(runtimeConn)
let! result = query.AsyncExecute()
return result
|> Seq.map (fun x ->
{ Name = x.name
id = x.id })
|> Seq.toArray
} |> Async.StartAsTask
Is there any way I can reduce the number of duplication while maintaining the raw performance of type safe raw queries?
I've tried using
type GetCompany = SqlEnumProvider<baseQuery, conn, provider>
Then I can write this code
GetCompany.Items |> Seq.where(fun (id, name) -> name = "xxx") |> printfn "%A"
So I can avoid duplication and pass a generic predicate fun (id, name) -> to the only GetCompany type.
Similarly,
GetCompany.Items |> Seq.where(fun (id, name) -> id = "1")
would work as well.
Note
Anyway, notice that the above will filter in memory. In fact Dynamic SQL is not permitted with this library, because it relies only on static analysis at compile time. If you need to construct queries dynamically, you can revert to raw ADO.NET or other light ORM.