I am getting this error when I try to retreive data from a database:
thread 'main' panicked at 'error retrieving column 2: error deserializing column 2: cannot convert between the Rust type `alloc::string::String` and the Postgres type `timestamp`'
Db structure:
CREATE TABLE IF NOT EXISTS table_(
id SERIAL PRIMARY KEY,
data VARCHAR NOT NULL,
date_saved TIMESTAMP
)
struct MyType{
local_id: i32,
data: String,
date_saved: String
}
let records = client.query("SELECT id,data,date_saved FROM table_",&[])?;
let mut the_records : Vec<MyType> = vec![];
for record in records {
let saved_data = MyType {
local_id: record.get(0),
data: record.get(1),
date_saved: record.get(2),
};
println!("{:?}",saved_data.data);
the_records.push(saved_data);
}
I found out that there is no possible conversion between Postgres Timestamp and String according to https://docs.rs/postgres/0.17.5/postgres/types/trait.FromSql.html but we need to use std::time::SystemTime.
So MyType will be:
struct MyType{
local_id: i32,
data: String,
date_saved: std::time::SystemTime
}
And I can manipulate time from there.
The above answer is good, but if you want a quicker solution (for e.g. want to just print to screen / log etc.), just cast the Timestamp to a TEXT within Postgres and then rust wouldn't complain.
So for e.g. this would throw ERROR:
SELECT now(); -- Will throw error
But this wouldn't
SELECT now()::TEXT; -- Will work fine
Related
I'm triying to do an Api REST with rust and postres but I cant make it work because the relation between these two.
The actual problem is that I have a column in postgres as jsonb and when I return the data and try to save it in a struct always gives error. Same problem when I try to save the data.
This are the models.(The option is only because I'm testing thing, it should return a value)
#[derive(Debug, Serialize, Deserialize)]
pub struct CategoryView {
pub id: i32,
pub category_name: String,
pub category_custom_fields: Option<serde_json::Value>,
}
#[derive(Debug, Serialize, Deserialize)]
pub struct CategoryPayload {
pub category_name: String,
pub category_custom_fields: Option<serde_json::Value>,
}
This are the postgres queries:
fn find_all(conn: &mut DbPooled) -> Result<Vec<CategoryView>, DbError> {
let mut query = "SELECT id, category_name, category_custom_fields FROM accounting.categories".to_owned();
query.push_str(" WHERE user_id = $1");
query.push_str(" AND is_deleted = false");
let items = conn.query(&query, &[&unsafe { CURRENT_USER.to_owned() }])?;
let items_view: Vec<CategoryView> = items
.iter()
.map(|h| CategoryView {
id: h.get("id"),
category_name: h.get("category_name"),
category_custom_fields: h.get("category_custom_fields"),
})
.collect();
Ok(items_view)
}
fn add(payload: &CategoryPayload, conn: &mut DbPooled) -> Result<CategoryView, DbError> {
let mut query =
"INSERT INTO accounting.categories (user_id, category_name, category_custom_fields, create_date, update_date)"
.to_owned();
query.push_str(" VALUES ($1, $2, $3, now(), now())");
query.push_str(" RETURNING id");
let item_id = conn
.query_one(
&query,
&[
&unsafe { CURRENT_USER.to_owned() },
&payload.category_name,
&payload.category_custom_fields,
],
)?
.get(0);
let inserted_item = CategoryView {
id: item_id,
category_name: payload.category_name.to_string(),
category_custom_fields: payload.category_custom_fields,
};
Ok(inserted_item)
}
with update happens to but I think is the same solution that the one form the add function.
The error is:
the trait bound `serde_json::Value: ToSql` is not satisfied
the following other types implement trait `ToSql`:
&'a T
&'a [T]
&'a [u8]
&'a str
Box<[T]>
Box<str>
Cow<'a, str>
HashMap<std::string::String, std::option::Option<std::string::String>, H>
and 17 others
required for `std::option::Option<serde_json::Value>` to implement `ToSql`
required for the cast from `std::option::Option<serde_json::Value>` to the object type `dyn ToSql + Sync`rustcClick for full compiler diagnostic`
For what I read serde_json::Value is the equivalent to jsonb so I don't understand it.
I had a similar problem previously trying to work with a decimal value in postgres, I had to change it to integer and save the value multiplied in the database. Is a money column so maybe if you help me with that too I will change it back.
I was hopping some could explain to me how to fix it and why this happens so I can avoid have to ask for help with the datatypes in the future.
The problem was in the depencies.
It looks like some dependencies have features that add aditional functionablility.
I had installed the dependencie without any feature so when I added the features it started to work without issues.
Only had to change from:
[dependencies]
postgres = "0.19.4"
to:
[dependencies]
postgres = { version = "0.19.4", features = ["with-chrono-0_4", "with-serde_json-1"] }
Chrono for dates and serde_json for jsonb.
I'll check the decimal problem but I think will be the same solution.
I have created table like this :
CREATE TABLE MyTable
(
id uuid,
Test BOOLEAN NOT NULL,
end_date TIMESTAMP NULL DEFAULT NULL,
PRIMARY KEY (id)
);
My struct
type Issue struct {
ID uuid.UUID
Test bool
EndDate time.Time `db:"due_date"`
}
Now case is there was some date without EndDate in live database so now where I query to get all data I am getting this error
ERROR: sql: Scan error on column index 11, name "i.end_date": unsupported Scan, storing driver.Value type <nil> into type *time.Time
I Don't getting any idea where issue can be .
UPDATE
If I use sql.NullTime and then I made a respoce mode like this
return &model.Issue{
AssetOwnerID: id,
DueDate : time.Now().UTC().Truncate(time.Second)
}
I am getting this error
Cannot use 'time.Now().UTC().Truncate(time.Second)' (type Time) as type sql.NullTime
You could use the sql.NullTime type, as example:
import (
"database/sql"
)
type Issue struct {
ID uuid.UUID
Test bool
EndDate sql.NullTime `db:"due_date"`
}
Then, you could use as examples:
Read operation:
if i.EndDate.Valid {
fmt.Println(i.EndDate.Time.Unix())
} else {
fmt.Println("nil endDate")
}
Write operation:
i.EndTime.Valid = true
i.EndTime.Time = time.Unix(iEndTime, 0)
UPDATE:
You can create the struct as:
return &model.Issue{
AssetOwnerID: id,
DueDate: sql.NullTime{
Time: time.Now().UTC().Truncate(time.Second),
Valid: true,
}
I'm using the rust-postgres crate to ingest data. This is a working example adding rows successfully:
let name: &str = "hello from rust";
let val: i32 = 123;
let now: DateTime<Utc> = Utc::now();
let timestamp = now.format("%Y-%m-%dT%H:%M:%S%.6f").to_string();
client.execute(
"INSERT INTO trades VALUES(to_timestamp($1, 'yyyy-MM-ddTHH:mm:ss.SSSUUU'),$2,$3)",
&[×tamp, &name, &val],
)?;
This doesn't look so nice as I have to do this forward and back string conversion, I would like to be able to write something like
let name: &str = "hello from rust";
let val: i32 = 123;
let now: DateTime<Utc> = Utc::now();
client.execute(
"INSERT INTO trades VALUES($1,$2,$3)",
&[&now, &name, &val],
)?;
What's the most performant way of ingesting timestamps in this way?
Edit:
Here's the returned error from the second example above
Error: Error { kind: ToSql(0), cause: Some(WrongType { postgres: Timestamp, rust: "chrono::datetime::DateTime<chrono::offset::utc::Utc>" }) }
And my cargo.toml looks like this (which has the chrono feature enabled for the rust postgres crate):
[dependencies]
chrono = "0.4.19"
postgres={version="0.19.0", features=["with-serde_json-1", "with-bit-vec-0_6", "with-chrono-0_4"]}
I think the problem is a mismatch between your postgres schema and your Rust type: the error seems to say that your postgres type is timestamp, while your rust type is DateTime<Utc>.
If you check the conversion table, DateTime<Utc> converts to a TIMESTAMP WITH TIME ZONE. The only types which convert to TIMESTAMP are NaiveDateTime and PrimitiveDateTime.
As per Masklinn's response, I needed to pass a NaiveDateTime type for this to work, the full example with naive_local looks like:
use postgres::{Client, NoTls, Error};
use chrono::{Utc};
use std::time::SystemTime;
fn main() -> Result<(), Error> {
let mut client = Client::connect("postgresql://admin:quest#localhost:8812/qdb", NoTls)?;
// Basic query
client.batch_execute("CREATE TABLE IF NOT EXISTS trades (ts TIMESTAMP, date DATE, name STRING, value INT) timestamp(ts);")?;
// Parameterized query
let name: &str = "rust example";
let val: i32 = 123;
let utc = Utc::now();
let sys_time = SystemTime::now();
client.execute(
"INSERT INTO trades VALUES($1,$2,$3,$4)",
&[&utc.naive_local(), &sys_time, &name, &val],
)?;
// Prepared statement
let mut txn = client.transaction()?;
let statement = txn.prepare("insert into trades values ($1,$2,$3,$4)")?;
for value in 0..10 {
let utc = Utc::now();
let sys_time = SystemTime::now();
txn.execute(&statement, &[&utc.naive_local(), &sys_time, &name, &value])?;
}
txn.commit()?;
println!("import finished");
Ok(())
}
I have a table with timestamp TIMESTAMP, data TEXT columns. I have a failing test because I can't get a timestamp value out of postgresql without time zone annotation. Here's an abridged version of what I've done in my Go application:
type Datapoint struct {
Timestamp string
Data sql.NullString
}
var testData = Datapoint{Timestamp:'2018-12-31 00:00:00', Data:'test'}
db.Exec("CREATE TABLE mytable (id SERIAL, timestamp TIMESTAMP, data TEXT);")
db.Exec("INSERT INTO mytable(timestamp, data) VALUES ($1, $2);", testData.Timestamp, testData.Data)
datapoints, err = db.Exec("SELECT timestamp::TIMESTAMP WITHOUT TIME ZONE, data FROM mytable;")
This trouble is that this query (after about 20 lines of error checking and row.Scan; golang's a bit verbose like that...) gives me:
expected 2018-12-31 00:00:00, received 2018-12-31T00:00:00Z
I requested without timezone (and the query succeeds in psql), so why am I getting the extra T and Z in the string?
Scan into a value of time.Time instead of string, then you can format the time as desired.
package main
import (
"database/sql"
"fmt"
"log"
"time"
)
type Datapoint struct {
Timestamp time.Time
Data sql.NullString
}
func main() {
var db *sql.DB
var dp Datapoint
err := db.QueryRow("SELECT timestamp, data FROM mytable").Scan(
&dp.Timestamp, &dp.Data,
)
switch {
case err == sql.ErrNoRows:
log.Fatal("No rows")
case err != nil:
log.Fatal(err)
default:
fmt.Println(dp.Timestamp.Format("2006-01-02 15:04:05"))
}
}
What you are receiving is an ISO 8601 representation of time.
T is the time designator that precedes the time components of the representation.
Z is used to represent that it is in UTC time, with Z representing zero offset.
In a way you are getting something without a timezone but it can be confusing, especially as you haven't localised your time at any point. I would suggest you consider using ISO times, or you could convert your time to a string like this
s := fmt.Sprintf("%d-%02d-%02d %02d:%02d:%02d\n",
t.Year(), t.Month(), t.Day(),
t.Hour(), t.Minute(), t.Second())
Situation:
I'm using a postgres database and have the following struct:
type Building struct {
ID int `json:"id,omitempty"`
Name string `gorm:"size:255" json:"name,omitempty"`
Lon string `gorm:"size:64" json:"lon,omitempty"`
Lat string `gorm:"size:64" json:"lat,omitempty"`
StartTime time.Time `gorm:"type:time" json:"start_time,omitempty"`
EndTime time.Time `gorm:"type:time" json:"end_time,omitempty"`
}
Problem:
However, when I try to insert this struct into the database, the following error occurs:
parsing time ""10:00:00"" as ""2006-01-02T15:04:05Z07:00"": cannot
parse "0:00"" as "2006""}.
Probably, it doesn't recognize the StartTime and EndTime fields as Time type and uses Timestamp instead. How can I specify that these fields are of the type Time?
Additional information
The following code snippet shows my Building creation:
if err = db.Create(&building).Error; err != nil {
return database.InsertResult{}, err
}
The SQL code of the Building table is as follows:
DROP TABLE IF EXISTS building CASCADE;
CREATE TABLE building(
id SERIAL,
name VARCHAR(255) NOT NULL ,
lon VARCHAR(31) NOT NULL ,
lat VARCHAR(31) NOT NULL ,
start_time TIME NOT NULL ,
end_time TIME NOT NULL ,
PRIMARY KEY (id)
);
While gorm does not support the TIME type directly, you can always create your own type that implements the sql.Scanner and driver.Valuer interfaces to be able to put in and take out time values from the database.
Here's an example implementation which reuses/aliases time.Time, but doesn't use the day, month, year data:
const MyTimeFormat = "15:04:05"
type MyTime time.Time
func NewMyTime(hour, min, sec int) MyTime {
t := time.Date(0, time.January, 1, hour, min, sec, 0, time.UTC)
return MyTime(t)
}
func (t *MyTime) Scan(value interface{}) error {
switch v := value.(type) {
case []byte:
return t.UnmarshalText(string(v))
case string:
return t.UnmarshalText(v)
case time.Time:
*t = MyTime(v)
case nil:
*t = MyTime{}
default:
return fmt.Errorf("cannot sql.Scan() MyTime from: %#v", v)
}
return nil
}
func (t MyTime) Value() (driver.Value, error) {
return driver.Value(time.Time(t).Format(MyTimeFormat)), nil
}
func (t *MyTime) UnmarshalText(value string) error {
dd, err := time.Parse(MyTimeFormat, value)
if err != nil {
return err
}
*t = MyTime(dd)
return nil
}
func (MyTime) GormDataType() string {
return "TIME"
}
You can use it like:
type Building struct {
ID int `json:"id,omitempty"`
Name string `gorm:"size:255" json:"name,omitempty"`
Lon string `gorm:"size:64" json:"lon,omitempty"`
Lat string `gorm:"size:64" json:"lat,omitempty"`
StartTime MyTime `json:"start_time,omitempty"`
EndTime MyTime `json:"end_time,omitempty"`
}
b := Building{
Name: "test",
StartTime: NewMyTime(10, 23, 59),
}
For proper JSON support you'll need to add implementations for json.Marshaler/json.Unmarshaler, which is left as an exercise for the reader 😉
As mentioned in "How to save time in the database in Go when using GORM and Postgresql?"
Currently, there's no support in GORM for any Date/Time types except timestamp with time zone.
So you might need to parse a time as a date:
time.Parse("2006-01-02 3:04PM", "1970-01-01 9:00PM")
I am have come across the same error. It seems like there is a mismatch between type of the column in the database and the Gorm Model
Probably the type of the column in the database is text which you might have set earlier and then changed the column type in gorm model.