While trying to insert into postgres db via gorm It is unable to convert time.Time information into timestamptz.
The error message shown is:
unable to encode time.Date(2023, time.January, 8, 1, 34, 22, 4178000, time.Local) into binary format for _timestamptz (OID 1185): cannot find encode plan
I created the struct as follows:
type Instance struct {
UpdatedAt time.Time `gorm:"index"`
InstanceId string `gorm:"primaryKey"`
InstanceName string
InstanceType string
InstanceState string
LaunchTime time.Time
ExitTime time.Time
PausedTimes []time.Time `gorm:"type:timestamptz[]"`
ResumedTimes []time.Time `gorm:"type:timestamptz[]"`
}
To Insert I tried running:
dbConnection.Clauses(clause.OnConflict{
UpdateAll: true,
}).Create(&model.InstanceDB{
InstanceId: "1",
PausedTimes: []time.Time{time.Now(), time.Now().Add(50000)},
ResumedTimes: []time.Time{time.Now()},
})
I though about using pq.Array but I couldn't find if it supports TimeArray.
Alternative approach would be to use unix timestamps using pq.Int64Array but I wanted to see if I could resolve this and keep using time.Time.
Got the same issue. Only downgrading helped:
gorm.io/driver/postgres v1.4.4
gorm.io/gorm v1.23.7
Possibly related:
https://github.com/go-gorm/gorm/issues/5960
https://github.com/go-gorm/postgres/issues/152
Best make a bug report.
My struct looks like:
type AdvertContent struct {
Id string `gorm:"column:id;primaryKey;type:uuid;default:uuid_generate_v4()" json:"id" example:"4ff8eb91-640b-4e26-a50f-3bcd1f933d0c"`
FromTime *time.Time `gorm:"column:from_time;type:time;" json:"fromTime,omitempty" example:"HH:MM"`
ToTime *time.Time `gorm:"column:to_time;type:time;" json:"toTime,omitempty" example:"HH:MM"`
} //#name AdvertContent
func (this AdvertContent) TableName() string {
return "advert_content"
}
When I use gorm.AutoMigrate, table fields from_time and to_time created with type timestampz, not time.
Gorm debug mode:
CREATE TABLE "advert_content" ("id" uuid DEFAULT uuid_generate_v4(),"from_time" timestamptz,"to_time" timestamptz)
How can I create table with time type fields?
when using specified database data type, it needs to be a full
database data type, for example: MEDIUMINT UNSIGNED NOT NULL
AUTO_INCREMENT
You should use the database data type like this
I have the following struct:
type TypeIncidence struct {
Number int bson:"number" json:"number"
Description string bson:"description" json:"description"
Date_time_stamp string bson:"dateTimeStamp" json:"date_time_stamp"
}
and I want insert a document in a collection:
type TypeIncidence struct {
Number int `bson:"number" json:"number"`
Description string `bson:"description" json:"description"`
Date_time_stamp **string?**
}
var incidence TypeIncidence
incidence.Number = 1
Description =" Text"
Date_time_stamp = **string?**
What data type would I have to use in a Golang structure to store date_time_struct a string?
If I want to store with the following format 'YYYY-MM-DD hh:mm:ss', what module and/or function should I use in golang? (in local machine or server converting zone time)
Thanks in advance
You can use time.Time:
CreatedAt time.Time `json:"created_at" bson:"created_at"`
However, I would recommend that you store Epoch Unix timestamp (the number of seconds since Jan 1st 1970) because it is universal:
CreatedAt int64 `json:"created_at" bson:"created_at"`
I have tried in the past to store time.Time in MongoDB through Golang but then I had trouble when I parsed the same information into a datetime object in Python. If you would like to be compatible across languages and technologies, storing the Epoch Unix timestamp would be a great option.
I have the following model...
type User struct {
ID string `sql:"type:uuid;primary_key;default:uuid_generate_v4()"`
FirstName string `form:"first_name" json:"first_name,omitempty"`
LastName string `form:"last_name" json:"last_name,omitempty"`
Password string `form:"password" json:"password" bindind:"required"`
Email string `gorm:"type:varchar(110);unique_index" form:"email" json:"email,omitempty" binding:"required"`
Location string `form:"location" json:"location,omitempty"`
Avatar string `form:"avatar" json:"avatar,omitempty"`
BgImg string `form:"bg_img" json:"bg_img,omitempty"`
CreatedAt time.Time
UpdatedAt time.Time
DeletedAt time.Time
}
I've tried several different ways, but this way throws (pq: relation "users" does not exist). I have no related models, it's literally just that one model.
I've tried using...
func (user *User) BeforeCreate(scope *gorm.Scope) error {
scope.SetColumn("ID", uuid.NewV4())
return nil
}
Along with a uuid lib, but had no luck with that either.
Turns out I was trying to store the UUID as the wrong type, I was doing...
func (user *User) BeforeCreate(scope *gorm.Scope) error {
scope.SetColumn("ID", uuid.NewV4())
return nil
}
When it needed to be...
func (user *User) BeforeCreate(scope *gorm.Scope) error {
scope.SetColumn("ID", uuid.NewV4().String())
return nil
}
For postgresql, here is what I did:
go get github.com/google/uuid
Use uuid.UUID (from "github.com/google/uuid"), as type,
e.gID uuid.UUID `gorm:"type:uuid;default:uuid_generate_v4()"`
Add uuid-ossp extension for postgres database,
e.g
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
Then, when you call DB's Create() method, the uuid is generated automatically.
Update: pg14+ gen_random_uuid()
(as mentioned in Doron Segal's comment)
pg 14 has built-in function gen_random_uuid() to generate uuid v4, e.g:
create table:
create table uuid_test (uid text default gen_random_uuid());
insert a row:
insert into uuid_test(uid) values (DEFAULT);
Then uid column is generated automatically.
Similiar, in go you can use the function as defaul value I think, e.g:
ID uuid.UUID gorm:"type:uuid;default:gen_random_uuid()"
BTW, the gen_random_uuid() function only support uuid v4 now, to use other versions, you still need uuid-ossp extension.
For this you will need gorm and go.uuid
go get github.com/jinzhu/gorm
go get github.com/satori/go.uuid
Try creating your own model base model in place of gorm.Model like so:
type Base struct {
ID string `sql:"type:uuid;primary_key;default:uuid_generate_v4()"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
DeletedAt *time.Time `sql:"index" json:"deleted_at"`
}
You would then populate this field using a method called before creation of any record, like so:
func (base *Base) BeforeCreate(scope *gorm.Scope) error {
id, err := uuid.NewV4()
if err != nil {
return err
}
return scope.SetColumn("ID", uuid.String())
}
Therefore, for your particular case, you would have:
type User struct {
Base
FirstName string `form:"first_name" json:"first_name,omitempty"`
LastName string `form:"last_name" json:"last_name,omitempty"`
Password string `form:"password" json:"password" bindind:"required"`
Email string `gorm:"type:varchar(110);unique_index" form:"email" json:"email,omitempty" binding:"required"`
Location string `form:"location" json:"location,omitempty"`
Avatar string `form:"avatar" json:"avatar,omitempty"`
BgImg string `form:"bg_img" json:"bg_img,omitempty"`
}
More details on this can be found here
This was my solution for Gorm v1.21
go get gorm.io/gorm
go get gorm.io/driver/postgres
go get github.com/google/uuid
import (
"gorm.io/gorm"
"github.com/google/uuid"
)
type User struct {
Id: string `gorm:"primaryKey"`
}
// Note: Gorm will fail if the function signature
// does not include `*gorm.DB` and `error`
func (user *User) BeforeCreate(tx *gorm.DB) (err error) {
// UUID version 4
user.Id = uuid.NewString()
return
}
Notes:
For the Google UUID package, the methods uuid.New() and uuid.NewString() use UUID version 4. This is not clearly stated in the documentation (http://pkg.go.dev/github.com/google/uuid), but by looking into the source code, you can see that these are wrappers around uuid.NewRandom() which is stated as being UUID version 4.
While some recommend the Satori UUID package (https://github.com/satori/go.uuid), benchmarks show that it has 3.3x lower performance than the Google UUID package
(https://gist.github.com/mattes/69a4ab7027b9e8ee952b5843e7ca6955)
The error (pq: relation "users" does not exist) usually means that, the table users does not exists in the database. It has nothing to do with the relationship between two models.
So basically, You first need to create the table in the database (Or auto migrate the database As per #Apin suggest). And try to re-run the same code.
None of these worked for me using gorm v1.21.
Here was my solution. Note that I'm using the satori/go.uuid library for generating UUID, but code with google's library is near identical.
type UUIDBaseModel struct {
ID uuid.UUID `gorm:"primary_key" json:"id"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
DeletedAt *time.Time `sql:"index" json:"deleted_at"`
}
func (base *UUIDBaseModel) BeforeCreate(tx *gorm.DB) error {
uuid := uuid.NewV4().String()
tx.Statement.SetColumn("ID", uuid)
return nil
}
Now I used Gorm 2.0 and this worked:
go get github.com/satori/go.uuid
type Tablename struct {
ID string `sql:"type:uuid;primary_key;default:uuid_generate_v4()"`
}
Why gorm is ignoring sql:"index" tags? No indexes got created.
Database in use here is PostgreSQL (importing _ "github.com/lib/pq"). This Model struct is used (because default gorm.Model uses an auto increment number - serial - as primary key and I wanted to set id myself):
type Model struct {
ID int64 `sql:"type:bigint PRIMARY KEY;default:0"`
CreatedAt time.Time
UpdatedAt time.Time
DeletedAt *time.Time `sql:"index"`
}
And one of actual models is:
type TUHistory struct {
Model
TUID int64 `json:"tu_id,string" gorm:"column:tu_id" sql:"index"`
}
func (x *TUHistory) TableName() string {
return "tu_history"
}
And the table is created by db.CreateTable(&TUHistory{}) which creates the table correctly except for indexes.
As a temporary work around, I do db.Model(&TUHistory{}).AddIndex("ix_tuh_tu_id", "tu_id") to create indexes.
From my experience, the db.CreateTable only creates the table and it's fields. You are better off using the AutoMigrate function with the model structure that you want to migrate:
db, err := gorm.Open("postgres", connectionString)
...
// error checking
...
db.AutoMigrate(&Model)
Also, I tried AutoMigrating the model you posted and got an error saying that multiple primary keys are not allowed, so I changed the model to:
type Model struct {
Id int64 `sql:"type:bigint;default:0"`
CreatedAt time.Time
UpdatedAt time.Time
DeletedAt *time.Time `sql:"index"`
}
and the AutoMigration created all PKs and indexes just fine.
Edit:
Checking the GORM's README, on this example, the Email structure goes as:
type Email struct {
ID int
UserID int `sql:"index"` // Foreign key (belongs to), tag `index` will create index for this field when using AutoMigrate
Email string `sql:"type:varchar(100);unique_index"` // Set field's sql type, tag `unique_index` will create unique index
Subscribed bool
}
Notice the comment on the UserId field saying it will create the index when using AutoMigrate.
Also, it's worth taking a look at how the AutoMigrate does it's job:
// Automating Migration
db.AutoMigrate(&User{})
db.Set("gorm:table_options", "ENGINE=InnoDB").AutoMigrate(&User{})
db.AutoMigrate(&User{}, &Product{}, &Order{})
// Feel free to change your struct, AutoMigrate will keep your database up-to-date.
// AutoMigrate will ONLY add *new columns* and *new indexes*,
// WON'T update current column's type or delete unused columns, to protect your data.
// If the table is not existing, AutoMigrate will create the table automatically.