How to update postgres JSONB with variable params in golang? - postgresql

I have a table in cockroachdb/postgres as below:
column_name | data_type | is_nullable | column_default | generation_expression | indices | is_hidden
+-------------+-----------+-------------+----------------+--------------------+-----------+-----------+
id | STRING | false | NULL | | {primary} | false
student | JSONB | true | NULL | | {} | false
(2 rows)
id | student
+--------------------+----------------------------------------------+
1 | {"name": "Albert"}
2 | {"name": "Bob", "state": "CA"}
I am trying to update student with variable number of params - for example, some times update age, some times update age and country etc. How do we do this in golang?
Here is what I have so far, but it does not work.
package main
import (
"database/sql"
"fmt"
_ "github.com/lib/pq"
)
var (
DB = "testdb"
DBUSER = "testuser"
TESTTABLE = "ttab"
CONNSTR = "dbname=" + DB + " user=" + DBUSER + " host=localhost port=26257 sslmode=disable"
DBDRIVER = "postgres"
)
var (
NAME = "name"
AGE = "age"
STATE = "state"
COUNTRY = "country"
)
func update(si map[string]string, id string) error {
silen := len(si)
if silen == 0 {
return fmt.Errorf("si cannot be empty")
}
keys := []string{}
values := []string{}
for k, v := range si {
keys = append(keys, k)
values = append(values, v)
}
db, err := sql.Open(DBDRIVER, CONNSTR)
if err != nil {
return err
}
defer db.Close()
sqlCmd := "UPDATE " + TESTTABLE + " SET student = student || jsonb_object($1, $2) WHERE id = $3"
_, err := db.Exec(sqlCmd, keys, values, id)
return err
}
func main() {
s := make(map[string]string)
s[AGE] = "22"
s[COUNTRY] = "USA"
if err := updateFast3DB(s3, "1"); err != nil {
fmt.Printf("err: %v\n", err)
}
}
Error:
[root#bin]# ./updatedb
update error: sql: converting argument $1 type: unsupported type []string, a slice of string
[root#bin]#

For anyone coming to this later, make sure to read libpq's documentation on passing array values to the driver.
To pass an array into the libpq driver, you must wrap the slice in a pq.Array first. In the example above, the code should look like this:
_, err := db.Exec(sqlCmd, pq.Array(keys), pq.Array(values), id)

Related

Is there such function like IsCallable?

I'd like to have in AutoHotkey function named, for example IsCallable, that can tell me whether an object (or anything that can be stored in AHK variable) is callable.
A callable object includes:
a normal function represented as a string.
a function object by Func("somefuncname").
a BoundFunc Object.
The key point is, if fnobj is callable, then the user can write %fnobj%(...) to actually call something.
Clarify my idea with code below:
test_check_callable()
{
fnstr := "fnhello"
fnobjA := Func("fnhello")
fnobjB := fnobjA.bind("0th")
fnobjC := fnobjB
%fnstr%()
%fnobjA%("A1")
%fnobjB%("B1")
fnobjC.("C1", "C2")
r1 := IsCallable(fnstr) ; true
r2 := IsCallable(fnobjA) ; true
r3 := IsCallable(fnobjB) ; true
r4 := IsCallable(fnobjC) ; true
e1 := IsCallable("NoSuch") ; false
e2 := IsCallable({key1:"value1"}) ; false
}
fnhello(args*)
{
Dbgwin_Output("fnhello() parameters: " args.Length())
for i,arg in args
{
Dbgwin_Output(Format(" param{} = {}", i, arg))
}
}
IsCallable(pobj)
{
; How to implement this? IsFunc? IsObject?
}
I hope r1, r2, r3, r4 should all get true, and e1, e2 get false.
I'm using Autohotkey 1.1.36.2 .
PS: Dbgwin_Output() is implemented here: https://github.com/chjfth/AmHotkey/blob/master/libs/debugwin.ahk
If you used AHKv2, you could make use of HasMethod. I'd recommend the usage of AHKv2, it's already at the RC3 stage.
Something like this should work pretty well to cover all basic use cases:
fnStr := "fnHello"
fnObjA := %fnStr%
fnObjB := fnobjA.bind("0th")
fnObjC := fnObjB
class HelloClass
{
Call() => MsgBox("Hello")
}
fnClass := HelloClass()
class NotCallable
{
}
no1 := "NoSuch"
no2 := {key: "value"}
classNo := NotCallable()
MsgBox(
"`"fnHello`": " IsCallable("fnHello") "`n" ;support pure strings
"fnStr: " IsCallable(fnStr) "`n" ;support string objects
"fnObjA: " IsCallable(fnObjA) "`n" ;support user-defined function objects
"fnObjB: " IsCallable(fnObjB) "`n" ;support bound function objects
"fnObjC: " IsCallable(fnObjC) "`n" ;same as fnObjA
"`"MsgBox`": " IsCallable("MsgBox") "`n" ;support built-in functions as pure strings
"MsgBox: " IsCallable(MsgBox) "`n" ;support built-in functions
"fnClass: " IsCallable(fnClass) "`n`n" ;support user defined classes
"`"NoSuch`": " IsCallable("NoSuch") "`n"
"no1: " IsCallable(no1) "`n"
"no2: " IsCallable(no2) "`n"
"classNo: " IsCallable(classNo) "`n"
)
fnHello(param := "")
{
MsgBox("hi " param)
}
IsCallable(inp)
{
if (HasMethod(inp))
return true
try
{
inp := %inp%
}
catch
{
return false
}
return HasMethod(inp)
}
Result:
"fnHello": 1
fnStr: 1
fnObjA: 1
fnObjB: 1
fnObjC: 1
"MsgBox": 1
MsgBox: 1
fnClass: 1
"NoSuch": 0
no1: 0
no2: 0
classNo: 0

conn closed when trying to make an insert into postgresql

I try to insert a user into postgres database and get three parameters back for futher processing but always get the error 'conn closed':
package db
import (
"context"
"os"
"github.com/jackc/pgx/v4"
)
const (
insertSql = "INSERT into users (name, email, created, status, role, password, activation_code) VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING name, email, activation_code;"
)
type userRepository struct {
connect *pgx.Conn
}
func NewUserRepository(c *pgx.Conn) domain.UserRepository {
return &userRepository{
connect: c,
}
}
func (r *userRepository) Save(u *domain.User) (string, string, string, *resterrors.RestErr) {
var name, email, activation_code string
if r.connect == nil {
os.Exit(1)
}
if err := r.connect.QueryRow(context.Background(), insertSql, u.Name, u.Email, u.Created, u.Status, u.Role, u.Password, u.ActivationCode).Scan(&name, &email, &activation_code); err != nil {
logger.Error("error saving user to the database: " + err.Error())
return "", "", "", resterrors.NewIntenalServerError(resterrors.DB_ERROR)
}
return name, email, activation_code, nil
}
connect is inserted in application.go when respository is created
repository := db.NewUserRepository(postgresql.Connect)
userHandler := controller.NewUserHandler(service.NewUserService(repository))
Connect is globally declared in postgresql package
package postgresql
import (
"context"
"fmt"
"net/url"
"github.com/jackc/pgx/v4"
)
var (
Connect *pgx.Conn
username = "postgres"
password = "***********"
host = "127.0.0.1"
port = "5432"
dbName = "test"
)
func init() {
//prepare URL to connect to database
var err error
datasourceName := fmt.Sprintf("postgres://%s:%s#%s:%s/%s", username, password, host, port, dbName)
Connect, err = pgx.Connect(context.Background(), datasourceName)
if err != nil {
logger.Error("unable to connect to database: " + err.Error())
panic(fmt.Errorf("unable to connect to database: %w", err))
}
logger.Info("successsfully connected to postgres database")
// to close DB connection
defer Connect.Close(context.Background())
}
What could be wrong with it?
A single DB Connection is not concurrency safe and has no mechanism for reestablishing connections. Your issue is likely that concurrent access borked the connection, or it was closed for some other reason.
You want a DB Connection Pool so that it can create new connections for concurrent operations (requests for net/http servers handle connections concurrently). From https://pkg.go.dev/github.com/jackc/pgx/v4#hdr-Connection_Pool:
*pgx.Conn represents a single connection to the database and is not concurrency safe. Use sub-package pgxpool for a concurrency safe connection pool.
You should have better results with pgxpool - see https://pkg.go.dev/github.com/jackc/pgx/v4#v4.14.1/pgxpool#hdr-Establishing_a_Connection
The reason for the conn closed error was that the connection was closed within init() function straight after the application start.
defer Connect.Close(context.Background())

on every api call on db, gorm scope keep adding to previous one

Below is my api handler.
On first run it gives me correct output and corresponding call in db is also correct.
2020-07-04 14:51:04.512 IST [32129] postgres#postgres LOG: execute <unnamed>: SELECT count(*) FROM "access_owners" WHERE ("access_owners"."area" = $1)
2020-07-04 14:51:04.512 IST [32129] postgres#postgres DETAIL: parameters: $1 = '4th Floor'
2020-07-04 14:51:04.513 IST [32129] postgres#postgres LOG: execute <unnamed>: SELECT * FROM "access_owners" WHERE ("access_owners"."area" = $1) LIMIT 10 OFFSET 2
2020-07-04 14:51:04.513 IST [32129] postgres#postgres DETAIL: parameters: $1 = '4th Floor'
this is the API response I get
{
"skip": 2,
"limit": 10,
"total": 3,
"resources": [
{
"id": 26,
"name": "Test_User_5",
"email": "test_user_5#gmailcom",
"area": "4th Floor"
}
]
}
But if I call the same API again with the same parameters it errors out adding one more where condition in DB.
2020-07-04 14:51:22.560 IST [32129] postgres#postgres LOG: execute <unnamed>: SELECT count(*) FROM "access_owners" WHERE ("access_owners"."area" = $1) AND ("access_owners"."area" = $2) LIMIT 10 OFFSET 2
2020-07-04 14:51:22.560 IST [32129] postgres#postgres DETAIL: parameters: $1 = '4th Floor', $2 = '4th Floor'
And this the error I get
sql: no rows in result set
This is my API handler
func GetAccessOwners(db *gorm.DB) echo.HandlerFunc {
return func(c echo.Context) error {
page := new(models.ResourcePage)
var ao = new(models.AccessOwner)
var err error
if err = c.Bind(ao); err != nil {
return err
}
var skip int
var limit int
skipParam := c.QueryParam("skip")
limitParam := c.QueryParam("limit")
if strings.TrimSpace(skipParam) != "" {
skip, err = strconv.Atoi(skipParam)
if err != nil {
return merry.Wrap(err)
}
} else {
skip = 0
}
if strings.TrimSpace(limitParam) != "" {
limit, err = strconv.Atoi(limitParam)
if err != nil {
return merry.Wrap(err)
}
} else {
limit = 10
}
accessOwners := []models.AccessOwner{}
page.Resources = []models.AccessOwner{}
page.Skip = skip
page.Limit = limit
db = db.Scopes(GetCount(*ao)).Model(page.Resources).Count(&page.Total)
db = db.Offset(skip)
db = db.Limit(limit)
if err := db.Find(&accessOwners).Error; err != nil {
return merry.Wrap(err)
}
page.Resources = accessOwners
return c.JSON(201, page)
}
}
func GetCount(ao models.AccessOwner) func(db *gorm.DB) *gorm.DB {
return func(db *gorm.DB) *gorm.DB {
return db.Where(ao)
}
From doc
When using multiple immediate methods with GORM, later immediate
method will reuse before immediate methods’s query conditions
You are setting db that's why in the second request query condition of first request already exists in db.
db = db.Scopes(GetCount(*ao)).Model(page.Resources).Count(&page.Total)
db = db.Offset(skip)
db = db.Limit(limit)
So, don't set db rather you can do chaining multiple operations or use separately
db.Scopes(GetCount(*ao)).Model(page.Resources).Count(&page.Total)
.Offset(skip).Limit(limit).Find(&accessOwners)

Execute powershell command in running container via Docker API

I want to execute a powershell command in a docker container running on a windows host.
The specific command I want to execute is "powershell Get-PSDrive C | Select-Object Used,Free"
I have implemented this using the Docker API for python and it is simple like calling:
cmd = "powershell Get-PSDrive C | Select-Object Used,Free"
output = container.exec_run(cmd)
This works as intended, but I need to implement this in golang.
But somehow, it is not clear for me how to interact with the Docker API for golang. I looked into the API and was confused by the hijackedSession. How do I have to setup the calls for ContainerExecCreate, ContainerExecAttach and ContainerExecStart ?
I expect the golang script to deliver the same results like the python code does:
Used Free
---- ----
199181606912 307151622144
Which then can be parsed by me.
The HijackedResponse struct:
type HijackedResponse struct {
Conn net.Conn
Reader *bufio.Reader
}
You need to copy the response from the resp.Reader,here is my code:
package main
import (
"bytes"
"context"
"fmt"
"github.com/docker/docker/api/types"
"github.com/docker/docker/client"
"github.com/docker/docker/pkg/stdcopy"
"strings"
)
func readFromCommand() (string, error) {
cli, err := client.NewEnvClient()
if err != nil {
return "", err
}
ctx := context.Background()
config := types.ExecConfig{
Cmd: strings.Split("powershell Get-PSDrive C | Select-Object Used,Free", " "),
AttachStdout: true,
AttachStderr: true,
}
response, err := cli.ContainerExecCreate(ctx,
// container id
"cf59d65ab1", config)
if err != nil {
return "", err
}
execID := response.ID
resp, err := cli.ContainerExecAttach(ctx, execID, config)
if err != nil {
return "", err
}
defer resp.Close()
stdout := new(bytes.Buffer)
stderr := new(bytes.Buffer)
_, err = stdcopy.StdCopy(stdout, stderr, resp.Reader)
if err != nil {
return "", err
}
s := stdout.String()
fmt.Println(s)
i := stderr.String()
fmt.Println(i)
return s, nil
}
Do remember to change the container id.

How do I solve «panic: sql: unknown driver "postgres" (forgotten import?)»?

I'm trying to INSERT data into POSTGRES from a .csv (pre-fixed width / tabular ) with GO.
What I've done:
package main
import (
"bufio"
"database/sql"
"encoding/csv"
"encoding/json"
"fmt"
"io"
"log"
"os"
)
type Consumidor struct {
CPF string `json:"CPF"`
Private string `json:"Private"`
Incompleto string `json:"Incompleto"`
Compras *Compras `json:"Compras,omitempty"`
}
type Compras struct {
DataUltimacompra string `json:"DataUltimacompra"`
TicketMedio string `json:"TicketMedio"`
TicketUltimaCompra string `json:"TicketUltimaCompra"`
LojaMaisFrequente string `json:"LojaMaisFrequente"`
LojaUltimaCompra string `json:"LojaUltimaCompra"`
}
const (
host = "localhost"
port = 5432
user = "postgres"
password = ""
dbname = "neoway"
)
func main() {
csvFile, _ := os.Open("data.csv")
reader := csv.NewReader(bufio.NewReader(csvFile))
var dadosinsert []Consumidor
for {
line, error := reader.Read()
if error == io.EOF {
break
} else if error != nil {
log.Fatal(error)
}
dadosinsert = append(dadosinsert, Consumidor{
CPF: line[0],
Private: line[1],
Incompleto: line[2],
Compras: &Compras{
DataUltimacompra: line[3],
TicketMedio: line[4],
TicketUltimaCompra: line[5],
LojaMaisFrequente: line[6],
LojaUltimaCompra: line[7],
},
})
}
peopleJson, _ := json.Marshal(dadosinsert)
fmt.Println(string(peopleJson))
psqlInfo := fmt.Sprintf("host=%s port=%d user=%s "+
"password=%s dbname=%s sslmode=disable",
host, port, user, password, dbname)
db, err := sql.Open("postgres", psqlInfo)
if err != nil {
panic(err)
}
defer db.Close()
sqlStatement := `
INSERT INTO base_teste (CPF,"PRIVATE","INCOMPLETO","DATA DA ÚLTIMA COMPRA","TICKET MÉDIO","TICKET DA ÚLTIMA COMPRA","LOJA MAIS FREQUÊNTE","LOJA DA ÚLTIMA COMPRA")
)
VALUES ($1, $2, $3, $4, $5, $6, 7$, 8$)
RETURNING id`
id := 0
err = db.QueryRow(sqlStatement, 30, "a", "b", "c").Scan(&id)
if err != nil {
panic(err)
}
fmt.Println("New record ID is:", id)
}
when I run, I get this error
[{"CPF":"xxxxx","Private":"TRUE","Incompleto":"FALSE","Compras":{"DataUltimacompra":"12/10/2018","TicketMedio":"200","TicketUltimaCompra":"250","LojaMaisFrequente":"111.111.111-99","LojaUltimaCompra":"111.111.111-88"}}]
panic: sql: unknown driver "postgres" (forgotten import?)
goroutine 1 [running]: main.main() C:/Users/Willian/Desktop/NEOWAY
PROJECT/neoway csv prefixed width importer/main.go:70 +0xbed
Process finished with exit code 2
You imported the sql/database, a package contains generic interface for sql-related operation.
Since it's only generic interface, you need to import the concrete implementation of the interface, and in this context, it's the database driver.
From your code: sql.Open("postgres", psqlInfo), I presume you are using postgresql database. There are some postgresql drivers for golang available, one of them is https://github.com/lib/pq driver. So add it to the import statement.
package main
import (
"bufio"
"database/sql"
"encoding/csv"
"encoding/json"
"fmt"
"io"
"log"
"os"
_ "github.com/lib/pq" // <------------ here
)
The database driver pq must be imported with _ character in front of the import statement. It's because we don't use it explicitly on the code meanwhile it's still required by database/sql package. For more details see this related SO question What does an underscore in front of an import statement mean?.
More information about golang sql: https://pkg.go.dev/database/sql.
.