This question already has answers here:
golang what is import side effect
(1 answer)
What does an underscore in front of an import statement mean?
(5 answers)
Import side effects
(1 answer)
Closed 4 years ago.
I continue to get this issue and I do not understand why
package models
import (
"database/sql"
"fmt"
_ "github.com/lib/pq"
)
const (
host = "localhost"
port = 5432
user = "postgres"
password = "postgres"
dbname = "postgres"
)
var db *sql.DB
func InitDB() {
psqlInfo := fmt.Sprintf("host=%s port=%d user=%s "+
"password=%s dbname=%s sslmode=disable",
host, port, user, password, dbname)
db, err := sql.Open("postgres", psqlInfo)
if err != nil {
panic(err)
}
defer db.Close()
err = db.Ping()
if err != nil {
panic(err)
}
fmt.Println("Successfully connected!")
}
Apologies as I am new to go and attemping to understand it. Thank you for any help. I have attempted to pull the library as well as move it around
Assuming you're referring to
import (
_ "github.com/lib/pq"
)
This is importing for a side effect. The underscore is in the alias position. This allows you to circumvent the error (or automatic removal if using something like goimports) of an unused import.
When github.com/lib/pq is imported it runs an init() function that registers the postgres database driver.
This init function can be seen here https://github.com/lib/pq/blob/d34b9ff171c21ad295489235aec8b6626023cd04/conn.go#L48-L50
func init() {
sql.Register("postgres", &Driver{})
}
If you needed to refer to something in the pq package itself it is fine to remove the underscore and then make direct reference to the package. However, make sure you do not unintentionally remove the import entirely or your attempt to open a connection to the postgres database will fail.
Related
This question already has answers here:
Is mongodb client driver concurrent safe?
(1 answer)
goroutine create multiple mongodb connection
(1 answer)
Closed 21 days ago.
I've managed to connect to MongoDB and I've got this client object, can this be passed around and reused?
Or does it need to be created each time I want to interact with the database?
If it can be reused then ill move the database connection out to its own function and run it just once at startup.
package main
import (
"context"
"fmt"
"log"
"main/api"
"os"
"time"
"github.com/joho/godotenv"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
type User struct {
Username string
Password string `bson:"password,omitempty"`
}
func main() {
if err := godotenv.Load(); err != nil {
log.Println("No .env file found")
}
uri := os.Getenv("MONGODB_URI")
serverAPIOptions := options.ServerAPI(options.ServerAPIVersion1)
clientOptions := options.Client().
ApplyURI(uri).
SetServerAPIOptions(serverAPIOptions)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := mongo.Connect(ctx, clientOptions)
if err != nil {
log.Fatal(err)
}
usersColl := client.Database("sampleShop").Collection("users")
newUser := User{Username: "8282", Password: "Korean"}
result, err := usersColl.InsertOne(context.TODO(), newUser)
if err != nil {
panic(err)
}
fmt.Println(result)
fmt.Println(uri)
api.GetPaymentSession()
}
I am pretty new to Go and I am trying to find the best way to set up my db communication. Essentially I remember from my previous workplaces that in PHP you can create a class that represents a SQL table and when you need to insert data into your db you would create an object of that class with all the necessary data, call insert(), pass your object and it would insert that data into a corresponding table without you writing any SQL code, update() works in a very similar way except it would update instead of inserting. Unfortunately, I don't remember the name of that PHP framework but maybe someone knows a way to achieve something like that in Go or is it not a thing?
Lets say I have a struct:
type Patients struct {
ID int
Name string
Image string
}
Now I want to have a function that takes Patients objet as a parameter and inserts it into a patients postgres table automatically converting patient into what postgres expects:
func (patients *Patients) insert(patient Patients) {
}
And then update() would take a Patients object and basically perform this chunk of code without me writing it:
stmt := `update patients set
name = $1,
image = $2,
where id = $3
`
_, err := db.ExecContext(ctx, stmt,
patient.Name,
patient.Image,
patient.ID
)
You are looking for something called an ORM (Object Relational Mapper). There are a few in Go, but the most popular is GORM. It's a bit of a controversial topic, but I think it's a good idea to use an ORM if you're new to Go and/or databases. It will save you a lot of time and effort.
The alternative is to use the database/sql package and write your own SQL queries. This is a good idea if you're an experienced Go developer and/or database administrator. It will give you more control over your queries and will be more efficient. Recommended reading: https://www.alexedwards.net/blog/organising-database-access. Recommended libraries for this approach include sqlx and pgx.
Here is what your struct would look like as a GORM model:
type Patient struct {
ID int `gorm:"primaryKey"`
Name string
Image string
}
And here is an example program for how to insert a patient into the database:
package main
import (
"fmt"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
type Patient struct {
ID int `gorm:"primaryKey"`
Name string
Image string
}
func main() {
dsn := "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable TimeZone=UTC"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
panic("failed to connect database")
}
db.AutoMigrate(&Patient{})
patient := Patient{
Name: "John Smith",
Image: "https://example.com/image.png",
}
result := db.Create(&patient)
if result.Error != nil {
panic(result.Error)
}
fmt.Println(patient)
}
If instead you wanted to use sqlx, you would write something like this:
package main
import (
"database/sql"
"fmt"
"log"
_ "github.com/lib/pq"
)
type Patient struct {
ID int
Name string
Image string
}
func main() {
dsn := "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable TimeZone=UTC"
db, err := sql.Open("postgres", dsn)
if err != nil {
log.Fatal(err)
}
defer db.Close()
_, err = db.Exec(`
CREATE TABLE IF NOT EXISTS patients (
id SERIAL PRIMARY KEY,
name TEXT,
image TEXT
)
`)
if err != nil {
log.Fatal(err)
}
patient := Patient{
Name: "John Smith",
Image: "https://example.com/image.png",
}
_, err = db.Exec(`
INSERT INTO patients (name, image) VALUES ($1, $2)
`, patient.Name, patient.Image)
if err != nil {
log.Fatal(err)
}
fmt.Println(patient)
}
Of course, managing your database schema is a bit more complicated with an ORM. You can use migrations, but I prefer to use a tool called goose. It's a bit of a pain to set up, but it's very powerful and flexible. Here is an example of how to use it:
package main
import (
"fmt"
"log"
"github.com/pressly/goose"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
type Patient struct {
ID int `gorm:"primaryKey"`
Name string
Image string
}
func main() {
dsn := "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable TimeZone=UTC"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
panic("failed to connect database")
}
goose.SetDialect("postgres")
goose.SetTableName("schema_migrations")
err = goose.Run("up", db.DB(), "migrations")
if err != nil {
log.Fatal(err)
}
patient := Patient{
Name: "John Smith",
Image: "https://example.com/image.png",
}
result := db.Create(&patient)
if result.Error != nil {
panic(result.Error)
}
fmt.Println(patient)
}
where your migrations directory looks like this:
migrations/
00001_create_patients.up.sql
00001_create_patients.down.sql
and your migrations look like this:
-- 00001_create_patients.up.sql
CREATE TABLE patients (
id SERIAL PRIMARY KEY,
name TEXT,
image TEXT
);
-- 00001_create_patients.down.sql
DROP TABLE patients;
I hope this helps! Let me know if you have any questions.
I think what you're looking for is an ORM. An ORM is a library that essentially does this, taking language structures and automatically handling the SQL logic for you.
The most popular library for this in Go is GORM. Here's a link to their home page: https://gorm.io/. I've used it heavily in production and it's been a good experience!
The docs have a good example of what it'll look like.
Hope this helps.
I have a set of functions in my web API app. They perform some operations on the data in the Postgres database.
func CreateUser () {
db, err := sql.Open("postgres", "user=postgres password=password dbname=api_dev sslmode=disable")
// Do some db operations here
}
I suppose functions should work with db independently from each other, so now I have sql.Open(...) inside each function. I don't know if it's a correct way to manage db connection.
Should I open it somewhere once the app starts and pass db as an argument to the corresponding functions instead of opening the connection in every function?
Opening a db connection every time it's needed is a waste of resources and it's slow.
Instead, you should create an sql.DB once, when your application starts (or on first demand), and either pass it where it is needed (e.g. as a function parameter or via some context), or simply make it a global variable and so everyone can access it. It's safe to call from multiple goroutines.
Quoting from the doc of sql.Open():
The returned DB is safe for concurrent use by multiple goroutines and maintains its own pool of idle connections. Thus, the Open function should be called just once. It is rarely necessary to close a DB.
You may use a package init() function to initialize it:
var db *sql.DB
func init() {
var err error
db, err = sql.Open("yourdriver", "yourDs")
if err != nil {
log.Fatal("Invalid DB config:", err)
}
}
One thing to note here is that sql.Open() may not create an actual connection to your DB, it may just validate its arguments. To test if you can actually connect to the db, use DB.Ping(), e.g.:
func init() {
var err error
db, err = sql.Open("yourdriver", "yourDs")
if err != nil {
log.Fatal("Invalid DB config:", err)
}
if err = db.Ping(); err != nil {
log.Fatal("DB unreachable:", err)
}
}
I will use a postgres example
package main
import necessary packages and don't forget the postgres driver
import (
"database/sql"
_ "github.com/lib/pq" //postgres driver
)
initialize your connection in the package scope
var db *sql.DB
have an init function for your connection
func init() {
var err error
db, err = sql.open("postgres", "connectionString")
//connectioString example => 'postgres://username:password#localhost/dbName?sslmode=disable'
if err != nil {
panic(err)
}
err = db.Ping()
if err != nil {
panic(err)
}
// note, we haven't deffered db.Close() at the init function since the connection will close after init. you could close it at main or ommit it
}
main function
func main() {
defer db.Close() //optional
//run your db functions
}
checkout this example
https://play.golang.org/p/FAiGbqeJG0H
I'm new to golang and MongoDB i've been following the official blog of mongoDb for getting started with mongo-db driver for go, and i'm not able to connect to my mongodb for some reason
here is the code ,the error is "context deadline exceeded", thnx in advance
package main
import (
"context"
"fmt"
"log"
"time"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
// "go.mongodb.org/mongo-driver/mongo/readpref"
)
func main() {
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb+srv://chandru:<Heregoesmypassword>#cluster0-9jkaf.mongodb.net/test?retryWrites=true&w=majority"))
if err != nil {
log.Fatal(err)
}
ctx, _ := context.WithTimeout(context.Background(), 20*time.Second)
err = client.Connect(ctx)
if err != nil {
log.Fatal(err)
}
defer client.Disconnect(ctx)
databases, err := client.ListDatabaseNames(ctx, bson.M{})
if err != nil {
log.Fatal(err)
}
fmt.Println(databases)
}
One possible cause is that you have not whitelisted your IP with MongoDB Atlas
Please refer to the following guide: https://docs.atlas.mongodb.com/security-whitelist/#view-whitelist-entries
Keep in mind that if you connect from your local machine, your IP address might change. You can get a static IP address from your ISP or connect to through another machine.
You can also enter 0.0.0.0/0 to allow access from any IP, but this is of course less secure.
I have an AWS instance with MongoDB running. I am trying to do a small db operation which seems to work when the files included below was written inside a single go file. When I try to split it I get the following error
the Insert operation must have a Deployment set before Execute can be called
The split files are given below
connect.go
package db
import (
"context"
"fmt"
"log"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
var Client1 mongo.Client
func Connect() {
// Set client options
clientOptions := options.Client().ApplyURI("remote_url")
// Connect to MongoDB
Client1, err := mongo.Connect(context.TODO(), clientOptions)
if err != nil {
log.Fatal(err)
}
// Check the connection
err = Client1.Ping(context.TODO(), nil)
if err != nil {
log.Fatal(err)
}
fmt.Println("Connected to MongoDB!")
}
func main() {
fmt.Println("Connection to MongoDB done.")
}
main.go
package main
import (
"context"
"fmt"
"log"
"db"
"go.mongodb.org/mongo-driver/bson"
)
// You will be using this Trainer type later in the program
type Trainer struct {
Name string
Age int
City string
}
func main() {
db.Connect()
collection := db.Client1.Database("test2").Collection("trainers")
_ = collection
fmt.Println("Created collection", _)
ash := Trainer{"Ash", 10, "Pallet Town"}
// misty := Trainer{"Misty", 10, "Cerulean City"}
// brock := Trainer{"Brock", 15, "Pewter City"}
insertResult, err := collection.InsertOne(context.TODO(), ash)
if err != nil {
log.Fatal(err)
}
fmt.Println("Inserted a single document: ", insertResult)
err = db.Client1.Disconnect(context.TODO())
if err != nil {
log.Fatal(err)
}
fmt.Println("Connection to MongoDB closed.")
}
They are placed in the following structure
/src -> main.go
/src -> /db/connect.go
I believe that your problem is caused by variable shadowing (wiki) and you're initializing a local variable and not the global mongo.Client object, hence throwing the error that you're getting.
It happens in your connect.go file, where you have defined two different Client1 variables with the same name:
One at global scope
Another in Connect() that gets declared+initialised when calling mongo.Connect()
var Client1 mongo.Client // Client1 at global scope
func Connect() {
// Set client options
clientOptions := options.Client().ApplyURI("remote_url")
// Connect to MongoDB
Client1, err := mongo.Connect(context.TODO(), clientOptions) // Client1 at local scope within Connect()
That causes that the one at global scope is never initialised so main.go crashes when trying to use it because it's nil.
There are several ways to solve this, for example by using a different name for the variable at local scope and assign the client to the global one:
var Client1 mongo.Client
func Connect() {
// Set client options
clientOptions := options.Client().ApplyURI("remote_url")
// Connect to MongoDB
Client1Local, err := mongo.Connect(context.TODO(), clientOptions)
Client1 = *Client1Local
Or avoid declaring the local variable and directly initialise the one at global scope:
var Client1 *mongo.Client // Note that now Client1 is of *mongo.Client type
func Connect() {
// Set client options
clientOptions := options.Client().ApplyURI("remote_url")
// Connect to MongoDB
var err error
Client1, err = mongo.Connect(context.TODO(), clientOptions) // Now it's an assignment, not a declaration+assignment anymore
More about Golang's variable shadowing discussion at Issue#377 proposal
Try to include like, since the you have omitted the db directory , that might have be the issue
import "db/db"
Also there is main() in connect.go there should be only one main package and main() function for a project.
---------------- Try what is discussed below exactly ------------
Ok this is my test setup with directory tree of code similar to yours:
[user#devsetup src]$ tree test
test
├── dbtest
│ └── db
│ └── connect.go
├── main // This is executable file
└── main.go
2 directories, 3 files
In connect.go i havent changed anything but removed the main() as mentioned above.
Make sure you use only exported functions and variables , Exported functions/Variables starts with Capital letter.
In main.go looks like the code below.
* Go to dbtest/db directory and run go install command. then go the project directory , ie test here in this case, and run go build main.go or go build .
package main
import (
"context"
"fmt"
"log"
"test/dbtest/db"
// "go.mongodb.org/mongo-driver/bson"
)
// You will be using this Trainer type later in the program
type Trainer struct {
Name string
Age int
City string
}
func main() {
db.Connect()
collection := db.Client1.Database("test2").Collection("trainers")
_ = collection
// fmt.Println("Created collection", _)
ash := Trainer{"Ash", 10, "Pallet Town"}
// misty := Trainer{"Misty", 10, "Cerulean City"}
// brock := Trainer{"Brock", 15, "Pewter City"}
insertResult, err := collection.InsertOne(context.TODO(), ash)
if err != nil {
log.Fatal(err)
}
fmt.Println("Inserted a single document: ", insertResult)
err = db.Client1.Disconnect(context.TODO())
if err != nil {
log.Fatal(err)
}
fmt.Println("Connection to MongoDB closed.")
}
This is my binary output since i havent given any input url , it is throwing an error.Seems working. I have commented out a package and a line in main.go
[user#devsetup test]$ ./test
20xx/yy/zz aa:bb:cc error parsing uri: scheme must be "mongodb" or "mongodb+srv"