Cannot connect to AWS RDS PostgreSQL with Golang using Helm Charts - postgresql

I am unable to connect to AWS RDS PostgeSQL when running Helm Chart for a Go App (GORM). All credentials are stored in a kubernetes secret, and the secret is being used in the helm chart.
Few Points:
Able to connect locally just fine.
The PostgreSQL database is already created in RDS, and made sure that the kubernetes secret as matches with the same creds.
Docker image is pushed and pulled from Gitlab without any errors.
Command "helm ls" displays the deployment status as "DEPLOYED"
When taking "kubectl get pod", I get STATUS as "CrashLoopBackoff"
When taking "kubectl describe pod", I get back MESSAGE "Back-off restarting failed container"
I then take "kubectl logs pod_name" to track the error, and get back the following:
failed to connect to the database
dial tcp 127.0.0.1:5432: connect: connection refused
(Not sure why it is still specifying "127.0.0.1" when i have the secret mounted)
Unable to exec into the pod because it is not running.
I have tried:
Secure a connection in the same cluster from a different pod using psql to ensure that the creds in the secret are in sync with what was set up in RDS PostgreSQL
Changing the api from DB_HOST, to host=%s
Tried connecting using fmt.Sprintf, as well as os.Getenv
"Versions"
GO version:
go1.11.1 darwin/amd64
DOCKER version:
Client:
Version: 18.06.1-ce
API version: 1.38
API.GO (file)
package controllers
import (
"fmt"
"log"
"net/http"
"os"
"time"
"github.com/gorilla/mux"
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/postgres"
_ "gitlab.torq.trans.apps.ge.com/503081542/torq-auth-api/models"
)
var err error
type API struct {
Database *gorm.DB
Router *mux.Router
}
func (api *API) Initialize(opts string) {
// Initialize DB
dbinfo := os.Getenv("DB_HOST, DB_USER, DB_PASSWORD, DB_NAME,
DB_PORT sslmode=disable")
// dbinfo := os.Getenv("host=%s user=%s password=%s dbname=%s port=%s sslmode=disable")
api.Database, err = gorm.Open("postgres", dbinfo)
if err != nil {
log.Print("failed to connect to the database")
log.Fatal(err)
}
// Bind to a port and pass our router in
// log.Fatal(http.ListenAndServe(":8000", handlers.CORS() .
(api.Router)))
fmt.Println("Connection established")
log.Printf("Postgres started at %s PORT", config.DB_PORT)
// MODELS
type application struct {
ID string `json:"id" gorm:"primary_key"`
CreatedAt time.Time `json:"-"`
UpdatedAt time.Time `json:"-"`
Name string `json:"name"`
Ci string `json:"ci"`
}
type Cloud struct {
ID string `json:"id" gorm:"primary_key"`
Name string `json:"name"`
}
fmt.Println("Tables are created")
// Enable this line for DB logging
api.Database.LogMode(true)}
// Initialize Router
api.Router = mux.NewRouter()
api.Router.HandleFunc("/api/v1/applications",
api.HandleApplications)
api.Router.HandleFunc("/api/v1/application/{id}",
api.HandleApplication)
api.Router.HandleFunc("/api/v1/clusters", api.handleClusters)
}
I am not exactly sure where the issue could be here, this is a learning experience for myself. Any ideas would be appreciated.
Thanks in advance!

I think your database initialization code is not doing what you want. Try something like this
var driver = "postgres"
var name = os.Getenv("DB_NAME")
var host = os.Getenv("DB_HOST")
var user = os.Getenv("DB_USER")
var pass = os.Getenv("DB_PASSWORD")
var conn = fmt.Sprintf("host=%v user=%v password=%v dbname=%v sslmode=disable", host, user, pass, name)
api.Database, err := gorm.Open(driver, conn)

Related

How to connect PostgreSQL database using Gorm

I am unable to connect to PostgreSQL.
var db *gorm.DB
var err error
func main() {
router := mux.NewRouter()
var err error
db, err = gorm.Open("postgres", "host=localhost port=5432 user=postgres dbname=dvdrental password=12345")
if err != nil {
panic("failed to connect database")
}
I am expecting database to be connected and fetching api.
This is the best way I found to connect to postgres.
import _ "github.com/jinzhu/gorm/dialects/postgres"
dsn := url.URL{
User: url.UserPassword(conf.User, conf.Password),
Scheme: "postgres",
Host: fmt.Sprintf("%s:%d", conf.Host, conf.Port),
Path: conf.DBName,
RawQuery: (&url.Values{"sslmode": []string{"disable"}}).Encode(),
}
db, err := gorm.Open("postgres", dsn.String())
Followed the official docs and it worked for me,
Visit https://gorm.io/docs/connecting_to_the_database.html
import (
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
dsn := "host=localhost user=gorm password=gorm dbname=gorm port=9920 sslmode=disable TimeZone=Asia/Shanghai"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
I just had to remove the host variable and enter rest of the details and it works.
First of all, the question keeps spaces for imagination.
Anyway, here're the possible use-cases why you're not able to connect gorm to postgres:
Make sure your postgres is well running in the host you have specified (in this case: localhost)
If it's alright, make sure your user and password is alright
If everything is ok, you can check in the CLI:
$ sudo -i -u postgres
$ psql
If it's successful, that means something else in the code. As far as I can see, you did not show the import section. Did you silently import the postgres gorm dialects?
import _ "github.com/jinzhu/gorm/dialects/postgres"

CreateSession: no reachable servers - mgo

I'm trying to connect to MongoDB Atlas free cluster using mgo.
Golang Code -
package main
import (
"fmt"
"gopkg.in/mgo.v2"
"time"
"log"
)
const (
AuthDatabase = "mydatabase"
AuthUserName = "databaseadmin"
AuthPassword = "databasepassword"
ReplicaSetName = "myproject-shard-0"
)
func main(){
MongoDBHosts := []string{
"myproject-shard-00-00-w4vds.mongodb.net:27017",
"myproject-shard-00-01-w4vds.mongodb.net:27017",
"myproject-shard-00-02-w4vds.mongodb.net:27017",
}
mongoDBDialInfo := &mgo.DialInfo{
Addrs: MongoDBHosts,
Timeout: 60 * time.Second,
Database: AuthDatabase,
Username: AuthUserName,
Password: AuthPassword,
ReplicaSetName: ReplicaSetName,
}
mongoSession, err := mgo.DialWithInfo(mongoDBDialInfo)
if err != nil {
log.Fatalf("CreateSession: %s\n", err)
}
defer mongoSession.Close()
fmt.Printf("Connected to replica set %v!\n", mongoSession.LiveServers())
}
Error Message -
CreateSession: no reachable servers
Environment
I'm using mongodb free cluster with Google App Engine GO SDK
To connect to MongoDB Atlas, you need SSL
Prerequisites
TLS/SSL
Clients must have support for TLS/SSL to connect to an Atlas cluster.
Clients must have support for the SNI TLS extension to connect to an Atlas M0 Free Tier cluster.
Whitelist
To access a cluster, you must connect from an IP address on the Atlas group’s IP whitelist. If you need to add an IP address to the whitelist, you can do so in the Connect dialog. You can also add the IP address from the Security tab.
Thus I had to change the following piece of code
mongoDBDialInfo := &mgo.DialInfo{
Addrs: MongoDBHosts,
Timeout: 60 * time.Second,
Database: AuthDatabase,
Username: AuthUserName,
Password: AuthPassword,
ReplicaSetName: ReplicaSetName,
}
mongoDBDialInfo.DialServer = func(addr *mgo.ServerAddr) (net.Conn, error) {
conn, err := tls.Dial("tcp", addr.String(), tlsConfig)
return conn, err
}
I had to import the following too
"crypto/tls"
"net"

Error when creating container with golang docker engine

I am creating a project in Go and I am using both "github.com/docker/docker/client" and "github.com/docker/docker/api/types", but when I try and create a container I get the following error:
ERROR: 2016/10/03 22:39:26 containers.go:84: error during connect: Post https://%2Fvar%2Frun%2Fdocker.sock/v1.23/containers/create: http: server gave HTTP response to HTTPS client
I can't understand why this is happening and it only happened after using the new golang docker engine(the old "github.com/docker/engine-api" is now deprecated).
The code isn't anything complicated, so I wonder if I am missing something:
resp, err := cli.Pcli.ContainerCreate(context.Background(), initConfig(), nil, nil, "")
if err != nil {
return err
}
And the initConfig that is called does the following:
func initConfig() (config *container.Config) {
mount := map[string]struct{}{"/root/host": {}}
return &container.Config{Image: "leadis_image", Volumes: mount, Cmd: strslice.StrSlice{"/root/server.py"}, AttachStdout: true}}
Also here is my dockerfile
FROM debian
MAINTAINER Leadis Journey
LABEL Description="This docker image is used to compile and execute user's program."
LABEL Version="0.1"
VOLUME /root/host/
RUN apt-get update && yes | apt-get upgrade
RUN yes | apt-get install gcc g++ python3 make
COPY container.py /root/server.py
EDIT
Just tried to test it with a simpler program
package main
import (
"fmt"
"os"
"io/ioutil"
"github.com/docker/docker/client"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/strslice"
"golang.org/x/net/context"
)
func initConfig() (config *container.Config) {
mount := map[string]struct{}{"/root/host": {}}
return &container.Config{Image: "leadis_image", Volumes: mount, Cmd: strslice.StrSlice{"/root/server.py"}, AttachStdout: true}
}
func main() {
client, _ := client.NewEnvClient()
cwd, _ := os.Getwd()
ctx, err := os.Open(cwd+"/Dockerfile.tar.gz")
if err != nil {
fmt.Println(err)
return
}
build, err := client.ImageBuild(context.Background(), ctx, types.ImageBuildOptions{Tags: []string{"leadis_image"}, Context: ctx, SuppressOutput: false})
if err != nil {
fmt.Println(err)
return
}
b, _ := ioutil.ReadAll(build.Body)
fmt.Println(string(b))
_, err = client.ContainerCreate(context.Background(), initConfig(), nil, nil, "")
if err != nil {
fmt.Println(err)
}
}
Same dockerfile, but I still get the same error:
error during connect: Post
https://%2Fvar%2Frun%2Fdocker.sock/v1.23/containers/create: http:
server gave HTTP response to HTTPS client
client.NewEnvClient()
Last time I tried, this API expects environment variables like DOCKER_HOST in a different syntax from than the normal docker client.
From the client.go:
// NewEnvClient initializes a new API client based on environment variables.
// Use DOCKER_HOST to set the url to the docker server.
// Use DOCKER_API_VERSION to set the version of the API to reach, leave empty for latest.
// Use DOCKER_CERT_PATH to load the TLS certificates from.
// Use DOCKER_TLS_VERIFY to enable or disable TLS verification, off by default.
To use this, you need DOCKER_HOST to be set/exported in one of these formats:
unix:///var/run/docker.sock
http://localhost:2375
https://localhost:2376

Can't connect with Mlab database but I can to my local mongo with mgo golang

I am trying to connect to a mlab mongo database in my Golang application but I always get 'auth failed'.
If I use my local mongo, I have no problems (my local doesn't have authentication)
EDIT: I do have created a database user in mLab and I can log in with that user in RoboMongo
My database package looks like this:
package database
import (
"os"
"fmt"
"sync"
"labix.org/v2/mgo"
"time"
)
type DB struct {
Database *mgo.Database
}
const (
MongoDBHosts = "mlabHost:mlabPort"
AuthDatabase = "mydatabase"
AuthUserName = "mlabUser"
AuthPassword = "mlabPassword"
)
var _init_ctx sync.Once
var _instance *DB
func New() *mgo.Database {
_init_ctx.Do(func() {
_instance = new(DB)
mongoDBDialInfo := &mgo.DialInfo{
Addrs: []string{MongoDBHosts},
Timeout: 600 * time.Second,
Database: AuthDatabase,
Username: AuthUserName,
Password: AuthPassword,
}
// Create a session which maintains a pool of socket connections
// to our MongoDB.
session, err := mgo.DialWithInfo(mongoDBDialInfo)
if err != nil {
fmt.Printf("Error en mongo: %+v\n", err)
os.Exit(1)
}
_instance.Database = session.DB(AuthDatabase)
})
return _instance.Database
}
With that code, I always get 'auth failed', but if I change the const values to connect to my local like this:
const (
MongoDBHosts = "localhost:27017"
AuthDatabase = "mydatabase"
AuthUserName = ""
AuthPassword = ""
)
Everything is good.
I can even connect to my mLab database through RoboMongo, but one thing I noticed was that trying to connect from the command line like this:
mongo mLabHost:mLabPort/mydatabase -u mLabUser -p mLabPassword
The prompt asks again for the password and then I get (mypassword = mLabPassword I enter):
2016-06-25T16:07:10.822-0500 E - [main] file [mypassword] doesn't exist
failed to load: mypassword
I tried connecting to mLab in several different ways, but I can't find what is my problem.
Thanks
Found the problem
I was importing the wrong go module.
I had:
labix.org/v2/mgo
instead of:
gopkg.in/mgo.v2
After importing gopkg.in/mgo.v2 it made the connection

Connection refused with Go + Postgres on Heroku

I am trying to connect to Heroku's Postgres using a Go. All is working fine locally.
The error I am receiving on Heroku is dial tcp 127.0.0.1:5432: connection refused.
I've confirmed my ability to connect to the database via psql on heroku's command line, and have confirmed that the database url config is correct. The code is clear enough, so I wonder if there is a lower-level problem.
The code is straightforward enough:
import (
"database/sql"
"github.com/coopernurse/gorp"
_ "github.com/lib/pq"
"os"
)
func openDb() *sql.DB {
connection := os.Getenv("DATABASE_URL")
db, err := sql.Open("postgres", connection)
if err != nil {
log.Println(err)
}
return db
}
...and am importing github.com/lib/pq. Go version is 1.1.2.
Heroku + Go is pretty particular about the connection strings. The URL-style doesn't seem to allow specification of sslmode=require, which Heroku insists upon.
The modified version uses pq to parse the URL into a traditional Postgres connection string, and appends the parameter:
import (
"database/sql"
"github.com/lib/pq"
"os"
)
func openDb() *sql.DB {
url := os.Getenv("DATABASE_URL")
connection, _ := pq.ParseURL(url)
connection += " sslmode=require"
db, err := sql.Open("postgres", connection)
if err != nil {
log.Println(err)
}
return db
}