Log in using IAM database authentication failed in Google CloudSQL - postgresql

Question

I tried to use Log in using IAM database authentication
with service account. but failed
Cloud SQL IAM service account authentication failed for user
I was totally stacled
procedure
1. create postgreSQL
instance test-auth
and set flag
databaseFlags:
- name: cloudsql.iam_authentication
value: on
2. create db
dbname: db_hoge
3. create service account
sa-cloudsql#myproject.iam.gserviceaccount.com
and add it as user. also, I added my own email ( which is the owner of this GCP project)
it looked added correctly.
$ gcloud sql users list --instance test-auth
NAME HOST TYPE PASSWORD_POLICY
postgres CLOUD_IAM_USER
maiil#mydomain.co.jp CLOUD_IAM_USER
sa-cloudsql#myproject.iam CLOUD_IAM_SERVICE_ACCOUNT
4. running cloud sql proxy ( v 1.31.0)
$ ./cloud_sql_proxy -version
Cloud SQL Auth proxy: 1.31.0+linux.amd64
$ ./cloud_sql_proxy -enable_iam_login -instances=myproject:asia-northeast1:test-auth=tcp:127.0.0.1:5432
5. DB connection
with service account -> failed
psql "sslmode=disable dbname=db_hoge host=127.0.0.1 user=sa-cloudsql#myproject.iam"
psql: error: FATAL: Cloud SQL IAM service account authentication failed for user "sa-cloudsql#myproject.iam"
with user (myself) -> success
psql "sslmode=disable dbname=db_hoge host=127.0.0.1 user=maiil#mydomain.co.jp"
psql (12.11 (Ubuntu 12.11-0ubuntu0.20.04.1), server 14.2)
WARNING: psql major version 12, server major version 14.
Some psql features might not work.
Type "help" for help.
db_hoge=>
IAM role
maiil#mydomain.co.jp => owner
sa-cloudsql#myproject.iam => roles/cloudsql.client ,　roles/cloudsql.instanceUser
I sincerely ask someone's kindly help.
Thank you in advance
update
following advice, I deployed app for testing connection on GKE
But now, I got error "FATAL: empty password returned by client".
When using IAM database authentication, we should not need password for that user.
I don't know it's because of App, GKE, IAM, or CloudSQL.
Deployment and App
Using sidecar, I runned cloudSQL proxy. and service-account for this pod has been binded google-service-account which has CloudSQL instance user & CloudSQL Client.
app.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
spec:
selector:
matchLabels:
run: web
template:
metadata:
labels:
run: web
spec:
containers:
- image: cloudproxy-test:1.0.0
imagePullPolicy: Always
name: web
ports:
- containerPort: 8080
protocol: TCP
- command:
- /cloud_sql_proxy
- -enable_iam_login
- -instances=my-project:asia-northeast1:test-auth=tcp:5432
image: gcr.io/cloudsql-docker/gce-proxy:1.25.0
name: cloudsql-proxy
serviceAccountName: ksa-workload-identity-binded
---
I made app with go. for postgreSQL, use dialers/postgres module.
package main
import (
"context"
"database/sql"
"fmt"
"log"
"net/http"
"github.com/gin-gonic/gin"
_ "github.com/GoogleCloudPlatform/cloudsql-proxy/proxy/dialers/postgres"
)
func main() {
engine := gin.Default()
engine.GET("/", func(c *gin.Context) {
ctx := context.Background()
if err := showRecords(ctx, "my-project:asia-northeast1:test-auth", "db_hoge", "sa-cloudsql#my-project.iam"); err != nil {
log.Fatal(err)
}
c.JSON(http.StatusOK, gin.H{
"How": "itwork",
})
})
engine.Run(":8080")
}
func showRecords(ctx context.Context, dbAddress, dbName, dbUser string) error {
dsn := fmt.Sprintf("host=%s user=%s dbname=%s sslmode=disable", dbAddress, dbUser, dbName)
fmt.Println(dsn)
db, err := sql.Open("cloudsqlpostgres", dsn)
if err != nil {
log.Fatal(err)
return err
}
defer db.Close()
return nil
}
When I used default user postgres with password and remove -enable_iam_login option from sidecar command, I could connect with DB.
So Application itself seemed not wrong.