I am building a lambda api connecting to AWS Aurora Serverless Postgres.
Follow AWS instruction to setup an Aurora Serverless with Postgres compatible (for MySql but still useful for Postgres): https://aws.amazon.com/getting-started/tutorials/configure-connect-serverless-mysql-database-aurora/
I added port 5432 in inbound of security group
I also use data-api-client to query the database (https://github.com/jeremydaly/data-api-client)
Built API lambda in Serverless Framework, set timeout is 1 min, added role AmazonRDSDataFullAccess
My lambda code (built in Serverless framework) is simple:
async function query_db(_sql) {
const data = require('data-api-client')({
secretArn: constants.DBSecretsStoreArn,
resourceArn: constants.DBAuroraClusterArn,
database: constants.DatabaseName
});
try {
let result = await data.query(_sql);
return result.records;
} catch (error) {
console.log('Lambda :: query_db :: Error: ' + error);
return error;
}
}
async function run() {
let sql = 'SELECT * FROM products LIMIT 10';
let result = await query_db(sql);
console.log('result: '+ JSON.stringify(result));
return callback(null, {
headers: {
'Access-Control-Allow-Origin': '*'
},
statusCode: 200,
body: JSON.stringify({msg: 'done})
});
}
Result:
It ran successfully in local (serverless-offline)
After deploying, it ran timeout, returned 502, error: "Internal server error"
Any suggestion is appreciated.
The data-api-client doesn't officially support Postgres, yet.
https://github.com/jeremydaly/data-api-client/issues/27
Related
#app.route('/signup', methods=['POST'])
def signup():
info = request.args
if info["password"] == info["password2"] and info["name"] and info["email"] and info["password"] and info["password2"]:
password = os.getenv("password")
link = 'mongodb+srv://yakov:' + password + '#cluster0.irzzw.mongodb.net/myAuctionDB?retryWrites=true&w=majority'
client = MongoClient(link)
db = client.get_database('myAuctionDB')
users = db.users
users.insert_one({
'name': info["name"],
'email': info["email"],
'password': info["password"],
'sales': [],
'offers': [],
'saved': []
})
return jsonify({"status": "ok", "message": " welcome to {} {} ".format(info["name"], info["email"])})
else:
return jsonify({"status": "error", "message": "you are missing some arguments"})
this is my code, it works when i run it locally from my computer.
i saved it on a host called pythonanywhere, and the code works, but it does not insert the json to mongoDB, it gives me this error "500 Internal Server Error".
this is the response when i run it locally:
this is the response of the same code when i run it through pythonanywhere:
Using a free acount in pythonanywhere, does not allow to connect to mongodb atlas, so when it runs on pythonanywhere, it does not connect to the database, and does not work
Thanks reading my issue.
Currently, I am using postgres (hobby-dev) on Heroku and facing this issue every time that I connect to the database.
error: Uncaught (in promise) Error: Unknown auth message code 1397113172
throw new Error(`Unknown auth message code ${code}`);
^
at Connection.handleAuth (connection.ts:197:15)
at Connection.startup (connection.ts:155:16)
at async Pool._createConnection (pool.ts:32:5)
at async pool.ts:61:7
at async Promise.all (index 0)
at async Pool._startup (pool.ts:63:25)
My application using Deno now
import { Pool } from "https://deno.land/x/postgres/mod.ts";
import { config } from "./config.ts";
const port = config.DB_PORT ? parseInt(config.DB_PORT || "") : undefined;
const POOL_CONNECTIONS = 20;
const dbPool = new Pool({
port,
hostname: config.DB_HOST,
user: config.DB_USER,
database: config.DB_NAME,
password: config.DB_PASS
}, POOL_CONNECTIONS);
export { dbPool };
Here is debug screen.
I have found this issue post and it mentioned about lacking ssl. Not sure how to do it on heroku.
I have tried some solutions, even change lib to pg and it still not work. I am very appreciated if any clue or help to fix this issue.
Note:
I read a document on heroku about "Heroku Postgres Connection Pooling is not available for Hobby-tier databases.". Then I switched to use Client with syntax similar like this to connect to Heroku postgres this:
import { Client } from "https://deno.land/x/postgres/mod.ts";
let config;
config = {
hostname: "localhost",
port: 5432,
user: "user",
database: "test",
applicationName: "my_custom_app"
};
// alternatively
config = "postgres://user#localhost:5432/test?application_name=my_custom_app";
const client = new Client(config);
await client.connect();
await client.end();
ref: https://deno-postgres.com/#/
Unable to connect to postgres in deno.js.
Here is the configuration:
const dbCreds = {
applicationName: "appname",
user: "user_sfhjwre",
database: "d9iu8mve7nen",
password: "68790f31eelkhlashdlkagsvADSDa52f9d8faed894c037ef6f9c9f09885603",
hostname: "ec2-345-34-97-212.eu-east-1.xx.amazonaws.com",
port: 5432,
};
export { dbCreds };
Usage:
import { Client } from "https://deno.land/x/postgres/mod.ts";
import { dbCreds } from "../config.ts";
const client = new Client(dbCreds);
await client.connect();
Also tried:
config = "postgres://user#localhost:5432/test?application_name=my_custom_app";
const client = new Client(config);
await client.connect();
Same result:
Uncaught Error: Unknown auth message code 1397113172
Is there anything wrong with the syntax, I can connect to the same db using prisma.
I have the PostgreSQL server in a remote server and, each time my public IP changes, I need to change the pg_hba.conf in order to set my new public IP as authorized for remote access.
Hope this helps.
Best regards.
I'm getting an exception while connecting to PostgreSQL 12 by creating a data source via wildfly admin console.
It works fine on my another machine which has PostgreSQL 9.5. and it seems to work fine when I connect without data-source.
try (Connection conn = DriverManager.getConnection(
"jdbc:postgresql://127.0.0.1:5432/registrar", "postgres", "postgres")) {
if (conn != null) {
System.out.println("Connected to the database!");
Statement statement = conn.createStatement();
ResultSet rs = statement.executeQuery("Select * FROM public.user LIMIT 10");
while(rs.next())
System.out.println(rs.getString(2));
} else {
System.out.println("Failed to make connection!");
}
But when I create a data-source like this:
and this the exception that I get when I try to test the connection from UI.
Request
{
"address" => [
("subsystem" => "datasources"),
("data-source" => "RegistrarDS")
],
"operation" => "test-connection-in-pool"
}
Response
Internal Server Error
{
"outcome" => "failed",
"failure-description" => "WFLYJCA0040: failed to invoke operation: WFLYJCA0047: Connection is not valid",
"rolled-back" => true,
"response-headers" => {"process-state" => "reload-required"}
}
Has anything changed in PostgreSQL 12? What could be the issue?
Could be an issue with the datasource-class defined in your driver as described here on issues.redhat.com
If a datasource-class is defined then it takes precedence to create the datasource and as such it can only use connection-properties.
Since there is not a single connection-url property for all drivers, you have to use connection-properties if you are defining the datasource-class.
Another thing you could try is to set "Connection Properties" instead of "Connection URL" and make sure you are using the correct/latest driver jar file.
Also check your Postgresql logs for further info on the exception..
-> https://www.postgresql.org/about/news/2000/
I am trying to implement some integration tests using mocha for functions that interact with a postgres database through knex, in a nodejs express app.
The functions work outside of mocha - I can start the app in node or nodemon, submit requests through Postman, retrieve records from the database, add new records, etc. But when I try to test the code through mocha, I get errors like the following for any functions that try to access the database:
select * from "item" where "user_id" = $1 - relation "item" does not exist
The environment variable for the database connection is set to connect to the right database; when I manually test the app end-to-end, everything works; I get data back from the database.
I've included what I think are the relevant code snippets below: the test script for one of the tests that won't work, the function I'm trying to test, and the modules that that function relies on.
TEST SCRIPT
const Item = require('../db/item');
const chai = require('chai');
const chaiAsPromised = require('chai-as-promised');
// set up the middleware
chai.use(chaiAsPromised);
var should = require('chai').should()
describe('Item.getByUser', function() {
contex`enter code here`t('With valid id', function() {
const item_id = 1;
const expectedResult = "Canoe";
it('should return items', function() {
return Item
.getByUser(item_id)
.then(items => {
items[0].name.should.equal(expectedResult);
});
});
});
SNIPPET FROM THE ITEM.GETBYUSER FUNCTION:
const knex = require('./connection');
module.exports = {
getByUser: function(id) {
return knex('item').where('user_id', id);
},
SNIPPET FROM THE CONNECTION MODULE:
require('dotenv-safe').config();
const environment = process.env.NODE_ENV || 'development';
const config = require('../knexfile')[environment];
module.exports = require('knex')(config);
SNIPPET FROM THE KNEXFILE MODULE:
module.exports = {
development: {
client: 'pg',
connection: process.env.DATABASE_URL
},
production: {
client: 'pg',
connection: process.env.DATABASE_URL
}
};
The error message I get for the above test is:
1) Item.getByUser
With valid id
should return items:
select * from "item" where "user_id" = $1 - relation "item" does not exist
error: relation "item" does not exist
at Connection.parseE (node_modules\pg\lib\connection.js:567:11)
at Connection.parseMessage (node_modules\pg\lib\connection.js:391:17)
at Socket.<anonymous> (node_modules\pg\lib\connection.js:129:22)
at addChunk (_stream_readable.js:284:12)
at readableAddChunk (_stream_readable.js:265:11)
at Socket.Readable.push (_stream_readable.js:220:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
Okay, it's looking like this was in fact related to the environment variable for the database in some way I can't quite figure out. Although I had the database connection set to 'postgres://localhost/mydatabase' the database actually being used when I tested the db live, including migrating and seeding the database through knex commands, was 'postgres://localhost/username' - a database with the same name as the Owner of 'mydatabase'. But I think the mocha tests were trying to connect to mydatabase, which was still empty at that point, since the migrate and seed affected the the database 'username.'
So, I think this can be closed. I'll try to replicate the problem where I was connected to the wrong db; not sure how that could have happened, as I never intentionally set the connection, or any environment variable to 'postgres://localhost/username'.