pythonanywhere is not updating mongoDB - mongodb
#app.route('/signup', methods=['POST'])
def signup():
info = request.args
if info["password"] == info["password2"] and info["name"] and info["email"] and info["password"] and info["password2"]:
password = os.getenv("password")
link = 'mongodb+srv://yakov:' + password + '#cluster0.irzzw.mongodb.net/myAuctionDB?retryWrites=true&w=majority'
client = MongoClient(link)
db = client.get_database('myAuctionDB')
users = db.users
users.insert_one({
'name': info["name"],
'email': info["email"],
'password': info["password"],
'sales': [],
'offers': [],
'saved': []
})
return jsonify({"status": "ok", "message": " welcome to {} {} ".format(info["name"], info["email"])})
else:
return jsonify({"status": "error", "message": "you are missing some arguments"})
this is my code, it works when i run it locally from my computer.
i saved it on a host called pythonanywhere, and the code works, but it does not insert the json to mongoDB, it gives me this error "500 Internal Server Error".
this is the response when i run it locally:
this is the response of the same code when i run it through pythonanywhere:
Using a free acount in pythonanywhere, does not allow to connect to mongodb atlas, so when it runs on pythonanywhere, it does not connect to the database, and does not work
Related
Searching for Keycloak user via attribute - searchForUserByUserAttribute - how is it fast?
Not experienced with keycloak, and haven't been able to find answers after a hefty google. Quick question - I have a custom attribute, userOrg, which is an uuid. It maps to a user organisation that lives outside of keycloak, in another database and contains the full details about the organisation (e.g. name, location). I'm reviewing some code and see a previous team mate has written custom keycloak api extension, findUsersByAttribute, which uses session.users().searchForUserByUserAttribute to locate all user with a specified userOrg. I'm guessing this would actually be a full table walk? Or am I wrong and keycloak somehow provides indexing over attributes to allow fast lookup? Next question - does keycloak provide a way of indexing over attributes/idea of user organisations. Or should that logic be outside of keycloak (e.g. in another database have a mapping of users and orgs). Is it spelled out anywhere in the docs? Thanks
You can search a user by user attribute via Admin REST API.
Endpoint. It depends on your Keycloak version
{keycloak URL}/auth/admin/realms/{my-realm}/users/?q=userOrg:{uuid}
or
{keycloak URL}/admin/realms/{my-realm}/users/?q=userOrg:{uuid}
In, Get users Returns a stream of users, filtered according to query parameters. section
I demoed it in local Keycloak v18.0.2 and Postman
1 I made 3 users by UI
2 Each users has own userOrg
3 Get master token
4 search user by UUID
http://localhost:8180/auth/admin/realms/my-realm/users/?q=userOrg:3db51e81-7569-4a05-aaf3-91058450c63e
Keycloak response matched user by attribute search API
I tested with 10K users. it take 10~12 msec by my laptop.
Here is my user created program by python
import json
import admin
import random
import uuid
admin = admin.Admin()
token = admin.get_master_token()
first_names = [ \
"AARON","ADAM","ALAN","ALBERT","ANDREW","ANTHONY" \
,"ANTONIO","ARTHUR","BENJAMIN","BILLY","BOBBY" \
,"BRANDON","BRIAN","BRUCE","CARL","CARLOS" \
,"CHARLES","CHRIS","CHRISTOPHER","CLARENCE","CRAIG" \
,"DANIEL","DAVID","DENNIS","DONALD","DOUGLAS" \
,"EARL","EDWARD","ERIC","ERNEST","EUGENE" \
,"FRANK","FRED","GARY","GEORGE","GERALD" \
,"GREGORY","HAROLD","HARRY","HENRY","HOWARD" \
,"JACK","JAMES","JASON","JEFFREY","JEREMY" \
,"JERRY","JESSE","JIMMY","JOE","JOHN" \
,"JOHNNY","JONATHAN","JOSE","JOSEPH","JOSHUA" \
,"JUAN","JUSTIN","KEITH","KENNETH","KEVIN" \
,"LARRY","LAWRENCE","LOUIS","MARK","MARTIN" \
,"MATTHEW","MICHAEL","NICHOLAS","PATRICK","PAUL" \
,"PETER","PHILIP","PHILLIP","RALPH","RANDY" \
,"RAYMOND","RICHARD","ROBERT","ROGER","RONALD" \
,"ROY","RUSSELL","RYAN","SAMUEL","SCOTT" \
,"SEAN","SHAWN","STEPHEN","STEVE","STEVEN" \
,"TERRY","THOMAS","TIMOTHY","TODD","VICTOR" \
,"WALTER","WAYNE","WILLIAM","WILLIE"]
last_names = [ \
"Adams","Allen","Alvarez","Anderson","Bailey",\
"Baker","Bennet","Brooks","Brown","Campbell",\
"Carter","Castillo","Chavez","Clark","Collins",\
"Cook","Cooper","Cox","Cruz","Davis",\
"Diaz","Edwards","Evans","Flores","Foster",\
"Garcia","Gomez","Gonzales","Gray","Green",\
"Gutierrez","Hall","Harris","Hernandez","Hill",\
"Howard","Hughes","Jackson","James","Jimenez",\
"Johnson","Jones","Kelly","Kim","King"\
"Lee","Lewis","Long","Lopez","Martin",\
"Martinez","Mendoza","Miller","Mitchell","Moore",\
"Morales","Morgan","Morris","Murphy","Myers",\
"Nelson","Nguyen","Ortiz","Parker","Patel"\
"Perez","Peterson","Phillips","Price","Ramirez",\
"Ramos","Reed","Reyes","Richardson","Rivera",\
"Roberts","Robinson","Rodriguez","Rogers","Ross",\
"Ruiz","Sanchez","Sanders","Scott","Smith",\
"Stewart","Taylor","Thomas","Thompson","Torres",\
"Turner","Walker","Ward","Watson","White",\
"Williams","Wilson","Wood","Wright","Young",]
user_list = []
index = 1
user_data = {}
access_item = {}
user_list = []
max_user_count = 10000
size_first_name = len(first_names)
size_last_name = len(last_names)
for index in range(1,max_user_count+1):
first_name_index = random.randint(0, size_first_name)
last_name_index = random.randint(0, size_last_name)
user_data['enabled'] = True
user_data['groups'] = []
user_data['emailVerified'] = ''
user_data['firstName'] = first_names[first_name_index-1].capitalize()
user_data['lastName'] = last_names[last_name_index-1]
user_data['username'] = 'user'+str(index)
user_data['email'] = 'user'+str(index)+'#test.com'
user_data['attributes'] = { 'userOrg' : [ str(uuid.uuid4())]}
user_list.append(user_data)
print(json.dumps(user_data))
user_data = {}
# add user if not exist
for user in user_list:
if (not admin.is_user_exist(token, 'test', user['username'])):
admin.add_user(token, user, 'test')
print('Add User', user['username'])
Get token code, name should be admin.py
from urllib.error import HTTPError
import requests
import ast
import json
class Admin:
# Keycloak master realm URL
url = 'http://localhost:8180/auth/realms/master/protocol/openid-connect/token'
# Keycloak master credential
params = {
'client_id': 'admin-cli',
'grant_type': 'password',
'username' : 'admin',
'password': 'admin'
}
def get_master_token(self):
try:
response = requests.post(self.url, self.params, verify=False).content.decode('utf-8')
except HTTPError as http_err:
print(f'HTTP error occurred: {http_err}') # Python 3.6
except Exception as err:
print(f'Other error occurred: {err}') # Python 3.6
print('Keycloak container is not running, Please check your docker container!')
raise SystemExit
else:
return ast.literal_eval(response)['access_token']
def add_user(self, token, user, realm_name):
url ='http://localhost:8180/auth/admin/realms/'+realm_name+'/users'
headers = {
'content-type': 'application/json',
'Authorization' : 'Bearer '+ str(token)
}
params = {
'username': user['username'],
'enabled': True,
'totp': False,
'emailVerified': True,
'firstName': user['firstName'],
'lastName': user['lastName'],
'email': user['email'],
'attributes': user['attributes'],
'disableableCredentialTypes': [],
'requiredActions': [],
'notBefore': 0,
'access': {
'manageGroupMembership': True,
'view': True,
'mapRoles': True,
'impersonate': True,
'manage': True
},
'realmRoles': [ realm_name ]
}
x = requests.post(url, headers=headers, json=params)
return x.content
def is_user_exist(self, token, realm_name, user_name):
url ='http://localhost:8180/auth/admin/realms/'+realm_name+'/users/?username='+user_name.replace(" ", "%20")
headers = {
'content-type': 'application/json',
'Authorization' : 'Bearer '+ str(token)
}
try:
response = requests.get(url, headers=headers)
response.raise_for_status()
except HTTPError as http_err:
print(f'HTTP error occurred: {http_err}') # Python 3.6
except Exception as err:
print(f'Other error occurred: {err}') # Python 3.6
else:
if len(response.content) == 2: # []
return False
if (json.loads(response.text)[0]['username'] == user_name.lower()):
return True
else:
return False
Unknown auth message code 1397113172 when connect to Heroku postgres
Thanks reading my issue.
Currently, I am using postgres (hobby-dev) on Heroku and facing this issue every time that I connect to the database.
error: Uncaught (in promise) Error: Unknown auth message code 1397113172
throw new Error(`Unknown auth message code ${code}`);
^
at Connection.handleAuth (connection.ts:197:15)
at Connection.startup (connection.ts:155:16)
at async Pool._createConnection (pool.ts:32:5)
at async pool.ts:61:7
at async Promise.all (index 0)
at async Pool._startup (pool.ts:63:25)
My application using Deno now
import { Pool } from "https://deno.land/x/postgres/mod.ts";
import { config } from "./config.ts";
const port = config.DB_PORT ? parseInt(config.DB_PORT || "") : undefined;
const POOL_CONNECTIONS = 20;
const dbPool = new Pool({
port,
hostname: config.DB_HOST,
user: config.DB_USER,
database: config.DB_NAME,
password: config.DB_PASS
}, POOL_CONNECTIONS);
export { dbPool };
Here is debug screen.
I have found this issue post and it mentioned about lacking ssl. Not sure how to do it on heroku.
I have tried some solutions, even change lib to pg and it still not work. I am very appreciated if any clue or help to fix this issue.
Note:
I read a document on heroku about "Heroku Postgres Connection Pooling is not available for Hobby-tier databases.". Then I switched to use Client with syntax similar like this to connect to Heroku postgres this:
import { Client } from "https://deno.land/x/postgres/mod.ts";
let config;
config = {
hostname: "localhost",
port: 5432,
user: "user",
database: "test",
applicationName: "my_custom_app"
};
// alternatively
config = "postgres://user#localhost:5432/test?application_name=my_custom_app";
const client = new Client(config);
await client.connect();
await client.end();
ref: https://deno-postgres.com/#/
Aurora Serverless (Postgresql) - execute statement timeout 502
I am building a lambda api connecting to AWS Aurora Serverless Postgres.
Follow AWS instruction to setup an Aurora Serverless with Postgres compatible (for MySql but still useful for Postgres): https://aws.amazon.com/getting-started/tutorials/configure-connect-serverless-mysql-database-aurora/
I added port 5432 in inbound of security group
I also use data-api-client to query the database (https://github.com/jeremydaly/data-api-client)
Built API lambda in Serverless Framework, set timeout is 1 min, added role AmazonRDSDataFullAccess
My lambda code (built in Serverless framework) is simple:
async function query_db(_sql) {
const data = require('data-api-client')({
secretArn: constants.DBSecretsStoreArn,
resourceArn: constants.DBAuroraClusterArn,
database: constants.DatabaseName
});
try {
let result = await data.query(_sql);
return result.records;
} catch (error) {
console.log('Lambda :: query_db :: Error: ' + error);
return error;
}
}
async function run() {
let sql = 'SELECT * FROM products LIMIT 10';
let result = await query_db(sql);
console.log('result: '+ JSON.stringify(result));
return callback(null, {
headers: {
'Access-Control-Allow-Origin': '*'
},
statusCode: 200,
body: JSON.stringify({msg: 'done})
});
}
Result:
It ran successfully in local (serverless-offline)
After deploying, it ran timeout, returned 502, error: "Internal server error"
Any suggestion is appreciated.
The data-api-client doesn't officially support Postgres, yet. https://github.com/jeremydaly/data-api-client/issues/27
Lua socket HTTP getting connection refused
I'm trying to create a function that creates an issue to a Github repository using Lua socket.http, but I'm getting connection refused everytime. The documentation for socket is a bit unclear and I couldn't find more helpful information on why is the request getting refused every time. I tried the following:
local config = {
token = "oauth token from github"
}
local http, ltn12 = require("socket.http"), require("ltn12")
local payload = '{"title": "Test", "body": "Test body", "labels": ["bug"]}'
local response, status, headers, line = http.request("https://api.github.com/repos/<username>/<repository>/issues?access_token=" .. config.token, payload)
So I checked again how to do and there is a second form to do a request. I'm trying the following:
local response = {}
local _, status, headers, line = http.request{
url = "https://api.github.com/repos/<username>/<repository>/issues",
sink = ltn12.sink.table(response),
method = "POST",
headers = {
["Authorization"] = "token " .. config.token,
["Content-Length"] = payload:len()
},
source = ltn12.source.string(payload)
}
According to socket documentation, this should make POST request to the URL sending the payload as body. If I print(status) it prints connection refused.
I'm ignoring the first return value as it always is 1.
I tried manually issuing the request using curl:
curl -H "Authorization: token <oauth token from github>" https://api.github.com/repos/<username>/<repository>/issues -XPOST -d '{"title": "Test", "body": "{"title": "Test", "body": "Test body", "labels": ["bug"]}'
And it posted the issue properly. I still can't figure it out what is happening that the connection is getting refused.
How to copy password protected database from remote server in mongodb?
I am running mongodb 2.4.8 and need to copy a database from remote server. Server has auth enabled in combination with a user having privileges the database. I have tried copydb but it didn't work. I guess it failed because of remote server using auth in combination with role based user(mentioned under authentication section of documentation).
host = "myhost.com"
mynonce = db.runCommand( { copydbgetnonce : 1, fromhost: host } ).nonce
username = "myuser"
password = "mypassword"
password_hash = hex_md5(mynonce + username + hex_md5(username + ":mongo:" + password))
db.runCommand({
copydb: 1,
fromdb: "test",
todb: "test",
fromhost: host,
username: username,
key: password_hash
})
# output: { "ok" : 0, "errmsg" : "" }
# but nothing really gets copied
What other options do I have? I would prefer a solution which can work from within the mongo shell as I do not have ssh access to server.
try db.copyDatabase(fromdb, todb, fromhost, username, password). as manual said: http://docs.mongodb.org/manual/reference/method/db.copyDatabase/