I have been pulling my hair trying to figure out what's wrong with mlflow. Iam deploying mlflow v1.26 in google cloudRun . back end artitfactory is google storage and backend database is google cloudsql postgres v13 instance.
here is my entrypoint using pg8000 v1.21.3 (I tried latest version as well) and psycopg2-binary v2.9.3
set -e
export ARTIFACT_URL="gs://ei-cs-dev01-ein-sb-teambucket-chaai-01/mlflow/"
export DATABASE_URL="postgresql+pg8000://mlflow:change2022#10.238.139.37:5432/mlflowps" #"$(python3 /app/get_secret.py --project="${GCP_PROJECT}" --secret=mlflow_database_url)"
if [[ -z "${PORT}" ]]; then
export PORT=8080
fi
exec mlflow server -h 0.0.0.0 -w 4 -p ${PORT} --default-artifact-root ${ARTIFACT_URL} --backend-store-uri ${DATABASE_URL}
now when I open mlflow ui page I see this error happening:
(
BAD_REQUEST: (pg8000.dbapi.ProgrammingError) {'S': 'ERROR', 'V':
'ERROR', 'C': '42883', 'M': 'operator does not exist: integer =
character varying', 'H': 'No operator matches the given name and
argument types. You might need to add explicit type casts.', 'P':
'382', 'F': 'parse_oper.c', 'L': '731', 'R': 'op_error'} [SQL: SELECT
DISTINCT runs.run_uuid..
)
You should use psycopg2 instead, e.g.:
postgresql+psycopg2://<username>:<password>#/<dbname>?host=/cloudsql/<my-project>:<us-central1>:<dbinstance>
It works for me, with versions:
mlflow==1.26.1
psycopg2-binary==2.9.3
Related
I am trying to deploy a Django REST API using Azure App Service on Linux. I am using a postgresql Database and deploy via pipeline. Azure has postgresql 9.6. After running my pipeline, the Website shows an Server Error (500).
The AppLogs show, that the Container couldn't be started due an failed import of psycopg2.
[ERROR] Exception in worker process
Traceback (most recent call last):
File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 25, in
import psycopg2 as Database
File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/psycopg2/__init__.py", line 50, in
from psycopg2._psycopg import ( # noqa
ImportError: /home/site/wwwroot/antenv/lib/python3.7/site-packages/psycopg2/_psycopg.cpython-37m-x86_64-linux-gnu.so: undefined symbol: PQencryptPasswordConn
In the Build-stage of my pipeline, I set up my environment (python3.7) like this:
- script: |
python -m venv antenv
source antenv/bin/activate
python -m pip install --upgrade pip
pip install setup
pip install -r requirements.txt
Where requirements.txt looks like this:
Django==3.0.2
djangorestframework==3.11.0
psycopg2-binary==2.8.4
pandas==0.25.3
pytest==5.3.5
pytest-django==3.8.0
pytest-mock==2.0.0
python-dateutil==2.8.1
sqlparse==0.3.0
whitenoise==5.0.1
BuildJob and DeploymentJob seem to run flawless. the Build-logs indicate that psycopg2_binary-2.8.4-cp37-cp37m-manylinux1_x86_64.whl was correctly downloaded and installed.
Also the App runs fine on my machine when using the database on azure by configuring in the settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'databasename',
'USER': 'user#postgresqlserver',
'PASSWORD': 'Password',
'HOST': 'postgresqlserver.postgres.database.azure.com',
'PORT': '',
'OPTIONS': {'sslmode': 'require'}
}
} # Of course the info is actually saved in environment variables
This gives me the feeling, that something with the psycopg2 installation is not working... For others the *psycopg2-binary seemed to do the trick but unfortunateley not for me.
Am I right to assume that on azure I'm nether able to install postgresql10 as suggested here https://github.com/psycopg/psycopg2/issues/983 nor can install from source like suggested here https://github.com/psycopg/psycopg2/issues/1018?
There must be something I am missing, I would be grateful for any advice!
EDIT:
Taking a look at the library (as suggested here https://stackoverflow.com/a/59652816/13183775) I found that I don't have a PQencryptPasswordConn function but only a PQencryptPassword function. I have the feeling that this is expected for Postgresql9.6 (https://github.com/psycopg/psycopg2/blob/cb3353be1f10590cdc2a894ada42c3b4c171feb7/psycopg/psycopgmodule.c#L466).
To check, whether there are multiple versions libpq:
/>find . -name "libpq*"
./var/lib/dpkg/info/libpq5:amd64.symbols
./var/lib/dpkg/info/libpq5:amd64.shlibs
./var/lib/dpkg/info/libpq5:amd64.list
./var/lib/dpkg/info/libpq5:amd64.triggers
./var/lib/dpkg/info/libpq-dev.list
./var/lib/dpkg/info/libpq5:amd64.md5sums
./var/lib/dpkg/info/libpq-dev.md5sums
./usr/share/doc/libpq5
./usr/share/doc/libpq-dev
./usr/share/locale/ko/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/it/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/pl/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/zh_TW/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/tr/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/cs/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/de/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/ru/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/sv/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/pt_BR/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/fr/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/es/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/zh_CN/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/ja/LC_MESSAGES/libpq5-9.6.mo
./usr/lib/x86_64-linux-gnu/pkgconfig/libpq.pc
./usr/lib/x86_64-linux-gnu/libpq.so.5
./usr/lib/x86_64-linux-gnu/libpq.so
./usr/lib/x86_64-linux-gnu/libpq.so.5.9
./usr/lib/x86_64-linux-gnu/libpq.a
./usr/include/postgresql/libpq-events.h
./usr/include/postgresql/libpq-fe.h
./usr/include/postgresql/libpq
./usr/include/postgresql/libpq/libpq-fs.h
./usr/include/postgresql/internal/libpq
./usr/include/postgresql/internal/libpq-int.h>
Sadly I'm not able to see here wether there are multiple libpq versions...
Wondering if anyone knows of a validator that can validate cloud-init config similar to this. We tried it and it gives a bunch of errors that are not applicable e.g., it does not recognize package_update which is a standard keyword that can be found here in the official documentation for example. So we are looking for a validator that works and recognizes the keywords in the official documentation.
I hit this one recently too and found a nice way to do it.
In Ubuntu you can use the follwoing to validate the file parses correctly.
cloud-init schema --config-file bob.txt
This will give you the following notice if the files are valid or invalid
:~$ sudo cloud-init schema --config-file bob.txt
Valid cloud-config file bob.txt
$ nano bob.txt #edited the yaml to make it invalid
:~$ sudo cloud-init schema --config-file bob.txt
Cloud config schema errors: format-l2.c1: File bob.txt is not valid yaml. while parsing a block mapping
in "<byte string>", line 2, column 1:
package_upgrade: true
^
expected <block end>, but found '-'
in "<byte string>", line 6, column 1:
- 'curl -fsSL https://get.docker ...
^
While your working with cloud init files cloud-init status --wait and many of the other commands on the cli are super useful.
Starting from version 22.2, the cloud-init command to validate config file is now:
cloud-init schema --config-file conf.yaml
For a given user foo with password bar, I can successfully authenticate against the admin database and get the desired result from one of our shard nodes with the following command:
mongo PRIMARY --host exampleShard1.com \
--port 27018 \
--username foo \
-p bar \
--authenticationDatabase 'admin' \
--eval 'db.serverStatus().connections'
However, when attempting to do the exact same command via a third party library such as the Java's MongoDB Driver or Python's pymongo, the authentication passes successfully but I'm unable to execute any commands successfully due to "not authorized" errors.
For example, here's a pymongo version of the working command above:
#!/usr/bin/python3
from pymongo import MongoClient
mongoClient = MongoClient("exampleShard1.com", 27018)
mongoClient.admin.authenticate("foo", "bar")
primaryDb = mongoClient["PRIMARY"]
print(primaryDb.eval("db.serverStatus().connections"))
Despite this appearing to be equivalent to the origin working command, this version fails with a version of the following error:
not authorized on local to execute command
{
$eval: db.serverStatus().connections,
args: [],
lsid: {
id: UUID("f5a936ee-71ad-4568-8bf2-a45c69424200")
},
$clusterTime: {
signature: {
keyId: 111,
hash: BinData(0, 6F)
},
clusterTime: Timestamp(1536848021, 159)
},
$db: "local",
$readPreference: {
mode: "primaryPreferred"
}
}
This is despite the authenticate() method returning "True", proving a successful authentication.
My guess would be there's a required value being set automatically from the command line that isn't being set from the script version, or that the command line version is somehow bypassing an extra level of authentication that the script version is caught in.
At this point, I'm unfortunately out of things to try, and I've exhausted Google for help. Any guidance would be greatly appreciated!
For each of the following 3 databases I want to check if the database exists.
"psql -d #{dbname} -c 'SELECT 1 as check_;'" on its own, works as expected and returns 0 if it exists and 2 if it doesn't.
But getting chef to do something with that test is difficult.
I can't use guard not_if because I really need to control a number of steps from that existence:
load the database
create user
grant user
etc...
So what I was really planning to do was to do a next and iterate to the next database in the loop if it existed, via next.
%w{ hcm91dmo ksysdb pgfin92}.each do |dbname|
begin
execute "check_exist_db" do
command "psql -d #{dbname} -c 'SELECT 1 as check_;'"
user "postgres"
ignore_failure true
dbexist2 = true
end
dbexist1 = true
Chef::Log.info("jlp:execute OK")
rescue
Chef::Log.info("jlp:execute rescue")
dbexist1 = false
dbexist2 = false
end
The rescue doesn't seem to matter much. And I still get errors despite the ignore_failure:
==> default: STDERR: psql: FATAL: database "pgfin92" does not exist
==> default: ---- End output of psql -d pgfin92 -c 'SELECT 1 as check_;' ----
==> default: Ran psql -d pgfin92 -c 'SELECT 1 as check_;' returned 2; ignore_failure is set, continuing
==> default:
==> default: ================================================================================
==> default: Error executing action `run` on resource 'execute[check_exist_db]'
==> default: ================================================================================
==> default:
==> default: Mixlib::ShellOut::ShellCommandFailed
==> default: ------------------------------------
==> default: Expected process to exit with [0], but received '2'
==> default: ---- Begin output of psql -d pgfin92 -c 'SELECT 1 as check_;' ----
==> default: STDOUT:
==> default: STDERR: psql: FATAL: database "pgfin92" does not exist
==> default: ---- End output of psql -d pgfin92 -c 'SELECT 1 as check_;' ----
==> default: Ran psql -d pgfin92 -c 'SELECT 1 as check_;' returned 2
For those who are curious, I am using Chef to gradually tweak a server to its desired state. A lot of time the code and configuration changes a bit, but without the databases needing any updates. In fact, since the databases get updated by user input, I want user changes to persist through Chef runs. So I don't want to drop and reload the databases without a good reason.
Solution, adapted from #coderanger 's answer:
#chef load: save list of databases that don't exist
dbtodo = []
%w{ hcm91dmo ksysdb pgfin92}.each do |dbname|
#note that you need to specify a user that works for postgres
if (shell_out("psql", "-d", dbname, "-c", "SELECT 1 as check_;",:user => "postgres").error?)
dbtodo.push(dbname)
end
end
#now, just loop thru databases that didn't exist
#I know I need to perform all the actions on each one of these.
dbtodo.each do |dbname|
#execute phase, no conditional needed because we're running off
#saved database list
postgresql_database dbname do
connection postgresql_connection_info
action :create
end
...
end
execute is a resource, which doesn't really help when you want to get some kind of result from running the command. What you want is to use the shell_out helper method:
shell_out("psql", "-d", dbname, "-c", "SELECT 1 as check_;").error?
or something similar to that. Unlike a resource which has the usual two-pass loading and execution process, shell_out (and shell_out!) are imperative Ruby code so when that line runs, it runs the command and gets you the result right then. Check out https://coderanger.net/two-pass/ for more info on the loading model and why your example doesn't work.
I use MAMP server with PHP 7.1.5; Symfony Framework 3.4.2, PostgreSQL 10 for a new project. I also use PostGIS to store space data with geometry data type in PostgreSQL. Therefore I installed and used: "jsor/doctrine-postgis": "^1.5"
Following is part of my composer.json:
"require": {
"php": ">=5.5.9",
"doctrine/doctrine-bundle": "^1.6",
"doctrine/orm": "^2.5",
"incenteev/composer-parameter-handler": "^2.0",
"jsor/doctrine-postgis": "^1.5",
"sensio/distribution-bundle": "^5.0.19",
"sensio/framework-extra-bundle": "^5.0.0",
"symfony/monolog-bundle": "^3.1.0",
"symfony/polyfill-apcu": "^1.0",
"symfony/swiftmailer-bundle": "^2.6.4",
"symfony/symfony": "3.4.*",
"twig/twig": "^1.0||^2.0"
The parameters.yml:
server_ver: 10.0
database_driver: pdo_pgsql
database_host: localhost
database_port: 5432
database_name: qtqg
database_path: ~
database_user: postgres
database_password: admin
The config.yml:
doctrine:
dbal:
default_connection: default
connections:
default:
mapping_types:
_text: string
server_version: %server_ver%
driver: %database_driver%
host: %database_host%
port: %database_port%
dbname: %database_name%
path: %database_path%
user: %database_user%
password: %database_password%
persistent: true
charset: UTF8
logging: %kernel.debug%
profiling: %kernel.debug%
# if using pdo_sqlite as your database driver:
# 1. add the path in parameters.yml
# e.g. database_path: '%kernel.project_dir%/var/data/data.sqlite'
# 2. Uncomment database_path in parameters.yml.dist
# 3. Uncomment next line:
#path: '%database_path%'
types:
geography:
class: 'Jsor\Doctrine\PostGIS\Types\GeographyType'
commented: false
geometry:
class: 'Jsor\Doctrine\PostGIS\Types\GeometryType'
commented: false
raster:
class: 'Jsor\Doctrine\PostGIS\Types\RasterType'
commented: false
Everything work well, I can use cmd to generate Entity:
php bin/console doctrine:mapping:import --force AppBundle annotation
And after that generate CRUD:
php bin/console generate:doctrine:crud AppBundleMonitoringAdminBundle:coquan -n --format=annotation --with-write
The Running PHP is PHP 7.1.5 and I also checked php.ini file in C:\Windows and loaded php.ini in MAMP server. php -m command shows:
PDO
pdo_mysql
PDO_ODBC
pdo_pgsql
pdo_sqlite
I don't think any problem with data driver because it can connect and generate Entities, CRUD....
But after generate CRUD and try to access the controller to list all item in one Entity, I got the Error:
An exception occurred in driver: could not find driver
One of the error line is:
AbstractPostgreSQLDriver->convertException('An exception occurred in driver: could not find driver', object(PDOException)) in vendor\doctrine\dbal\lib\Doctrine\DBAL\DBALException.php (line 176)
I tried many way including changing mamp to xamp, wamp server, all recommendation about how to config dbal... but the error is still there
Could anyone help me!
the problem is with PostgreSQL 10
Inside vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/PostgreSqlSchemaManager.php line line 292 change
$data = $this->_conn->fetchAll('SELECT min_value, increment_by FROM ' . $this->_platform->quoteIdentifier($sequenceName));
For
$version = floatval($this->_conn->getWrappedConnection()->getServerVersion());
if ($version >= 10) {
$data = $this->_conn->fetchAll('SELECT min_value, increment_by FROM pg_sequences WHERE schemaname = \'public\' AND sequencename = '.$this->_conn->quote($sequenceName));
}
else
{
$data = $this->_conn->fetchAll('SELECT min_value, increment_by FROM ' . $this->_platform->quoteIdentifier($sequenceName));
}
I was have the same issue with postgres 9.6.4 with WAMP on windows 10
phpinfo() would display pdo_pgsql as loaded, and it worked fine in another PHP application. php -m from the gitbash would not show pdo_pgsql
In the end, I had to edit the php.ini that was loaded and uncomment these 2 lines
extension=php_pdo_pgsql.dll
extension=php_pgsql.dll
restart apache and now it works!!!!