I provisioned a MongoDB for my Heroku app and I can connect to the database fine (using a third-party MongoDB UI tool). I created a new user and got the following message back:
Successfully added user: {
"user" : "username",
"roles" : [
{
"role" : "dbAdmin",
"db" : "heroku_340171"
}
]
}
However, when running my Django app I get the following error:
File "g:\Git\ChatterbotTest\chatterbottest\urls.py", line 3, in <module>
from views import FbView
File "g:\Git\ChatterbotTest\chatterbottest\views.py", line 33, in <module>
database_uri="mongodb://username:password#ds23123.mlab.com:13938/heroku_340171"
File "g:\Python\lib\site-packages\chatterbot\chatterbot.py", line 37, in __init__
self.storage = utils.initialize_class(storage_adapter, **kwargs)
File "g:\Python\lib\site-packages\chatterbot\utils.py", line 33, in initialize_class
return Class(**kwargs)
File "g:\Python\lib\site-packages\chatterbot\storage\mongodb.py", line 102, in __init__
self.statements.create_index('text', unique=True)
File "g:\Python\lib\site-packages\pymongo\collection.py", line 1529, in create_index
self.__create_index(keys, kwargs)
File "g:\Python\lib\site-packages\pymongo\collection.py", line 1430, in __create_index
parse_write_concern_error=True)
File "g:\Python\lib\site-packages\pymongo\collection.py", line 232, in _command
collation=collation)
File "g:\Python\lib\site-packages\pymongo\pool.py", line 419, in command
collation=collation)
File "g:\Python\lib\site-packages\pymongo\network.py", line 116, in command
parse_write_concern_error=parse_write_concern_error)
File "g:\Python\lib\site-packages\pymongo\helpers.py", line 210, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: not authorized on chatterbot-database to execute command { createIndexes: "statements", indexes: [ { unique: true, name: "text_1", key: { text: 1 } } ] }
dbAdmin should have the permissions to run createIndexes. What's going wrong?
Related
I'm triyng run the crawler with docker image, but it's returning this error.
PS C:\Users\Santosgab\Desktop\documentation> docker run -it --env-file=.env -e "CONFIG=$(cat config.json | jq -r tostring)" algolia/docsearch-scraper
Traceback (most recent call last):
File "/root/src/config/config_loader.py", line 101, in _load_config
data = json.loads(config, object_pairs_hook=OrderedDict)
File "/usr/lib/python3.6/json/__init__.py", line 367, in loads
return cls(**kw).decode(s)
File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.6/json/decoder.py", line 355, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
i used the config provided by Algolia, just made some small changes to run in my project, my config.json has no problem.
{
"index_name": "Sequor",
"sitemap_urls": ["http://localhost:3000/sitemap.xml"],
"sitemap_alternate_links": true,
"stop_urls": ["/tests"],
"selectors": {
"lvl0": {
"selector": "(//ul[contains(#class,'menu__list')]//a[contains(#class, 'menu__link menu__link--sublist menu__link--active')]/text() | //nav[contains(#class, 'navbar')]//a[contains(#class, 'navbar__link--active')]/text())[last()]",
"type": "xpath",
"global": true,
"default_value": "Documentation"
},
"lvl1": "header h1",
"lvl2": "article h2",
"lvl3": "article h3",
"lvl4": "article h4",
"lvl5": "article h5, article td:first-child",
"lvl6": "article h6",
"text": "article p, article li, article td:last-child"
},
"strip_chars": " .,;:#",
"custom_settings": {
"separatorsToIndex": "_",
"attributesForFaceting": ["language", "version", "type", "docusaurus_tag"],
"attributesToRetrieve": [
"hierarchy",
"content",
"anchor",
"url",
"url_without_anchor",
"type"
]
},
"conversation_id": ["833762294"],
"nb_hits": 46250
}
Two things I notice:
There's no path on your config file. Can you try it with pathing (i.e. ./config.json)
Your config is missing any start_urls as entry points to crawling.
I am running into an issue with MySQL python connection.
Traceback (most recent call last):
File "./expconfig.py", line 176, in <module>
cnx = mysql.connector.connect(**config)
File "/usr/lib/python2.6/site-packages/mysql/connector/__init__.py", line 179, in connect
return MySQLConnection(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/mysql/connector/connection.py", line 95, in __init__
self.connect(**kwargs)
File "/usr/lib/python2.6/site-packages/mysql/connector/abstracts.py", line 728, in connect
self._open_connection()
File "/usr/lib/python2.6/site-packages/mysql/connector/connection.py", line 228, in _open_connection
self._ssl)
File "/usr/lib/python2.6/site-packages/mysql/connector/connection.py", line 150, in _do_auth
ssl_options.get('cipher'))
File "/usr/lib/python2.6/site-packages/mysql/connector/network.py", line 420, in switch_to_ssl
ssl_version=ssl.PROTOCOL_TLSv1, ciphers=cipher)
TypeError: wrap_socket() got an unexpected keyword argument 'ciphers'
MySQL server has "ssl_diabled" , so the client doesn't need SSL connection. However it invokes
I have following,
Python : 2.6.6 MySql Python connector mysql-connector-python-2.1.7
OS : RHEL 6.6
MySQL server Server version : 5.7.17-enterprise-commercial-advanced
Code
try:
flags = [ClientFlag.FOUND_ROWS,-ClientFlag.SSL]
config = {
'user' : 'ed30_user',
'password' : 'mypassword',
'host' : options.remHost,
'database' : 'config',
'client_flags': [-ClientFlag.SSL],
'ssl_disabled' : False
}
cnx = mysql.connector.connect(**config)
cur=cnx.cursor(dictionary=True)
The reason for the error was that Pythong 2.6.6 has "ssl" module with different method signature than expected by the MysqlConnector version 2.1.7.
The MySQL web https://dev.mysql.com/doc/connector-python/en/connector-python-versions.html had in correct version table
I dropped back to connector version mysql-connector-python-2.0.5-1.el6.noarch.rpm and the error went away.
just set 'ssl_disabled' : True if using config file
or ssl_disabled='True' if your config is in arguments to mysql.connector.connect function
I have deployed a cluster to Azure successfully. Now trying to set up kubectl to work with it.
Running:
az acs kubernetes get-credentials --resource-group=group --name=cluster
Results in:
scp: .kube/config: No such file or directory
Traceback (most recent call last):
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/main.py", line 36, in main
cmd_result = APPLICATION.execute(args)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/application.py", line 212, in execute
result = expanded_arg.func(params)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 377, in __call__
return self.handler(*args, **kwargs)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 626, in _execute_command
reraise(*sys.exc_info())
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 603, in _execute_command
result = op(client, **kwargs) if client else op(**kwargs)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/custom.py", line 814, in k8s_get_credentials
_k8s_get_credentials_internal(name, acs_info, path, ssh_key_file)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/custom.py", line 835, in _k8s_get_credentials_internal
'.kube/config', path_candidate, key_filename=ssh_key_file)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/acs_client.py", line 64, in secure_copy
scp.get(src, dest)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/scp.py", line 198, in get
self._recv_all()
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/scp.py", line 348, in _recv_all
raise SCPException(asunicode(msg[1:]))
SCPException: scp: .kube/config: No such file or directory
File ~/.kube/config exists, and I can ssh to the master node of the cluster.
Any idea what on can be wrong with what I am doing?
UPDATE.
Tried running the command per one of the comments here:
az acs kubernetes get-credentials --resource-group=group --name=cluster -f /Users/me/.kube/config --debug
And received the following info:
Password for private key:
paramiko.transport : starting thread (client mode): 0x9659890L
paramiko.transport : Local version/idstring: SSH-2.0-paramiko_2.3.1
paramiko.transport : Remote version/idstring: SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.2
paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
paramiko.transport : kex algos:[u'curve25519-sha256#libssh.org', u'ecdh-sha2-nistp256', u'ecdh-sha2-nistp384', u'ecdh-sha2-nistp521', u'diffie-hellman-group-exchange-sha256', u'diffie-hellman-group14-sha1'] server key:[u'ssh-rsa', u'rsa-sha2-512', u'rsa-sha2-256', u'ecdsa-sha2-nistp256', u'ssh-ed25519'] client encrypt:[u'chacha20-poly1305#openssh.com', u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'aes128-gcm#openssh.com', u'aes256-gcm#openssh.com'] server encrypt:[u'chacha20-poly1305#openssh.com', u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'aes128-gcm#openssh.com', u'aes256-gcm#openssh.com'] client mac:[u'umac-64-etm#openssh.com', u'umac-128-etm#openssh.com', u'hmac-sha2-256-etm#openssh.com', u'hmac-sha2-512-etm#openssh.com', u'hmac-sha1-etm#openssh.com', u'umac-64#openssh.com', u'umac-128#openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-sha1'] server mac:[u'umac-64-etm#openssh.com', u'umac-128-etm#openssh.com', u'hmac-sha2-256-etm#openssh.com', u'hmac-sha2-512-etm#openssh.com', u'hmac-sha1-etm#openssh.com', u'umac-64#openssh.com', u'umac-128#openssh.com', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-sha1'] client compress:[u'none', u'zlib#openssh.com'] server compress:[u'none', u'zlib#openssh.com'] client lang:[u''] server lang:[u''] kex follows?False
paramiko.transport : Kex agreed: ecdh-sha2-nistp256
paramiko.transport : HostKey agreed: ecdsa-sha2-nistp256
paramiko.transport : Cipher agreed: aes128-ctr
paramiko.transport : MAC agreed: hmac-sha2-256
paramiko.transport : Compression agreed: none
paramiko.transport : kex engine KexNistp256 specified hash_algo <built-in function openssl_sha256>
paramiko.transport : Switch to new keys ...
paramiko.transport : Trying SSH key 765f702f5fff7fe30260d53b5bbb57eb
paramiko.transport : userauth is OK
paramiko.transport : Authentication (publickey) successful!
paramiko.transport : [chan 0] Max packet in: 32768 bytes
paramiko.transport : Received global request "hostkeys-00#openssh.com"
paramiko.transport : Rejecting "hostkeys-00#openssh.com" global request from server.
paramiko.transport : [chan 0] Max packet out: 32768 bytes
paramiko.transport : Secsh channel 0 opened.
paramiko.transport : [chan 0] Sesch channel 0 request ok
paramiko.transport : [chan 0] EOF received (0)
paramiko.transport : [chan 0] EOF sent (0)
scp: .kube/config: No such file or directory
Traceback (most recent call last):
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/main.py", line 36, in main
cmd_result = APPLICATION.execute(args)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/application.py", line 212, in execute
result = expanded_arg.func(params)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 377, in __call__
return self.handler(*args, **kwargs)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 626, in _execute_command
reraise(*sys.exc_info())
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 603, in _execute_command
result = op(client, **kwargs) if client else op(**kwargs)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/custom.py", line 814, in k8s_get_credentials
_k8s_get_credentials_internal(name, acs_info, path, ssh_key_file)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/custom.py", line 835, in _k8s_get_credentials_internal
'.kube/config', path_candidate, key_filename=ssh_key_file)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/acs_client.py", line 64, in secure_copy
scp.get(src, dest)
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/scp.py", line 198, in get
self._recv_all()
File "/Users/me/lib/azure-cli/lib/python2.7/site-packages/scp.py", line 348, in _recv_all
raise SCPException(asunicode(msg[1:]))
SCPException: scp: .kube/config: No such file or directory
Version of azure cli is 2.0.18. Would appreciate advice.
Try Using --file -f flag to specify the absolute path of the "config" file in .kube directory and you can use --debug to debug further.
az acs kubernetes get-credentials --resource-group=group --name=cluster -f /.../.../.kube/config --debug
In my case, .kube/config file wasn't created on the master. There was an error during the attempt to create .kube/config file, which I have noticed by ssh-ing to my master node and looking into /var/log/azure directory. The error was related to my service principal password. With a new service principal, new clusters work just fine.
I am connecting to mongo with a user with clusterAdmin and backup roles, but I get the error:
2017-02-09 17:51:23,254 [ERROR] mongo_connector.util:96 - Fatal Exception
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/mongo_connector/util.py", line 94, in wrapped
func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/mongo_connector/connector.py", line 370, in run
'listShards')['shards']:
File "/usr/lib/python2.7/site-packages/mongo_connector/util.py", line 78, in retry_until_ok
return func(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/pymongo/database.py", line 494, in command
codec_options, **kwargs)
File "/usr/lib64/python2.7/site-packages/pymongo/database.py", line 406, in _command
parse_write_concern_error=parse_write_concern_error)
File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 419, in command
collation=collation)
File "/usr/lib64/python2.7/site-packages/pymongo/network.py", line 116, in command
parse_write_concern_error=parse_write_concern_error)
File "/usr/lib64/python2.7/site-packages/pymongo/helpers.py", line 210, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
OperationFailure: not authorized on admin to execute command { listShards: 1 }
This page under Required Permissions says The simplest way to get mongo-connector running is to create a user with the backup role:
https://github.com/mongodb-labs/mongo-connector/wiki/Usage-with-Authentication
db.getSiblingDB("admin").createUser({ user:"backup",pwd:"password_here", roles: ["backup"] })
But I cant even connect with such a user (Authentication error):
2017-02-10 16:52:01,448 [ERROR] mongo_connector.util:96 - Fatal Exception
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/mongo_connector/util.py", line 94, in wrapped
func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/mongo_connector/connector.py", line 398, in run
hosts, replicaSet=repl_set)
File "/usr/lib/python2.7/site-packages/mongo_connector/connector.py", line 299, in create_authed_client
client['admin'].authenticate(self.auth_username, self.auth_key)
File "/usr/lib64/python2.7/site-packages/pymongo/database.py", line 1048, in authenticate
connect=True)
File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 505, in _cache_credentials
sock_info.authenticate(credentials)
File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 523, in authenticate
auth.authenticate(credentials, self)
File "/usr/lib64/python2.7/site-packages/pymongo/auth.py", line 470, in authenticate
auth_func(credentials, sock_info)
File "/usr/lib64/python2.7/site-packages/pymongo/auth.py", line 450, in _authenticate_default
return _authenticate_scram_sha1(credentials, sock_info)
File "/usr/lib64/python2.7/site-packages/pymongo/auth.py", line 201, in _authenticate_scram_sha1
res = sock_info.command(source, cmd)
File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 419, in command
collation=collation)
File "/usr/lib64/python2.7/site-packages/pymongo/network.py", line 116, in command
parse_write_concern_error=parse_write_concern_error)
File "/usr/lib64/python2.7/site-packages/pymongo/helpers.py", line 210, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
OperationFailure: Authentication failed.
When I log into mongos with both these users and run the command
db.getSiblingDB("admin").runCommand( { listShards: 1 } )
I get a shard listing no probs
{
"shards" : [
{
"_id" : "shard001",
"host" : "shard001/timgrhlmdb01:27020,timgrhlmdb02:27020",
"state" : 1
},
{
"_id" : "shard002",
"host" : "shard002/timgrhlmdb03:27020,timgrhlmdb04:27020",
"state" : 1
}
],
"ok" : 1
}
So what does this mean:
OperationFailure: not authorized on admin to execute command { listShards: 1 }
Update
I rebuilt the cluster from scratch and still have the same problem: OperationFailure: not authorized on admin to execute command { listShards: 1 }
I have also tried the user 'backup' with only the roles 'clusterManager' and 'readAnyDatabase'. This allows the user to list shards, but now mongo-connector fails with 'Authentication failed':
{ "_id" : "admin.backup", "user" : "backup", "db" : "admin", "credentials" : { "SCRAM-SHA-1" : { "iterationCount" : 10000, "salt" : "pWcEU7uFqfHPgGe8z+E9Wg==", "storedKey" : "k2tapXQPtM2dHlxYnJiWVxO/rtg=", "serverKey" : "EGG8M4i27OYBy+fLYaL13+Nn4mc=" } }, "roles" : [ { "role" : "readAnyDatabase", "db" : "admin" }, { "role" : "clusterManager", "db" : "admin" } ] }
Check out users by running this command:
db.system.users.find({})
Make sure that the user you created is with a backup role,if you can log in as the backup user and you can also run those commands,that means backup user was created and granted a role and its privileges.
Make sure that you have the role of clusterManager to perform this.
Provides management and monitoring actions on the cluster. A user with
this role can access the config and local databases, which are used in
sharding and replication, respectively.
Provides the following actions on the cluster as a whole:
addShard
appendOplogNote
applicationMessage
cleanupOrphaned
flushRouterConfig
listShards
removeShard
etc
Have a look at built-in-roles.
By the way,have a look at this issue.Hope this helps.
Response from bug submitted to mongodb-labs/mongo-connector:
This is indeed a subtle bug introduced in #563. We changed a find on
config.shards into a call to listShards assuming that it would have no
change in behavior. Unfortunately (and annoyingly), the backup role
has privileges to read the list of shards in the config.shards
collection but, as you can see, does not have the privilege to run the
listShards command. I'll revert this change to fix the problem in
the upcoming 2.5.1 bug-fix release.
In the meantime, you will need to grant the mongo-connector user the
backup AND clusterMonitor roles.
An important point that is not yet mentioned in the documentation is
that the user must be created on a mongos and all the shards. This
enables mongo-connector to authenticate to the cluster as a whole and
to each shard individually.
This now works! yay
That will teach me for following the manual lol!
I connect to remote sharded DB cluster. It connects OK, but if I insert document I get error:
File "/home/df/SlickJump/WishNuServer/server.py", line 522, in POST
unit = wndb.pages.find_one({'pagehash': thash, 'version': Version})
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 598, in find_one
for result in self.find(spec_or_id, *args, **kwargs).limit(-1):
File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 814, in next
if len(self.__data) or self._refresh():
File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 763, in _refresh
self.__uuid_subtype))
File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 720, in __send_message
self.__uuid_subtype)
File "/usr/local/lib/python2.7/dist-packages/pymongo/helpers.py", line 105, in _unpack_response
error_object["$err"])
OperationFailure: database error: setShardVersion failed host: db1s0:27017 { errmsg: "exception: remote client 5.101.101.150:46117 tried to initialize this host (db1
7) as shard shard0000, but shard name was previously initialized...", code: 13298, ok: 0.0 }
In MongoDB logs in looks:
2014-07-09T13:26:40.666+0000 [conn11] Assertion failed while processing query op for wishnudb.pages :: caused by :: 10429 setShardVersion failed host: db1s0:27017 {
"exception: remote client 5.101.101.150:46219 tried to initialize this host (db1s0:27017) as shard shard0000, but shard name was previously initialized...", code: 1
k: 0.0 }
Why do i get this error? There is a reason?
It happened to me when I had two shards that are actually one mongod:
{ "_id" : "shard0000", "host" : "localhost:27027" }
{ "_id" : "shard0001", "host" : "127.0.0.1:27027" }
When I removed the second shard, the error gone.