How to enable text search in mongo? - mongodb

I tried so many things..
# in replica set configuration, specify the name of the replica set
# replSet = setname
setParameter=textSearchEnabled=true
This is the part of config file. Still after setting this the text search is not enabled.
Am using pymongo for text searching
This is my code
db.command("text", 'tracks' ,search=request.POST['content_search'], limit = 12)['results']]
My mongo version is 2.4.10. Please guide me.
This is the traceback
Traceback (most recent call last):
File "/home/nidhin/social-media-widget/env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 114, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/nidhin/social-media-widget/env/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 57, in wrapped_view
return view_func(*args, **kwargs)
File "/home/nidhin/social-media-widget/socialmedia/widget/views.py", line 84, in monitor
data = [i['obj'] for i in db.command("text", 'tracks' ,search=request.POST['content_search'], filter = test_data, limit = 12)['results']]
File "/home/nidhin/social-media-widget/env/local/lib/python2.7/site-packages/pymongo/database.py", line 435, in command
uuid_subtype, compile_re, **kwargs)[0]
File "/home/nidhin/social-media-widget/env/local/lib/python2.7/site-packages/pymongo/database.py", line 341, in _command
msg, allowable_errors)
File "/home/nidhin/social-media-widget/env/local/lib/python2.7/site-packages/pymongo/helpers.py", line 178, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
OperationFailure: command SON([('text', 'tracks'), ('filter', {'publisher_desc': u'Blogs'}), ('search', u'box'), ('limit', 12)]) failed: text search not enabled

Adding this line to config file should work:
setParameter=textSearchEnabled=true
How do you start mongdb?
Edit:
I recommend you to check that:
You have mongodb started with this config.
You could check it by calling db.runCommand("getCmdLineOpts") in MongoDb shell
via MongoDb Shell db.runCommand({getParameter:1, textSearchEnabled: 1}) returns textSearchEnabled:true

Related

"InterfaceError: connection already closed" when using multiprocessing.Pool on black box function that queries PostgreSQL database

I've been given a Python (2.7) function that takes 3 strings as arguments, and returns a list of dictionaries. Due to the nature of the project, I can't alter the function, which is quite complex, calling several other non-standard Python modules and querying a PostgreSQL database using psychopg2. I think that it's the Postgres functionality that's causing me problems.
I want to use the multiprocessing module to speed up calling the function hundreds of times. I've written a "helper" function so that I can use multiprocessing.Pool (which takes only 1 argument) with my function:
from function_script import function
def function_helper(args):
return function(*args)
And my main code looks like this:
from helper_script import function_helper
from multiprocessing import Pool
argument_a = ['a0', 'a1', ..., 'a99']
argument_b = ['b0', 'b1', ..., 'b99']
argument_c = ['c0', 'c1', ..., 'c99']
input = zip(argument_a, argument_b, argument_c)
p = Pool(4)
results = p.map(function_helper, input)
print results
What I'm expecting is a list of lists of dictionaries, however I get the following errors:
Traceback (most recent call last):
File "/local/python/2.7/lib/python2.7/site-packages/variantValidator/variantValidator.py", line 898, in validator
vr.validate(input_parses)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/validator.py", line 33, in validate
return self._ivr.validate(var, strict) and self._evr.validate(var, strict)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/validator.py", line 69, in validate
(res, msg) = self._ref_is_valid(var)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/validator.py", line 89, in _ref_is_valid
var_x = self.vm.c_to_n(var) if var.type == "c" else var
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/variantmapper.py", line 223, in c_to_n
tm = self._fetch_TranscriptMapper(tx_ac=var_c.ac, alt_ac=var_c.ac, alt_aln_method="transcript")
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/decorators/lru_cache.py", line 176, in wrapper
result = user_function(*args, **kwds)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/variantmapper.py", line 372, in _fetch_TranscriptMapper
self.hdp, tx_ac=tx_ac, alt_ac=alt_ac, alt_aln_method=alt_aln_method)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/transcriptmapper.py", line 69, in __init__
self.tx_identity_info = hdp.get_tx_identity_info(self.tx_ac)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/decorators/lru_cache.py", line 176, in wrapper
result = user_function(*args, **kwds)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/dataproviders/uta.py", line 353, in get_tx_identity_info
rows = self._fetchall(self._queries['tx_identity_info'], [tx_ac])
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/dataproviders/uta.py", line 216, in _fetchall
with self._get_cursor() as cur:
File "/local/python/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/dataproviders/uta.py", line 529, in _get_cursor
cur.execute("set search_path = " + self.url.schema + ";")
File "/local/python/2.7/lib/python2.7/site-packages/psycopg2/extras.py", line 144, in execute
return super(DictCursor, self).execute(query, vars)
DatabaseError: SSL error: decryption failed or bad record mac
And:
Traceback (most recent call last):
File "/local/python/2.7/lib/python2.7/site-packages/variantValidator/variantValidator.py", line 898, in validator
vr.validate(input_parses)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/validator.py", line 33, in validate
return self._ivr.validate(var, strict) and self._evr.validate(var, strict)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/validator.py", line 69, in validate
(res, msg) = self._ref_is_valid(var)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/validator.py", line 89, in _ref_is_valid
var_x = self.vm.c_to_n(var) if var.type == "c" else var
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/variantmapper.py", line 223, in c_to_n
tm = self._fetch_TranscriptMapper(tx_ac=var_c.ac, alt_ac=var_c.ac, alt_aln_method="transcript")
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/decorators/lru_cache.py", line 176, in wrapper
result = user_function(*args, **kwds)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/variantmapper.py", line 372, in _fetch_TranscriptMapper
self.hdp, tx_ac=tx_ac, alt_ac=alt_ac, alt_aln_method=alt_aln_method)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/transcriptmapper.py", line 69, in __init__
self.tx_identity_info = hdp.get_tx_identity_info(self.tx_ac)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/decorators/lru_cache.py", line 176, in wrapper
result = user_function(*args, **kwds)
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/dataproviders/uta.py", line 353, in get_tx_identity_info
rows = self._fetchall(self._queries['tx_identity_info'], [tx_ac])
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/dataproviders/uta.py", line 216, in _fetchall
with self._get_cursor() as cur:
File "/local/python/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/local/python/2.7/lib/python2.7/site-packages/hgvs/dataproviders/uta.py", line 526, in _get_cursor
conn.autocommit = True
InterfaceError: connection already closed
Does anybody know what might cause the Pool function to behave like this, when it seems so simple to use in other examples that I've tried? If this isn't enough information to go on, can anyone advise me on a way of getting to the bottom of the problem (this is the first time I've worked with someone else's code)? Alternatively, are there any other ways that I could use the multiprocessing module to call the function hundreds of times?
Thanks
I think what may be happening is that your connection object is used across all workers and when 1 worker has completed all its tasks it closes the connection and meanwhile the other workers are still working and the connection is closed so when one of those workers tries to use the db it is already closed.

Address already in use in targetcli iser enable

kindly help me to fix this problem
I am using "targetcli-2.0rc1"
I have created the portal with IP address 192.168.1.11 and port no : 3260
When I am doing "iser_enable" then it throws following error
/iscsi/iqn.20...168.1.11:3260> iser_enable
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/configshell/shell.py", line 983, in run_interactive
self._cli_loop()
File "/usr/lib/python2.7/dist-packages/configshell/shell.py", line 806, in _cli_loop
self.run_cmdline(cmdline)
File "/usr/lib/python2.7/dist-packages/configshell/shell.py", line 927, in run_cmdline
self._execute_command(path, command, pparams, kparams)
File "/usr/lib/python2.7/dist-packages/configshell/shell.py", line 902, in _execute_command
result = target.execute_command(command, pparams, kparams)
File "/usr/lib/python2.7/dist-packages/targetcli/ui_node.py", line 216, in execute_command
return UINode.execute_command(self, command, pparams, kparams)
File "/usr/lib/python2.7/dist-packages/targetcli/ui_node.py", line 101, in execute_command
pparams, kparams)
File "/usr/lib/python2.7/dist-packages/configshell/node.py", line 1405, in execute_command
result = method(*pparams, **kparams)
File "/usr/lib/python2.7/dist-packages/targetcli/ui_target.py", line 1042, in ui_command_iser_enable
self.portal._set_iser_attr(True)
File "/usr/lib/python2.7/dist-packages/rtslib/target.py", line 870, in _set_iser_attr
fwrite(path, "1")
File "/usr/lib/python2.7/dist-packages/rtslib/utils.py", line 106, in fwrite
file_fd.close()
IOError: [Errno 98] Address already in use
when i am using Soft-RoCE, iser_enable has no issues...
when I am trying with Soft-iWarp i got the above issue
my ibv_devices o/p is as follows
root#VCHN145:~# ibv_devices
device node GUID
------ ----------------
siw_wlan0 c038969609070000
siw_eth0 3417eb90714e0000
siw_lo 7369775f6c6f0000
root#VCHN145:~#
kindly help me to fix this problem

Error while start openerp

i have installed openerp. i have created postgres database. While i open ¨localhost:8060¨ in browser, display the followin error:
Client Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/http.py", line 204, in dispatch
response["result"] = method(self, **self.params)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/controllers/main.py", line 761, in get_list
monodb = db_monodb(req)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/controllers/main.py", line 129, in db_monodb
return db_redirect(req, True)[0]
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/controllers/main.py", line 109, in db_redirect
dbs = db_list(req, True)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/controllers/main.py", line 90, in db_list
dbs = proxy.list(force)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/session.py", line 30, in proxy_method
result = self.session.send(self.service_name, method, *args)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/session.py", line 103, in send
raise xmlrpclib.Fault(openerp.tools.ustr(e), formatted_info)
Server Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/session.py", line 89, in send
return openerp.netsvc.dispatch_rpc(service_name, method, args)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/netsvc.py", line 292, in dispatch_rpc
result = ExportService.getService(service_name).dispatch(method, params)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/service/web_services.py", line 122, in dispatch
return fn(*params)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/service/web_services.py", line 359, in exp_list
cr = db.cursor()
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/sql_db.py", line 484, in cursor
return Cursor(self._pool, self.dbname, serialized=serialized)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/sql_db.py", line 182, in __init__
self._cnx = pool.borrow(dsn(dbname))
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/sql_db.py", line 377, in _locked
return fun(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/sql_db.py", line 440, in borrow
result = psycopg2.connect(dsn=dsn, connection_factory=PsycoConnection)
File "/usr/lib/python2.7/dist-packages/psycopg2/__init__.py", line 179, in connect
connection_factory=connection_factory, async=async)
OperationalError: FATAL: role "bala" does not exist
otherwise i started like following ¨openerp-server start¨, error following like
Traceback (most recent call last):
File "/usr/local/bin/openerp-server", line 5, in <module>
pkg_resources.run_script('openerp==7.0-20140110-002122', 'openerp-server')
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 499, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1235, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/EGG-INFO/scripts/openerp-server", line 5, in <module>
openerp.cli.main()
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/cli/__init__.py", line 51, in main
__import__(m)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/modules/module.py", line 133, in load_module
mod = imp.load_module('openerp.addons.' + module_part, f, path, descr)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/l10n_si/__init__.py", line 22, in <module>
import account_wizard
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/l10n_si/account_wizard.py", line 22, in <module>
import tools
ImportError: No module named tools
Anyone help for error correction. i cannot clear this error even two days i spend for this.
OperationalError: FATAL: role "bala" does not exist
A user that you have told OpenERP to use does not exist in the database. You probably need to create it. I expect you missed one or more installation steps.
Look for a CREATE USER, CREATE ROLE, or createuser command in the instructions.
GRANT commands or a database ownership assignment may also be nececssary depending on what the user is supposed to be able to do.
(It might also be assumed that you'll know to create the user to connect to the DB as).

handle crash of flask app on database restart

Currently have made a flask application which crashes when I do a postgres database restart, because the cursor which was opened is stale ...
How do I handle this situation. Currently connecting the flask app to postgres via psycopg2....
I am not a database expert...
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1701, in __call__
return self.wsgi_app(environ, start_response)
File "/var/www/flaskapps/capp/override.py", line 15, in __call__
return self.app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1689, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1687, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1360, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1356, in full_dispatch_request
rv = self.preprocess_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1539, in preprocess_request
rv = func()
File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 321, in _load_user
self.reload_user()
File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 350, in reload_user
user = self.user_callback(user_id)
File "/var/www/flaskapps/capp/login_setup.py", line 163, in load_user
cursor.execute(qstr)
File "/usr/share/pyshared/psycopg2/extras.py", line 123, in execute
return _cursor.execute(self, query, vars)
InterfaceError: cursor already closed
This is one of the many cases where your code needs to detect a transient failure and re-try the transaction, re-opening the connection if necessary.
Other cases include deadlocks and serialization failures.
The sqlstate on the exception will let you determine which error cases to retry and how. See the PostgreSQL documentation on error codes for guidance on the meaning of the sqlstate codes.
Sometimes your database interface will through a typed exception that tells you enough just by its data type, too. This doesn't look like one of those cases.

MongoDB Assertion Error: starting_from == self.__retrieved (pymongo driver)

MongoDB Question:
We're using a sharded replicaset, running pymongo 2.2 against mongo (version: 2.1.1-pre-). We're getting a traceback when a query returns more than one result document.
Traceback (most recent call last):
File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner
self.run()
File "/opt/DCM/mods/plugin.py", line 25, in run
self._mod.collect_metrics_dcm()
File "/opt/DCM/plugins/res.py", line 115, in collect_metrics_dcm
ms.updateSpecificMetric(metricName, value, timestamp)
File "/opt/DCM/mods/mongoSaver.py", line 155, in updateSpecificMetric
latestDoc = self.getLatestDoc(metricName)
File "/opt/DCM/mods/mongoSaver.py", line 70, in getLatestDoc
for d in dlist:
File "/usr/lib64/python2.6/site-packages/pymongo/cursor.py", line 747, in next
if len(self.__data) or self._refresh():
File "/usr/lib64/python2.6/site-packages/pymongo/cursor.py", line 698, in _refresh
self.__uuid_subtype))
File "/usr/lib64/python2.6/site-packages/pymongo/cursor.py", line 668, in __send_message
assert response["starting_from"] == self.__retrieved
AssertionError
The code that give what dlist is is a simple find(). I've tried reIndex(), no joy. I've tried stopping and starting the mongo server, no joy.
This is easily replicable for me. Any ideas?
Ok, so traced this down a bit, and I have a SOLUTION for this assertion error.
There is a BUG in Mongo. When querying a sharded replicaset, Mongo returns an incorrect value for 'starting_from'. Instead of returning 0 on the first query, it's returning the number of records received instead of the offset value. I have a patch for pymongo to protect against this bad info:
File is site-packages/pymongo/cursor.py.
[user#hostname]$ diff cursor.py.orig cursor.py
631,632c631,634
< if not self.__tailable:
< assert response["starting_from"] == self.__retrieved
---
> if ((not self.__tailable) and (self.__retrieved != 0) and (response["starting_from"] != self.__retrieved)):
> from pprint import pformat
> msg = "Server response of 'starting_from' is '%s', but self__retrieved (which is only set to nonzero below here) is '%s'." % (pformat(response), pformat(self.__retrieved))
> assert False, msg
The 'starting_from' comes from helpers.py decoding the response from Mongo:
result["starting_from"] = struct.unpack("<i", response[12:16])[0]
So, it's the 12th thru the 15th byte of Mongo's response.
This is a bug in the 2.1.1 development release of mongos. See https://jira.mongodb.org/browse/SERVER-5844