No handlers could be found for logger "mongo_connector.util" - mongodb

I always got the error below when I try to run mongo-connector with neo4j doc manager. I also tried with a config file as in https://github.com/mongodb-labs/mongo-connector/blob/master/config.json
Where is the problem?
mongo-connector -m localhost:27017 -t http://localhost:7474/db/data -d neo4j_doc_manager
No handlers could be found for logger "mongo_connector.util"
Traceback (most recent call last):
File "/usr/bin/mongo-connector", line 11, in <module>
sys.exit(main())
File "/usr/lib/python2.6/site-packages/mongo_connector/util.py", line 85, in wrapped
func(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/mongo_connector/connector.py", line 1041, in main
conf.parse_args()
File "/usr/lib/python2.6/site-packages/mongo_connector/config.py", line 118, in parse_args
option, dict((k, values.get(k)) for k in option.cli_names))
File "/usr/lib/python2.6/site-packages/mongo_connector/connector.py", line 824, in apply_doc_managers
module = import_dm_by_name(dm['docManager'])
File "/usr/lib/python2.6/site-packages/mongo_connector/connector.py", line 803, in import_dm_by_name
module = __import__(full_name, fromlist=(name,))
File "/usr/lib/python2.6/site-packages/mongo_connector/doc_managers/neo4j_doc_manager.py", line 16, in <module>
from py2neo import Graph, authenticate
File "/usr/lib/python2.6/site-packages/py2neo/__init__.py", line 28, in <module>
from py2neo.database import *
File "/usr/lib/python2.6/site-packages/py2neo/database/__init__.py", line 65
parameters = {k: v for k, v in parameters.items() if k not in presub_parameters}

It looks like you are using Python 2.6. I'm not sure if that version is officially supported for this project. I would suggest upgrading to Python 2.7 or preferably 3.4 and trying to reproduce.

Related

Google Cloud SDK installation - gsutil error

I have installed Google Cloud SDK but having an issue running "gsutil". Here are the error I'm getting:
~/gcloud/google-cloud-sdk#> gsutil
Traceback (most recent call last):
File "/Users/gonyi/Desktop/gonyyi/gcloud/google-cloud-sdk/bin/bootstrapping/gsutil.py", line 13, in <module>
import bootstrapping
File "/Users/gonyi/Desktop/gonyyi/gcloud/google-cloud-sdk/bin/bootstrapping/bootstrapping.py", line 40, in <module>
from googlecloudsdk.core import execution_utils
File "/Users/gonyi/Desktop/gonyyi/gcloud/google-cloud-sdk/lib/googlecloudsdk/core/execution_utils.py", line 33, in <module>
from googlecloudsdk.core import log
File "/Users/gonyi/Desktop/gonyyi/gcloud/google-cloud-sdk/lib/googlecloudsdk/core/log.py", line 810, in <module>
_log_manager = _LogManager()
File "/Users/gonyi/Desktop/gonyyi/gcloud/google-cloud-sdk/lib/googlecloudsdk/core/log.py", line 526, in __init__
self._file_formatter = _LogFileFormatter()
File "/Users/gonyi/Desktop/gonyyi/gcloud/google-cloud-sdk/lib/googlecloudsdk/core/log.py", line 315, in __init__
super(_LogFileFormatter, self).__init__(fmt=_LogFileFormatter.FORMAT)
TypeError: must be type, not classobj
~/gcloud/google-cloud-sdk#>
I have Python 2.7 installed, also tried to install this with "brew cask" as well as download it directly from Google Cloud site and install, but no luck either.
I googled this error but it seems like I'm the only one with this error..
"gcloud" command works just fine; but it's just "gsutil" that's not working.
Thank you

Apache Beam Python wordcount example errors on Windows10

I am running Anaconda - conda virtual env with Python 2.7
I have followed Apache Beam Python SDK Quickstart
When I run -
'python -m apache_beam.examples.wordcount --input C:\Users\simon_6dagkya\OneDrive\ProgrammingCore\Apache Beam\examples\wordcount\kinglear.txt --output C:\Users\simon_6dagkya\OneDrive\ProgrammingCore\Apache Beam\examples\wordcount\output.txt'
I get following error:
INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
Traceback (most recent call last):
File "C:\Users\simon_6dagkya\Anaconda3\envs\apachebeam\lib\runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "C:\Users\simon_6dagkya\Anaconda3\envs\apachebeam\lib\runpy.py", line 72, in _run_code
exec code in run_globals
File "C:\Users\simon_6dagkya\Anaconda3\envs\apachebeam\lib\site-packages\apache_beam\examples\wordcount.py", line 136, in <module>
run()
File "C:\Users\simon_6dagkya\Anaconda3\envs\apachebeam\lib\site-packages\apache_beam\examples\wordcount.py", line 90, in run
lines = p | 'read' >> ReadFromText(known_args.input)
File "C:\Users\simon_6dagkya\Anaconda3\envs\apachebeam\lib\site-packages\apache_beam\io\textio.py", line 524, in __init__
skip_header_lines=skip_header_lines)
File "C:\Users\simon_6dagkya\Anaconda3\envs\apachebeam\lib\site-packages\apache_beam\io\textio.py", line 119, in __init__
validate=validate)
File "C:\Users\simon_6dagkya\Anaconda3\envs\apachebeam\lib\site-packages\apache_beam\io\filebasedsource.py", line 121, in __init__
self._validate()
File "C:\Users\simon_6dagkya\Anaconda3\envs\apachebeam\lib\site-packages\apache_beam\options\value_provider.py", line 133, in _f
return fnc(self, *args, **kwargs)
File "C:\Users\simon_6dagkya\Anaconda3\envs\apachebeam\lib\site-packages\apache_beam\io\filebasedsource.py", line 181, in _validate
'No files found based on the file pattern %s' % pattern)
IOError: No files found based on the file pattern C:\Users\simon_6dagkya\OneDrive\ProgrammingCore\Apache
Any help most appreciated.
IOError: No files found based on the file pattern C:\Users\simon_6dagkya\OneDrive\ProgrammingCore\Apache
Your input string has a space in it. Add quotes.

Librabbitmq 2.0.0 with Python 3 gives TypeError: can't pickle memoryview objects

I am using the latest master branch of the git repo https://github.com/celery/librabbitmq and installing librabbitmq==2.0.0 for Python 3.6 by following the instructions in the readme
Using the development version
You can clone the repository by doing the following:
$ git clone git://github.com/celery/librabbitmq.git
Then install it by doing the following:
$ cd librabbitmq
$ make install # or make develop
This works fine (after installing certain binaries for c compliation in the OS), but when I then make a small a+b add task and call it with add.delay(2,2) it fails with the following error. I looked up and saw that Celery 4 uses json as serializer, so clearly it is not because if pickle serialization
Changing from librabbitmq to pyamqp broker works normally
Same exact situation in both MacOS and Ubuntu 16
[2018-04-30 23:40:02,956: CRITICAL/MainProcess] Unrecoverable error:
SystemError(' returned a result with an error set',) Traceback (most
recent call last): File
"/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/kombu/messaging.py",
line 624, in _receive_callback
return on_m(message) if on_m else self.receive(decoded, message) File
"/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 570, in on_task_received
callbacks, File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/strategy.py",
line 145, in task_message_handler
handle(req) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/worker.py",
line 221, in _process_task_sem
return self._quick_acquire(self._process_task, req) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/kombu/async/semaphore.py",
line 62, in acquire
callback(*partial_args, **partial_kwargs) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/worker.py",
line 226, in _process_task
req.execute_using_pool(self.pool) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/request.py",
line 531, in execute_using_pool
correlation_id=task_id, File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/concurrency/base.py",
line 155, in apply_async
**options) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/billiard/pool.py",
line 1486, in apply_async
self._quick_put((TASK, (result._job, None, func, args, kwds))) File
"/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/concurrency/asynpool.py",
line 813, in send_job
body = dumps(tup, protocol=protocol) TypeError: can't pickle memoryview objects
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File
"/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/worker.py",
line 203, in start
self.blueprint.start(self) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/bootsteps.py",
line 119, in start
step.start(parent) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/bootsteps.py",
line 370, in start
return self.obj.start() File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 320, in start
blueprint.start(self) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/bootsteps.py",
line 119, in start
step.start(parent) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 596, in start
c.loop(*c.loop_args()) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/loops.py",
line 88, in asynloop
next(loop) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/kombu/async/hub.py",
line 354, in create_loop
cb(*cbargs) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/kombu/transport/base.py",
line 236, in on_readable
reader(loop) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/kombu/transport/base.py",
line 218, in _read
drain_events(timeout=0) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/librabbitmq-2.0.0-py3.6-macosx-10.6-intel.egg/librabbitmq/init.py",
line 227, in drain_events
self._basic_recv(timeout) SystemError: returned a result with an error set
This library is not recommended to use as rabbitmq broker with celery. Instead please try py-amqp. this is more maintained and less buggy.

google-cloud-storage library from nosetests using testbed

I have google-cloud-storage pip installed into a lib directory and vendored in. It's running just fine locally during development of my python appengine app. However, when trying to run unit tests via nose and testbed I'm getting "The 'google-cloud-core' distribution was not found and is required by the application". Here is the stack:
Traceback (most recent call last):
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/jason/dev/gain-data/data/storage/__init__.py", line 4, in <module>
from google.cloud.storage import Blob, Client
File "/Users/jason/dev/gain-data/lib/google/cloud/storage/__init__.py", line 42, in <module>
from google.cloud.storage.batch import Batch
File "/Users/jason/dev/gain-data/lib/google/cloud/storage/batch.py", line 30, in <module>
from google.cloud.storage.connection import Connection
File "/Users/jason/dev/gain-data/lib/google/cloud/storage/connection.py", line 17, in <module>
from google.cloud import connection as base_connection
File "/Users/jason/dev/gain-data/lib/google/cloud/connection.py", line 31, in <module>
get_distribution('google-cloud-core').version)
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 557, in get_distribution
dist = get_provider(dist)
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 431, in get_provider
return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 968, in require
needed = self.resolve(parse_requirements(requirements))
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 854, in resolve
raise DistributionNotFound(req, requirers)
DistributionNotFound: The 'google-cloud-core' distribution was not found and is required by the application
Any thoughts?
I had the same issue with google-cloud-translate, I was forced to also install the package "globally", i.e. pip install google-cloud-translate.
After struggling a lot with this same issue I found out that the error was because the vendor pip lib wasn't in the PYTHONPATH before calling the nosetests.
Try adding the vendor lib to the PYTHONPATH and then run the tests.
export PYTHONPATH="$(HOME)/Projects/myproject/pip_lib:$$PYTHONPATH"; \
nosetests .

how to properly resolve gsutil error

I just installed gsutil on OS X, exactly following Google's instruction, and am seeing errors of the following format when running any gsutil command:
Traceback (most recent call last):
File "/Users//gsutil/gsutil", line 22, in <module>
gsutil.RunMain()
File "/Users//gsutil/gsutil.py", line 101, in RunMain
sys.exit(gslib.__main__.main())
File "/Users//gsutil/gslib/__main__.py", line 175, in main
command_runner = CommandRunner()
File "/Users//gsutil/gslib/command_runner.py", line 107, in __init__
self.command_map = self._LoadCommandMap()
File "/Users//gsutil/gslib/command_runner.py", line 113, in _LoadCommandMap
__import__('gslib.commands.%s' % module_name)
File "/Users//gsutil/gslib/commands/disablelogging.py", line 16, in <module>
from gslib.command import COMMAND_NAME
ImportError: cannot import name COMMAND_NAME
This error occurs on several modules in the commands directory. The only thing I could do to get rid of these errors is to remove the following modules from the directory which reference COMMAND_NAME: disablelogging, enablelogging, getacl, getcors, getdefacl, getlogging, setacl, setcors, setdefacl.
Did I do the right thing here? Is this a bug in gsutil?