I am new to working with Chatterbot. I am attempting to run chatterbot on my raspberry pi 4. After installing it, I attempted a basic program from chatterbot's documentation website:
from chatterbot import ChatBot
from chatterbot.trainers import ChatterBotCorpusTrainer
chatbot = ChatBot('Bob')
######### Create a new trainer for the chatbot
trainer = ChatterBotCorpusTrainer(chatbot)
######### Train the chatbot based on the english corpus
trainer.train("chatterbot.corpus.english")
# Get a response to an input statement
responce = chatbot.get_response("Hello, how are you today?")
print(responce)
When I run this program I get this error message:
Traceback (most recent call last):
File "/home/pi/Matthew.codes/JenkinsProject/chatbot_test.py", line 4, in
chatbot = ChatBot('Bob')
File "/home/pi/.local/lib/python3.7/site-packages/chatterbot/chatterbot.py", line 28, in init
self.storage = utils.initialize_class(storage_adapter, **kwargs)
File "/home/pi/.local/lib/python3.7/site-packages/chatterbot/utils.py", line 33, in initialize_class
return Class(*args, **kwargs)
File "/home/pi/.local/lib/python3.7/site-packages/chatterbot/storage/sql_storage.py", line 20, in init
super().init(**kwargs)
File "/home/pi/.local/lib/python3.7/site-packages/chatterbot/storage/storage_adapter.py", line 23, in init
'tagger_language', languages.ENG
File "/home/pi/.local/lib/python3.7/site-packages/chatterbot/tagging.py", line 20, in init
import spacy
File "/home/pi/.local/lib/python3.7/site-packages/spacy/init.py", line 6, in
from .errors import setup_default_warnings
File "/home/pi/.local/lib/python3.7/site-packages/spacy/errors.py", line 2, in
from .compat import Literal
File "/home/pi/.local/lib/python3.7/site-packages/spacy/compat.py", line 38, in
from thinc.api import Optimizer # noqa: F401
File "/home/pi/.local/lib/python3.7/site-packages/thinc/api.py", line 2, in
from .initializers import normal_init, uniform_init, glorot_uniform_init, zero_init
File "/home/pi/.local/lib/python3.7/site-packages/thinc/initializers.py", line 4, in
from .backends import Ops
File "/home/pi/.local/lib/python3.7/site-packages/thinc/backends/init.py", line 7, in
from .ops import Ops
File "/home/pi/.local/lib/python3.7/site-packages/thinc/backends/ops.py", line 15, in
from .cblas import CBlas
File "thinc/backends/cblas.pyx", line 1, in init thinc.backends.cblas
File "/home/pi/.local/lib/python3.7/site-packages/blis/init.py", line 3, in
from .cy import init
ImportError: /home/pi/.local/lib/python3.7/site-packages/blis/cy.cpython-37m-arm-linux-gnueabihf.so: undefined symbol: __atomic_load_8
I have tried adding storage adapters to the name class, but that did not work. As far as I can tell, I have installed all the dependencies.
Linux Mint 20.3
I try to run pgadmin4 like this:
./pgadmin4env/bin/pgadmin4&
But get error:
[1] 3994127
(pgadmin4env) alexeij#workstation:~/pgadmin4/pgadmin4_dir$ Traceback (most recent call last):
File "./pgadmin4env/bin/pgadmin4", line 5, in <module>
from pgadmin4.pgAdmin4 import main
File "/home/alexeij/pgadmin4/pgadmin4_dir/pgadmin4env/lib/python3.8/site-packages/pgadmin4/pgAdmin4.py", line 35, in <module>
import config
File "/home/alexeij/pgadmin4/pgadmin4_dir/pgadmin4env/lib/python3.8/site-packages/pgadmin4/config.py", line 25, in <module>
from pgadmin.utils import env, IS_WIN, fs_short_path
File "/home/alexeij/pgadmin4/pgadmin4_dir/pgadmin4env/lib/python3.8/site-packages/pgadmin4/pgadmin/__init__.py", line 21, in <module>
from flask import Flask, abort, request, current_app, session, url_for
File "/home/alexeij/pgadmin4/pgadmin4_dir/pgadmin4env/lib/python3.8/site-packages/flask/__init__.py", line 19, in <module>
from jinja2 import Markup, escape
ImportError: cannot import name 'Markup' from 'jinja2' (/home/alexeij/pgadmin4/pgadmin4_dir/pgadmin4env/lib/python3.8/site-packages/jinja2/__init__.py)
import io
import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="ip-boundary-996e9662df4c.json"
from google.cloud import speech
I am trying to load the library required to use Google STT, but an error occurs. Here's what the error says:
Traceback (most recent call last):
File "/home/yugwon/IP_boundary/server/stt.py", line 5, in <module>
from google.cloud import speech
File "/home/yugwon/IP_boundary/venv/lib/python3.7/site-packages/google/cloud/speech/__init__.py", line 17, in <module>
from google.cloud.speech_v1 import SpeechClient
File "/home/yugwon/IP_boundary/venv/lib/python3.7/site-packages/google/cloud/speech_v1/__init__.py", line 17, in <module>
from .services.speech import SpeechClient
File "/home/yugwon/IP_boundary/venv/lib/python3.7/site-packages/google/cloud/speech_v1/services/speech/__init__.py", line 16, in <module>
from .client import SpeechClient
File "/home/yugwon/IP_boundary/venv/lib/python3.7/site-packages/google/cloud/speech_v1/services/speech/client.py", line 43, in <module>
from google.api_core import operation # type: ignore
File "/home/yugwon/IP_boundary/venv/lib/python3.7/site-packages/google/api_core/operation.py", line 45, in <module>
from google.longrunning import operations_pb2
File "/home/yugwon/IP_boundary/venv/lib/python3.7/site-packages/google/longrunning/operations_pb2.py", line 26, in <module>
from google.longrunning.operations_proto_pb2 import *
File "/home/yugwon/IP_boundary/venv/lib/python3.7/site-packages/google/longrunning/operations_proto_pb2.py", line 31, in <module>
from google.api import client_pb2 as google_dot_api_dot_client__pb2
ImportError: cannot import name 'client_pb2' from 'google.api' (/home/yugwon/IP_boundary/python-generated/google/api/__init__.py)
As a workaround for this, I tried the following, but still the same problem appeared and failed.
pip install --upgrade googleapis-common-protos
Below is my venv pip list.
Do you have any other solution?
I'm trying to use Hyperopt parallel search with MongoDB, and encountered some issues with Mongotrials, which have been discussed here. I've tried all their methods, and I am still unable to find solutions to my specific problem. The specific model I'm trying to minimize is RadomForestRegressor from sklearn.
I've followed this tutorial. And I'm able to print out the calculated "fmin" with no issue.
Here are my steps so far:
1) Activate a virtual environment called "tensorflow" (I've installed all my libraries there)
2) Start MongoDB:
(tensorflow) bash-3.2$ mongod --dbpath . --port 1234 --directoryperdb --journal --nohttpinterface
3) Initiate workers:
(tensorflow) bash-3.2$ hyperopt-mongo-worker --mongo=localhost:1234/foo_db --poll-interval=0.1
4) Run my python code, and my python code is as follows:
import numpy as np
import pandas as pd
from sklearn.metrics import mean_absolute_error
from hyperopt import hp, fmin, tpe, STATUS_OK, Trials
from hyperopt.mongoexp import MongoTrials
# Preprocessing data
train_xg = pd.read_csv('train.csv')
n_train = len(train_xg)
print "Whole data set size: ", n_train
# Creating columns for features, and categorical features
features_col = [x for x in train_xg.columns if x not in ['id', 'loss', 'log_loss']]
cat_features_col = [x for x in train_xg.select_dtypes(include=['object']).columns if x not in ['id', 'loss', 'log_loss']]
for c in range(len(cat_features_col)):
train_xg[cat_features_col[c]] = train_xg[cat_features_col[c]].astype('category').cat.codes
# Use this to train random forest regressor
train_xg_x = np.array(train_xg[features_col])
train_xg_y = np.array(train_xg['loss'])
space_rf = { 'min_samples_leaf': hp.choice('min_samples_leaf', range(1,100)) }
trials = MongoTrials('mongo://localhost:1234/foo_db/jobs', exp_key='exp1')
def minMe(params):
# Hyperopt tuning for hyperparameters
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestRegressor
from hyperopt import STATUS_OK
try:
import dill as pickle
print('Went with dill')
except ImportError:
import pickle
def hyperopt_rf(params):
rf = RandomForestRegressor(**params)
return cross_val_score(rf, train_xg_x, train_xg_y).mean()
acc = hyperopt_rf(params)
print 'new acc:', acc, 'params: ', params
return {'loss': -acc, 'status': STATUS_OK}
best = fmin(fn=minMe, space=space_rf, trials=trials, algo=tpe.suggest, max_evals=100)
print "Best: ", best
5) After I run the above Python code, I get the following errors:
INFO:hyperopt.mongoexp:Error while unpickling. Try installing dill via "pip install dill" for enhanced pickling support.
INFO:hyperopt.mongoexp:job exception: 'module' object has no attribute 'minMe'
Traceback (most recent call last):
File "/Users/WernerChao/tensorflow/bin/hyperopt-mongo-worker", line 6, in <module>
sys.exit(hyperopt.mongoexp.main_worker())
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1302, in main_worker
return main_worker_helper(options, args)
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1249, in main_worker_helper
mworker.run_one(reserve_timeout=float(options.reserve_timeout))
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1064, in run_one
domain = pickle.loads(blob)
AttributeError: 'module' object has no attribute 'minMe'
INFO:hyperopt.mongoexp:PROTOCOL mongo
INFO:hyperopt.mongoexp:USERNAME None
INFO:hyperopt.mongoexp:HOSTNAME localhost
INFO:hyperopt.mongoexp:PORT 1234
INFO:hyperopt.mongoexp:PATH /foo_db/jobs
INFO:hyperopt.mongoexp:DB foo_db
INFO:hyperopt.mongoexp:COLLECTION jobs
INFO:hyperopt.mongoexp:PASS None
INFO:hyperopt.mongoexp:Error while unpickling. Try installing dill via "pip install dill" for enhanced pickling support.
INFO:hyperopt.mongoexp:job exception: 'module' object has no attribute 'minMe'
Traceback (most recent call last):
File "/Users/WernerChao/tensorflow/bin/hyperopt-mongo-worker", line 6, in <module>
sys.exit(hyperopt.mongoexp.main_worker())
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1302, in main_worker
return main_worker_helper(options, args)
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1249, in main_worker_helper
mworker.run_one(reserve_timeout=float(options.reserve_timeout))
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1064, in run_one
domain = pickle.loads(blob)
AttributeError: 'module' object has no attribute 'minMe'
INFO:hyperopt.mongoexp:PROTOCOL mongo
INFO:hyperopt.mongoexp:USERNAME None
INFO:hyperopt.mongoexp:HOSTNAME localhost
INFO:hyperopt.mongoexp:PORT 1234
INFO:hyperopt.mongoexp:PATH /foo_db/jobs
INFO:hyperopt.mongoexp:DB foo_db
INFO:hyperopt.mongoexp:COLLECTION jobs
INFO:hyperopt.mongoexp:PASS None
INFO:hyperopt.mongoexp:Error while unpickling. Try installing dill via "pip install dill" for enhanced pickling support.
INFO:hyperopt.mongoexp:job exception: 'module' object has no attribute 'minMe'
Traceback (most recent call last):
File "/Users/WernerChao/tensorflow/bin/hyperopt-mongo-worker", line 6, in <module>
sys.exit(hyperopt.mongoexp.main_worker())
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1302, in main_worker
return main_worker_helper(options, args)
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1249, in main_worker_helper
mworker.run_one(reserve_timeout=float(options.reserve_timeout))
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1064, in run_one
domain = pickle.loads(blob)
AttributeError: 'module' object has no attribute 'minMe'
INFO:hyperopt.mongoexp:PROTOCOL mongo
INFO:hyperopt.mongoexp:USERNAME None
INFO:hyperopt.mongoexp:HOSTNAME localhost
INFO:hyperopt.mongoexp:PORT 1234
INFO:hyperopt.mongoexp:PATH /foo_db/jobs
INFO:hyperopt.mongoexp:DB foo_db
INFO:hyperopt.mongoexp:COLLECTION jobs
INFO:hyperopt.mongoexp:PASS None
INFO:hyperopt.mongoexp:no job found, sleeping for 0.7s
INFO:hyperopt.mongoexp:Error while unpickling. Try installing dill via "pip install dill" for enhanced pickling support.
INFO:hyperopt.mongoexp:job exception: 'module' object has no attribute 'minMe'
Traceback (most recent call last):
File "/Users/WernerChao/tensorflow/bin/hyperopt-mongo-worker", line 6, in <module>
sys.exit(hyperopt.mongoexp.main_worker())
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1302, in main_worker
return main_worker_helper(options, args)
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1249, in main_worker_helper
mworker.run_one(reserve_timeout=float(options.reserve_timeout))
File "/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1064, in run_one
domain = pickle.loads(blob)
AttributeError: 'module' object has no attribute 'minMe'
INFO:hyperopt.mongoexp:exiting with N=9223372036854775803 after 4 consecutive exceptions
6) Then Mongo workers would shut off.
Things I've tried:
install "dill" as the error suggested -> didn't work
Put global imports into the objective function so it can pickle -> didn't work
Put try except with "dill" or "pickle" as import -> didn't work
Does anyone have similar issues? I'm running out of ideas to try, and have been working on this for 2 days in vain. I think I am missing something really simple here, just can't seem to find it.
What am I missing?
Any suggestion is welcomed please!
Had the same problem in python 3.5. Installing Dill didn't help, nor dir setting workdir in MongoTrials or hyperopt-mongo-worker cli. hyperopt-mongo-worker doesn't seem to have access to __main__ where the function was defined:
AttributeError: Can't get attribute 'minMe' on <module '__main__' from ...hyperopt-mongo-worker
As #jaikumarm suggested, I circumvented the problem by writing a module file with all the required functions. However, instead of soft-linking it into the bin directory, I extended the PYTHONPATH before running hyperopt-mongo-worker:
export PYTHONPATH="${PYTHONPATH}:<dir_with_the_module.py>"
hyperopt-mongo-worker ...
That way, the hyperopt-monogo-worker is able to import the module containing minMe.
I fought with this for several days before coming up with a workable solution. there are two problems:
1. the mongo worker spawns off a separate process to run the optimizer so any context from your original python file is lost and unavailable for this new process.
2. the imports on this new process happen in the context of the hyperopt-mongo-worker scipy, which is in your case will be /Users/WernerChao/tensorflow/bin/.
So my solution is to make this new optimizer function completely self sufficient
optimizer.py
import numpy as np
import pandas as pd
from sklearn.metrics import mean_absolute_error
# Preprocessing data
train_xg = pd.read_csv('train.csv')
n_train = len(train_xg)
print "Whole data set size: ", n_train
# Creating columns for features, and categorical features
features_col = [x for x in train_xg.columns if x not in ['id', 'loss', 'log_loss']]
cat_features_col = [x for x in train_xg.select_dtypes(include=['object']).columns if x not in ['id', 'loss', 'log_loss']]
for c in range(len(cat_features_col)):
train_xg[cat_features_col[c]] = train_xg[cat_features_col[c]].astype('category').cat.codes
# Use this to train random forest regressor
train_xg_x = np.array(train_xg[features_col])
train_xg_y = np.array(train_xg['loss'])
def minMe(params):
# Hyperopt tuning for hyperparameters
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestRegressor
from hyperopt import STATUS_OK
try:
import dill as pickle
print('Went with dill')
except ImportError:
import pickle
def hyperopt_rf(params):
rf = RandomForestRegressor(**params)
return cross_val_score(rf, train_xg_x, train_xg_y).mean()
acc = hyperopt_rf(params)
print 'new acc:', acc, 'params: ', params
return {'loss': -acc, 'status': STATUS_OK}
wrapper.py
from hyperopt import hp, fmin, tpe, STATUS_OK, Trials
from hyperopt.mongoexp import MongoTrials
import optimizer
space_rf = { 'min_samples_leaf': hp.choice('min_samples_leaf', range(1,100)) }
best = fmin(fn=optimizer.minMe, space=space_rf, trials=trials, algo=tpe.suggest, max_evals=100)
print "Best: ", best
trials = MongoTrials('mongo://localhost:1234/foo_db/jobs', exp_key='exp1')
Once you have this code link the optimizer.py to the bin folder
ln -s /Users/WernerChao/Git/test/optimizer.py /Users/WernerChao/tensorflow/bin/
now run the wrapper.py and then the mongo worker it should be able to import the optimizer from its local context and run the minMe function.
Try to install Dill in the Python environment of your tensorflow (or possibly the worker):
/Users/WernerChao/tensorflow/lib/python2.7/site-packages/hyperopt
Your aim is to get rid of the hyperopt error message:
hyperopt.mongoexp:Error while unpickling. Try installing dill via "pip install dill" for enhanced pickling support.
This is because the Python by default cannot marshal a function. It requires dill library to extend Python's pickling module for serialising/de-serialising Python objects. In your case, it failed to serialise your function minMe().
I made a separate file which calculates the loss and copied it to /anaconda2/bin/
and
/anaconda2/lib/python2.7/site-packages/hyperopt
it is working fine.
This was my Traceback
Traceback (most recent call last):
File "/home/greatskull/anaconda2/bin/hyperopt-mongo-worker", line 6, in <module>
sys.exit(hyperopt.mongoexp.main_worker())
File "/home/greatskull/anaconda2/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1302, in main_worker
return main_worker_helper(options, args)
File "/home/greatskull/anaconda2/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1249, in main_worker_helper
mworker.run_one(reserve_timeout=float(options.reserve_timeout))
File "/home/greatskull/anaconda2/lib/python2.7/site-packages/hyperopt/mongoexp.py", line 1073, in run_one
with temp_dir(workdir, erase_created_workdir), working_dir(workdir):
File "/home/greatskull/anaconda2/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/home/greatskull/anaconda2/lib/python2.7/site-packages/hyperopt/utils.py", line 229, in temp_dir
os.makedirs(dir)
File "/home/greatskull/anaconda2/lib/python2.7/os.py", line 150, in makedirs
makedirs(head, mode)
File "/home/greatskull/anaconda2/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
Today I have tried use docker-compose and got next error:
>>docker-compose
Traceback (most recent call last):
File "/usr/bin/docker-compose", line 9, in <module>
load_entry_point('docker-compose==1.7.1', 'console_scripts', 'docker-compose')()
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 558, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2682, in load_entry_point
return ep.load()
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2355, in load
return self.resolve()
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2361, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 14, in <module>
from . import errors
File "/usr/lib/python2.7/site-packages/compose/cli/errors.py", line 9, in <module>
from docker.errors import APIError
File "/usr/lib/python2.7/site-packages/docker/__init__.py", line 20, in <module>
from .client import Client, AutoVersionClient, from_env # flake8: noqa
File "/usr/lib/python2.7/site-packages/docker/client.py", line 22, in <module>
import websocket
File "/usr/lib/python2.7/site-packages/websocket/__init__.py", line 22, in <module>
from ._core import *
File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 39, in <module>
from ._socket import *
File "/usr/lib/python2.7/site-packages/websocket/_socket.py", line 28, in <module>
from ._ssl_compat import *
AttributeError: 'module' object has no attribute 'ssl'
OS: OpenSuse Leap 42.1
Docker-compose: 1.7.1
Docker: 1.11.2
libopenssl-devel installed
Downgrade OpennSSL to version 1.0.1i-15.1 from 1.0.1i-2.36.1 has fixed my problem.