Fetching data from mongo-tracker in ' Rasa' and storing it in mongoDB at runtime like sender_id,timestamp - mongodb

Could you please tell me how are you fetching specific details from the tracker store. Elaborating my doubt below:
in my run_app.py (socketIO class) i have used mongotracker like this-
db = MongoTrackerStore(domain=“d.yml”,host=‘host ip’, db=‘xyz’, username=“x”,password=“x”,collection=“x”,event_broker=None)
agent = Agent.load(‘models/dialogue’, interpreter=‘models/current/nlu’,action_endpoint = action_endpoint,tracker_store=db)
now i want to fetch some data like db.sender_id or db.event. the reason of doing it is to store it column wise on my mongodb.Please help me solving this problem.

This information should already be stored in your mongodb, so there should be no extra need for you to store it.
Maybe see the documentation for this https://rasa.com/docs/core/tracker_stores/ and make sure your endpoints.yml file includes the correct information:
tracker_store:
store_type: mongod
url: <url to your mongo instance, e.g. mongodb://localhost:27017>
db: <name of the db within your mongo instance, e.g. rasa>
username: <username used for authentication>
password: <password used for authentication>
auth_source: <database name associated with the user’s credentials>
For information on how to fetch specific details from your mongodb maybe have a look at the mongodb docs https://docs.mongodb.com/manual/reference/method/db.collection.find/.

look at this Example
I am using Pymongo to connect monogoDB. try to understand my code
from typing import Any, Text, Dict, List
from pymongo.database import Database
from pymongo import MongoClient
from rasa_sdk import Action, Tracker
from rasa_sdk.executor import CollectingDispatcher
import pymongo
# url="http://localhost:3000/api"
client = pymongo.MongoClient("localhost", 27017)
db=client.sample
class mercdesCarAction(Action):
def name(self):
return "mercdesCarAction"
def run(self,dispatcher,tracker,domain):
res = db.datas.find({'action':'mercdesCarAction'})
for i in res:
dispatcher.utter_button_message(i['text'],i['buttons'])
return []

Related

How can I use FastAPI Routers with FastAPI-Users and MongoDB?

I can use MongoDB with FastAPI either
with a global client: motor.motor_asyncio.AsyncIOMotorClient object, or else
by creating one during the startup event per this SO answer which refers to this "Real World Example".
However, I also want to use fastapi-users since it works nicely with MongoDB out of the box. The downside is it seems to only work with the first method of handling my DB client connection (ie global). The reason is that in order to configure fastapi-users, I have to have an active MongoDB client connection just so I can make the db object as shown below, and I need that db to then make the MongoDBUserDatabase object required by fastapi-users:
# main.py
app = FastAPI()
# Create global MongoDB connection
DATABASE_URL = "mongodb://user:paspsword#localhost/auth_db"
client = motor.motor_asyncio.AsyncIOMotorClient(DATABASE_URL, uuidRepresentation="standard")
db = client["my_db"]
# Set up fastapi_users
user_db = MongoDBUserDatabase(UserDB, db["users"])
cookie_authentication = CookieAuthentication(secret='lame secret' , lifetime_seconds=3600, name='cookiemonster')
fastapi_users = FastAPIUsers(
user_db,
[cookie_authentication],
User,
UserCreate,
UserUpdate,
UserDB,
)
After that point in the code, I can import the fastapi_users Routers. However, if I want to break up my project into FastAPI Routers of my own, I'm hosed because:
If I move the client creation to another module to be imported into both my app and my routers, then I have different clients in different event loops and get errors like RuntimeError: Task <Task pending name='Task-4' coro=<RequestResponseCycle.run_asgi() running at /usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py:389> cb=[set.discard()]> got Future <Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.8/asyncio/futures.py:360]> attached to a different loop (touched on in this SO question)
If I user the solutions of the "Real World Example", then I get stuck on where to build my fastapi_users object in my code example: I can't do it in main.py because there's no db object yet.
I considered making the MongoDBUserDatabase object as part of the startup event code (ie within async def connect_to_mongo() from the Real World Example), but I'm not able to get that to work either since I can't see how to make it work.
How can I either
make a global MongoDB client and FastAPI-User object in a way that can be shared among my main app and several routers without "attached to a different loop" errors, or
create fancy wrapper classes and functions to set up FastAPI users with the startup trigger?
I don't think my solution is complete or correct, but I figured I'd post it in case it inspires any ideas, I'm stumped. I have run into the exact dilemma, almost seems like a design flaw..
I followed this MongoDB full example and named it main.py
At this point my app does not work. The server starts up but result results in the aforementioned "attached to a different loop" whenever trying to query the DB.
Looking for guidance, I stumbled upon the same "real world" example
In main.py added the startup and shudown event handlers
# Event handlers
app.add_event_handler("startup", create_start_app_handler(app=app))
app.add_event_handler("shutdown", create_stop_app_handler(app=app))
In dlw_api.db.events.py this:
import logging
from dlw_api.user import UserDB
from fastapi import FastAPI
from fastapi_users.db.mongodb import MongoDBUserDatabase
from motor.motor_asyncio import AsyncIOMotorClient
LOG = logging.getLogger(__name__)
DB_NAME = "dlwLocal"
USERS_COLLECTION = "users"
DATABASE_URI = "mongodb://dlw-mongodb:27017" # protocol://container_name:port
_client: AsyncIOMotorClient = None
_users_db: MongoDBUserDatabase = None
def get_users_db() -> MongoDBUserDatabase:
return _users_db
async def connect_to_db() -> None:
global _users_db
# logger.info("Connecting to {0}", repr(DATABASE_URL))
client = AsyncIOMotorClient(DATABASE_URI)
db = client[DB_NAME]
collection = db[USERS_COLLECTION]
_users_db = MongoDBUserDatabase(UserDB, collection)
LOG.info(f"Connected to {DATABASE_URI}")
async def close_db_connection(app: FastAPI) -> None:
_client.close()
LOG.info("Connection closed")
And dlw_api.events.py:
from typing import Callable
from fastapi import FastAPI
from dlw_api.db.events import close_db_connection, connect_to_db
from dlw_api.user import configure_user_auth_routes
from fastapi_users.authentication import CookieAuthentication
from dlw_api.db.events import get_users_db
COOKIE_SECRET = "THIS_NEEDS_TO_BE_SET_CORRECTLY" # TODO: <--|
COOKIE_LIFETIME_SECONDS: int = 3_600
COOKIE_NAME = "c-is-for-cookie"
# Auth stuff:
_cookie_authentication = CookieAuthentication(
secret=COOKIE_SECRET,
lifetime_seconds=COOKIE_LIFETIME_SECONDS,
name=COOKIE_NAME,
)
auth_backends = [
_cookie_authentication,
]
def create_start_app_handler(app: FastAPI) -> Callable:
async def start_app() -> None:
await connect_to_db(app)
configure_user_auth_routes(
app=app,
auth_backends=auth_backends,
user_db=get_users_db(),
secret=COOKIE_SECRET,
)
return start_app
def create_stop_app_handler(app: FastAPI) -> Callable:
async def stop_app() -> None:
await close_db_connection(app)
return stop_app
This doesn't feel correct to me, does this mean all routes that use Depends for user-auth have to be included on the server startup event handler??
The author (frankie567) of fastapi-users created a repl.it showing a solution of sorts. My discussion about this solution may provide more context but the key parts of the solution are:
Don't bother using FastAPI startup trigger along with Depends for your MongDB connectivity management. Instead, create a separate file (ie db.py) to create your DB connection and client object. Import this db object whenever needed, like your Routers, and then use it as a global.
Also create a separate users.py to do 2 things:
Create globally used fastapi_users = FastAPIUsers(...) object for use with other Routers to handle authorization.
Create a FastAPI.APIRouter() object and attach all the fastapi-user routers to it (router.include_router(...))
In all your other Routers, import both db and fastapi_users from the above as needed
Key: split your main code up into
a main.py which only import uvicorn and serves app:app.
an app.py which has your main FastAPI object (ie app) and which then attaches all our Routers, including the one from users.py with all the fastapi-users routers attached to it.
By splitting up code per 4 above, you avoid the "attached to different loop" error.
I faced similar issue, and all I have to do to get motor and fastapi run in the same loop is this:
client = AsyncIOMotorClient()
client.get_io_loop = asyncio.get_event_loop
I did not set on_startup or whatsoever.

Getting javax.script.ScriptException: com.mongodb.MongoException: not authorized for query on default.paymentHeader

I am using Groovy to query on MongoDB and I am getting not authorized in querying in MongoDB. Can you please check if there is a problem on my script?
DBName default
Collection paymentHeader
import com.mongodb.*;
import com.mongodb.DB;
import com.mongodb.MongoCredential;
import org.apache.jmeter.protocol.mongodb.config.MongoDBHolder;
import com.mongodb.BasicDBObject;
import com.mongodb.DBCollection;
//Get DB
DB db = MongoDBHolder.getDBFromSource("SITDB","${DBName}","${username}","${password}");
boolean auth = db.authenticate("${username}","${password}".toCharArray());
//Get Collection
DBCollection collection = db.getCollection("${Collection}");
//Find ApplicationNum
//collection.find({applicationNum: "${applicationId}"});
BasicDBObject query = new BasicDBObject("applicationNum", "${applicationId}");
DBObject result = collection.findOne(query);
SampleResult.setResponseData(result.toString().getBytes());
Response code: 500
Response message: javax.script.ScriptException: com.mongodb.MongoException: not authorized for query on default.paymentHeader
This doesn't have anything with JMeter, you need to provide read permissions for the ${username} onto default.paymentHeader collection.
Also be aware that you should not reference JMeter Functions or Variables as ${username}, you should use vars shorthand for JMeterVariables class instead like vars.get('username')
More information: JSR223 Sampler Documentation, in particular:
JMeter processes function and variable references before passing the script field to the interpreter, so the references will only be resolved once. Variable and function references in script files will be passed verbatim to the interpreter, which is likely to cause a syntax error. In order to use runtime variables, please use the appropriate props methods, e.g.
props.get("START.HMS");
props.put("PROP1","1234");

Connection to MongoDB from MATLAB

I want to create a connection to my database in MongoDB from Matlab R2015a. I have tried with both the drivers for C# and Java but none of them seem to work and I don't know what the problem is.
For Java:
Code:
javaaddpath('/%path%/mongodb-driver-3.0.0.jar')
import com.mongodb.*;
mongoClient = MongoClient();
db = mongoClient.getDB('myDB');
colls = db.getCollectionNames();
coll = db.getCollection('myCollection');
Error:
No appropriate method, property, or field 'getDB' for class 'MongoDB.Driver.MongoClient'.
For C#:
Code:
NET.addAssembly('%path%\CSharpDriver-2.0.0\MongoDB.Driver.dll');
import MongoDB.Driver.*;
mongoClient = MongoDB.Driver.MongoClient();
mongoServer = mongoClient.GetServer();
db = mongoClient.GetDatabase('myDB');
collection = db.GetCollection('myCollection');
Errors:
1. No appropriate method, property, or field 'GetServer' for class 'MongoDB.Driver.MongoClient'.
2. If I comment the GetServer line I get: No appropriate method, property, or field 'GetCollection' for class 'MongoDB.Driver.MongoDatabaseImpl'.
I don't know if I am missing something and it would be really helpful if I could make it work.
I have also tried with the driver for Matlab but I couldn't make it create the .dll.
Thanks.
You have to open the client with:
import com.mongodb.*;
mongoClient = MongoClient('myIP', 'myPort');
I'm using the java version with Matlab 2015b. I assume you have done the import correct. Otherwise, the MongoClient class would not be found.

How to declare a Mongo Binary Object in Flask

I'm building a flask application where I will be serving small images. These images are stored in MongoDB as BinaryData. In a helper function, I can store the data with these lines of python:
a = {"file_name": f, "payload": Binary(article.read())}
ARTICLES.insert(a)
I'm trying to build a class that contains the image. However, I cannot find the correct field declaration
class BinaryFile(mongo.Document):
created_at = mongo.DateTimeField(default=datetime.datetime.now, required=True)
file_name = mongo.StringField(max_length=255, required=True)
payload = mongo.Binary()
producing this error:
AttributeError: 'MongoEngine' object has no attribute 'Binary'
Can anyone suggest the correct way to declare this value or am I completely off base? This page does not provide a way to declare a field as Binary: http://api.mongodb.org/python/current/api/bson/index.html
Thanks!
Gabe helped get me on the right path.
First, I had to decide whether to use standard Binary format or move to GridFS, I chose to stick with regular Binary.
What I didn't understand was that the DateTimeField and StringField were being provided by MongoEngine. Gabe's comment got me going that way and I found the MongoEngine fields docs: http://docs.mongoengine.org/apireference.html#fields
I called that and got the error here: mongoengine.fields.ImproperlyConfigured: PIL library was not found which was fixed by doing
pip install Pillow
So now I have
import datetime
from app import mongo
from flask import url_for
from bson.binary import Binary
class BinaryFile(mongo.Document):
created_at = mongo.DateTimeField(default=datetime.datetime.now, required=True)
file_name = mongo.StringField(max_length=255, required=True)
payload = mongo.ImageField(required=False)
and I'm on to the next error! See you soon!

Pylons with Mongodb

According to the pymongo documentation,
PyMongo is thread-safe and even provides built-in connection pooling for threaded applications.
I normally initiate my mongodb connection like this:
import pymongo
db = pymongo.Connection()['mydb']
and then I can just use it like db.users.find({'name':..})...
Does this mean that I can actually place that two lines in the lib/apps_global.py like:
class Globals(object):
def __init__(self, config):
self.cache = CacheManager(**parse_cache_config_options(config))
import pymongo
self.db_conn = pymongo.connection()
self.db = self.db_conn['simplesite']
and then in my base controller:
class BaseController(WSGIController):
def __call__(self, environ, start_response):
"""Invoke the Controller"""
# WSGIController.__call__ dispatches to the Controller method
# the request is routed to. This routing information is
# available in environ['pylons.routes_dict']
ret = WSGIController.__call__(self, environ, start_response)
# Don't forget to release the thread for mongodb
app_globals.db_conn.end_request()
return ret
And start calling app_global's db variable throughout my controllers?
I hope it is really that easy.
Ben Bangert, an author of Pylons, has written his blog engine with mongodb.
You can browse it's source code online.