In Meteor Mongo, how to specify the readPref to primary|secondary in Meteor Mongo Query.
I hope the following provides a better understanding of the relationship between Meteor and Mongo.
Meteor collections for more comfort
Meteor provides you with the full mongo functionality. However for comfort it provides a wrapped API of a mongo collection that integrates best with the Meteor environment. So if you import Mongo via
import { Mongo } from 'meteor/mongo'
you primarily import the wrapped mongo collection where operations are executed in a Meteor fiber. The cursor that is returned by queries of these wrapped collections are also not the "natural" cursors but also wrapped cursors to be Meteor optimized.
If you try to access a native feature on these instances that is not implemented you will receive an error. In your case:
import { Meteor } from 'meteor/meteor';
import { Random } from 'meteor/random';
const ExampleCollection = new Mongo.Collection('examples')
Meteor.startup(() => {
// code to run on server at startup
ExampleCollection.insert({ value: Random.id() })
const docsCursor = ExampleCollection.find();
docsCursor.readPref('primary')
});
Leads to
TypeError: docsCursor.readPref is not a function
Accessing the node mongo driver collections
The good news is, you can access the layer underneath via Collection.rawCollection() where you have full access to the node Mongo driver. This is because under the hood Meteor's Mongo.Collection and it's Cursor are making use of this native driver in the end.
Now you will find two other issues:
readPref is named in a node-mongo cursor cursor.setReadPreference (3.1 API).
Cursor.fetch does not exist but is named cursor.toArray which (as many native operations do) returns a Promise
So to finally answer your question
you can do the following:
import { Meteor } from 'meteor/meteor';
import { Random } from 'meteor/random';
const ExampleCollection = new Mongo.Collection('examples')
Meteor.startup(() => {
// code to run on server at startup
ExampleCollection.insert({ value: Random.id() })
const docsCursor = ExampleCollection.rawCollection().find();
docsCursor.setReadPreference('primary')
docsCursor.toArray().then((docs) => {
console.log(docs)
}).catch((err)=> console.error(err))
});
Summary
By using collection.rawCollection() you an have access to the full spectrum of the node mongo driver API
You are on your own to integrate the operations, cursors and results (Promises) into your environment. Good helpers are Meteor.bindEnvironment and Meteor.wrapAsync
Beware of API changes of the node-mongo driver. On the one hand the mongo version that is supported by the driver, on the other hand the driver version that is supported by Meteor.
Note that it is easier to "mess up" things with the native API but it also gives you a lot of new options. Use with care.
Related
I am playing around with FastAPI a bit and wanted to connect it to a MongoDB database. I however am confused which ODM to choose between motor which is async and mongoengine. Also, in the NoSQL example here they have created a new bucket and also the called the code to connect to db every time it is used. However, both motor and mongoengine seem to prefer a global connection. So what would be a good way to connect to mongodb?
I believe you already got your answers in the issue forums of the Fastapi project on Github: Issue 452 (closed). But I'll recap the solutions here for future reference:
In short, you can use either motor or mongoengine, Fastapi supports both and you can reuse a global client object that's started and ended with your app process.
Some context details to (hopefully) clarify these technologies and their relationships:
The official MongoDB driver for Python is pymongo. Under the hoods, both MongoEngine and Motor use Pymongo. Pymongo implements a direct client for MongoDB (daemons) and offers a Python API to make requests.
If you wanted to, you could use pymongo with Fastapi directly. (On th SQL side of things, this would be equivalent to using psycopg2 in Flask directly without going through something like SQLAlchemy.)
MongoEngine is an ODM (Object-Document Mapper). It offers a Python object-oriented API that you can use in your application to work more comfortably and when it comes to the actual DB requests, MongoEngine will use pymongo.
Motor is a wrapper for pymongo that makes it non-blocking (allowing async/await). It uses an event-loop, either through Tornado or through asyncio. If you are using Fastapi with uvicorn, uvicorn will implement async functionality with uvloop. In short, using Motor with FastAPI, async should "just work". Unfortunately, Motor does not implement an ODM. In this sense it is more similar to pymongo.
Fastapi handles the requests from clients (using Starlette), but it will let you implement your own connection to MongoDB. So you are not restricted to any particular choice, but you are mostly on your own (a la Flask).
You can use the startup/shutdown hooks of your FastAPI app to start/stop your Motor/MongoEngine client. You don't need to worry about your client object not persisting due to multiprocess issues, because Fastapi is single-threaded.
#app.on_event("startup")
async def create_db_client():
# start client here and reuse in future requests
#app.on_event("shutdown")
async def shutdown_db_client():
# stop your client here
An example implementation of motor with Fastapi can be found here.
I recently created an Async Mongo ODM well suited for FastAPI: ODMantic.
app = FastAPI()
engine = AIOEngine()
class Tree(Model):
"""This model can be used either as a Pydantic model or
saved to the database"""
name: str
average_size: float
discovery_year: int
#app.get("/trees/", response_model=List[Tree])
async def get_trees():
trees = await engine.find(Tree)
return trees
#app.put("/trees/", response_model=Tree)
async def create_tree(tree: Tree):
await engine.save(tree)
return tree
You can have a look to the FastAPI example for a more detailed example.
I disagree with accepted answer saying fastapi is single threaded.
As far as I can tell fastapi is only single threaded if you handle the requestions using an async function. So if you dont use async under the hood starlette should use mutliple threads from a threadpool to handle multiple requests.
For more information on threading model and async in fastapi/starlette this is a good intro https://fastapi.tiangolo.com/async/
https://stackoverflow.com/questions/70446584/how-does-fastapi-uvicorn-parallelize-requests#:~:text=For%20endpoints%20defined%20with%20def%20%28not%20async%20def%29%2C,can%20be%20controlled.%20This%20question%20is%20addressed%20here.
I've created a JSON API with Express.js, Mongoose and MongoDB. Currently, there's no way for the clients of the API to check if the data in a collection has changed - they would need to download the whole collection periodically.
How could I allow the clients of the API to check for changes to a collections (inserts, updates, deletions) without downloading the collection itself?
Is there a way of getting the version number of the collection, the last change timestamp or a hash of the collection with Mongoose? What is the best practice solution to this problem?
In current and earlier versions of MongoDB, you have to do it on the application side, maybe using polling.
MongoDB 3.6 has a brand new feature called change stream that allows you to listen changes happening on your collections in real time.
The sample code to listen selected changes happening on your collection is below:
var MongoClient = require('mongodb').MongoClient
, assert = require('assert');
MongoClient.connect("mongodb://172.16.0.110:27017/myproject?readConcern=majority").then(function(client){
var db = client.db('myproject')
var changeStreams = db.collection('documents').watch()
changeStreams.on('change', function(change){
console.log(change)
})
})
If you are using node.js, you need to use following node module to get it working:
"dependencies": {
"mongodb": "mongodb/node-mongodb-native#3.0.0"
}
Can I write a procedure from server side which later on gets stored in Db and used for further transactions.
If yes can you provide me a sample code which shows how to write the js from server side in java.
Can I write a procedure from server side which later on gets stored in Db and used for further transactions.
No but as #Philipp states you can write a block of JavaScript which will be evaled within the bult in JavaScript engine in MongoDB (spidermonkey atm unles you compile with V8).
I should be clear this IS NOT A STORED PROCEDURE AND IT DOES NOT RUN "SERVER SIDE" as SQL procedures do.
You must also note that the JS engine is single threaded and eval (when used) locks, and about a tonne of other problems.
Really the whole ability to store functions in the system collection is to store repeating code for tasks such as MR.
It is possible to do that, but 10gen advises that you shouldn't do it. Javascript functions can be stored in the special collections system.js and invoced through the eval command.
The rest of this post is copy&pasted from the official documentation: http://www.mongodb.org/display/DOCS/Server-side+Code+Execution#Server-sideCodeExecution-Storingfunctionsserverside
Note: we recommend not using server-side stored functions when possible. As these are code it is likely best to store them with the rest of your code in a version control system.
There is a special system collection called system.js that can store JavaScript functions to be reused. To store a function, you would do:
db.system.js.save( { _id : "foo" , value : function( x , y ){ return x + y; } } );
_id is the name of the function, and is unique per database.
Once you do that, you can use foo from any JavaScript context (db.eval, $where, map/reduce)
Here is an example from the shell:
> db.system.js.save({ "_id" : "echo", "value" : function(x){return x;} })
> db.eval("echo('test')")
test
See http://github.com/mongodb/mongo/tree/master/jstests/storefunc.js for a full example.
In MongoDB 2.1 you will also be able to load all the scripts saved in db.system.js into the shell using db.loadServerScripts()
>db.loadServerScripts()
>echo(3)
3
Well you can use morphia. It is an ODM for mongo in Java. I know it is not like a PL/SQL kind of solution which you exactly want. But mongo has no extensive support for constraints or triggers as in Oracle or SQL Server. Validations and stuff needs to be done through code. ODM's like Morphia in Java, Mongoose in Node.js or Mongoengine in Python are pretty decent emerging libraries you can use for such tasks, but these constraints are on app side.
I am wondering if it is possible to use a mongodb geospatial index with Meteor architecture.
Minimongo does not implement geospatial indices, but does this mean that we cannot use this mongo feature on the server side?
For example, with the todos app, if we use location on the todo, will it be possible to do:
// Publish complete set of lists to all clients.
Meteor.publish('todos', function (lon,lat) {
return Todos.find({loc: {$near:[lon,lat]}}).limit(2);
});
And on the client side :
Meteor.subscribe('todos', lon, lat );
Yes, you can use the MongoDB geospatial index within Meteor, and you can create that index from within your Meteor app too.
- Geospatial Search
I'm using the $within operator below, as opposed to the $near operator mentioned above, but this still applies:
Meteor.publish('places', function(box) {
return Places.find({ loc : { $within : { $box : box }}});
});
Reminder: These kinds of geo queries are only available on the server (currently).
- Creating a Geospatial Index from within Meteor (rather than in a MongoDB shell)
Places._ensureIndex({ loc : "2d" });
e.g. You could use the above in your bootstrap.js.
Also, you'll probably want to put your ensureIndex in Meteor.startup, or perhaps when you're inserting some initial data.
Warning: As mentioned here, the above method of calling ensureIndex is a work around for want of an official way to call it, so please expect that this might change.
Update: now reflects changes in Meteor 0.5.0, see #Dror's comment below.
Yes, I think you can.
On the server side, Meteor delegates find/update/.. into node-mongo-native call. You can take a look the code in packages/mongo-livedata/mongo_driver.js. And as I know, node-mongo-native supports geospatial index.
Is there a way for Node.js to listen to a change in a particular data in a collection of MongoDB, and fire an event if a change happens?
Well, this is an old question, but I was struggling with the same thing. I found a number of tidbits that helped me put together a solution, and I've published it as a library:
https://github.com/TorchlightSoftware/mongo-watch
The library is written in coffeescript. Here's an example in javascript, for those that prefer.
var MongoWatch = require('mongo-watch'),
watcher = new MongoWatch({parser: 'pretty'});
watcher.watch('test.users', function(event) {
return console.log('something changed:', event);
});
MongoDB apparently now supports triggers and watch, this answer is outdated.
[Original] I believe you are looking for a database trigger.
Unfortunately, MongoDB has no support for them yet, so I don't think you can listen for changes directly from the database. You'll need to setup some sort of notification system (e.g. pub/sub) that alerts interested parties when a collection has changed.
MongoDB 3.6 introduced change streams, which are designed to solve precisely this problem.
Here's an example: https://github.com/thakkaryash94/mongodb-change-streams-nodejs-example
const pipeline = [
{
$project: { documentKey: false }
}
];
const db = client.db("superheroesdb");
const collection = db.collection("superheroes");
const changeStream = collection.watch(pipeline);
// start listen to changes
changeStream.on("change", function (change) {
console.log(change);
});
Well, kind of late.
But still, if someone looking to notify the MongoDB changes to the node.js then they can use mubsub library.
This is active library and very easy to integrate with nodejs.
It uses Mongo's tailable cursors and capped collections.
It works in pub/sub fashion to notify the subscriber when the document inserted into the MongoDB.
Check on Github for details.