trying to write a Mongo query that will Base64 Decode a field that is Base64 encoded and then perform a simple "like" on the decoded value. I'm following a couple of different posts as well as the Mongo docs, but can't seem to get the syntax correct. I basically want to do a query like this :
db.getCollection('my-collection').find (
{ base64Decode(edmDocumentId): /ni-billing-retro/ }
);
Where base64Decode() is a custom function inserted into system.js.
Posts:
----------------
Export text stored as Bindata in mongodb
How to query MongoDB with "like"?
What I've done so far :
I saved the base64Decode() function to the system.js, and I can see the function....https://docs.mongodb.com/manual/tutorial/store-javascript-function-on-server/.
db.system.js.insertOne( {
_id: "base64Decode",
value : function (s) {
var e={},i,k,v=[],r='',w=String.fromCharCode,u=0;
var n=[[65,91],[97,123],[48,58],[43,44],[47,48]];
for(z in n){for(i=n[z][0];i<n[z][1];i++){v.push(w(i));}}
for(i=0;i<64;i++){e[v[i]]=i;}
function a(c){
if(c<128)r+=w(c);else if(c>=192)u=c;else r+=w(((u&31)<<6)+(c&63));
}
for(i=0;i<s.length;i+=72){
var b=0,c,x,l=0,o=s.substring(i,i+72);
for(x=0;x<o.length;x++){
c=e[o.charAt(x)];b=(b<<6)+c;l+=6;
while(l>=8)a((b>>>(l-=8))%256);
}
}
return r;
}
});
I've tried using $where, to no avail...returns ReferenceError: edmDocumentId is not. Added the db.loadServerScripts(); to fix the base64Decode() Reference error.
db.loadServerScripts();
db.getCollection('rapid-document-meta').find (
{ $where: (base64Decode(edmDocumentId) == /ni-billing/) }
);
I've tried doing a straight find (), Unexpected token : Line 2
db.getCollection('rapid-document-meta').find (
{ base64Decode(edmDocumentId): /ni-billing-retro/ }
);
Tried the following from Calling db.system.js Function in $where : ReferenceError: edDocumentId is not defined, even though edmDocumentId is on every single record.
db.loadServerScripts();
db.getCollection('rapid-document-meta').mapReduce (
base64Decode(edDocumentId),
function() {},
{ "out": { "inline": 1 } }
);
Does someone have an example of a find query that uses a custom function from system.js??? Mongo version 4.0.8.
Related
I'm trying to use find() but in function system.js in mongo db but when i run the script the result is
this is my findstored() function
db.system.js.save({
_id:"findstored",
value:function(){
var data = db.counters.find({}, (err, data) => {
data.forEach((data) => {
print("Customer Name: "+data.first_name)
})
return data.first_name;
})
}
})
I just want to display the result with function in system.js. Thank you
Not sure what is the purpose of your function, but this is the right syntax for that function to be saved in system.js:
db.system.js.save({
_id:"findstored",
value:function(){
var ret = "Not found";
db.counters.find().forEach(function(data){
print("Customer Name: "+data.first_name)
ret = data.first_name;
})
return ret;
}
})
And make sure you run db.loadServerScripts(); after saving the function to have it properly loaded.
Do not store application logic in the database. There are performance limitations to running JavaScript inside of MongoDB.
db.system.js.save({
_id:"findUserInformation",
value:function(x){
return db.users.findOne({ "_id" : x }, { firstName:1 }, (err, data) => {
return data
})
}
})
On other shell or command use this findUserInformation function like this
db.loadServerScripts();
findUserInformation("5d7b4ef6f691b71b5097e9cb");
In your question please check return type and run query first and then return data.
In findstored function Application code also is typically most effective when it shares version control with the application itself.
I have a basic schema that implements getters/setters:
var User = new Schema( {
username : { type : String , required : true , get: getUsername, set: setUsername }
} );
User.set('toObject', { getters: true , virtuals : true } );
User.set('toJSON', { getters: true , virtuals : true } );
UserModel = mongoose.model('User', User);
setUsername is fairly simple but "randomizes" the username, and obviously getUsername retrieves the original value. Now, everytime I call setUsername, I will get a different "random" string (I mean it's pretty much a hash function).
My question is, can I somehow use the getter in a query?
Like
UserModel.$where( 'this.username.get() == "<somestring>"' ).exec( function( err , users ) {
// do something with the users
}
} ); // here I get MongoError: TypeError: this.username.get is not a function
or
UserModel.find( { $where: function() {
return( getUsername( this.username ) == "<somestring>" )
} } , function( err , users ) {
// do something with the users
} ); // here I get MongoError: ReferenceError: getUsername is not defined
I tried adding a getUsername method to the schema as well, adding virtuals, etc., but I cannot seem to find a solution. i've tried about 15 different syntaxes but I either get an error, an empty array or all users.
Basically if I have a user whose username is "a", it'll be saved in my DB as *"b", "c", "d", or whatever else the setUsername function sets it to.
I'd like to retrieve the user knowing only that their username is "a".
Thank you!
Yes, this should theoretically be possible using query middleware, but you might have to think a little differently:
Let's say you want to run a query like this:
UserModel.find({user: 'a'})
where 'a' is the unscrambled input and you actually want to automatically lookup {user: getUser('a')}, which we'll say is {user: 'b'}
Then you could use this middleware to modify your Query like this:
const userFilter = function (next) {
const query = this.getQuery();
if (query.userName) {
// this transforms the query in-place from {user: 'a'} to {user : 'b'}
query.userName = setUserName(query.userName)};
}
return next();
};
// let's hook this one in!
User.pre('find', userFilter);
Note: This will hook on all queries including ".userName" If you only use UserModel.find() then this will suffice, but otherwise you might also want to register this hook on ['count', 'find', 'findOne', 'findOneAndRemove','findOneAndUpdate', 'remove', 'update', 'updateOne', 'updateMany'] and that some queries (iirc e.g. findByIdAndUpdate) is supposed to bypass mongoose and directly use the mongodbDriver, which might also bypass this hook... Even though it still seems to work currently, so you might want to test all kinds of functions with it)
EDIT:
Looking back at your question, I am not sure, if I understood your question correctly. What does the setUsername function actually return? Is it always the same output for the same input? If so, you basically always to query {userName: setUserName('a')}, right?
I am using SailsJS on a project and I need to use native() for certain querys. The problem I have is that I can't find a proper way to instantiate a Waterline Model Object from the mongo collection find result. I have being searching information about this and the only thing I have found is the following:
var instance = new Model._model(mongo_result_item);
This should work properly, but when I do instance.save(function(err, ins){}); the model throws an error because of the "_id" field, that should be "id".
I have took a look into sails-mongo code and I found that for the "find" method they do this:
// Run Normal Query on collection
collection.find(where, query.select, queryOptions).toArray(function(err, docs) {
if(err) return cb(err);
cb(null, utils.normalizeResults(docs, self.schema));
});
So the normalizeResults does the magic with the "_id" attribute, and other stuff.
The way I am doing this right now is to require the sails-mongo utils.js file to have access to this method.
Full sample:
var mongoUtils = require('sails-mongo/lib/utils.js');
SampleModel.native(function(nativeErr, collection){
collection.find({ 'field' : value }).toArray(function(collectionErr, results){
if (!results || results.length == 0) return res.restfullInvalidFieldValue({ msg : 'INVALID_VALUE' });
var norm_results = mongoUtils.normalizeResults(results);
var instance = new SampleModel._model(norm_results[0]);
});
});
Is there a better / proper way to achieve this ?
I need to do a native search because I have found a problem with Waterline find() method using strings, where the search should be case sensitive. Every string field on the model is being used as a regular expression match of the form : /^{string}$/i
Searching by a regular expression with the case insensitive flag will give me problems. In the other hand, doing { field : { $regex : new RegExp('^'+regexp_escaped_string+'$') } } could be possible, but I think it will perform worst than { field : value }.
If someone have found a different workaround for the case insensitive problem, please, point me in the right direction.
Thanks in advance.
$regex might help you to search case insensitive string using option paramteter as "i", you can also specify custom regex instead for more information see $regex mongodb documentation.
/**
* PetController
*
* #description :: Server-side logic for managing pets
* #help :: See http://links.sailsjs.org/docs/controllers
*/
module.exports = {
searchByName: function (req,res) {
Pet
.native(function(err, collection) {
if (err) return res.serverError(err);
collection.find(
{
name: { $regex: /like-my-name/, $options: "i" } // here option "i" defines case insensitive
},
{
name: true
})
.toArray(function (err, results) {
if (err) return res.serverError(err);
return res.ok(results);
});
});
}
};
See here for more on native mongo query - https://stackoverflow.com/a/54830760/1828637
I have the following Document:
{
"_id": 100,
"Version": 1,
"Data": "Hello"
}
I have a function which return a number from a sequence:
function getNextSequence(name) {
var ret = db.Counter.findAndModify(
{
query: { _id: name },
update: { $inc: { value: 1 } },
new: true,
upsert: true
}
);
return ret.value;
}
I can use this for optimistic concurrency by performing the following Mongo command:
db.CollectionName.findAndModify({
query: { "_id" : NumberLong(100), "Version" : 1 },
update: { "$set" : {
"Data": "Here is new data!",
"Version" : db.eval('getNextSequence("CollectionName")') }
},
new: true
}
);
This will update the document (as the _id and Version) match, with the new Data field, and also the new number out of the eval call.
It also returns a modified document, from which I can retrieve the new Version value if I want to make another update later (in the same 'session').
My problem is:
You cannot create an Update document using the MongoDB C# client that will serialize to this command.
I used:
var update = Update.Combine(
new UpdateDocument("$set", doc),
Update.Set(versionMap.ElementName, new BsonJavaScript("db.eval('getNextSequence(\"Version:CollectionName\")')")))
);
If you use what I first expected to perform this task, BsonJavascript, you get the following document, which incorrectly sets Version to a string of javascript.
update: { "$set" : {
"Data": "Here is new data!",
"Version" : { "$code" : "db.eval('getNextSequence(\"Version:CollectionName\")')" }
}
}
How can I get MongoDB C# client to serialize an Update document with my db.eval function call in it?
I have tried to add a new BsonValue type in my assembly which I would serialize down to db.eval(''); However there is a BsonType enum which I cannot modify, without making a mod to MongoDB which I would not like to do incase of any issues with the change, compatibility etc.
I have also tried simply creating the Update document myself as a BsonDocument, however FindAndModify will only accept an IMongoUpdate interface which a simply a marker that at present I find superfluous.
I have just tried to construct the command manually by creating a BsonDocument myself to set the Value: db.eval, however I get the following exception:
A String value cannot be written to the root level of a BSON document.
I see no other way now than drop down to the Mongo stream level to accomplish this.
So I gave up with trying to get Mongo C# Client to do what I needed and instead wrote the following MongoDB function to do this for me:
db.system.js.save(
{
_id : "optimisticFindAndModify" ,
value : function optimisticFindAndModify(collectionName, operationArgs) {
var collection = db.getCollection(collectionName);
var ret = collection.findAndModify(operationArgs);
return ret;
}
}
);
This will get the collection to operate over, and execute the passed operationArgs in a FindAndModify operation.
Because I could not get the shell to set a literal value (ie, not a "quoted string") on a javascript object, I had to to this in my C# code:
var counterName = "Version:" + CollectionName;
var sequenceJs = string.Format("getNextSequence(\"{0}\")", counterName);
var doc = entity.ToBsonDocument();
doc.Remove("_id");
doc.Remove(versionMap.ElementName);
doc.Add(versionMap.ElementName, "SEQUENCEJS");
var findAndModifyDocument = new BsonDocument
{
{"query", query.ToBsonDocument()},
{"update", doc},
{"new", true},
{"fields", Fields.Include(versionMap.ElementName).ToBsonDocument() }
};
// We have to strip the quotes from getNextSequence.
var findAndModifyArgs = findAndModifyDocument.ToString();
findAndModifyArgs = findAndModifyArgs.Replace("\"SEQUENCEJS\"", sequenceJs);
var evalCommand = string.Format("db.eval('optimisticFindAndModify(\"{0}\", {1})');", CollectionName, findAndModifyArgs);
var modifiedDocument = Database.Eval(new EvalArgs
{
Code = new BsonJavaScript(evalCommand)
});
The result of this is that I can now call my Sequence Javascript, the getNextSequence function, inside the optimisticFindAndModify function.
Unforunately I had to use a string replace in C# as again there is no way of setting a BsonDocument to use the literal type db.eval necessary, although Mongo Shell likes it just fine.
All is now working.
EDIT:
Although, if you really want to push boundaries, and are actually awake, you will realize this same action can be accomplished by performing an $inc on the Version field.... and none of this is necessary....
However: If you want to follow along to the MongoDB tutorial on how they to say to implement concurrency, or you just want to use a function in a FindAndModify, this will help you. I know I'll probably refer back to it a few times in this project!
I am trying to read out a file in my MongoDB database. In the console the response is correct while in my application I get the following error:
Uncaught TypeError: Cannot read property 'iati' of undefined
I defined a template helper which should return a certain sub-field within my MongoDB collection. However the following does not seem to work (I get the beforementioned error).
Template.hello.helpers({
test: function() {
return Test.findOne().iati;
}
});
What does seem to work is to return the entire object:
Template.hello.helpers({
test: function() {
return Test.findOne();
}
});
And then call the specific field within the template:
{{test.iati}}
However, I want to use the data within the JavaScript script. What am I doing wrong?
Collection methods like Tests.findOne() return the documents that are already fetched to the client's Minimongo copy. Before your document is fetched, findOne() will return null.
To safeguard against this, simply check the result in the helper before you proceed with the calculation:
Template.hello.helpers({
test: function() {
if(! Test.findOne()) return;
return Test.findOne().iati;
},
});
You can also wait for the subscription in the Iron Router to ensure the proper documents are loaded:
this.route('routeName', {
...
onBeforeAction: function() {
this.subscribe('subscriptionName').wait();
...
},
...
});