How to replace prefetch source in Bloodhound with a 'real time' alternative? - mongodb

I've been using Bloodhound with the prefetch [docs] option defined.
This works fine, except when I add content to the json file being prefetched, it is not available as a search result unless I restart the browser.
So I am trying to make the search results reflect the updated file content in 'real time'.
I tried simply replacing prefetch with remote but this causes the search functionality not to work as intended (it shows non-matched results).
Below is the code I am using with prefetch.
Version info: typeahead.bundle.min.js at v0.10.5.
function searchFunction() {
var template =
"<p class=\"class_one\">{{area}}</p><p class=\"class_two\">{{title}}</p><p class=\"class_three\">{{description}}</p>";
var compiled_template = Hogan.compile(template);
var dataSource = new Bloodhound({
datumTokenizer: function(d) {
return Bloodhound.tokenizers.whitespace(d.tokens.join(
' '));
},
queryTokenizer: Bloodhound.tokenizers.whitespace,
prefetch: '/static/my_file.json'
# remote: '/search'
});
dataSource.initialize();
$('.my_lookup .typeahead').typeahead({}, {
source: dataSource.ttAdapter(),
name: 'courses',
displayKey: 'title',
templates: {
suggestion: compiled_template.render.bind(
compiled_template)
}
}).focus().on('typeahead:selected', function(event, selection) {
var title = selection.title
// do things with the title variable
});
}
Edit:
I started thinking perhaps I need some server side logic to perform a search on a database that contains the content previously within the local json file.
Using the code posted below, the following works:
Searches database in real time.
All matches are returned.
The following does not work:
It does not offer suggestions, you have to type the full token name.
If searching for apple, it will search after typing a, then p etc, if it doesn't get any results, it shows this error in Firebug: TypeError: data is null. After a few of these errors, it stops triggering searches and no error is displayed.
And, the results from the database are in the following format, and I don't know how to apply the Hogan template for the suggestions to each result:
{
"matches": [{
"tokens": ["apple", "orange"],
"area": "Nautical",
"_id": {
"$oid": "4793765242f9d1337be3d538"
},
"title": "Boats",
"description": "Here is a description"
}, {
"tokens": ["apple", "pineapple"],
"area": "Aviation",
"_id": {
"$oid": "4793765242f9d1337be3d539"
},
"title": "Planes",
"description": "Here is a description."
}]
}
JS
function searchFunction() {
var engine = new Bloodhound({
remote: {
url: '/search?q=%QUERY%',
wildcard: '%QUERY%'
},
datumTokenizer: Bloodhound.tokenizers.whitespace('q'),
queryTokenizer: Bloodhound.tokenizers.whitespace,
});
engine.initialize();
$('.my_lookup .typeahead').typeahead({
}, {
source: engine.ttAdapter(),
name: 'courses',
displayKey: 'title',
templates: {
suggestion: function (data) {
return "// not sure how to apply markup to each match"
}
}
}).focus().on('typeahead:selected', function(event, selection) {
console.log(selection);
var title = "// again not sure how to access individual match data"
// do things with the title variable
});
}
MongoDB Schema
Database: courses
Collection: courses
Documents:
{
"_id" : ObjectId("4793765242f9d1337be3d538"),
"tokens" : [
"apple",
"orange"
],
"area" : "Nautical",
"title" : "Boats",
"description" : "Here is a description."
}
and:
{
"_id" : ObjectId("4793765242f9d1337be3d539"),
"tokens" : [
"apple",
"pineapple"
],
"area" : "Aviation",
"title" : "Planes",
"description" : "Here is a description."
}
etc
Python (using Bottle routes)
#route('/search')
def search():
"""
Query courses database for matches in tokens field.
"""
# get the query
query = request.GET.q
# define the database
dbname = 'courses'
db = connection[dbname]
# define the collection
collection = db.courses
# make the query
matches = collection.find({"tokens":query})
# send back results
results = {}
results['matches'] = matches
response.content_type = 'application/json'
return dumps(results)

Related

MongoDB Atlas Search - Multiple terms in search-string with 'and' condition (not 'or')

In the documentation of MongoDB Atlas search, it says the following for the autocomplete operator:
query: String or strings to search for. If there are multiple terms in
a string, Atlas Search also looks for a match for each term in the
string separately.
For the text operator, the same thing applies:
query: The string or strings to search for. If there are multiple
terms in a string, Atlas Search also looks for a match for each term
in the string separately.
Matching each term separately seems odd behaviour to me. We need multiple searches in our app, and for each we expect less results the more words you type, not more.
Example: When searching for "John Doe", I expect only results with both "John" and "Doe". Currently, I get results that match either "John" or "Doe".
Is this not possible using MongoDB Atlas Search, or am I doing something wrong?
Update
Currently, I have solved it by splitting the search-term on space (' ') and adding each individual keyword to a separate must-sub-clause (with the compound operator). However, then the search query no longer returns any results if there is one keyword with only one character. To account for that, I split keywords with one character from those with multiple characters.
The snippet below works, but for this I need to save two generated fields on each document:
searchString: a string with all the searchable fields concatenated. F.e. "John Doe Man Streetstreet Citycity"
searchArray: the above string uppercased & split on space (' ') into an array
const must = [];
const searchTerms = 'John D'.split(' ');
for (let i = 0; i < searchTerms.length; i += 1) {
if (searchTerms[i].length === 1) {
must.push({
regex: {
path: 'searchArray',
query: `${searchTerms[i].toUpperCase()}.*`,
},
});
} else if (searchTerms[i].length > 1) {
must.push({
autocomplete: {
query: searchTerms[i],
path: 'searchString',
fuzzy: {
maxEdits: 1,
prefixLength: 4,
maxExpansions: 20,
},
},
});
}
}
db.getCollection('someCollection').aggregate([
{
$search: {
compound: { must },
},
},
]).toArray();
Update 2 - Full example of unexpected behaviour
Create collection with following documents:
db.getCollection('testing').insertMany([{
"searchString": "John Doe ExtraTextHere"
}, {
"searchString": "Jane Doe OtherName"
}, {
"searchString": "Doem Sarah Thisistestdata"
}])
Create search index 'default' on this collection:
{
"mappings": {
"dynamic": false,
"fields": {
"searchString": {
"type": "autocomplete"
}
}
}
}
Do the following query:
db.getCollection('testing').aggregate([
{
$search: {
autocomplete: {
query: "John Doe",
path: 'searchString',
fuzzy: {
maxEdits: 1,
prefixLength: 4,
maxExpansions: 20,
},
},
},
},
]).toArray();
When a user searches for "John Doe", this query returns all the documents that have either "John" OR "Doe" in the path "searchString". In this example, that means all 3 documents. The more words the user types, the more results are returned. This is not expected behaviour. I would expect more words to match less results because the search term gets more precise.
An edgeGram tokenization strategy might be better for your use case because it works left-to-right.
Try this index definition take from the docs:
{
"mappings": {
"dynamic": false,
"fields": {
"searchString": [
{
"type": "autocomplete",
"tokenization": "edgeGram",
"minGrams": 3,
"maxGrams": 10,
"foldDiacritics": true
}
]
}
}
}
Also, add change your query clause from must to filter. That will exclude the documents that do not contain all the tokens.

FormBuilder accessing second-level attributes (dot notation)

I'm using Angular2's FromBuilder to build a query to send to my backend (MongoDB). The problem is, I'm trying to access second-level attributes. For example I have a Song which has a Genres array in it. I want to access genres.hipHop for example but I can't set a form group to genres.hipHop. All I can do is use one word.
Here is an example of a query I want to build
{
"where": {
"and": [
{"genres.house": {"exists": "true"}}
]
},
"order": "created DESC"
};
My FormBilder looks like this:
this.queryForm = this._formBuilder.group({
where: this._formBuilder.group({
and: this._formBuilder.array([
this._formBuilder.group({
genres : this._formBuilder.group({ <-- need it be genres.hipHop
exists: true
})
})
]),
order: 'created DESC',
})
});
Which creates:
{
"where": {
"and": [
{"genres": {"exists": "true"}}
]
},
"order": "created DESC"
};
I can't add the .hipHop after genres.
I tried nesting the genres(hipHop) inside of genres but when I send it to my backend, it doesn't recognize hipHop. This is the query that was built:
{ "where": {
"and": [
{ "genres":
{ "hipHop": { "exists": true } }
}
],
"order": "created DESC" }
}
This was the error I got:
"MongoError: unknown operator: $hipHop\n
Here is an example of a song object:
{
"title": "Sweet Talk feat. Quinn XCII (Evan Gartner Remix)",
"artist": "Academy",
"audio": "https://api.soundcloud.com/tracks/270500010/stream?client_id=90d140308348273b897fab79f44a7c89",
"image": "https://i1.sndcdn.com/artworks-000168676452-qkxqul-t500x500.jpg",
"download": "http://stereoload.com/academy/quinn-xcii-x-academy-sweet-talk-evan-gartner-remix",
"url": "http://soundcloud.com/academy401/sweettalkremix",
"created": "2016-12-13T09:21:28.071Z",
"genres": {
"hipHop": 30,
"house": 30,
"pop": 40
},
"rank": 0
}
Thank you!
I don't think you'll be able to use the dot notation here, as it won't be a valid key.
Looking at this line in the source:
https://github.com/angular/angular/blob/2.4.1/modules/%40angular/forms/src/form_builder.ts#L75
It looks like Angular iterates over the keys to build the controls. As you can't do "x.y" as a key in javascript, this fails.
Example
var x = {y.z = "a"} //--> Uncaught SyntaxError: Unexpected token .
I think you'll have to make another set of keys inside your genre FormGroup -- one of the keys could be hiphop, for instance. Which might be what you're looking to do anyway if there are more genres you want to add :)
EDIT:
Example --
this.queryForm = this._formBuilder.group({
where: this._formBuilder.group({
and: this._formBuilder.array([
this._formBuilder.group({
genres : this._formBuilder.group({
hiphop : this._formBuilder.group({
exists: true
})
})
})
]),
order: 'created DESC',
})
});
Did you tried define object key as a string "genres.hipHop"?
this.queryForm = this._formBuilder.group({
where: this._formBuilder.group({
and: this._formBuilder.array([
this._formBuilder.group({
"genres.hipHop" : this._formBuilder.group({
exists: true
})
})
]),
order: 'created DESC',
})
});

Meteor. Sorting my collection by a deeply nested value

In my application I have a list of tiles representing each project in a portfolio. This is the main list view for the app and all projects are fetched from the collection without any sorting or ordering.
When I have an optional slug parameter specified in my route (for the category assigned to the project) I want to be able to display the projects within the UI that match that category first, and then display the other ones that don't match the category.
For reference, I have included the code for the route below:
/**
* Project list view (all projects) with optional
* filter parameter for showing projects only by
* their category name.
*/
this.route('list', {
path: '/:_category_slug?',
template: 'template_main',
action: function() {
if(this.ready()) {
this.render();
}
},
waitOn: function() {
return [
Meteor.subscribe('projects'),
Meteor.subscribe('formations'),
Meteor.subscribe('categories')
];
},
data: function() {
if(this.params._category_slug) {
/**
* Building up the query given the category slug and the language
*/
var query = {};
query['slug.' + App.language] = this.params._category_slug;
/**
* Grab the category given the query, so we can get its 'id'
*/
var category = App.models.categories.findOne(query);
/**
* This is the query I need to work on so that I can achieve what I want
*/
return App.models.projects.find({}).fetch();
}
else {
return App.models.projects.find({}).fetch();
}
},
yieldTemplates: {
'components_header': {to: 'header'},
'views_list': {to: 'content'},
'components_footer': {to: 'footer'}
}
});
For reference, I have also included a sample of the data for three projects that is relevant to this question.
{
"id": 10,
"slug": {
"en": "sample-english-slug",
},
"title": {
"en": "Sample English Title",
},
"description": {
"en": "A good description.",
},
"category_ids": [
{
"id": 5
},
{
"id": 6
}
],
},
{
"id": 12,
"slug": {
"en": "another-sample-slug",
},
"title": {
"en": "Another sample title",
},
"description": {
"en": "Sample description three",
},
"category_ids": [
{
"id": 1
},
{
"id": 4
}
],
},
{
"id": 11,
"slug": {
"en": "another-sample-slug",
},
"title": {
"en": "A sample title",
},
"description": {
"en": "Sample description",
},
"category_ids": [
{
"id": 2
},
{
"id": 5
}
],
}
So what I would want to do is make sure that given a category with an ID of 5, I want those first two projects to be the first two that appear.
Can this be done in meteor, without having to resort to writing extra logic in JS? One approach I did have once was to update each project from within the Client side collection (something I no longer do) and set a few extra attributes, then sort after that.
When dealing with syncing client and server collections, this is not really feasible.
From the mongodb docs:
Use the dot notation to match by specific fields in an embedded document. Equality matches for specific fields in an embedded document will select documents in the collection where the embedded document contains the specified fields with the specified values. The embedded document can contain additional fields.
I don't know if you can do it with a single query, but you can concat two complementary queries that use dot notation.
var selected = App.models.projects.find({'category_ids.id': category._id}).fetch();
var other = App.models.projects.find({'category_ids.id': {$ne: category._id}}).fetch();
return selected.concat(other);

Sorting by document values in couchbase and scala

I am using couchbase and I have a document (product) that looks like:
{
"id": "5fe281c3-81b6-4eb5-96a1-331ff3b37c2c",
"defaultName": "default name",
"defaultDescription": "default description",
"references": {
"configuratorId": "1",
"seekId": "1",
"hsId": "1",
"fpId": "1"
},
"tenantProducts": {
"2": {
"adminRank": 1,
"systemRank": 15,
"categories": [
"3"
]
}
},
"docType": "product"
}
I wish to get all products (this json is product) that belong to certain category, So i've created the following view:
function (doc, meta) {
if(doc.docType == "product")
{
for (var tenant in doc.tenantProducts) {
var categories = doc.tenantProducts[tenant].categories
// emit(categories, doc);
for(i=0;i<categories.length;i++)
{
emit([tenant, categories[i]], doc);
}
}
}
}
So i can run the view with keys like:
[["tenantId", "Category1"]] //Can also have: [["tenant1", "Category1"],["tenant1", "Category2"] ]
My problem is that i receive the document, but i wish to sort the documents by their admin rank and system rank, these are 2 fields that exists in the "value".
I understand that the only solution would be to add those fields to my key, determine that my key would be from now:
[["tenantId", "Category1", "systemRank", "adminRank"]]
And after i get documents, i need to sort by the 3rd and 4th parameters of the key ?
I just want to make sure i understand this right.
Thanks

Extract Individual particular element from Nested item in MongoDB

I am new to MongoDB.
I have created a collection in MongoDB and stored the following
Q1UsefulStatementsList: [{
Q1UsefulStatement: "Useful Sentence"
Q1ActionsList: [{
Q1Verb: "Verb in the sentence"
Q1NP: "The Noun phrase"
Q1PP: "The Preposition phrase"
}]
}]
Q2UsefulStatementsList: [{
Q2UsefulStatement: "Useful Sentence"
Q2ActionsList: [{
Q2Verb: "Verb in the Sentence"
Q2NP: "The Noun phrase"
Q2PP: "The preposition Phrase"
}]
}]
I need to loop through this collection and get all the Verbs from Q1UsefulStatementsList and Q2UsefulStatementsList.
Ex:
Q1UsefulStatementsList: [{
Q1UsefulStatement: "My dog also likes eating sausage"
Q1ActionsList: [{
Q1Verb: "likes"
Q1NP: "My dog"
Q1PP: "n / a"
}]
} {
Q1UsefulStatement: "The disabling of log helps"
Q1ActionsList: [{
Q1Verb: "disabling"
Q1NP: "disaabling of logs"
Q1PP: "of"
}]
}]
Q2UsefulStatementsList: [{
Q2UsefulStatement: "Log analysis failed"
Q2ActionsList: [{
Q2Verb: "failed"
Q2NP: "Log analysis"
Q2PP: "n / a"
}]
}]
I would like to get 'likes' and 'disabling' as output when I run through Q1UsefulStatementsList.
I have tried it using the code below. But is there a easier way to do these sort of things in MongoDB?
I tried using the 'dot operator' like (Q1UsefulStatementsList.Q1UsefulStatement) but what it gives me is a entire BSON(JSON) object. What I need is actual individual values directly.
Do suggest any easier way if any.
The Java code that I have written to extract the values
if (object.get("Q1UsefulStatementsList") != null) {
BasicDBList qUseStatementList = (BasicDBList)(object.get("Q1UsefulStatementsList"));
for (Object qUsefulStatement: qUseStatementList) {
DBObject tmp = (DBObject) qUsefulStatement;
if (tmp.get("Q1ActionsList") != null) {
BasicDBList qActionsList = (BasicDBList) tmp.get("Q1ActionsList");
for (Object qVerbs: qActionsList) {
DBObject tmpQVerbs = (DBObject) qVerbs;
String verb = (tmpQVerbs.get("Q1Verb").toString());
String nP = (tmpQVerbs.get("Q1NP").toString());
String pP = (tmpQVerbs.get("Q1PP").toString());
}
}
}
}
You can use Aggregation Framework to get the verbs you want. You can do it with the following code.
db.myObject.aggregate(
{ $unwind : "$Q1UsefulStatementsList"},
{ $unwind : "$Q1UsefulStatementsList.Q1ActionsList"},
{$group:{ _id: "$_id", verbs : {$addToSet : "$Q1UsefulStatementsList.Q1ActionsList.Q1Verb"}}}
);
This will return results as follow :
"result" : [
{
"_id" : ObjectId("5253e46ae3a2c44e082642c9"),
"verbs" : ["disabling", "likes"]
}
]
By looping the result array you can extract verbs array and add it to the Set in order to get unique verbs.
You can easily convert it to Java code.