mongoengine query to nested listfield - mongodb

I made article system with python flask.
To communicate with mongodb, use flask_mongoengine
Here is my model.
class SubComment(EmbeddedDocument):
no = SequenceField()
body = StringField()
class Comment(EmbeddedDocument):
no = SequenceField()
body = StringField()
sub_comment = ListField(EmbeddedDocumentField(SubComment))
class Article(Document):
title = StringField()
body = StringField()
comments = ListField(EmbeddedDocumentField(Comment))
SubComment model stored into Comment model and Comment model stored into Article model.
So, this is the output that I want.
{
"_id" : ObjectId("5c0641d81b48d9fe50dfdd7f"),
"title" : "test",
"body" : "gogo",
"comments" : [
{
"no" : 1,
"body" : "first comment",
"sub_comment" : [
{
"no": 1,
"body": "sub comm"
}
]
}
]
}
When I insert Comment model to Article model, just use below code.
comment = Comment(
body='first comment'
)
article = Article.objects(body='gogo').first()
article.comments.append(comment)
article.save()
But when I try to insert SubComment to Comment, it throw errors -> AttributeError: 'BaseList' object has no attribute 'sub_comment'
Below is the code I used.
comment = SubComment(
body='sub comment'
)
article = Article.objects(title='test', comments__no=1).first()
article.comments.sub_comment.append(comment)
article.save()
After some searched, people said there is no way to insert nested field.
Is there any solution here? I have to use raw query?
Thanks!

[SOLVED]
I solved with for loop like this.
count = 0
article = Article.objects(title='test', comments__no=1).first()
for comment in article.comments:
if comment.no == 1:
print(comment.no)
count = comment.no - 1
article.comments[count].sub_comment.append(sub_comment)
article.save()
But I don't know it is the right way.
If you have any other solution, please comment it.

Related

How to add the new field to Object in Purescript

I am first about the Purescript.
I am going to add the new field to Object and send it as a function param.
But I can not find a good solution for this.
For example.
oldFiled = {
title : "title",
description : "d"
}
newField = {
time : "time"
}
//result after added new field
oldFiled = {
title : "title",
description : "d",
time : "time"
}
How can I do it?
If it's just about adding a single field you can use https://pursuit.purescript.org/packages/purescript-record/2.0.1/docs/Record#v:insert like so:
import Data.Record as Record
import Data.Symbol (SProxy(..))
oldFiled = {
title : "title",
description : "d"
}
newFiled = Record.insert (SProxy :: _ "time") "time" oldFiled
If you're merging records look at the merge union and disjointUnion functions in the Data.Record module

Mongo DB - map relational data to document structure

I have a dataset containing 30 million rows in a mongo collection. An example set of records would be:
{"_id" : ObjectId("568bc0f2f7cd2653e163a9e4"),
"EmailAddress" : "1234#ab.com",
"FlightNumber" : 1043,
"FlightTime" : "10:00"},
{"_id" : ObjectId("568bc0f2f7cd2653e163a9e5"),
"EmailAddress" : "1234#ab.com",
"FlightNumber" : 1045,
"FlightTime" : "12:00"},
{"_id" : ObjectId("568bc0f2f7cd2653e163a9e6"),
"EmailAddress" : "5678#ab.com",
"FlightNumber" : 1045,
"FlightTime" : "12:00"},
This has been imported directly from SQL server, hence the relational'esque nature of the data.
How can I best map this data to another collection so that all the data is then grouped by EmailAddress with the FlightNumbers nested? An example of the output would then be:
{"_id" : ObjectId("can be new id"),
"EmailAddress" : "1234#ab.com",
"Flights" : [{"Number":1043, "Time":"10:00"},{"Number":1045, "Time":"12:00"}]},
{"_id" : ObjectId("can be new id"),
"EmailAddress" : "5678#ab.com",
"Flights" : [{"Number":1045, "Time":"12:00"}]},
I've been working on an import routing that iterates through each record in the source collection and then bulk inserts into the second collection. This is working fine however doesn't allow me to group the data unless I back process through the records which adds a huge time overhead to the import routine.
The code for this would be:
var sourceDb = db.getSiblingDB("collectionSource");
var destinationDb = db.getSiblingDB("collectionDestination");
var externalUsers=sourceDb.CRM.find();
var index = 0;
var contactArray = new Array();
var identifierArray = new Array();
externalUsers.forEach(function(doc) {
//library code for NewGuid omitted
var guid = NewGuid();
//buildContact and buildIdentifier simply create 2 js objects based on the parameters
contactArray.push(buildContact(guid, doc.EmailAddress, doc.FlightNumber));
identifierArray.push(buildIdentifier(guid, doc.EmailAddress));
index++;
if (index % 1000 == 0) {
var now = new Date();
var dif = now.getTime() - startDate.getTime();
var Seconds_from_T1_to_T2 = dif / 1000;
var Seconds_Between_Dates = Math.abs(Seconds_from_T1_to_T2);
print("Written " + index + " items (" + Seconds_Between_Dates + "s from start)");
}
//bulk insert in batches
if (index % 5000 == 0) {
destinationDb.Contacts.insert(contactArray);
destinationDb.Identifiers.insert(identifierArray);
contactArray = new Array();
identifierArray = new Array();
}
});
Many thanks in advance
Hey there and welcome to MongoDB. In this situation you may want to consider using two different Collections -- one for users and one for flights.
User:
{
_id:
email:
}
Flight:
{
_id:
userId:
number: // if number is unique, you can actually specify _id as number
time:
}
In your forEach loop, you would first check to see if a user document with that specific email address already exists. If it doesn't, create it. Then use the User document's unique identifier to insert a new document into the Flights collection, storing the identifier under the field userId (or maybe passengerId?).

How to create a river using NEST client for ElasticSearch

I'm beginning to work with Elastic Search on our MVC5 project.
We have continuous integration build in place, and would like to configure Elastic Search via NEST just like Entity Framework Code-First.
I'm struggling to find how I can invoke this request through NEST.
PUT _river/mytype/_meta
{
"type":"jdbc",
"jdbc": {
"driver":"com.microsoft.sqlserver.jdbc.SQLServerDriver",
"url" : "jdbc:sqlserver://127.0.0.1:1433;databaseName=MyDatabase;",
"user" : "user",
"password" : "password123",
"sql":"select * from RiverView",
"poll":"30s",
"index":"myindex",
"type": "mytype",
"strategy": "simple"
}
}
Does anyone have any idea how to do this?
Thank you in advance.
I was able to make this work with using ElasticClient.Raw
var client = CreateClient();
var river = new
{
type = "jdbc",
jdbc = new
{
driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver",
url = String.Format("jdbc:sqlserver://{0}:1433;databaseName={1};", "127.0.0.1", "myDatabase"),
user = "user",
password = "password",
sql = "select * from mySqlView",
poll = "30s",
index = "index",
type = "type",
strategy = "simple"
}
};
string body = client.Serializer.Serialize(river, Formatting.Indented);
//HACK: Second parameter is a hack, appended _meta to type because NEST doesn't implement MetaPut()
var response = client.Raw.IndexPut("_river", "mytype/_meta", body);
Might not be the most elegant but worked for my requirement.

How can I search an embedded collection within MongoDB?

For example suppose I have a Post schema as follows:
var Schema = new db.Schema({
title : String
, text : String
// Denormalize by using user name
, author : String
, tags: [String]
, comments: [Comment]
});
I'm trying to match a particular tag as follows:
function getPostByTag(tag, callback) {
Post.find().where({"tags.value": tag}).run(callback);
}
But it is returning everything. What am I doing wrong?
Where are you getting the "value" part of "tags.value". Shouldn't it just be:
Post.find().where({tags:tag}).run(callback);

Nested document insert into MongoDB with C#

I am trying to insert nested documents in to a MongoDB using C#. I have a collection called categories. In that collection there must exist documents with 2 array, one named categories and one named standards. Inside those arrays must exist new documents with their own ID's that also contain arrays of the same names listed above. Below is what I have so far but I am unsure how to proceed. If you look at the code what I want to do is add the "namingConventions" document nested under the categories array in the categories document however namingConventions must have a unique ID also.
At this point I am not sure I have done any of this the best way possible so I am open to any and all advice on this entire thing.
namespace ClassLibrary1
{
using MongoDB.Bson;
using MongoDB.Driver;
public class Class1
{
public void test()
{
string connectionString = "mongodb://localhost";
MongoServer server = MongoServer.Create(connectionString);
MongoDatabase standards = server.GetDatabase("Standards");
MongoCollection<BsonDocument> categories = standards.GetCollection<BsonDocument>("catagories");
BsonDocument[] batch = {
new BsonDocument { { "categories", new BsonArray {} },
{ "standards", new BsonArray { } } },
new BsonDocument { { "catagories", new BsonArray { } },
{ "standards", new BsonArray { } } },
};
categories.InsertBatch(batch);
((BsonArray)batch[0]["categories"]).Add(batch[1]);
categories.Save(batch[0]);
}
}
}
For clarity this is what I need:
What I am doing is building a coding standards site. The company wants all the standards stored in MongoDB in a tree. Everything must have a unique ID so that on top of being queried as a tree it can be queried by itself also. An example could be:
/* 0 */
{
"_id" : ObjectId("4fb39795b74861183c713807"),
"catagories" : [],
"standards" : []
}
/* 1 */
{
"_id" : ObjectId("4fb39795b74861183c713806"),
"categories" : [{
"_id" : ObjectId("4fb39795b74861183c713807"),
"catagories" : [],
"standards" : []
}],
"standards" : []
}
Now I have written code to make this happen but the issue seems to be that when I add object "0" to the categories array in object "1" it is not making a reference but instead copying it. This will not due because if changes are made they will be made to the original object "0" so they will not be pushed to the copy being made in the categories array, at least that is what is happening to me. I hope this clears up what I am looking for.
So, based on your latest comment, it seems as though this is the actual structure you are looking for:
{
_id: ObjectId(),
name: "NamingConventions",
categories: [
{
id: ObjectId(),
name: "Namespaces",
standards: [
{
id: ObjectId(),
name: "TitleCased",
description: "Namespaces must be Title Cased."
},
{
id: ObjectId().
name: "NoAbbreviations",
description: "Namespaces must not use abbreviations."
}
]
},
{
id: ObjectId(),
name: "Variables",
standards: [
{
id: ObjectId(),
name: "CamelCased",
description: "variables must be camel cased."
}
]
}
]
}
Assuming this is correct, then the below is how you would insert one of these:
var collection = db.GetCollection("some collection name");
var root = new BsonDocument();
root.Add("name", "NamingConventions");
var rootCategories = new BsonArray();
rootCategories.Add(new BsonDocument
{
{ "id": ObjectId.GenerateNewId() },
{ "name", "Namespaces" },
{ "standards", new BsonArray() }
});
root.Add("categories", rootCategories);
//etc...
collection.Save(root);
Hope that helps, if not, I give up :).
So, I guess I'm confused by what you are asking. If you just want to store the namingConventions documents inside the array, you don't need a collection for them. Instead, just add them to the bson array and store them.
var categoriesCollection = db.GetCollection<BsonDocument>("categories");
var category = new BsonDocument();
var namingConventions = new BsonArray();
namingConventions.Add(new BsonDocument("convention1", "value"));
category.Add("naming_conventions", namingConventions);
categoriesCollection.Insert(category);
This will create a new document for a category, create an array in it called naming_conventions with a single document in it with an element called "convention1" and a value of "value".
I also am not quite sure what you are trying to accomplish. Perhaps if you posted some sample documents in JSON format we could show you the C# code to write documents that match that.
Alternatively, if you wish to discuss your schema, that could also be better done in the context of JSON rather than C#, and once a schema has been settled on then we can discuss how to write documents to that schema in C#.
One thing that didn't sound right in your original description was the statement "in that collection must exist 2 arrays". A collection can only contain documents, not arrays. The documents themselves can contain arrays if you want.
var filter = Builders<CollectionDefination>.Filter.Where(r => r._id== id);
var data = Builders<CollectionDefination>.Update.Push(f=>
f.categories,categoriesObject);
await _dbService.collection.UpdateOneAsync( filter,data);
Note: Make sure embedded document type [Categories] is array.