My application is a registration application, where a user will fills out a big registration form with 100 fields and submits it.
The way data stored is:
A submission document which stores a submissionId, name, email and datetime of submission.
Submission value documents, which is nothing but the data of each field field in the registration form.
When a user fills out the registration form and submits it, 1 submission document is created with a submissionId, and each of its fields are stored as a separate submissionValue document with its key having the submissionId, ie, 100 submissionValue documents.
An example submissionValue document is
{
"_id": "0027ec9c-83d1-45a7-bd2c-ee41f4i5d692/citizenship[0].country_of_birth[0].label/1551683896469",
"_rev": "1-4fdf63cb121e475d96c160b80a0598de",
"$doctype": "submissionValue",
"value": "India"
}
Here the first part of the id (0027ec9c-83d1-45a7-bd2c-ee41f4i5d692) is the submissionId, followed by the key name.
I have an Export to CSV feature where I am facing a timeout issue. I have around 10000 registrations in my application. When I click on the Export button, I am passing all the 10000 submissionIds and then for each submissionId I am calling a View to fetch all the submissionValues. Below is the code for the View:
function(doc) {
if(doc.$doctype == "submissionValue" && doc._id.indexOf("/") > 0) {
var submissionId = doc._id.split("/")[0]
emit(submissionId , doc);
}
}
In the above View, I am returning all the submissionValue documents for each submission, then in the .NET side I am grouping the records based on the submissionId and writing to Excel.
I am getting timeout exception. I don' know if the problem is with the way data is stored in the documents or the view I am trying to fetch the data.
Related
Writing this question I'm working with .Net's LiteDB but I think the question applies to entire NoSQL matters.
One of collection in my db contains documents that don't have fixed structure. I want to let user add his own values, of whatever name and value he wants.
So, for example, document at first would have following structure:
{
"_id": 1,
"creatorId": 10
}
But user would be able to specify new value and choose whether it will be int or boolean.
{
"_id": 1,
"creatorId": 10,
"customValue": false
}
Next time user open my app, he maybe will want to use values of the same kind as he used before, so I need to show him some kind of form with inputs named based on his previous activity. So if he previously added value named "customValue", I want to show him TextView named "customValue" next time he opens page with form.
Is there a way of retrieving structure of such document based on every record from collection? Or do I need to somehow track names of added values and save them in separate collection?
In LiteDB you can use BsonDocument class to read collection documents. BsonDocument is a generic way to implement document in BSON format (with all BSON data type available).
If you use:
var col = db.GetCollection("mycol");
var doc = col.FindById(1);
foreach(var key in doc.Keys)
{
var value = doc[key];
var valueDataType = value.Type; // return an enumerable with all data types
}
I have a page that shows notifications. On this page as part of the layout is a small notifications icon that also shows the count of notifications.
To facilitate this I have two subscriptions running at the same time. Both on the users collection.
The first looks like this:
this.subscribe("user", {_id: Meteor.userId()}, {
fields: {
"notifications._id":1
}
});
the second like this:
this.subscribe("user", {_id: Meteor.userId()}, {
fields: {
"notifications.title": 1,
"notifications._id": 1,
"notifications.level": 1
}
});
The issue I'm having is that if the first subscription runs first (which it does), the second subscription doesn't publish the additional fields.
Is there a clean way of allowing multiple projections of the same collection (cleaner than this: Meteor: publish 2 different projections of array elements)
The publication user takes in a selector and set of fields and merges it with what the current user is allowed to see for the requested user - however this requirement makes no difference, even if I use different publications the issue remains.
It's likely that what I want can't be done, and I should just refactor and create a notification collection
I asked this question a couple of days ago, but deleted it and I am adding more clarification here in what I'm looking at.
So what I have is a process where a user uploads a CSV and the CSV is then parsed by PapaParse and then sent server side and ultimately inserted into MongoDb.
My problem is that none of these uploads are linked to the specific user, so anyone will have access to every upload the way things look now.
What I tried to do is loop through the upload data, which looks like this;
var document = [{object}, {object}, {object}, {object}, {object}... ];
I used a for loop to loop through each of the objects and add an _id field this contains the user's id via var currentUser = this.userId;
Meteor.methods({
insert: function(document){
var currentUser = this.userId;
var newDocument = document;
for(var i = 0; i < newDocument.length; i++){
newDocument[i]._id = currentUser;
}
Bank.insert(newDocument);
}
Problem is that memory allocation is an issue for larger uploads and meteor simply crashes trying to loop through all the objects and individually adding the _id key to each object in each cell of the array.
When the document is inserted into MongoDB, it looks like this:
I know in my previous post, someone mentioned that MongoDB's insert method doesn't take an array as input, but somehow in my case, it does because the above screenshot is exactly how the document looks before being inserted into MongoDB. So basically, each object is a new document inside a MongoDB. I'm trying to find a way to bind the user's userID with each document in the databse.
Is there another way to associate the upload with the unique current user other than looping through the entire data set, which could be in the tens of thousands on some users?
Why not just do?
Meteor.methods({
insert: function(document){
var currentUser = Meteor.userId();
var newDocument = document;
Bank.insert({userId: currentUser, data: newDocument});
}
});
Now each document in your collection will have two keys: userId and data. The latter will be your array.
Not quite sure what the best practice is if I have two collections, a user collection and a picture collection - I do not want to embed all my pictures into my user collection.
My client searches for pictures under a certain criteria. Let's say he gets 50 pictures back from the search (i.e. one single mongodb query). Each picture is associated to one user. I want the user name displayed as well. I assume there is no way to do a single search performance wise on the user collection returning the names of each user for each picture, i.e. I would have to do 50 searches. Which means, I could only avoid this extra performance load by duplicating data (next to the user_id, also the user_name) in my pictures collection?
Same question the other way around. If my client searches for users and say 50 users are returned from the search through one single query. If I want the last associated picture + title also displayed next to the user data, I would again have to add that to the users collection, otherwise I assume I need to do 50 queries to return the picture data?
Lets say the schema for your picture collection is as such:
Picture Document
{
_id: Objectid(123),
url: 'img1.jpg',
title: 'img_one',
userId: Objectid(342)
}
1) Your picture query will return documents that look like the above. You don't have to make 50 calls to get the user associated with the images. You can simply make 1 other query to the Users Collection using the user ids taken from the picture documents like such:
db.users.find({_id: {$in[userid_1,user_id2,userid_3,...,userid_n]}})
You will receive an array of user documents with the user information. You'll have to handle their display on the client afterwards. At most you'll need 2 calls.
Alternatively
You could design the schema as such:
Picture Document
{
_id: Objectid(123),
url: 'img1.jpg',
title: 'img_one',
userId: Objectid(342),
user_name:"user associated"
}
If you design it this way. You would only require 1 call, but the username won't be in sync with user collection documents. For example lets say a user changes their name. A picture that was saved before may have the old user name.
2) You could design your User Collection as such:
User Document
{
_id: Objectid(342),
name: "Steve jobs",
last_assoc_img: {
img_id: Object(342)
url: 'img_one',
title: 'last image title
}
}
You could use the same principles as mentioned above.
Assuming that you have a user id associated with every user and you're also storing that id in the picture document, then your user <=> picture is a loosely coupled relationship.
In order to not have to make 50 separate calls, you can use the $in operator given that you are able to pull out those ids and put them into a list to run the second query. Your query will basically be in English: "Look at the collection, if it's in the list of ids, give it back to me."
If you intend on doing this a lot and intend for it to scale, I'd either recommend using a relational database or a NoSQL database that can handle joins to not force you into an embedded document schema.
I have a document where i am saving all the user registration information with a particular random id associated to it.
my document looks like:
_id:xyz
_rev:9-a4d476aa9f4b41879d21a7f475a2f1d5
-742566755
course:gg
passwdt:t
phone:1313
clgnme:hfhjf
address:fjhfj
-884381686
course:gg
passwdt:t
phone:1313
clgnme:hfhjf
address:fjhfj
Now, i Want to retrieve the document with all information as shown in the document.
I want that it should be shown individually as i have done with getting the values on console. but i want it acording to each id individually.
Like if i want to get data of -884381686 id and it can be anything name, address anything. it should fetch data accordingly.
**NOTE:**PROGRAMMING TO BE DONE IN GWT
You mean you want to access each document via its ID? You can just make an HTTP GET request to (assuming CouchDb is run locally):
GET localhost:5984/<database>/<id>
That will return the entire document
Based on your comment below, the best way to get each item in the document is to create a map view in Couch. Say your document looks like this:
{
_id: 123,
rev: 22,
posts: [{ id: 2, name: hello}, {id: 3, name: world}]
}
If you wanted to list all of the posts, your Map view would look like this:
function(doc){
for(var post in doc.posts){
emit([doc._id, post], 1);
}
}
That will emit all of the posts in every single document. If you have different types of documents, you can add code to only emit for certain types of documents. Then you can go to the URL for the view to see all of the posts: http://localhost:5984/database/_design/ViewCategory/_view/ViewName?reduce=false