I am trying to find the best way to clean my Mongo DB from old rows. This is the row structure:
{
"_id": ObjectId("52adb7fb12e20f2e2400be38"),
"1": "2013-12-15 06: 07: 20",
"2": "1",
"3": "",
"4": "",
"5": "ID",
"6": "139.195.98.240",
"7": "",
"8": "youtube",
"9": NumberInt(0),
"10": "",
"11": ""
}
The date field is this.1. So I want to set a delte method for all rows older then 30 days.
So I figured out that a map can help with this task, if there are any other suggestion please feel free to suggest.
This is the map function that i am trying to run:
{
mapreduce : "delete_rows",
map : function () {
var delete_date = new Date();
delete_date.setDate(delete_date.getDate()-7);
row_date = new Date(this.1);
if(row_date < delete_date){
emit(this._id,{date: this.1}, {all_data: this});
}
},
out : {
"delete_rows"
},
keeptemp:false,
jsMode : false,
verbose : false
}
I get the following error at rockmongo query window:
Criteria must be a valid JSON object
Can anyone jelp me with this syntax?
Thanks
Related
I am new to dart, I have a requirement to group by multiple fields and get the minimum value of each student. I don't have an idea on how to implement since I am new in dart. Below is the example of the realtime firebase table structure.
"gameRanking" : {
"-MmvcDgrGsuCjhcsmXfP" : {
"game" : "Puzzle",
"score" : "105",
"student" : "John Doe",
},
"-MasdDgrGsuCjhcsmXfP" : {
"game" : "Puzzle",
"score" : "99",
"student" : "John Doe",
},
"-Mmw0kagqLrEbdWlkXg7" : {
"game" : "Puzzle",
"score" : "87",
"student" : "Mary Doe",
},
"-MmwC8ONbJUWzP_Wa7X0" : {
"game" : "Puzzle",
"score" : "95",
"student" : "Mary Doe",
}
},
Here is the expected output:
Puzzle John Doe 99
Puzzle Mary Doe 87
I won't answer the question entirely because it's clearly a "do my homework" kind of question.
But in order to assist you in getting started with programming, I thought it would be useful to post "nearly" the whole solution here. I hope you will be able to then read answers in other related questions, for example this one, to get to the final solution.
Here you go, this solves the "plumbing" part of the problem:
class Stats {
String game;
int score;
String student;
Stats(this.game, this.score, this.student);
#override
String toString() => "Stats($game, $score, $student)";
}
main() {
final map = {
"gameRanking": {
"-MmvcDgrGsuCjhcsmXfP": {
"game": "Puzzle",
"score": "105",
"student": "John Doe",
},
"-MasdDgrGsuCjhcsmXfP": {
"game": "Puzzle",
"score": "99",
"student": "John Doe",
},
"-Mmw0kagqLrEbdWlkXg7": {
"game": "Puzzle",
"score": "87",
"student": "Mary Doe",
},
"-MmwC8ONbJUWzP_Wa7X0": {
"game": "Puzzle",
"score": "95",
"student": "Mary Doe",
}
},
};
final rankingsById = map["gameRanking"] ?? {};
final rankings = rankingsById.map((id, data) {
return MapEntry(
id,
Stats(data["game"].coerceToString(), data["score"].coerceToInt(),
data["student"].coerceToString()));
}).values;
rankings.forEach((stats) => print(stats));
}
extension on Object? {
int coerceToInt() => int.parse(coerceToString());
String coerceToString() => this?.toString() ?? "?";
}
Running this prints:
Stats(Puzzle, 105, John Doe)
Stats(Puzzle, 99, John Doe)
Stats(Puzzle, 87, Mary Doe)
Stats(Puzzle, 95, Mary Doe)
Good luck finishing the rest of the owl :)
I needed the same... I don't know if this is the best solution...
Map<int,List<MyObject>> mom=groupBy(_myObjectList,(element)=>element.cornerCnt);//get a Map with cornerCnt as key and a List of myObjects for every key
int maxCornerCnt;
int max=0;
mom.forEach((key, moList) { //for every map entry
int len=moList.length; //get lenght of myObject-list
if (len>max) {
max=len;
maxCornerCnt=key; //write key(cornerCnt) to maxCornerCnt
}
});
I have the following document :
{
"recordKey": "FOO",
"channels": [{
"id": "CH1",
"blocks": []
}, {
"id": "CH2",
"blocks": []
}]
}
In my current use case, I'm doing two requests with addToSet operator for adding new blocks for the channel CH1 or CH2 For example for the channel CH1, I'm doing this:
selector =
{
"$and" : [ {
"recordKey" : "FOO"
}, {
"channels.id" : "CH1"
} ]
}
addChunkRequest = "$addToSet" : {
"channels.$.blocks" : {
"$each" : [ {
"startime" : 101000000,
"blockType" : "DATA",
"fileLoc" : "/tmp/f1",
"nsamples" : 1000
}
query1 = db.collection.update(selector, update)
I'm doing the same think for the channel CH2. Now I want to group the two requests in one request. How can I achieve that ?
Well you cannot of course "update multiple array elements in the one operation", because that is just not presently allowed and a restriction of the positional $ operator.
What you "can" do however is use Bulk Operations to issue "both" operations in a "single request" to the server:
var data = [
{
"channel": "CH1",
"blocks": [{
"startime" : 101000000,
"blockType" : "DATA",
"fileLoc" : "/tmp/f1",
"nsamples" : 1000
}]
},
{
"channel": "CH2",
"blocks": [{
"startime" : 202000000,
"blockType" : "DATA",
"fileLoc" : "/tmp/f2",
"nsamples" : 2000
}]
}
]
var ops = data.map(d => ({
"updateOne": {
"filter": { "recordKey": "FOO", "channels.id": d.channel },
"update": {
"$addToSet": { "channels.$.blocks": { "$each": d.blocks } }
}
}
});
db.collection.bulkWrite(ops);
So it's still "two" operations and that cannot be avoided, however it's only "one" request and response from the server, and that actually helps you quite a lot.
I am new to Mongo Aggregation and have this data
{
"_id": {
"$oid": "5654a8f0d487dd1434571a6e"
},
"ValidationDate": {
"$date": "2015-11-24T13:06:19.363Z"
},
"DataRaw": " WL 00100100012015-08-28 02:44:17+0000+ 16.81 8.879 1084.00",
"ReadingsAreValid": true,
"locationID": " WL 001",
"Readings": {
"pH": {
"value": 8.879
},
"SensoreDate": {
"value": {
"$date": "2015-08-28T02:44:17.000Z"
}
},
"temperature": {
"value": 16.81
},
"Conductivity": {
"value": 1084
}
},
"HMAC":"ecb98d73fcb34ce2c5bbcc9c1265c8ca939f639d791a1de0f6275e2d0d71a801"
}
My goal is to calculate the average temperature for every two hours interval, then per month, year and week. I have tried various queries but no luck. This is what i have tried so far
data.aggregate([{"$unwind":"$Readings"},
{"$project":{"HourRecord":{"$hour":"Readings.SensoreDate.value"},
"YearRecord":{"$year":"$Readings.SensoreDate.value"}}},
{'$group' : {'_id' : "$locationID",
'AverageTemp' : { '$avg' : '$Readings.temperature.value'}}}
])
and i got an empty results like this
{u'ok': 1.0, u'waitedMS': 0L, u'result': [{u'AverageTemp': None, u'_id': None}]}
I have tried several other combinations but still got empty results.
The following Queries Return the required result per hour, month etc but how to i group by two hours interval instead of one hour
test_Agg.aggregate([{"$unwind":"$Readings"},
{"$project":{ "HourRecord": { "$hour":"$Readings.SensoreDate.value"},
"YearRecord": {"$year":"$Readings.SensoreDate.value"},
"MonthRecord": {"$month":"$Readings.SensoreDate.value"},
"locationID" : 1,
"Readings.pH.value":1,
'Readings.temperature.value' : 1}
},
{'$group' : {'_id' :"$HourRecord",
'AverageTemp' : { '$avg' : '$Readings.temperature.value'}}
}])
The second pipeline $project in the query didn't project the required values used in the subsequent pipelines. Hence, the result was empty.
The required fields have been added in the $project. Try the below query.
db.temperature.aggregate([{"$unwind":"$Readings"},
{"$project":{ "HourRecord": { "$hour":"$Readings.SensoreDate.value"},
"YearRecord": {"$year":"$Readings.SensoreDate.value"},
locationID : 1,
'Readings.temperature.value' : 1}
},
{'$group' : {'_id' : "$locationID",
'AverageTemp' : { '$avg' : '$Readings.temperature.value'}}}
]);
Output for the one document provided in the post:-
{
"_id" : " WL 001",
"AverageTemp" : 16.81
}
I have data stored with a nested location object and can't figure out how to get elastic4s to return the location as part of the result of a search. I have data that when queried (from the REST endpoint) looks like this:
{
"_index": "us_large_cities",
"_type": "city",
"_id": "AU7ke-xU_N_KRYZ5Iii_",
"_score": 1,
"_source": {
"city": "Oakland",
"state": "CA",
"location": {
"lat": "37.8043722",
"lon": "-122.2708026"
}
}
}
When I try querying it using elastic4s like so:
search in "us_large_cities"->"city" fields("location", "city", ) query {
filteredQuery filter {
geoPolygon("location") point(37.9, -122.31) point(37.8, -122.31) point(37.8, -122.25) point(37.9, -122.25)
}
I get back results like this:
{
"_index" : "us_large_cities",
"_type" : "city",
"_id" : "AU7keH9l_N_KRYZ5Iig0",
"_score" : 1.0,
"fields" : {
"city" : [ "Berkeley" ]
}
}
Where I would expect to see "location" but don't. Does anyone know how I specify the fields so that I can actually get the location?
You should try using source filtering instead, as shown below. Note the use of sourceInclude instead of fields.
search in "us_large_cities"->"city" sourceInclude("location", "city") query {
filteredQuery filter {
geoPolygon("location") point(37.9, -122.31) point(37.8, -122.31) point(37.8, -122.25) point(37.9, -122.25)
}
I have a document per day per meter. How can I add another subdocument in the data array and create the whole document if he doesn't exists ?
{
"key": "20120418_123456789",
"data":[
{
"Meter": 123456789,
"Dt": ISODate("2011-12-29T16:00:00.0Z"),
"Energy": 25,
"PMin": 11,
"PMax": 16
}
],
"config": {"someparam": 4.5}
}
Can I use upsert for that purpose ?
The result will be if document exists :
{
"key": "20120418_123456789",
"data":[
{
"Meter": 123456789,
"Dt": ISODate("2011-12-29T16:00:00.0Z"),
"Energy": 25,
"PMin": 11,
"PMax": 16
},
{
"Meter": 123456789,
"Dt": ISODate("2011-12-29T16:15:00.0Z"),
"Energy": 22,
"PMin": 13,
"PMax": 17
}
],
"config": {"someparam": 4.5}
}
Thanks in advance
I think what you want is the $addToSet command - that will push an element to an array only if it does not already exist. I've simplified your example a bit for brevity:
db.meters.findOne()
{
"_id" : ObjectId("4f8e95a718bc9c7da1e6511a"),
"config" : {
"someparam" : 4.5
},
"data" : [
{
"Meter" : 123456789,
}
],
"key" : "20120418_123456789"
}
Now run:
db.meters.update({"key" : "20120418_123456789"}, {"$addToSet": {"data" : {"Meter" : 1234}}})
And we get the updated version:
db.meters.findOne()
{
"_id" : ObjectId("4f8e95a718bc9c7da1e6511a"),
"config" : {
"someparam" : 4.5
},
"data" : [
{
"Meter" : 123456789,
},
{
"Meter" : 1234
}
],
"key" : "20120418_123456789"
}
Run the same command again and the result is unchanged.
Note: you are likely going to be growing these documents, especially if this field is unbounded and causing frequent (relatively expensive) moves by updating in this way - you should have a look here for ideas on how to mitigate this:
http://www.mongodb.org/display/DOCS/Padding+Factor#PaddingFactor-ManualPadding