I work on a yaml file configuration of Fluentd. Input data comes from kafka as json file.
some of them are lists lik:
{ a:[ a1:"1", a2:"2", ... ], b:"10" }
How can Fluentd access a range of list items (such as first 10 items) of this json.
I wrote this config to access a specific item:
$.a[0]
and try some config to access first 10, such as:
$.a[0:10]
$.a[0-10]
but not worked!
can you help me?
I explain more by an example:
suppose we have this record in Fluentd:
"key1": 1,
"key2": 2,
"key3": {
"a": 1,
"b": 1,
"c": 1,
"d": [
{
"d1": 1,
"d2": 2,
},
{
"d1": 1,
"d2": 2,
},
{
"d1": 1,
"d2": 2,
}, ...
]
}
List d maybe has more similar items.
How can I remove/access all Key3.d.d2?
According to this , I handled this problem by:
$.Key3.d[0].d2,$.Key3.d[1].d2, $.Key3.d[2].d2,...
but this is not a good resolution because I don't know what is the length of list d.
Related
DB collection seatsObj:
{
"product_id": 46539040,
"freeSeating": false,
"tempTransId": "1ecae165f2d86315fea19963d0ded41a",
"seatLayout": {
"colAreas": {
"Count": 2,
"intMaxSeatId": 43,
"intMinSeatId": 2,
"objArea": [
{
"AreaDesc": "EXECUTIVE",
"AreaCode": "0000000003",
"AreaNum": "1",
"HasCurrentOrder": true,
"objRow": [
{
"GridRowId": 1,
"PhyRowId": "A",
"objSeat": [
{
"GridSeatNum": 1,
"SeatStatus": "1",
"seatNumber": 1,
"seatPrice": 400,
"ID": 111
},
{
"GridSeatNum": 2,
"SeatStatus": "0",
"seatNumber": 2,
"seatPrice": 450,
"ID": 112
},
I was able to find ways to locate and update specific fields using:
seatsObj.updateOne(
{"seatLayout.colAreas.objArea.0.objRow.0.objSeat.seatPrice": 470},
{$set: {"seatLayout.colAreas.objArea.0.objRow.0.objSeat.$.ID": 888}});
but i cannot find simple way to return a specific field value from objSeat array element based on search criteria (for example: get 400 as a result of querying seatPrice for the seat with ID = 111). Could anyone give me a direction? From my initial research I have to go into crazy nested $unwind -s and $objectToArray -s, etc... Isn't there a simpler way? Thank you!!
Do mongodb has any method to sort number before alphabet or character?
For example: 1, 2, 3, 4, -
I used collation({locale: "en_US", numericOrdering: true}) and its output -, 1, 2, 3, 4.
But I expect 1, 2, 3, 4, -
custom-sorting is the one you are looking for.
you can do it as below
{
$project:{
"customFieldToBeCreatedForSorting":{
$cond:[
{ $isNumber : "$FieldContainsMixedTypeData" },
NumberLong(1111111111), //This can be customized by finding max in the previous stage
$FieldContainsMixedTypeData
]
}
}
},
{
//$sort pipeline goes here
}
I'm trying to search any value that match with a "name" param, inside any object with any level in a MongoDB collection.
My BSON looks like this:
{
"name": "a",
"sub": {
"name": "b",
"sub": {
"name": "c",
"sub": [{
"name": "d"
},{
"name": "e",
"sub": {
"name": "f"
}
}]
}
}
}
I've created an index with db.collection.createIndex({"name": "text"}); and it seems to work, because it has created more than one.
{
"numIndexesBefore" : 1,
"numIndexesAfter" : 6,
"note" : "all indexes already exist",
"ok" : 1
}
But, when I use this db.collection.find({$text: {$search : "b"}}); to search, it does not work. It just searches at the first level.
I cannot do a search with precision, because the dimensions of the objects/arrays is dynamic and can grow or shrink at any time.
I appreciate your answers.
MongoDB cannot build an index on arbitrarily-nested objects. The index only occurs for the depth specified. In your case, the $text search will only check the top-level name field, but not the name field for any of the nested sub-documents. This is an inherent limitation for indexing.
To my knowledge, MongoDB has no support for handling these kinds of deeply-nested data structures. You really need to break your data out into separate documents in order to handle it correctly. For example, you could break it out into the following:
[
{
"_id": 0,
"name": "a",
"root_id": null,
"parent_id": null
},
{
"_id": 1,
"name": "b",
"root_id": 0,
"parent_id": 0
},
{
"_id": 2,
"name": "c",
"root_id": 0,
"parent_id": 1
},
{
"_id": 3,
"name": "d",
"root_id": 0,
"parent_id": 2
},
{
"_id": 4,
"name": "e",
"root_id": 0,
"parent_id": 2
},
{
"_id": 5,
"name": "f",
"root_id": 0,
"parent_id": 4
}
]
In the above structure, our original query db.collection.find({$text: {$search : "b"}}); will now return the following document:
{
"_id": 1,
"name": "b",
"root_id": 0,
"parent_id": 0
}
From here we can retrieve all related documents by retrieving the root_id value and finding all documents with an _id or root_id matching this value:
db.collection.find({
$or: [
{_id: 0},
{root_id: 0}
]
});
Finding all root-level documents is a simple matter of matching on root_id: null.
The drawback, of course, is that now you need to assemble these documents manually after retrieval by matching a document's parent_id with another document's _id because the hierarchical information has been abstracted away. Using a $graphLookup could help alleviate this somewhat by matching each subdocument with a list of ancestors, but you would still need to determine the nesting order manually.
Regardless of how you choose to structure your documents moving forward, this sort of restructure is going to be needed if you're going to query on arbitrarily-nested content. I would encourage you to consider different possibilities and determine which is most suited for your specific application needs.
Put my question more clear: I am using mongodb, each document in the collection looks like:
{
u'cell_latlng': {
u'type': u'Polygon',
u'coordinates': [
[
[
...
],
[
...
],
[
...
],
[
...
]
]
]
},
u'self': {
u'Bus Line': 1,
u'Meeting Room': 1,
u'Neighborhood': 1,
u'Home (private)': 1,
u'Food': 1,
u'High School': 2,
u'Elementary School': 1,
u'Beach': 1,
u'Other Repair Shop': 1
},
u'_id': ObjectId('5545bed4e139c7dcde660f5a'),
u'point_latlng': {
u'type': u'Point',
u'coordinates': [
...
]
},
u'area': 100
}
Now I need to find a specific doc in the collection (the collection is called citymap). I know the value of the field 'self' and the value is called feature[0], so I just use:
results = citymap.find({'self':feature[0]})
However,
results.count() == 0
I check the feature[0] again and I am sure it is:
feature[0] = {u'Bus Line': 1, u'Meeting Room': 1, u'Neighborhood': 1, u'Home (private)': 1, u'Food': 1, u'High School': 2, u'Elementary School': 1, u'Beach': 1, u'Other Repair Shop': 1}
which is exactly the same as that in the doc, so why cannot I find this doc using this sentence?
Great thanks to any help
Let's check this script. Suppose your feature object like this
var feature = {u'Bus Line': 1, u'Meeting Room': 1, u'Neighborhood': 1, u'Home (private)': 1, u'Food': 1, u'High School': 2, u'Elementary School': 1, u'Beach': 1, u'Other Repair Shop': 1}
Iterate over feature object and append every keys as self and push them into new object. Check below function:
var appendSelf = {};
function appendData(data) {
for (var k in data) {
if (data.hasOwnProperty(k)) {
appendSelf["self.".concat(k)] = data[k]; //append self to every keys and set it's respective values
}
}
}
Now call this function as appendData(feature) on console if you type appendSelf then it shows object with appending self like this
> appendSelf
{
"self.u'Bus Line": 1,
"self.u'Meeting Room": 1,
"self.u'Neighborhood": 1,
"self.u'Home (private)": 1,
"self.u'Food": 1,
"self.u'High School": 4,
"self.u'Elementary School": 1,
"self.u'Beach": 1,
"self.u'Other Repair Shop": 1
}
So use this appendSelf in find criteria as citymap.find(appendSelf[0]).pretty() it show expected result.
H,
I'm trying to update the version field in this object but I'm not able to make a query with 2 nested $match. So what I would like to do is get the record with file id 12 and version 1.
I would ask also if is it a good practice have more the one nested array in mongoDB (like this object)...
Query:
db.collection.find({"my_uuid":"434343"},{"item":{$elemMatch:{"file_id":12,"changes":{$elemMatch:{"version":1}}}}}).pretty()
Object:
{
"my_uuid": "434343",
"item": [
{
"file_id": 12,
"no_of_versions" : 1,
"changes": [
{
"version": 1,
"commentIds": [
4,
5,
7
]
},
{
"version": 2,
"commentIds": [
10,
11,
15
]
}
]
},
{
"file_id": 234,
"unseen_comments": 3,
"no_of_versions" : 2,
"changes": [
{
"version": 1,
"commentIds": [
100,
110,
150
]
}
]
}
]
}
Thank you
If you want the entire documents that satisfy the criteria returned in the result, then I think it's fine. But if you want to limit the array contents of item and changes to just the matching elements, then it could be a problem. That's because, you'll have to use the $ positional operator in the projection to limit the contents of the array and only one such operator can appear in the projection. So, you'll not be able to limit the contents of multiple arrays within the document.