Mongodb: Querying through multiple relatively unknown keys for a value - mongodb

Question

Right now, we are using mongodb 1.2.2 to create a database and store values. Our data types look like this:
"file" : "1" , "tools": { "foo": { "status": "pending"} }
"file" : "2" , "tools": { "bar": { "status": "pending" } }
"file" : "3" , "tools": { "foo": { "status": "running" } }
"file" : "4" , "tools": { "bar": { "status": "done" } }
"file" : "5" , "tools": { "foo": { "status": "done" } }
We want to query for every single one that has { "status" : "pending" }. We do not want to use {"tools.foo.status" : "pending"} because we will have many different variations other than foo and bar. To make it more clear we want to do something like this {"tools.*.status" : "pending"}

Answer 1

No, you can't do that. I'm afraid you'll have to maintain your own index for this. That is, for every insert/update to the files collection, do an upsert to the file_status_index collection to update current status.
Querying is also a two-step process: first query the index collection to get the ids, and then issue $in query to the files collection to get actual data.
This may sound scary, but that's a price you have to pay with this schema.

Answer 2

Firstly, you should upgrade your MongoDB. 1.2.2 is really an old version.
Secondly, you cannot do query you ask. You can do this with the Map/Reduce.

Answer 3

I think it's time to ask why you're storing things the way you are.
There is no efficient way to search this kind of structure; since there is no known keys-only path to get to the value you're filtering on, every single record needs to be expanded every single time, and that's very expensive, especially once your collection no longer fits in RAM.
IMO, you'd be better off with a secondary collection to hold these statuses. Yes, it makes your datastore more relational, but that's because your data is relational.
file_tools:
{ 'file_id' : 1, 'name' : 'foo', 'status' : 'pending' }
{ 'file_id' : 2, 'name' : 'bar', 'status' : 'pending' }
{ 'file_id' : 3, 'name' : 'foo', 'status' : 'running' }
{ 'file_id' : 4, 'name' : 'foo', 'status' : 'done' }
{ 'file_id' : 5, 'name' : 'foo', 'status' : 'done' }
files:
{ 'id': 1 }
{ 'id': 2 }
{ 'id': 3 }
{ 'id': 4 }
{ 'id': 5 }
> // find out which files have pending tools
> files_with_pending_tools = file_tools.find( { 'status' : 'pending' }, { 'file_id' : 1 } )
> //=> [ { 'file_id' : 1 }, { 'file_id' : 2 } ]
>
> // just get the ids
> file_ids_with_pending_tools = files_with_pending_tools.map( function( file_tool ){
> file_tool['file_id']
> })
> //=> [1,2]
>
> // query the files
> files.find({'id': { $in : file_ids_with_pending_tools }})
> //=> [ { 'id' : 1 }, { 'id' : 2 } ]