Geospatial index in MongoDB makes no difference to performance - mongodb

Question

I'm trying to find which documents are geo-located within a given rectangle. I have a Mongo collection looking a bit like:
{
...
"metadata" : {
...
"geometry" : { "type" : "Point", "coordinates" : [ -0.000, 51.477 ] }
}
}
And my query looks like:
db.my_coll.find({ "$query" : {
"metadata.geometry" : {
"$geoIntersects" : {
"$geometry" : { "type" : "Polygon", "coordinates" : [ [ [..., ...], ... ] ] }
} } }, "$explain":1})
With no geospatial index I get:
{
"cursor" : "BasicCursor",
"isMultiKey" : false,
"n" : 646,
"nscannedObjects" : 19539,
"nscanned" : 19539,
"nscannedObjectsAllPlans" : 19539,
"nscannedAllPlans" : 19539,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 152,
"nChunkSkips" : 0,
"millis" : 125,
...
With the geospatial index db.my_coll.ensureIndex({"metadata.geometry" : "2dsphere"}); I get:
{
"cursor" : "BtreeCursor metadata.geometry_2dsphere",
"isMultiKey" : false,
"n" : 646,
"nscannedObjects" : 18726,
"nscanned" : 18727,
"nscannedObjectsAllPlans" : 18726,
"nscannedAllPlans" : 18727,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 146,
"nChunkSkips" : 0,
"millis" : 161,
...
i.e.it's slower with the index when explaining. Querying from an outside application shows no significant difference in query time with or without the index (ms resolution). What am I doing wrong? Shouldn't the index make the query rather faster than this?
Thanks :-)