I recently saw this error in a Mongo 2.6 replicaset:
WARNING: the collection 'mydatabase.somecollection' lacks a unique index on _id. This index is needed for replication to function properly.
I assumed the _id index would be unique by default. But I am trying to check / set it. getIndexes shows there is no unique option set.
> db.somecollection.getIndexes()[0]
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "mydatabase.somecollection",
"name" : "_id_"
}
> db.somecollection.ensureIndex({"_id":1},{unique:true})
> { "numIndexesBefore" : 3, "note" : "all indexes already exist", "ok" : 1 }
> db.somecollection.getIndexes()[0]
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "mydatabase.somecollection",
"name" : "_id_"
}
I have tried .validate(true):
...
"valid" : true,
"errors" : [ ],
"ok" : 1
}
and also .reIndex() that runs without error. I am unable to remove the _id index to recreate it - how can I set the index to unique or what should I do to ensure data consistency in the RS? Note the RS was upgraded as per upgrade instructions from 2.2 --> 2.4 --> 2.6. I have found this MongoDB - Collection lacks a unique index on _id but there is nothing that resolves my issue in there.
I have seen this in the past when a new member to the replica set was added with a different Compatibility Version. Run db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } ) on all of your nodes in the replica set and if one is different, stop the replication on that node, change the CompatibilityVersion and then ready it to the replica.
So it turns out that the error came up when a new member was added to the existing replica set, and was only shown on that member. If I connect to the database and try add a duplicate _id, I get the usual E11000 duplicate key error index: ... even though getIndexes() doesn't indicate the unique constraint on the index (assuming it is implicit).
Related
I have an index which i need to modify.
{
"v" : 1,
"key" : {
"expectedDateTime" : 1
},
"name" : "expectedDateTime_1",
"ns" : "expectation.expectation_data",
"expireAfterSeconds" : 43200
}
The expireAfterSeconds is incorrect and needs to be changed to 432000.
When I dropped the index it seemed fine
db.expectation_data.dropIndex({"expectedDateTime":1})
{ "nIndexesWas" : 4, "ok" : 1 }
The getIndexes() shows that the index does not exist.
Then when i try to recreate the index i get this error
db.expectation_data.createIndex({"expectedDateTime":1},
{expireAfterSeconds:432000,name:"expectedDateTime"});
{
"ok" : 0,
"errmsg" : "Index with name: expectedDateTime already exists with different options",
"code" : 85
}
Now on running getIndexes(), i see that the index seems to have recreated with the old TTL. I tried repeating this process multiple times, but ran into the same issue again and again.
I cannot find any documentation which says that i cannot recreate an index of the same name. If i use a different name it works fine
db.expectation_data.createIndex({"expectedDateTime":1}, {expireAfterSeconds:432000});
.
.
>db.expectation_data.getIndexes()
.
.
{
"v" : 1,
"key" : {
"expectedDateTime" : 1
},
"name" : "expectedDateTime_1",
"ns" : "expectation.expectation_data",
"expireAfterSeconds" : 432000
}
Is there any restriction on recreating indexes with the same name ?
This looks like the index is recreated automatically after deletion. Make sure that no applications using ensureIndex or #Index-Annotations are connecting to the database.
As it turned out. This was due to an #Index annotation used in the entity with the old timeout. The application was still running when i made the index changes.
When i stopped the application, i was able to create the index as i originally expected
My query is failing to find all matching results. If I add an additional _id parameter to a specific matching example, I am getting results
> db.reviews.count({"contentProvider":"GLORP", "responses.0": {$exists: true}})
0
> db.reviews.count({_id: "1234", "contentProvider":"GLORP", "responses.0": {$exists: true}})
1
the first query is using index:
"indexName" : "contentProvider_1_reviewDetail_1_reviewerUserName_1_providerReviewId_1",
and the query with the _id is of course using the _id_ index:
"indexName" : "_id_"
Here is the index in question:
{
"v" : 1,
"key" : {
"contentProvider" : 1,
"reviewDetail" : 1,
"reviewerUserName" : 1,
"providerReviewId" : 1
},
"name" : "contentProvider_1_reviewDetail_1_reviewerUserName_1_providerReviewId_1",
"ns" : "test.reviews",
"background" : true
}
Using mongodb version 3.2.3
Is the index corrupted? Will dropping it and readding it likely fix the problem?
It's possible and you could certainly try it, however without knowing what version of MongoDB you are using and without seeing the index definition I cannot say for certain.
There are multiple different types of indexes as well as index properties like: sparse or partial that can change behavior and may explain why the index doesn't return the results you expect.
I'd recommend checking the index first and see if the index definition has any properties that would result in the document being excluded.
If not then you can always drop the index and recreate it.
I created mongo db collection index using java code
dbCollection.createIndex("accountNumber");
When i see indices using
db.accounts.getIndexes()
I am getting the index name as "accountNumber_1"
How to get the index name also same as document field? or how to give index name?
Is naming indices important or i can ignore this?
When we create index on the document users
> db.users.createIndex({name: 1})
{
"ok" : 0,
"errmsg" : "Index with name: name_1 already exists with different option
s",
"code" : 85
}
the name: name_1 is returned, then we can get the index through getIndexes()
> db.users.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "test.users"
},
{
"v" : 1,
"unique" : true,
"key" : {
"name" : 1
},
"name" : "name_1",
"ns" : "test.users",
"background" : true,
"safe" : null
}
]
We know, the name_1 is just the value of index name. and the key name is used to create index for document users. I think the name_1 is the value of name to meet BSON structure. We can ignore it...
You can create index with the name you wanted using the other variant of createIndex method, refer java API here.
public void createIndex(DBObject keys,
DBObject options)
Creates an index on the field specified, if that index does not already exist.
Prior to MongoDB 3.0 the dropDups option could be used with unique indexes allowing documents with duplicate values to be dropped when building the index. Later versions of MongoDB will silently ignore this setting.
Parameters:
keys - a document that contains pairs with the name of the field or fields to index and order of the index
options - a document that controls the creation of the index.
MongoDB documentation
Index Creation Tutorials
You can corresponding mongodb documentation here.
Basically the second parameter 'options' contain an option to supply index name explicitly.
I had executed this command to set a TTL Index on mongodb,
db.sessions.ensureIndex({'expiration':1},{"expireAfterSeconds" : 30})
but after 4 days,I found these documents were not removed.
I had confirmed command and document's field was correct.
I don't know how to fix it.
after executed db.serverStatus(), I got
localTime is 2015-01-16 11:03:05.554+08:00
and the following is some info of my collection
db.sessions.getIndexes()
{
"0" : {
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "meta.sessions"
},
"1" : {
"v" : 1,
"key" : {
"expiration" : 1
},
"name" : "expiration_1",
"ns" : "meta.sessions",
**"expireAfterSeconds" : 30**
}
}
db.sessions.find()
/* 0 */
{
"_id" : ObjectId("54b4c2e0f840238ca1436788"),
"data" : ...,
"expiration" : **ISODate("2015-01-13T16:02:33.947+08:00"),**
"sid" : "..."
}
/* 1 */
{
"_id" : ObjectId("54b4c333f840238ca1436789"),
"data" : ...,
"expiration" : ISODate("2015-01-13T16:06:56.942+08:00"),
"sid" : ".."
}
/* ... */
To expire data from a collection (Tested in version 3.2) you must create indexes:
db.my_collection.createIndex( { "createdAt": 1 }, { expireAfterSeconds: 3600 } )
After that, every document that you insert in this collection must have the "createdAt" with the current date:
db.my_collection.insert( {
"createdAt": new Date(),
"dataExample": 2,
"Message": "Success!"
} )
The document will be removed when the date createdAt value + expireAfterSeconds value arrive.
Note: This background task in MongoDB, by default, happens once every 60 seconds.
When you create TTL index in the foreground (like you did), MongoDB begins removing expired documents as soon as the index finishes building. Best to tail -f mongod.log during index creation to track the progress. You may wish to remove & recreate index if something went wrong.
If index was created in the background, the TTL thread can begin deleting documents while the index is building.
TTL thread that removes expired documents runs every 60 seconds.
If you created index on the replica that was taken out of the replica set and is running in standalone mode index WILL be created but documents will NOT be removed until you rejoin (or remove replica set) configuration. If this is the case you may get something similar to this in the mongod.log
** WARNING: mongod started without --replSet yet 1 documents are
** present in local.system.replset
** Restart with --replSet unless you are doing maintenance and no other
** clients are connected.
** The TTL collection monitor will not start because of this.
** For more info see http://dochub.mongodb.org/core/ttlcollections
I'd like to make sharding of my existing users collection. Users collection has already single ascending index by default {"_id" : 1}. I want to convert this index to "hashed" and to shard based on this hashed key according to the documentation:
I've tried "brute-force" solution to delete default index and then recreate it with "hashed" parameter but it doesn't allow to do that.
UPDATE: I've also tried db.users.ensureIndex({_id: "hashed"}). But after I run this command nothing really happens.
switched to db bg_shard_single
mongos> db.users.ensureIndex({_id:"hashed"});
mongos> db.users.getIndexes();
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "bg_shard_single.users",
"name" : "_id_"
}
]
It does not allow you to do so because you can not create an index from _id field. Instead of this you can do something like this db.collection.ensureIndex( { _id: "hashed" } ) to create a hashing index on this field.
Then you will see "name" : "_id_hashed" as your hashed index which you can use for sharding purposes later.
I've found what was the problem. Apparently, I was using the old version of mongodb. That's why mongos didn't want me to update '_id' to "hashed". After I've updated to 2.4.8 as #Salvador-Dali mentions it becomes "name" : "_id_hashed".