I created MongoDB river with index in Elasticsearch, but later on noticed that I don't need several fields and I want to use different username/password for river.
How to:
1.Initialize update of river settings for new username/password?
2.Delete/exclude useless fields from index from Elasticsearch without rebuilding whole index from the beginning?
I have like 20-30GB of indexed data and whole process of getting data through river may take long hours.
All I found is Delete and PUT, but there's no update for index fields or river mentioned either in docs or in google.
It's not currently possible to remove a field from a mapping. In order to remove all values of a field from all documents, you need to reindex all documents with this field removed.
Related
I'm reading mosql code and how it uses oplog collection from mongodb to copy to postgresql.
For updating case, for example:
I saw that mosql always update the whole document to postgresql instead of only the modified fields. That is really weird because there is not sense to update all the fields in postgresql table when I want to update only 1 or 2 fields. That is a problem because I'm using bigger documents.
Looking the code I saw that mosql uses the o field from oplog but it keeps the whole document and that is why mosql update all the fields in postgresql and there is not way to know what fields were updated.
Is there a way to figure out what fields were updated? to updating only that fields instead of the complete document?
What happens when I add a new document to my collection and run the createIndex() function. Does MongoDB build a new index over the whole collection (time and resource consuming)? Or is MongoDB just updating the index for the single document (time and resource saving)? I am not sure because I found this in the documentation (3.0):
By default, MongoDB builds indexes in the foreground, which prevents all read and write operations to the database while the index builds. Also, no operation that requires a read or write lock on all databases (e.g. listDatabases) can occur during a foreground index build.
I am asking because I need a dynamic 2dsphere index which will be updated continuous. If it needs to build the index everytime over the whole collection it would take too much time.
Building an index indexes all existing documents and also cause all future documents to get indexed as well. So if you insert new documents into an already indexed collection (or update the indexed fields of existing documents), the indexes will get updated.
When you call createIndex and an index of the same type over the same fields already exists, nothing happens.
In Zend_Service_Solr I can add or delete a record.
$solr->addDocument($document);
Is there any way that I can update a record. I couldn't find any document for that. Or is there any extension for doing the same.
In most cases updating a document in Solr is to add the same document again (with the same value for the uniqueKey field).
It's possible to perform certain updates in more recent versions of Solr, but these require all fields to be stored (so that the document can just be re-added internally) and a custom update syntax. There are also some work in progress with non-textual DocValues being updatable without having to resubmit the complete document, but this is currently not in any released version of Solr.
The best way to handle this is usually to just re-submit the document with updated values, and have a straight forward way of doing that in your application code.
My problem is with Elasticasearch, I have 1564 indexes and 1564 documents in MongoDB (after my last populating operation : in Symfony with Elasticabundle :php app/console foqs:elastica:populate)
but when I add a document manually the number of indexes remains 1564 where it should be 1565
Did I miss something ?
The functionality to update Elasticsearch indexes when Doctrine entities are modified is documented in the README file under Realtime, selective index update. The configuration option is listeners, which falls under the persistence option you should already have defined per model.
In the project I've been working on, I've added a new field - an embedded document. Will this addition affect data prior to the change? From what I've read this shouldn't affect prior data, and this is actually one of the benefits of using MongoDB. Its just the previous data won't have that field.
Thanks!
No it won't affect your previous documents - each document in your collection can have it's own unique fields if you want. It is up to you to handle this at application level.