MongoDB - Execute a command conditionally - mongodb

I need to execute a command that is part of my migration scripts (mongock java library) conditionally based on Mongo version
In case Mongo version is lower then 4.2 I need to execute this command. For Mongo 4.2 and higher it should not be executed.
db.getSiblingDB('admin').runCommand( { setParameter: 1, failIndexKeyTooLong: false } )
Is there a way how to do it?
Thank you,
Lukas

don't know if you already found a solution for this.
The idea is to figure out the Mongo's version inside the changeSet and then perform the required actions.
I will assume you are using the last Mongock's release for version 3.x.x, which is 3.3.2 or 4.0.x.alpha, so you should use MongockTemplate, instead of MongoTemplate. Don't worry, it's just a wrapper/decorator, you won't miss anything from the original API.
So you can do something like this:
#ChangeSet(id = "changeset-1", order = "001", author = "mongock")
public void changeSet(MongockTemplate mongockTemplate, ClientRepository clientRepository) {
String version = mongockTemplate.executeCommand(new Document("buildinfo", new BsonString("")))
.get("version")
.toString();
if(getNumericalVersion(version) < 42) {
//do your actions
} else {
//do you other actions
}
}
NOTE 1: We don't recommend using executeCommand. In this case, for taking the version it's fine, but in general we recommend not using it, specially for writes.
NOTE 2: Currently we are working on version 4, which will allow to scan multiple packages and even isolated ChangeLog classes. So another approach would be to take the version in the bean creation and inject the change or not depending on the MongoDB's version

Related

Wagtail 3.x postgres search returns no results

I recently updated from Wagtail 2.13.5 to 3.0.3. After the update, the search() method on Wagtail's PageQuerySet returns no results for search terms that clearly should return results (and which do when using the older version).
For example, under 2.13, where search_backend is an instance of wagtail.contrib.postgres_search.backend.PostgresSearchBackend, and qs is an instance of wagtail.core.query.PageQuerySet the following returns lots of results:
search_backend.search('popular-search-term', qs, fields=None, operator=None, order_by_relevance=True, partial_match=True)
But under 3.0.3, where search_backend is now an instance of wagtail.search.backends.database.postgres.postgres.PostgresSearchBackend and qs is an instance of wagtail.query.PageQuerySet, the same call to search() will return nothing (an empty queryset).
The data in the qs queryset is the same in both cases, so maybe I'm missing something in my configuration of the search backend? My "settings.py" file has:
WAGTAILSEARCH_BACKENDS = {
'default': {
'BACKEND': 'wagtail.search.backends.database',
'SEARCH_CONFIG': 'english',
},
}
and
INSTALLED_APPS = [
...
'wagtail.search',
'wagtail.search.backends.database.postgres',
...
]
I had to guess at the value for 'wagtail.search.backends.database.postgres'. AFAICT, Wagtail's docs don't mention what should go into INSTALLED_APPS. But the pre-upgrade value of 'wagtail.contrib.postgres_search' would clearly be wrong, as that module has been removed.
Anyone got an idea why calling search() on a PageQuerySet would incorrectly return no results?
The steps for changing the search backend are documented at https://docs.wagtail.org/en/stable/releases/2.15.html#database-search-backends-replaced. In particular:
You should remove wagtail.contrib.postgres_search from INSTALLED_APPS, and do not need to add anything in its place - the existing wagtail.search app is sufficient
After changing over, you need to re-run the ./manage.py update_index command to ensure that the new search index is populated.

How to set the offset.commit.policy to AlwaysCommitOffsetPolicy in debezium?

I created a Debezium Embedded engine to capture MySQL change data. I want to commit the offsets as soon as I can. In the code, the config is created including follows.
.with("offset.commit.policy",OffsetCommitPolicy.AlwaysCommitOffsetPolicy.class.getName())
Running this returns, java.lang.NoSuchMethodException: io.debezium.embedded.spi.OffsetCommitPolicy$AlwaysCommitOffsetPolicy.<init>(io.debezium.config.Configuration)
However, When I start the embedded engine with,
.with("offset.commit.policy",OffsetCommitPolicy.PeriodicCommitOffsetPolicy.class.getName()), the embedded engine works fine.
Note that the class OffsetCommitPolicy.PeriodicCommitOffsetPolicy constructor includes the config parameter while OffsetCommitPolicy.AlwaysCommitOffsetPolicy doesn't.
public PeriodicCommitOffsetPolicy(Configuration config) {
...
}
How to get the debezium embedded engine to use its AlwaysCommitOffsetPolicy?
Thanks for the report. This is partly bug (which we would appreciate if you could log into our Jira). You can solve this issue by calling a dedicated method embedded engine builder like `io.debezium.embedded.EmbeddedEngine.create().with(OffsetCommitPolicy.always())'
Tested with version 1.4.0Final:
new EmbeddedEngine.BuilderImpl() // create builder
.using(config) // regular config
.using(OffsetCommitPolicy.always()) // explicit commit policy
.notifying(this::handleEvent) // even procesor
.build(); // and finally build!

Create mongo change stream in the mongo shell

MongoDB introduced change streams in their 3.6 release.
I wanted to implement mongo change stream in my code and wanted to understand how it works. I will implement using the java driver and it's pretty clear.
But I wanted to know if there is there any way to open a change stream on in the mongo shell? Couldn't find much resources on that.
The db.collection.watch command opens a change stream cursor.
For example:
watchCursor = db.getSiblingDB("data").sensors.watch(
[
{ $match : {"operationType" : "insert" } }
]
)
while (!watchCursor.isExhausted()){
if (watchCursor.hasNext()){
print(tojson(watchCursor.next()));
}
}
Plenty more detail in the docs.

Meteor.subscribe on server side

I want to create a backend service which monitors a mongodb collection for new entries. As those are being created, I wish to run processing and update them.
I thought doing so with a Meteor service/app would be a wise idea because Meteor uses 'oplog tailing' which seems ideal for this purpose (I'd rather avoid polling if possible).
As such, I figured creating a minimal server-side-only app should solve it.
So basically, I need something along these lines:
if (Meteor.isServer) {
MyCollection = new Mongo.Collection('myCollection');
Meteor.publish('myCollectionPub', function () {
return MyCollection.find({ some: criteria... });
}
// is there such a thing?
Meteor.serverSideSubscribe('MyCollectionPub',
function (newDocs) {
// process/update newDocs
});
}
According to the Meteor docs, I cannot use Meteor.subscribe() on the server (and indeed it crashes if I try).
Question is:
Are there ways of 'subscribing' to collection updates on the server?
The PeerLibrary server-autorun package (along with it's dependant, reactive-mongo) will provide you with easy server-side observation of collections.
An alternative to #tarmes suggestion is the collection-hooks package, however as pointed out by David Weldon, it will only trigger in instance it is run in:
https://github.com/matb33/meteor-collection-hooks
MyCollection.after.insert(function (userId, doc) {
// ...
});
If you need it to run even when another instance makes a change in the mongo database, you can observe a cursor that is returned from your collection:
MyCollection.find({created_at : {$gt: some_current_time}}).observe({
added: function(item) {
// Alert code
}
});

How to change settings of an index after creation in elastic4s?

I'd need to disable index refreshing for the course of a batch indexing (of gigabytes) and set it back again after it's done. But from the source code of elastic4s I can't find a way to do it other than at index creation time... Is it possible? Or is there a workaround for this?
In java client :
client
.admin
.indices()
.prepareUpdateSettings()
.setSettings(settings)
.setIndices(indexName)
.execute()
.actionGet()
Natively :
curl -XPUT 'localhost:9200/my_index/_settings' -d '
{
"index" : {
"refresh_interval" : -1
}
}
'
This is how you do it in elastic4s (for example setting the refresh interval property).
client.execute {
update settings "myindex" set Map("index.refresh_interval" -> "10s")
}
Note: Not all settings can be changed at runtime or after the index is created.
Note 2: I have added this API in response to your question, and is only available in version 1.5.1 onwards.
Note 3: I could back port it to 1.4.x or 1.3.x if required by anyone.