When connecting to a MongoDB replication cluster, I want to know which node the query runs on.
I tried to use explain() in mongo shell, but the Java driver doesn't seem to support this command.
How can I achieve this using MongoDB Java Driver?
You can try utilizing Command Monitoring from MongoDB Java Driver;
public class CustomCommandListener implements CommandListener {
#Override
public void commandStarted(final CommandStartedEvent event) {
System.out.println(String.format("Sent command '%s:%s' with id %s to database" +
" '%s' on connection '%s' to server '%s'",
event.getCommandName(),
event.getCommand().get(event.getCommandName()),
event.getRequestId(),
event.getDatabaseName(),
event.getConnectionDescription().getConnectionId(),
event.getConnectionDescription().getServerAddress()));
}
#Override
public void commandSucceeded(CommandSucceededEvent event) {
//ignore
}
#Override
public void commandFailed(CommandFailedEvent event) {
//ignore
}
}
There are also CommandSucceededEvent and CommandFailedEvent, but regardless of its result, you can get some details with the CommandStartedEvent as above.
Then pass this custom listener into your MongoClient settings;
ClusterSettings clusterSettings = ClusterSettings.builder().hosts(hostList).build();
MongoClientSettings settings = MongoClientSettings.builder()
.addCommandListener(new CustomCommandListener())
.clusterSettings(clusterSettings)
// other settings
.build();
MongoClient client = MongoClients.create(settings);
More on MongoDB Java Driver docs
Related
Environment: I am running Apache Ignite v2.13.0 for the cache and the cache store is persisting to a Mongo DB v3.6.0. I am also utilizing Spring Boot (Java).
Question: When I have an expiration policy set, how do I remove the corresponding data from my persistent database?
What I have attempted: I have attempted to utilize CacheEntryExpiredListener but my print statement is not getting triggered. Is this the proper way to solve the problem?
Here is a sample bit of code:
#Service
public class CacheRemovalListener implements CacheEntryExpiredListener<Long, Metrics> {
#Override
public void onExpired(Iterable<CacheEntryEvent<? extends Long, ? extends Metrics>> events) throws CacheEntryListenerException {
for (CacheEntryEvent<? extends Long, ? extends Metrics> event : events) {
System.out.println("Received a " + event);
}
}
}
Use Continuous Query to get notifications about Ignite data changes.
ExecutorService mongoUpdateExecutor = Executors.newSingleThreadExecutor();
CacheEntryUpdatedListener<Integer, Integer> lsnr = new CacheEntryUpdatedListener<Integer, Integer>() {
#Override
public void onUpdated(Iterable<CacheEntryEvent<? extends Integer, ? extends Integer>> evts) {
for (CacheEntryEvent<?, ?> e : evts) {
if (e.getEventType() == EventType.EXPIRED) {
// Use separate executor to avoid blocking Ignite threads
mongoUpdateExecutor.submit(() -> removeFromMongo(e.getKey()));
}
}
}
};
var qry = new ContinuousQuery<Integer, Integer>()
.setLocalListener(lsnr)
.setIncludeExpired(true);
// Start receiving updates.
var cursor = cache.query(qry);
// Stop receiving updates.
cursor.close();
Note 1: EXPIRED events should be enabled explicitly with ContinuousQuery#setIncludeExpired.
Note 2: Query listeners should not perform any heavy/blocking operations. Offload that work to a separate thread/executor.
I have a springboot with springdata mongodb application where I am connecting to mongo change stream to save the changes to a audit collection. My application is running multiple instances (2 instances) and will be scaled up to n number instances when the load increased. When records are created in the original collection (“my collection”), the listeners will be triggered in all running instances and creates duplicate records. Following is my setup
build.gradle
…
// spring data mingodb version 3.1.5
implementation 'org.springframework.boot:spring-boot-starter-data-mongodb'
…
Listener config
#Configuration
#Slf4j
public class MongoChangeStreamListenerConfig {
#Bean
MessageListenerContainer changeStreamListenerContainer(
MongoTemplate template,
PartyConsentAuditListener consentAuditListener,
ErrorHandler errorHandler) {
MessageListenerContainer messageListenerContainer =
new MongoStreamListenerContainer(template, errorHandler);
ChangeStreamRequest<PartyConsentEntity> request =
ChangeStreamRequest.builder(consentAuditListener)
.collection("my-collection")
.filter(newAggregation(match(where("operationType").in("insert", "update", "replace"))))
.fullDocumentLookup(FullDocument.UPDATE_LOOKUP)
.build();
messageListenerContainer.register(request, MyEntity.class, errorHandler);
log.info("mongo stream listener is registered");
return messageListenerContainer;
}
#Bean
ErrorHandler getLoggingErrorHandler() {
return new ErrorHandler() {
#Override
public void handleError(Throwable throwable) {
log.error("error in creating audit records {}", throwable);
}
};
}
}
Listener container
public class MongoStreamListenerContainer extends DefaultMessageListenerContainer {
public MongoStreamListenerContainer(MongoTemplate template, ErrorHandler errorHandler) {
super(template, Executors.newFixedThreadPool(15), errorHandler);
}
#Override
public boolean isAutoStartup() {
return true;
}
}
ChangeListener
#Component
#Slf4j
#RequiredArgsConstructor
/**
* This class will listen to mongodb change stream and process changes. The onMessage will triggered
* when a record added, updated or replaced in the mongo db.
*/
public class MyEntityAuditListener
implements MessageListener<ChangeStreamDocument<Document>, MyEntity> {
#Override
public void onMessage(Message<ChangeStreamDocument<Document>, MyEntity > message) {
var update = message.getBody();
log.info("db change event received");
if (update != null) {
log.info("creating audit entries for id {}", update.getId());
// This will execute in all the instances and creating duplicating records
}
}
}
Is there a way to control the execution on one instance at a given time and share the load between nodes?. It would be really nice to know if there is a config from spring data mongodb to control the flow.
Also, I have checked the following post in stack overflow and I am not sure how to use this with spring data.
Mongo Change Streams running multiple times (kind of): Node app running multiple instances
Any help or tip to resolve this issue is highly appreciated. Thank you very much in advance.
I am currently running into an issue while creating a reactive mongoclient when I provide the URL with ssl=true option.
I am creating configuration class in spring boot where I create Reactive mongoclient using the following option:
MongoClients.create(Connections ring Conn)
Here when I try to connect to a DB with no ssl settings it works, but with ssl enabled option I am getting error saying NettyEventLoop class is not found.
Can anyone suggest what I can do to fix this issue
It seems that the API has changed, so since MongoDB driver v3.8, the method is "applyToSslSettings":
import com.mongodb.Block;
import com.mongodb.connection.SslSettings;
import com.mongodb.connection.SslSettings.Builder;
import com.mongodb.connection.netty.NettyStreamFactoryFactory;
import io.netty.channel.nio.NioEventLoopGroup;
#Configuration
public class Config {
private NioEventLoopGroup eventLoopGroup = new NioEventLoopGroup();
#Bean
public MongoClientSettingsBuilderCustomizer sslCustomizer() {
Block<SslSettings.Builder> sslSettingsBlock = new Block<SslSettings.Builder>() {
#Override
public void apply(Builder t) {
t.applySettings(SslSettings.builder()
.enabled(true)
.invalidHostNameAllowed(true)
.build());
}
};
return clientSettingsBuilder -> clientSettingsBuilder
.applyToSslSettings(sslSettingsBlock)
.streamFactoryFactory(NettyStreamFactoryFactory.builder()
.eventLoopGroup(eventLoopGroup).build());
}
#PreDestroy
public void shutDownEventLoopGroup() {
eventLoopGroup.shutdownGracefully();
}
}
I was able to overcome this issue by configuring MongoClientSettingsBuilderCustomizer and NioEventLoop Group.
Please find below the code:
private NioEventLoopGroup eventLoopGroup = new NioEventLoopGroup();
#Bean
public MongoClientSettingsBuilderCustomizer sslCustomizer() {
return clientSettingsBuilder -> clientSettingsBuilder
.sslSettings(SslSettings.builder()
.enabled(true)
.invalidHostNameAllowed(true)
.build())
.streamFactoryFactory(NettyStreamFactoryFactory.builder()
.eventLoopGroup(eventLoopGroup).build());
}
I am experimenting with Dropwizard (https://github.com/robertkuhar/dropwiz_get_start, https://github.com/robertkuhar/dropwiz_mongo_demo ) and am impressed with how easy it is to integrate with my IDE. To start my dropwizard app, I simply find the class with the main method and "Debug As...Java Application" and I'm on my way. Stopping the application is equally simple, just click the red "Terminate" button from the Debug view. I noticed, however, that I don't make it to the breakpoints in the stop() method of my Managed classes when I stop it in this manner.
How do I get Dropwizard to go through graceful shutdown when its running directly in eclipse's Debugger?
#Override
public void run( BlogConfiguration configuration, Environment environment ) throws Exception {
...
MongoManaged mongoManaged = new MongoManaged( mongo );
environment.manage( mongoManaged );
...
}
Breakpoints in the stop() of MongoManage never get hit.
public class MongoManaged implements Managed {
private final MongoClient mongo;
public MongoManaged( MongoClient mongo ) {
this.mongo = mongo;
}
#Override
public void start() throws Exception {
}
#Override
public void stop() throws Exception {
if ( mongo != null ) {
mongo.close();
}
}
}
Does it help if you use this java feature?
// In case vm shutdown
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run()
{
// what should be closed if forced shudown
// ....
LOG.info(String.format("--- End of ShutDownHook (%s) ---", APPLICATION_NAME));
}
});
Just add this before first breakpoint.
I have created GWT app, in which I have a Vertical Panel where I log the details.
Client side logging I'm doing using logger
sample code is:
public static VerticalPanel customLogArea = new VerticalPanel();
public static Logger rootLogger = Logger.getLogger("");
logerPanel.setTitle("Log");
scrollPanel.add(customLogArea);
logerPanel.add(scrollPanel);
if (LogConfiguration.loggingIsEnabled()) {
rootLogger.addHandler(new HasWidgetsLogHandler(customLogArea));
}
And I'm updating my vertical log panel using this code
rootLogger.log(Level.INFO,
"Already Present in Process Workspace\n");
But now my question is , I have to log server side details also into my vertical log panel.
My serverside GreetingServiceImpl code is:
public boolean createDirectory(String fileName)
throws IllegalArgumentException {
Boolean result = false;
try {
rootLogger.log(Level.INFO,
"I want to log this to my UI vertical log Panel");
system.out.println("log this to UI");
File dir = new File("D:/GenomeSamples/" + fileName);
if (!dir.exists()) {
result = dir.mkdir();
}
} catch (Exception e) {
e.printStackTrace();
}
return result;
}
Now I want to log sysoutprt statements to my UI from here. How can I achieve this. Now using rootLogger.log(Level.INFO,
"I want to log this to my UI vertical log Panel"); code it is logging this to eclipse console . But how to log this to my UI in client side.
Please let me know If anything wrong in this question.
If I understood you right, you want to see your server log entries in web interface. And of course, java logger and printStackTrace() won't help you in that: your gwt code is compiled to JavaScript and has nothing to do with console and log files. Besides, your server can't "push" log entries to client - it's up to client to make requests. So if you want to track new log entries and move it to client, you need to poll server for new entries. And yet another problem: you may have many clients polling your servlet and you should keep in mind this multi-threading.
This is how I see probable implementation (it's just concept, may contain some errors and misspellings):
Remote interface:
public interface GreetingService extends RemoteService {
List<String> getLogEntries();
boolean createDirectory(String fileName)throws IllegalArgumentException;
}
Remote Servlet:
public class GreetingServiceImpl extends RemoteServiceServlet implements GreetingService {
public static final String LOG_ENTRIES = "LogEntries";
public List<String> getLogEntries() {
List<String> entries = getEntriesFromSession();
List<String>copy = new ArrayList<String>(entries.size());
copy.addAll(entries);
//prevent loading the same entries twice
entries.clear();
return copy;
}
public boolean createDirectory(String fileName)throws IllegalArgumentException {
Boolean result = false;
try {
log("I want to log this to my UI vertical log Panel");
log("log this to UI");
File dir = new File("D:/GenomeSamples/" + fileName);
if (!dir.exists()) {
result = dir.mkdir();
}
} catch (Exception e) {
log("Exception occurred: " + e.getMessage());
}
return result;
}
private List<String> getEntriesFromSession() {
HttpSession session= getThreadLocalRequest().getSession();
List<String>entries = (List<String>)session.getAttribute(LOG_ENTRIES);
if (entries == null) {
entries = new ArrayList<String>();
session.setAttribute(LOG_ENTRIES,entries);
}
return entries;
}
private void log(String message) {
getEntriesFromSession().add(message);
}
Simple implementation of polling (gwt client-side):
Timer t = new Timer() {
#Override
public void run() {
greetingAsyncService.getLogEntries(new AsyncCallBack<List<String>>() {
void onSuccess(List<String>entries) {
//put entries to your vertical panel
}
void onFailure(Throwable caught){
//handle exceptions
}
});
}
};
// Schedule the timer to run once in second.
t.scheduleRepeating(1000);
greetingAsyncService.createDirectory(fileName, new AsyncCallBack<Void>(){
void onSuccess(List<String>entries) {
//no need to poll anymore
t.cancel();
}
void onFailure(Throwable caught){
//handle exceptions
}
});
}
As you can see, I have used session to keep log entries, because session is client-specific and so different clients will receive different logs. It's up to you to decide what to use - you may create your own Logger class that will track users itself and give appropriate logs to appropriate clients.
And also you may want to save level of your messages (INFO,ERROR etc.) and then display messages in different colors (red for ERROR, for instance). To do so, you need to save not List, but some your custom class.
You'd create a logging servlet that has the same methods as your logging framework to send log messages to your server via RPC.
Here are some sample RPC log methods you can use:
public interface LogService extends RemoteService {
public void logException(String logger, String priority, String message, String error, StackTraceElement[] stackTrace, String nativeStack);
}
public interface LogServiceAsync {
public void logException(String logger, String priority, String message, String error, StackTraceElement[] stackTrace, String nativeStack, AsyncCallback<Void> callback);
}
public class LogServiceImpl extends RemoteServiceServlet implements LogService {
public void logException(String loggerName, String priority, String logMessage, String errorMessage, StackTraceElement[] stackTrace, String nativeStack) {
Logger logger = getLogger(loggerName);
Level level = getLevel(priority);
// Create a Throwable to log
Throwable caught = new Throwable();
if (errorMessage != null && stackTrace != null) {
caught = new Throwable(errorMessage);
caught.setStackTrace(stackTrace);
}
//do stuff with the other passed arguments (optional)
logger.log(level, message, caught);
}
}
Although those implementations are very nice, forget about timers and repeated server queries. We've something better now.
It's possible to push data from server to client using Atmosphere which supports WebSockets.