I am using spring boot and use postgres , rabbit mq and deploying application on CF. We figured that we need to set connection pool, we found that whatever configuration we do, on CF we can its max 4 connections, not sure from where we get that number (probably something with buildpack or service config).
In order to resolve that I had to extend AbstractCloudConfig, and that is pain as it turns off other auto configuration so now I have to manually configure rabbit mq connection factory too:(. I have came up with below configuration, but not sure this is right way.
Spring boot version: 1.4
Please advise.
package com.example;
import javax.sql.DataSource;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.config.java.AbstractCloudConfig;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;
/**
* If we need to modify some some custom service configuration on cloud foundry
* e.g. setting up of connection pools. If we set normally and expose bean, it
* will work fine on local machine. But as it will go to Cloud foundry it
* somehow creates max 4 connections. (Not sure from where this number comes)
*
* Adding this configuration meaning we no longer want to leverage auto
* configuration. As soon as Spring boot sees this bean in cloud profile it will
* turn of auto configuration. Expectation is application is going to take care
* of all configuration. This normally works for most of the applications.
*
* For more information read: https://github.com/dsyer/cloud-middleware-blog
* https://docs.cloudfoundry.org/buildpacks/java/spring-service-bindings.html
*
* Hopefully future release of spring boot will allow us to hijack only
* configuration that we want to do ourselves and rest will be auto
* configuration specifically in context with CloudFoundry.
*
*/
#Configuration
#Profile("cloud")
public class CloudServicesConfig extends AbstractCloudConfig {
#Value("${vcap.services.postgres.credentials.jdbc_uri}")
private String postgresUrl;
#Value("${vcap.services.postgres.credentials.username}")
private String postgresUsername;
#Value("${vcap.services.postgres.credentials.password}")
private String postgresPassword;
#Value("${spring.datasource.driver-class-name}")
private String dataSourceDriverClassName;
#Primary
#Bean
public DataSource dataSource() {
org.apache.tomcat.jdbc.pool.DataSource dataSource = new org.apache.tomcat.jdbc.pool.DataSource();
dataSource.setDriverClassName(dataSourceDriverClassName);
dataSource.setUrl(postgresUrl);
dataSource.setUsername(postgresUsername);
dataSource.setPassword(postgresPassword);
dataSource.setInitialSize(10);
dataSource.setMaxIdle(5);
dataSource.setMinIdle(5);
dataSource.setMaxActive(25);
return dataSource;
}
// You can add rest of services configuration below e.g. rabbit connection
// factory, redis etc to centralize services configuration for cloud.
// This example did not use profile but that is what you should use to
// separate out cloud vs local configuraion to help run on local etc.
}
You don't need all that configuration just to customize the pool size. You should just need this code as shown in the documentation:
#Bean
public DataSource dataSource() {
PoolConfig poolConfig = new PoolConfig(5, 30, 3000);
DataSourceConfig dbConfig = new DataSourceConfig(poolConfig, null);
return connectionFactory().dataSource(dbConfig);
}
Related
I use spring boot 2.2.5 + micrometer 1.3.5 + starter-data-mongodb
Under "io.micrometer.core.instrument.binder.mongodb" I can see 2 classes CommandListener and ConnectionPoolListener. I would like to know what purpose these serve?
In actuator metrics endpoint, mongo metrics are not available.
How do I enable metrics for mongodb in actuator? For example, actuator automatically shows several metrics of RabbitMQ. I was expecting something similar in case of MongoDB as well.
Should I create my own metrics?
In order to enable Spring Boot applying its AutoConfiguration I suggest to use the customizer pattern:
Kotlin:
#Configuration
class MongoConfiguration {
#Bean
fun mongoClientSettingsBuilderCustomizer(meterRegistry: MeterRegistry) =
MongoClientSettingsBuilderCustomizer {
it.addCommandListener(MongoMetricCommandListener(meterRegistry))}
}
Java:
#Configuration
public class MongoConfiguration {
#Bean
public MongoClientSettingsBuilderCustomizer mongoClientSettingsBuilderCustomizer(MeterRegistry meterRegistry) {
return builder -> builder.addCommandListener(new MongoMetricsCommandListener(meterRegistry));
}
}
Please note you currenly will neither see a relation to the spring data repository nor to the mongo collection in the metrics. see open issue
EDIT (07/30/2021):
The issues has been fixed, so you likely get collection metrics in a current release.
Adding those listeners is not as straighforward as i thought and it's quite dependend on which properties you use for configuring Spring Data MongoDB.
The key for the integration is to customize the com.mongodb.MongoClientSettings instance which is used for creating the MongoClient. There are multiple possiblities to do so as documented in Connecting to MongoDB with Spring
The following is a working example (simplified from an application of ours) based on Spring Boot 2.3 which assumes you are using spring.data.mongodb.uri to specify the connection string in your application.properties.
package com.example.demo;
import com.mongodb.ConnectionString;
import com.mongodb.MongoClientSettings;
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.binder.mongodb.MongoMetricsCommandListener;
import io.micrometer.core.instrument.binder.mongodb.MongoMetricsConnectionPoolListener;
import org.springframework.boot.autoconfigure.mongo.MongoProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.mongodb.core.MongoClientFactoryBean;
#Configuration
public class MongoConfiguration {
#Bean
public MongoClientFactoryBean mongoClientFactoryBean(MongoProperties properties, MeterRegistry meterRegistry) {
MongoClientFactoryBean mongoClientFactoryBean = new MongoClientFactoryBean();
mongoClientFactoryBean.setConnectionString(new ConnectionString(properties.getUri()));
MongoClientSettings settings = MongoClientSettings.builder()
.addCommandListener(new MongoMetricsCommandListener(meterRegistry))
.applyToConnectionPoolSettings(builder ->
builder.addConnectionPoolListener(new MongoMetricsConnectionPoolListener(meterRegistry)))
.build();
mongoClientFactoryBean.setMongoClientSettings(settings);
return mongoClientFactoryBean;
}
}
Unfortunately a lot has in configuring MongoDB from Spring Boot 2.2 to 2.3. If you cannot use Spring Boot 2.3 and you are stuck in backporting this to 2.2 please let me know.
Short answer
Create a MongoClientOptions bean with addCommandListener and you are good to go.
#Configuration
public class MongoConfiguration {
#Autowired
private MeterRegistry meterRegistry;
#Bean
public MongoClientOptions myMongoClientOptions() {
return MongoClientOptions.builder()
.addCommandListener(new MongoMetricsCommandListener(meterRegistry)).build();
}
}
#chargue's answer won't work for certain version of spring-data-mongodb. Because org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration initializes MongoClient with MongoClientOptions, not MongoClientSettings. It might work in newer version of spring-data-mongodb cuz MongoClientSettings is the recommended way according to mongodb's documents.
#Bean
#ConditionalOnMissingBean(type = { "com.mongodb.MongoClient", "com.mongodb.client.MongoClient" })
public MongoClient mongo(MongoProperties properties, ObjectProvider<MongoClientOptions> options,
Environment environment) {
return new MongoClientFactory(properties, environment).createMongoClient(options.getIfAvailable());
}
Make sure prometheus and micrometer is correctlly setup. You should see mongo metrics in prometheus endpoint like below:
# HELP process_cpu_usage The "recent cpu usage" for the Java Virtual Machine process
# TYPE process_cpu_usage gauge
process_cpu_usage{application="",} 0.004362672325272289
# HELP mongodb_driver_commands_seconds_max Timer of mongodb commands
# TYPE mongodb_driver_commands_seconds_max gauge
mongodb_driver_commands_seconds_max{application="",cluster_id="60b0d12d73b6df671cb4d882",command="find",server_address="",status="SUCCESS",} 34.684200332
mongodb_driver_commands_seconds_max{application="",cluster_id="60b0d12d73b6df671cb4d882",command="buildInfo",server_address="",status="SUCCESS",} 0.263514375
# HELP mongodb_driver_commands_seconds Timer of mongodb commands
# TYPE mongodb_driver_commands_seconds summary
mongodb_driver_commands_seconds_count{application="",cluster_id="60b0d12d73b6df671cb4d882",command="find",server_address="",status="SUCCESS",} 1.0
mongodb_driver_commands_seconds_sum{application="",cluster_id="60b0d12d73b6df671cb4d882",command="find",server_address="",status="SUCCESS",} 34.684200332
mongodb_driver_commands_seconds_count{application="",cluster_id="60b0d12d73b6df671cb4d882",command="buildInfo",server_address="",status="SUCCESS",} 1.0
mongodb_driver_commands_seconds_sum{application="",cluster_id="60b0d12d73b6df671cb4d882",command="buildInfo",server_address="",status="SUCCESS",} 0.263514375
Thanks for Spring Boot Admin!
I am using it with Spring Cloud Kubernetes, our k8s pods only get discovered when we start the Spring Boot Admin, after the service pods have been started.
It seems looking at InstanceDiscoveryListener, the discovery of clients,
will happen based on events. Like ApplicationReadyEvent (When starting) and i.e. InstanceRegisteredEvent.
Is it correct to say, that Spring Boot Admin, will not try to discover periodically? If so how do I make sure an event is fired from the application to Spring boot admin picks it up and registers the instance?
Especially to make sure, that instanced are registered when they are started after spring boot admin, was started. (The order in which k8s pods are started is arbitrary/hard to control, something in general we don't want to do).
Thank You!
Christophew
Version:
springBootAdminVersion = '2.0.1'
springCloudVersion = 'Finchley.RELEASE'
springCloudK8s = '0.3.0.RELEASE'
Not sure if this is the best way to solve it but seems to work:
class TimedInstanceDiscoveryListener extends InstanceDiscoveryListener {
private static final Logger log = LoggerFactory.getLogger(TimedInstanceDiscoveryListener.class);
public TimedInstanceDiscoveryListener(DiscoveryClient discoveryClient, InstanceRegistry registry, InstanceRepository repository) {
super(discoveryClient, registry, repository);
log.info("Starting custom TimedInstanceDiscoveryListener");
}
#Scheduled(fixedRate = 5000)
public void periodicDiscovery() {
log.info("Discovering new pod / services");
super.discover();
}
}
#Bean
#ConfigurationProperties(prefix = "spring.boot.admin.discovery")
public InstanceDiscoveryListener instanceDiscoveryListener(ServiceInstanceConverter serviceInstanceConverter,
DiscoveryClient discoveryClient,
InstanceRegistry registry,
InstanceRepository repository) {
InstanceDiscoveryListener listener = new TimedInstanceDiscoveryListener(discoveryClient, registry, repository);
listener.setConverter(serviceInstanceConverter);
return listener;
}
We have developed an application in Spring Boot + spring data (backend) + MongoDB and used IBM Websphere Liberty as application Server. We were used "Application Managed DB Connection" in an yml file and enjoyed the benefit of Spring Boot autoconfiguration.
Due to policy changes, we would need to manage our DB Connection in Liberty Server(using mongo feature), in Server.xml. I spent whole day in finding out an good example to do this, but dont find any example in Spring with "Container Managed MongoDB Connection" in IBM Websphere Liberty Server.
Can someone please support here?
Check out this other stackoverflow solution. The following is an extension of how you would use that in your Spring Boot app.
You should be able to inject your datasource the same way. You could even inject it into your configuration and wrap it in a Spring DelegatingDataSource.
#Configuration
public class DataSourceConfiguration {
// This is the last code section from that link above
#Resource(lookup = "jdbc/oracle")
DataSource ds;
#Bean
public DataSource mySpringManagedDS() {
return new DelegatingDataSource(ds);
}
}
Then you should be able to inject the mySpringManagedDS DataSource into your Component, Service, etc.
In the past Liberty had a dedicated mongodb-2.0 feature for the server.xml, however this feature provided pretty minimal benefit, since you still needed to bring your own MongoDB libraries. Also, over time MongoDB made significant breaking changes to their API, including how MongoDB gets configured.
Since the MongoDB API is changing so drastically between releases, we found it better to not provide any new MongoDB features in Liberty and instead suggest that users simply use a CDI producer like this:
CDI producer (holds any configuration too):
#ApplicationScoped
public class MongoProducer {
#Produces
public MongoClient createMongo() {
return new MongoClient(new ServerAddress(), new MongoClientOptions.Builder().build());
}
#Produces
public MongoDatabase createDB(MongoClient client) {
return client.getDatabase("testdb");
}
public void close(#Disposes MongoClient toClose) {
toClose.close();
}
}
Example usage:
#Inject
MongoDatabase db;
#POST
#Path("/add")
#Consumes(MediaType.APPLICATION_JSON)
public void add(CrewMember crewMember) {
MongoCollection<Document> crew = db.getCollection("Crew");
Document newCrewMember = new Document();
newCrewMember.put("Name",crewMember.getName());
newCrewMember.put("Rank",crewMember.getRank());
newCrewMember.put("CrewID",crewMember.getCrewID());
crew.insertOne(newCrewMember);
}
This is just the basics, but the following blog post goes into much greater detail along with code examples:
https://openliberty.io/blog/2019/02/19/mongodb-with-open-liberty.html
I am trying Ribbon configuration with RestTemplate based on bookmark service example but without luck, here is my code:
#SpringBootApplication
#RestController
#RibbonClient(name = "foo", configuration = SampleRibbonConfiguration.class)
public class BookmarkServiceApplication {
public static void main(String[] args) {
SpringApplication.run(BookmarkServiceApplication.class, args);
}
#Autowired
RestTemplate restTemplate;
#RequestMapping("/hello")
public String hello() {
String greeting = this.restTemplate.getForObject("http://foo/hello", String.class);
return String.format("%s, %s!", greeting);
}
}
with error page as below:
Whitelabel Error Page
This application has no explicit mapping for /error, so you are seeing this as a fallback.
Tue Mar 22 19:59:33 GMT+08:00 2016
There was an unexpected error (type=Internal Server Error, status=500).
No instances available for foo
but if I remove annotation #RibbonClient, everything will be just ok,
#RibbonClient(name = "foo", configuration = SampleRibbonConfiguration.class)
and here is SampleRibbonConfiguration implementation:
public class SampleRibbonConfiguration {
#Autowired
IClientConfig ribbonClientConfig;
#Bean
public IPing ribbonPing(IClientConfig config) {
return new PingUrl();
}
#Bean
public IRule ribbonRule(IClientConfig config) {
return new AvailabilityFilteringRule();
}
}
Is it because RibbonClient can not work with RestTemplate together?
and another question is that does Ribbon configuration like load balancing rule could be configured via application.yml configuration file?
as from Ribbon wiki, seems we can configure Ribbon parameters like NFLoadBalancerClassName, NFLoadBalancerRuleClassName etc in property file, does Spring Cloud also supports this?
I'm going to assume you're using Eureka for Service Discovery.
Your particular error:
No instances available for foo
can happen for a couple of reasons
1.) All services are down
All of the instances of your foo service could legitimately be DOWN.
Solution: Try visiting your Eureka Dashboard and ensure all the services are actually UP.
If you're running locally, the Eureka Dashboard is at http://localhost:8761/
2.) Waiting for heartbeats
When you very first register a service via Eureka, there's a period of time where the service is UP but not available. From the documentation
A service is not available for discovery by clients until the
instance, the server and the client all have the same metadata in
their local cache (so it could take 3 heartbeats)
Solution: Wait a good 30 seconds after starting your foo service before you try calling it via your client.
In your particular case I'm going to guess #2 is likely what's happening to you. You're probably starting the service and trying to call it immediately from the client.
When it doesn't work, you stop the client, make some changes and restart. By that time though, all of the heartbeats have completed and your service is now available.
For your second question. Look at the "Customizing the Ribbon Client using properties" section in the reference documentation. (link)
I am new to EJB3 world. I want to create scheduler for file processing in EJB. I tried the following code..
package com.fks.nclp.ejb.scheduler;
import javax.ejb.Schedule;
import javax.ejb.Schedules;
import javax.ejb.Stateless;
#Stateless
public class AutoTimerBean {
#Schedules(
{
#Schedule(second="3",persistent=false)
}
)
public void executeOnEveryTwoSecond(){
System.out.println("THIS IS TESTING OF EJB SCHEDULER");
}
}
And deployed EAR application on GlassFish3.1. As per my requirement the scheduler should be fired at every three seconds.
But its not happening. Any suggestion ???
Thanks,
Gunjan.
Got the solution. In GlassFish server, we have to create EJB timer service from admin console.
Steps are as follow ..
Go to glassFish admin console -> Go to Configurations
-> Go to server config -> Go to EJB Container
Here set Timer DataSource = JDBC Default Resource pool.
Restart the server.
After adding JDBC default resource pool to Timer DataSource, the scheduler works fine.
Thanks,
Gunjan.