Overridden RabbitSourceConfiguration (app starters) does not work with Spring Cloud Edgware - spring-cloud

I'm testing an upgrade of my Spring Cloud DataFlow services from Spring Cloud Dalston.SR4/Spring Boot 1.5.9 to Spring Cloud Edgware/Spring Boot 1.5.9. Some of my services extend source (or sink) components from the app starters. I've found this does not work with Spring Cloud Edgware.
For example, I have overridden org.springframework.cloud.stream.app.rabbit.source.RabbitSourceConfiguration and bound my app to my overridden version. This has previously worked with Spring Cloud versions going back almost a year.
With Edgware, I get the following (whether the app is run standalone or within dataflow):
***************************
APPLICATION FAILED TO START
***************************
Description:
Field channels in org.springframework.cloud.stream.app.rabbit.source.RabbitSourceConfiguration required a bean of type 'org.springframework.cloud.stream.messaging.Source' that could not be found.
Action:
Consider defining a bean of type 'org.springframework.cloud.stream.messaging.Source' in your configuration.
I get the same behaviour with the 1.3.0.RELEASE and 1.2.0.RELEASE of spring-cloud-starter-stream-rabbit.
I override RabbitSourceConfiguration so I can set a header mapper on the AmqpInboundChannelAdapter, and also to perform a connectivity test prior to starting up the container.
My subclass is bound to the Spring Boot application with #EnableBinding(HeaderMapperRabbitSourceConfiguration.class). A cutdown version of my subclass is:
public class HeaderMapperRabbitSourceConfiguration extends RabbitSourceConfiguration {
public HeaderMapperRabbitSourceConfiguration(final MyHealthCheck healthCheck,
final MyAppConfig config) {
// ...
}
#Bean
#Override
public AmqpInboundChannelAdapter adapter() {
final AmqpInboundChannelAdapter adapter = super.adapter();
adapter.setHeaderMapper(new NotificationHeaderMapper(config));
return adapter;
}
#Bean
#Override
public SimpleMessageListenerContainer container() {
if (config.performConnectivityCheckOnStartup()) {
if (LOGGER.isInfoEnabled()) {
LOGGER.info("Attempting connectivity with ...");
}
final Health health = healthCheck.health();
if (health.getStatus() == Status.DOWN) {
LOGGER.error("Unable to connect .....");
throw new UnableToLoginException("Unable to connect ...");
} else if (LOGGER.isInfoEnabled()) {
LOGGER.info("Connectivity established with ...");
}
}
return super.container();
}
}

You really should never do stuff like healthCheck.health(); within a #Bean definition. The application context is not yet fully baked or started; it may, or may not, work depending on the order that beans are created.
If you want to prevent the app from starting, add a bean that implements SmartLifecycle, put the bean in a late phase (high value) so it's started after everything else. Then put your code in start(). autStartup must be true.
In this case, it's being run before the stream infrastructure has created the channel.
Some ordering might have changed from the earlier release but, in any case, performing activity like this in a #Bean definition is dangerous.
You just happened to be lucky before.
EDIT
I just noticed your #EnableBinding is wrong; it should be Source.class. I can't see how that would ever have worked - that's what creates the bean for the channels field of type Source.
This works fine for me after updating stream and the binder to 1.3.0.RELEASE...
#Configuration
public class MySource extends RabbitSourceConfiguration {
#Bean
#Override
public AmqpInboundChannelAdapter adapter() {
AmqpInboundChannelAdapter adapter = super.adapter();
adapter.setHeaderMapper(new MyMapper());
return adapter;
}
}
and
#SpringBootApplication
#EnableBinding(Source.class)
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}
If that doesn't work, please edit the question to show your POM.

Related

Spring kafka setErrorHandler deprecated replacement (boot 2.6.4)

On spring boot 2.6.4, this method is deprecated.
public ConcurrentKafkaListenerContainerFactory<Object, Object> kafkaListenerContainerFactory(
ConcurrentKafkaListenerContainerFactoryConfigurer configurer) {
var factory = new ConcurrentKafkaListenerContainerFactory<Object, Object>();
configurer.configure(factory, consumerFactory());
// deprecated
factory.setErrorHandler(new GlobalErrorHandler());
return factory;
}
The global error handler class
public class GlobalErrorHandler implements ConsumerAwareErrorHandler {
private static final Logger log = LoggerFactory.getLogger(GlobalErrorHandler.class);
#Override
public void handle(Exception thrownException, ConsumerRecord<?, ?> data, Consumer<?, ?> consumer) {
// my custom global logic (e.g. notify ops team via slack)
}
}
What is the replacement sample for this? The doc says I should use setCommonErrorHandler, but how to implements the CommonErrorHandler interface, as no method to be overriden there.
Point is, I have to send slack notification to ops team, based on certain condition (the message tpye, which is available on kafka message header)
This is not blocking, just an annoying deprecated message though.
Thanks
See the Spring for Apache Kafka documentation; legacy error handlers are replaced with CommonErrorHandler implementations.
What's New?
https://docs.spring.io/spring-kafka/docs/current/reference/html/#x28-eh
The legacy GenericErrorHandler and its sub-interface hierarchies for record an batch listeners have been replaced by a new single interface CommonErrorHandler with implementations corresponding to most legacy implementations of GenericErrorHandler. See Container Error Handlers for more information.
Container Error Handlers
https://docs.spring.io/spring-kafka/docs/current/reference/html/#error-handlers
Starting with version 2.8, the legacy ErrorHandler and BatchErrorHandler interfaces have been superseded by a new CommonErrorHandler. These error handlers can handle errors for both record and batch listeners, allowing a single listener container factory to create containers for both types of listener. CommonErrorHandler implementations to replace most legacy framework error handler implementations are provided and the legacy error handlers deprecated. The legacy interfaces are still supported by listener containers and listener container factories; they will be deprecated in a future release.
I was facing exactly the same problem, so I changed the method implementation ConsumerAwareErrorHandler by
CommonErrorHandler
and implemented
handleRecord
like described in the docs and it works!
public class GlobalErrorHandler implements CommonErrorHandler {
private static final Logger log = LoggerFactory.getLogger(GlobalErrorHandler.class);
#Override
public void handleRecord(
Exception thrownException,
ConsumerRecord<?, ?> record,
Consumer<?, ?> consumer,
MessageListenerContainer container) {
log.warn("Global error handler for message: {}", record.value().toString());
}
}
In KafkaConfig.class
#Bean(value = "kafkaListenerContainerFactory")
public ConcurrentKafkaListenerContainerFactory<Object, Object> kafkaListenerContainerFactory(
ConcurrentKafkaListenerContainerFactoryConfigurer configurer) {
var factory = new ConcurrentKafkaListenerContainerFactory<>();
configurer.configure(factory, consumerFactory());
factory.setCommonErrorHandler(new GlobalErrorHandler());
return factory;
}

EF Core Migrations in Azure Function startup

According to https://learn.microsoft.com/en-us/azure/azure-functions/functions-dotnet-dependency-injection the service provider should not be used until AFTER the startup has completed running. Indeed, if I try to get a registered service it will fail.
Example:
[assembly: FunctionsStartup(typeof(Startup))]
namespace Fx {
public sealed class Startup : FunctionsStartup {
public override void Configure(IFunctionsHostBuilder builder) {
var configurationBuilder = new ConfigurationBuilder();
configurationBuilder.AddEnvironmentVariables();
var configuration = configurationBuilder.Build();
builder.Services.AddInfrastructure(configuration);
builder.Services.AddApplication();
var serviceProvider = builder.Services.BuildServiceProvider();
DependencyInjection.AddDatabase(serviceProvider).GetAwaiter().GetResult();
}
}
}
public static class DependencyInjection {
public static async Task AddDatabase(IServiceProvider services) {
using var scope = services.CreateScope();
var serviceProvider = scope.ServiceProvider;
var context = serviceProvider.GetRequiredService<ApplicationDbContext>();
//Error generated here
if (context.Database.IsSqlServer()) {
await context.Database.MigrateAsync();
}
await ApplicationDbContextSeed.SeedSamplePersonnelDataAsync(context);
}
public static IServiceCollection AddInfrastructure(
this IServiceCollection services,
IConfiguration configuration) {
services.AddDbContext<ApplicationDbContext>(options =>
options.UseSqlServer(configuration.GetConnectionString("DefaultConnection"),
b => b.MigrationsAssembly(typeof(ApplicationDbContext).Assembly.FullName)));
services.AddScoped<IApplicationDbContext>(provider => provider.GetService<ApplicationDbContext>());
return services;
}
}
This produces the following error
Microsoft.EntityFrameworkCore: No database provider has been configured for this DbContext. A provider can be configured by overriding the DbContext.OnConfiguring method or by using AddDbContext on the application service provider. If AddDbContext is used, then also ensure that your DbContext type accepts a DbContextOptions<TContext> object in its constructor and passes it to the base constructor for DbContext.
Is there a good option for migrating and seeding during startup?
The easiest way I found to run code after startup was by registering a custom IWebJobsStartup by using the WebJobsStartupAttribute (the FunctionsStartupAttribute actually also inherits from this attribute). In the WebJobsStartup class you'll need to register your extension using the AddExtension where you are able to use dependency injection and seed your database. My code:
[assembly: WebJobsStartup(typeof(DbInitializationService), "DbSeeder")]
namespace Our.Database.Seeder
{
public class DbInitializationService : IWebJobsStartup
{
public void Configure(IWebJobsBuilder builder)
{
builder.AddExtension<DbSeedConfigProvider>();
}
}
[Extension("DbSeed")]
internal class DbSeedConfigProvider : IExtensionConfigProvider
{
private readonly IServiceScopeFactory _scopeFactory;
public DbSeedConfigProvider(IServiceScopeFactory scopeFactory)
{
_scopeFactory = scopeFactory;
}
public void Initialize(ExtensionConfigContext context)
{
using var scope = _scopeFactory.CreateScope();
var dbContext = scope.ServiceProvider.GetService<YourDbContext>();
dbContext.Database.EnsureCreated();
// Further DB seeding, etc.
}
}
}
According to your code, I assume that you're building something aligned to the CleanArchitecture Repository on Github. https://github.com/jasontaylordev/CleanArchitecture
The main difference between this repo and your apporach, is that you're obviously not using ASP.NET, which is not a problem at all, but requires a little bit more configuration work.
The article already mentioned (https://markheath.net/post/ef-core-di-azure-functions) refers another blogpost (https://dev.to/azure/using-entity-framework-with-azure-functions-50aa), which briefly explains that EntityFramework Migrations are not capable of auto-discovering your migrations in an Azure Function. Therefore, you need to implement an instance of IDesignTimeDbContextFactory. I also stumbled upon it in the microsoft docs:
https://learn.microsoft.com/en-us/ef/core/cli/dbcontext-creation?tabs=dotnet-core-cli#from-a-design-time-factory
You could for example place it inside your Infrastructure\Persistence\Configurations folder. (Once again, I'm only assuming that you're following the CleanArchitecture repo structure)
DI in AZURE Functions
Caveats
A series of registration steps run before and after the runtime processes the startup class. Therefore, keep in mind the following items:
The startup class is meant for only setup and registration. Avoid using services registered at startup during the startup process. For instance, don't try to log a message in a logger that is being registered during startup. This point of the registration process is too early for your services to be available for use. After the Configure method is run, the Functions runtime continues to register additional dependencies, which can affect how your services operate.
The dependency injection container only holds explicitly registered types. The only services available as injectable types are what are setup in the Configure method. As a result, Functions-specific types like BindingContext and ExecutionContext aren't available during setup or as injectable types

Detecting when the database session gets refreshed on a Spring Boot 2 application

I'm trying to execute the following SQL statement every time the Database Session gets refreshed. I have a Spring Boot 2.0.1.RELEASE with JPA application and a PostgreSQL Database.
select set_config('SOME KEY', 'SOME VALUE', false);
As the PostgreSQL documentation states the is_local parameter is used to indicate that this configuration value will apply just for the current transaction -if true- or will be attached to the session (as I require) -if false-
The problem is that I'm not aware when Hibernate/Hikari are refreshing the db session, so, in practice, the application start failing when it has a couple of minutes running, as you can imagine...
My approach -that is not working yet- is to implement a EmptyInterceptor, for that I have added a DatabaseCustomizer class to inject my hibernate.session_factory.interceptor properly in a way that Spring can fill out all my #Autowires
DatabaseInterceptor.class
#Component
public class DatabaseInterceptor extends EmptyInterceptor {
#Autowired
private ApplicationContext context;
#Override
public void afterTransactionBegin(Transaction tx) {
PersistenceService pc = context.getBean(PersistenceService.class);
try {
pc.addPostgresConfig("SOME KEY", "SOME VALUE");
System.out.println("Config added...");
} catch (Exception e) {
e.printStackTrace();
}
}
}
DatabaseCustomizer.class
#Component
public class DatabaseCustomizer implements HibernatePropertiesCustomizer {
#Autowired
private DatabaseInterceptor databaseInterceptor;
#Override
public void customize(Map<String, Object> hibernateProperties) {
hibernateProperties.put("hibernate.session_factory.interceptor", databaseInterceptor);
}
}
Obviously, there is a problem with this approach because when I #Override the afterTransactionBegin method to start another transaction I get an Infinite loop.
I tried to look something inside that Transaction tx that could help to be sure that this transaction is not being generated by my own addPostgresConfig but there is not much on it.
Is there something else I could try to achieve this?
Thanks in advance,

How to load spring application context even if Cassandra down

When using
#Configuration
#EnableCassandraRepositories(basePackages={"com.foo"})
public class CassandraConfig{
#Bean
public CassandraClusterFactoryBean cluster()
{
final CassandraClusterFactoryBean cluster = new CassandraClusterFactoryBean();
cluster.setContactPoints(nodesRead);
cluster.setPort(port);
return cluster;
}
Where in the com.foo package there is a interface that extends CrudRepository.
Is there a way to make it so that at startup time an exception is not thrown if the database is down?
Ideally what occurs is that we startup and anytime you call a method on the repository, it will first attempt to connect to the database and then if the database is still down return an error saying can't connect.
The behavior I currently observe is that NoHostAvailableException is thrown and the web container does not start up.
I was able to come up with a solution. I removed the #EnableCassandraRepositories(basePackages={"com.foo"}) annotation from the repository and defined a Bean in my Config that would return my repository. Removing the EnableCassandraRepositories allowed lazy loading of the repository. This new bean in my Config allowed me to instantiate my repository using the RepositoryFactorySupport getRepository() method. I annotated this bean as lazy and made sure references to the bean were also lazy.
Assume my repository looks like the following
public interface IBarRepository extends CrudRepository<Bar, BarKey>{}
My Config file now looks like
#Configuration
public class CassandraConfig{
#Bean
#Lazy(value=true)
public IBarRepository barRepository() throws Exception
{
final RepositoryFactorySupport support = CassandraRepositoryFactory(cassandraTemplate());
return support.getRepository(IBarRepository.class);
}
#Bean
#Lazy(value=true)
public CassandraClusterFactoryBean cluster()
{
final CassandraClusterFactoryBean cluster = new CassandraClusterFactoryBean();
cluster.setContactPoints(nodesRead);
cluster.setPort(port);
return cluster;
}
//More beans down here defining things like cluster, mappingContext, session, etc.

Test the remote client jndi lookup using arquillian

Setup: arquillian, jboss as 7.1.1.final as a managed Container
I am currently migrating an EJB application from EJB 2.x to 3.x and JBoss 3.x to JBoss AS 7.1.
During this process i would like to get most classes under test and stumbled over arquillian.
While arquillian seems to offer some nice features on inter-bean-functionality i cannot figure out whether or not the testing of remote client features using jndi lookups works or not.
I used the Arquillian Getting started guides on my beans which worked, but since these are using #Inject and in my application jndi lookups are used everywhere i (at least think that i) need to swerve from that path.
Here is the TestCase i created based on Arquillian Getting Started. I explicitly left in all attempts using jndi properties of which i thought they might help.
The Test
should_create_greeting()
works if the Greeter bean using a separate Producer.
#RunWith(Arquillian.class)
public class GreeterTest {
public static final String ARCHIVE_NAME = "test";
Logger logger = Logger.getLogger(GreeterTest.class.getName());
#Deployment
public static Archive<?> createDeployment() {
JavaArchive jar = ShrinkWrap.create(JavaArchive.class, ARCHIVE_NAME + ".jar").addPackage(Greeter.class.getPackage())
.addAsManifestResource("test-persistence.xml", "persistence.xml").addAsManifestResource("OracleGUIDS-ds.xml")
.addAsManifestResource(EmptyAsset.INSTANCE, "beans.xml");
return jar;
}
/**
* #Inject works using a producer with {#code #Produces}
*/
// #Inject
// Greeter greeter;
#ArquillianResource
Context context;
GreeterRemote greeter;
#Before
public void before() throws Exception {
Map<String, String> env = new HashMap<>();
env.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.as.naming.InitialContextFactory");
env.put("jboss.naming.client.ejb.context", "true");
// env.put("jboss.naming.client.connect.options.org.xnio.Options.SASL_POLICY_NOPLAINTEXT",
// "false");
// env.put("jboss.naming.client.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS",
// "false");
// env.put("jboss.naming.client.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED",
// "false");
for (Map.Entry<String, String> entry : env.entrySet()) {
context.addToEnvironment(entry.getKey(), entry.getValue());
}
greeter = (GreeterRemote) context.lookup(ARCHIVE_NAME + "/" + Greeter.class.getSimpleName() + "!"
+ GreeterRemote.class.getName());
}
#Test
public void should_create_greeting() {
Assert.assertEquals("Hello, Earthling!", greeter.createGreeting("Earthling"));
greeter.greet(System.out, "Earthling");
}
}
Is it possible to get this test running with jndi lookup? Am i missing something?
If you want to test the Remote features of a EJB you probably want to run on the client side and not in container.
You can configure the Deployment to be only client side by using #Deployment(testable=false). The #Test methods will then run as if you were a remote client.
Beyond that you can just lookup the bean via the injected Context if you want.
I had the same issue, so in a workaround i just added on the method to be tested the remoteejb as a parameter.
On my ejb:
public List localBean.obtain(RemoteEJB remoteEjb){
return remoteEjb.obtain();
}
Then on the arquillian test :
#Inject
private LocalBean localBean;
#Inject
private RemoteEJB remoteEjb;
#Test
public void test(){
List<Vo>voList = localBean.obtain(remoteEjb);
}
The best part is the remote ejb its injected and on the caller method original
#EJB(lookup="java:global/ear/ejb/RemoteEjb")
private RemoteEJB remoteEjb;