SpringCloudStream-HttpSource is returning 404 - spring-cloud

I am working on the Brooklyn.Release version of SpringCloudStream. My use case has HttpSource with Mutiple Sinks. When I added the Starter App dependency to the application and using it as below:
<dependency>
<groupId>org.springframework.cloud.stream.app</groupId>
<artifactId>spring-cloud-starter-stream-source-http</artifactId>
<version>1.0.4.RELEASE</version>
</dependency>
#SpringBootApplication
#Import(HttpSourceConfiguration.class)
public class SourceApplication {
public static void main(String[] args) {
SpringApplication.run(SourceApplication.class, args);
}
}
My Aggregate App is
#SpringBootApplication
public class Application {
public static void main(String[] args) {
new AggregateApplicationBuilder().from(SourceApplication.class)
.via(ProcessorApplication.class).to(SinkApplication.class).args().run(args);
}
}
Getting the following response as below all the time:
<Fault xmlns="http://localhost/">
<error>Not Found</error>
<message>No message available</message>
<path>/</path>
<status>404</status>
<timestamp>1477612242743</timestamp>
</Fault>
Added ComponentScan for HttpSourceConfiguration(Out of the Box); but no success.
#SpringBootApplication
#ComponentScan(value = {"com.xxx.xxx.stream","org.springframework.cloud.stream.app.http.source"})
public class Application {
public static void main(String[] args) {
new AggregateApplicationBuilder().from(SourceApplication.class)
.via(ProcessorApplication.class).to(SinkApplication.class).args().run(args);
}
}
If I use the same SourceApplication with Rabbit Binder, it is working as expected. Can you please guide me in getting this working?
Appreciate your help and time.
Regards
Karthik

The REST endpoints don't seem to be setup and it looks like you may be running into this issue. You would need #ComponentScan for the package where HttpSourceConfiguration is (since it is from a different package).
If I use the same SourceApplication with Rabbit Binder, it is working as expected.
Do you mean the out of the box http app with the rabbit binder or you just added the rabbit binder dependency in the above SourceApplication?
Usually you prefer using the http source application that comes out of the box (with any of the associated binder), but for the aggregate application you don't need the binder unless you have input or output of the aggregate app itself is bound to a broker.

Related

Spring kafka setErrorHandler deprecated replacement (boot 2.6.4)

On spring boot 2.6.4, this method is deprecated.
public ConcurrentKafkaListenerContainerFactory<Object, Object> kafkaListenerContainerFactory(
ConcurrentKafkaListenerContainerFactoryConfigurer configurer) {
var factory = new ConcurrentKafkaListenerContainerFactory<Object, Object>();
configurer.configure(factory, consumerFactory());
// deprecated
factory.setErrorHandler(new GlobalErrorHandler());
return factory;
}
The global error handler class
public class GlobalErrorHandler implements ConsumerAwareErrorHandler {
private static final Logger log = LoggerFactory.getLogger(GlobalErrorHandler.class);
#Override
public void handle(Exception thrownException, ConsumerRecord<?, ?> data, Consumer<?, ?> consumer) {
// my custom global logic (e.g. notify ops team via slack)
}
}
What is the replacement sample for this? The doc says I should use setCommonErrorHandler, but how to implements the CommonErrorHandler interface, as no method to be overriden there.
Point is, I have to send slack notification to ops team, based on certain condition (the message tpye, which is available on kafka message header)
This is not blocking, just an annoying deprecated message though.
Thanks
See the Spring for Apache Kafka documentation; legacy error handlers are replaced with CommonErrorHandler implementations.
What's New?
https://docs.spring.io/spring-kafka/docs/current/reference/html/#x28-eh
The legacy GenericErrorHandler and its sub-interface hierarchies for record an batch listeners have been replaced by a new single interface CommonErrorHandler with implementations corresponding to most legacy implementations of GenericErrorHandler. See Container Error Handlers for more information.
Container Error Handlers
https://docs.spring.io/spring-kafka/docs/current/reference/html/#error-handlers
Starting with version 2.8, the legacy ErrorHandler and BatchErrorHandler interfaces have been superseded by a new CommonErrorHandler. These error handlers can handle errors for both record and batch listeners, allowing a single listener container factory to create containers for both types of listener. CommonErrorHandler implementations to replace most legacy framework error handler implementations are provided and the legacy error handlers deprecated. The legacy interfaces are still supported by listener containers and listener container factories; they will be deprecated in a future release.
I was facing exactly the same problem, so I changed the method implementation ConsumerAwareErrorHandler by
CommonErrorHandler
and implemented
handleRecord
like described in the docs and it works!
public class GlobalErrorHandler implements CommonErrorHandler {
private static final Logger log = LoggerFactory.getLogger(GlobalErrorHandler.class);
#Override
public void handleRecord(
Exception thrownException,
ConsumerRecord<?, ?> record,
Consumer<?, ?> consumer,
MessageListenerContainer container) {
log.warn("Global error handler for message: {}", record.value().toString());
}
}
In KafkaConfig.class
#Bean(value = "kafkaListenerContainerFactory")
public ConcurrentKafkaListenerContainerFactory<Object, Object> kafkaListenerContainerFactory(
ConcurrentKafkaListenerContainerFactoryConfigurer configurer) {
var factory = new ConcurrentKafkaListenerContainerFactory<>();
configurer.configure(factory, consumerFactory());
factory.setCommonErrorHandler(new GlobalErrorHandler());
return factory;
}

Overridden RabbitSourceConfiguration (app starters) does not work with Spring Cloud Edgware

I'm testing an upgrade of my Spring Cloud DataFlow services from Spring Cloud Dalston.SR4/Spring Boot 1.5.9 to Spring Cloud Edgware/Spring Boot 1.5.9. Some of my services extend source (or sink) components from the app starters. I've found this does not work with Spring Cloud Edgware.
For example, I have overridden org.springframework.cloud.stream.app.rabbit.source.RabbitSourceConfiguration and bound my app to my overridden version. This has previously worked with Spring Cloud versions going back almost a year.
With Edgware, I get the following (whether the app is run standalone or within dataflow):
***************************
APPLICATION FAILED TO START
***************************
Description:
Field channels in org.springframework.cloud.stream.app.rabbit.source.RabbitSourceConfiguration required a bean of type 'org.springframework.cloud.stream.messaging.Source' that could not be found.
Action:
Consider defining a bean of type 'org.springframework.cloud.stream.messaging.Source' in your configuration.
I get the same behaviour with the 1.3.0.RELEASE and 1.2.0.RELEASE of spring-cloud-starter-stream-rabbit.
I override RabbitSourceConfiguration so I can set a header mapper on the AmqpInboundChannelAdapter, and also to perform a connectivity test prior to starting up the container.
My subclass is bound to the Spring Boot application with #EnableBinding(HeaderMapperRabbitSourceConfiguration.class). A cutdown version of my subclass is:
public class HeaderMapperRabbitSourceConfiguration extends RabbitSourceConfiguration {
public HeaderMapperRabbitSourceConfiguration(final MyHealthCheck healthCheck,
final MyAppConfig config) {
// ...
}
#Bean
#Override
public AmqpInboundChannelAdapter adapter() {
final AmqpInboundChannelAdapter adapter = super.adapter();
adapter.setHeaderMapper(new NotificationHeaderMapper(config));
return adapter;
}
#Bean
#Override
public SimpleMessageListenerContainer container() {
if (config.performConnectivityCheckOnStartup()) {
if (LOGGER.isInfoEnabled()) {
LOGGER.info("Attempting connectivity with ...");
}
final Health health = healthCheck.health();
if (health.getStatus() == Status.DOWN) {
LOGGER.error("Unable to connect .....");
throw new UnableToLoginException("Unable to connect ...");
} else if (LOGGER.isInfoEnabled()) {
LOGGER.info("Connectivity established with ...");
}
}
return super.container();
}
}
You really should never do stuff like healthCheck.health(); within a #Bean definition. The application context is not yet fully baked or started; it may, or may not, work depending on the order that beans are created.
If you want to prevent the app from starting, add a bean that implements SmartLifecycle, put the bean in a late phase (high value) so it's started after everything else. Then put your code in start(). autStartup must be true.
In this case, it's being run before the stream infrastructure has created the channel.
Some ordering might have changed from the earlier release but, in any case, performing activity like this in a #Bean definition is dangerous.
You just happened to be lucky before.
EDIT
I just noticed your #EnableBinding is wrong; it should be Source.class. I can't see how that would ever have worked - that's what creates the bean for the channels field of type Source.
This works fine for me after updating stream and the binder to 1.3.0.RELEASE...
#Configuration
public class MySource extends RabbitSourceConfiguration {
#Bean
#Override
public AmqpInboundChannelAdapter adapter() {
AmqpInboundChannelAdapter adapter = super.adapter();
adapter.setHeaderMapper(new MyMapper());
return adapter;
}
}
and
#SpringBootApplication
#EnableBinding(Source.class)
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}
If that doesn't work, please edit the question to show your POM.

spring cloud + ribbon + feign + zuul + eureka

How to loadbalance micro services via Ribbon (Not feign). I have 3 micro services "M1", "M2" and "M2_duplication", "M1" is communicating with "M2" via feign. I want if "M2" get too much traffic, the requests will be forwarded to the "M2_duplication". How is this possible via #ribbonclient ?
POM M1:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-ribbon</artifactId>
</dependency>
The feign call in M1:
//name is taken from Eureka(service registry)
#FeignClient(name = "M1")
public interface M1ServiceClient {
#RequestMapping(method = RequestMethod.GET, value = "/getAllM2")
Map<String, String> getAllM2();
}
Application M1:
#EnableConfigurationProperties()
#SpringBootApplication
#EnableEurekaClient
#EnableFeignClients
public class PortefeuilleApplication {
public static void main(String[] args) {
SpringApplication.run(PortefeuilleApplication.class, args);
}
}
Your question is rather vague ... for example, you specifically state 'not feign' and then show a FeignClient. Nevertheless it looks like you are asking how to implement an availability pattern for your ribbon load balancer. To do this, you create a Ribbon Configuration class and then override the load balancer strategy rule. For example:
#Configuration
public class MyConfiguration {
#Bean
public IRule ribbonRule() {
return new RoundRobinRule();
}
}
There are a number of existing Ribbon Rule strategies around availability e.g. AvailabilityFilteringRule or BestAvailableFilter, but if none of these suit you could also write your own Rule. Once you have your class, amend your RibbonClient annotation to reference it, e.g:
#RibbonClient(name = "myClient", configuration = MyConfiguration.class)
You can find more information here: https://github.com/Netflix/ribbon/wiki/Working-with-load-balancers

How to use regexmapper based routing in Zuul? PatternServiceRouteMapper not working?

What I try to achieve is a routing for example:
http://zuul-host:8080/v1/foo/hello to my service foo-v1, resource hello
I'm trying out the regexmapper example described at http://cloud.spring.io/spring-cloud-netflix/spring-cloud-netflix.html
My problem is that I see that a service called foo-v1 gets mapped to /v1/foo in the PatternServiceRouteMapper but then I'm not able to call that route. It's also no visible at /mappings. Do I have to activate that route somewhere?
Setup
Foo Service
application.properties
server.port=9092
spring.application.name=foo-v1
eureka.client.serviceUrl.defaultZone=http://localhost:8761/eureka/
eureka.instance.healthcheck.enable=true
Zuul
My configuration class Routings.java. I added some sysout log output for the service mapping and I get foo-v1 -> v1/foo in the log. Therefore this mapping should be active.
#Configuration
public class Routings {
#Bean
public PatternServiceRouteMapper serviceRouteMapper() {
return new PatternServiceRouteMapper(
"(?<name>^.+)-(?<version>v.+$)",
"${version}/${name}") {
#Override
public String apply(final String serviceId) {
String route = super.apply(serviceId);
System.out.println(serviceId + " -> " +route);
return route;
}
};
}
}
My ZuulApplication.java
#SpringBootApplication
#EnableZuulProxy
#ComponentScan
public class ZuulApplication {
public static void main(String[] args) {
SpringApplication.run(ZuulApplication.class, args);
}
#RefreshScope
#ConfigurationProperties("zuul")
public ZuulProperties zuulProperties() {
return new ZuulProperties();
}
}
Ok, found the solution.
Remove ignoredServices: '*' from the zuul config.
This happens if you work through the examples. They start with explicitly configured routes and ignore dynamic routings. It's in the documentation but made no sense to me at that point :-)
To skip having a service automatically added, set zuul.ignored-services to a list of service id patterns.
When using the regexmapper we start using services that get added automatically and that's the feature we disabled with ignoredServices: '*'

Swagger documentation with JAX-RS Jersey 2 and Grizzly

I have implementated a Rest web service (the function is not relevant) using JAX-RS. Now I want to generate its documentation using Swagger. I have followed these steps:
1) In build.gradle I get all the dependencies I need:
compile 'org.glassfish.jersey.media:jersey-media-moxy:2.13'
2) I documentate my code with Swagger annotations
3) I hook up Swagger in my Application subclass:
public class ApplicationConfig extends ResourceConfig {
/**
* Main constructor
* #param addressBook a provided address book
*/
public ApplicationConfig(final AddressBook addressBook) {
register(AddressBookService.class);
register(MOXyJsonProvider.class);
register(new AbstractBinder() {
#Override
protected void configure() {
bind(addressBook).to(AddressBook.class);
}
});
register(io.swagger.jaxrs.listing.ApiListingResource.class);
register(io.swagger.jaxrs.listing.SwaggerSerializers.class);
BeanConfig beanConfig = new BeanConfig();
beanConfig.setVersion("1.0.2");
beanConfig.setSchemes(new String[]{"http"});
beanConfig.setHost("localhost:8282");
beanConfig.setBasePath("/");
beanConfig.setResourcePackage("rest.addressbook");
beanConfig.setScan(true);
}
}
However, when going to my service in http://localhost:8282/swagger.json, I get this output.
You can check my public repo here.
It's times like this (when there is no real explanation for the problem) that I throw in an ExceptionMapper<Throwable>. Often with server related exceptions, there are no mappers to handle the exception, so it bubbles up to the container and we get a useless 500 status code and maybe some useless message from the server (as you are seeing from Grizzly).
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Response;
import javax.ws.rs.ext.ExceptionMapper;
public class DebugMapper implements ExceptionMapper<Throwable> {
#Override
public Response toResponse(Throwable exception) {
exception.printStackTrace();
if (exception instanceof WebApplicationException) {
return ((WebApplicationException)exception).getResponse();
}
return Response.serverError().entity(exception.getMessage()).build();
}
}
Then just register with the application
public ApplicationConfig(final AddressBook addressBook) {
...
register(DebugMapper.class);
}
When you run the application again and try to hit the endpoint, you will now see a stacktrace with the cause of the exception
java.lang.NullPointerException
at io.swagger.jaxrs.listing.ApiListingResource.getListingJson(ApiListingResource.java:90)
If you look at the source code for ApiListingResource.java:90, you will see
Swagger swagger = (Swagger) context.getAttribute("swagger");
The only thing here that could cause the NPE is the context, which scrolling up will show you it's the ServletContext. Now here's the reason it's null. In order for there to even be a ServletContext, the app needs to be run in a Servlet environment. But look at your set up:
HttpServer server = GrizzlyHttpServerFactory
.createHttpServer(uri, new ApplicationConfig(ab));
This does not create a Servlet container. It only creates an HTTP server. You have the dependency required to create the Servlet container (jersey-container-grizzly2-servlet), but you just need to make use of it. So instead of the previous configuration, you should do
ServletContainer sc = new ServletContainer(new ApplicationConfig(ab));
HttpServer server = GrizzlyWebContainerFactory.create(uri, sc, null, null);
// you will need to catch IOException or add a throws clause
See the API for GrizzlyWebContainerFactory for other configuration options.
Now if you run it and hit the endpoint again, you will see the Swagger JSON. Do note that the response from the endpoint is only the JSON, it is not the documentation interface. For that you need to use the Swagger UI that can interpret the JSON.
Thanks for the MCVE project BTW.
Swagger fixed this issue in 1.5.7. It was Issue 1103, but the fix was rolled in last February. peeskillet's answer will still work, but so will OP's now.