Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I want to debug some Kafka topics so I know if the consumer or producer is at fault here.
Is there a UI for Kafka where I can see what messages a topic contain?
A dumper would also be nice so I can search for stuff on my own.
We use Landoop's Kafka Topics UI, which is pretty good. You can see topic contents and information (e.g. number of partitions, configuration, etc) and also export topic contents.
I'll second Yoni Gibb's suggestion of the Landoop product. I also use it in development and find it very useful; although you may need to tweak a few settings around timeout and size in order to see all messages. Easy to install, just pull the Docker image.
Kafkacat is useful too, but it's not quite as good for being able to monitor many topics at once and be left running.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 19 days ago.
Improve this question
I have to implement a solution that consists of processing a large amount of data by applying business requirement rules. The input and the output will be a file.
I haven't been using Kafka before, I am wondering if I can use Kafka streams to process these rules or use spring batch combined with Kafka streams.
Is there any other frameworks/technologies that can be used in Java?
Thank you
Kafka Streams is a stream processing solution; what you're talking about is more of a batch workload. The difficulties you will encounter using KStreams are:
Kafka Streams doesn't have a good way of working with files as input and output.
In Stream Processing, there's no real concept of "beginning" and "end," whereas I gather from the nature of your question that you do indeed have a beginning and end in your use-case.
As such I would recommend another batch solution.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
I'm looking for a way implementing exponential backoff / retry policy with Celery and SQS.
I like to use the visibility timeout mechanism SQS is offering and not celery countdown feature, as it is based on worker being active / not replaced with a different one.
Is there any kind of implementation supporting this?
Cheers
As far as I know Celery already offers this since the 4.0 release. More about it in the "Automatic retry for exceptions" section of the Celery documentation. In the case you want more than that, then you are on your own I suppose...
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
How easy is to switch from Rabbit to Kafka in existing solution, to replace one implementation (Rabbit) with other (Kafka)? We are about to use Rabbit in our implementation but we want to see if it is possible in the future to replace it with Kafka.
It is possible, and I've seen people do it - but it is a big project.
Not only the APIs are different, but the semantics are different. So you need to rethink your data model, scaling model, error handling, etc. And then there's testing.
If you don't have tons of code to update, and the code is localized and you have both RabbitMQ and Kafka experts on the team you may be able to get it done in a month or two.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
Does scaladoc/javadoc for Apache Kafka exist somewhere on the web?
(yes, I can download the sources and generate it for personal use, but for quoting in emails/conversations a URL comes very handy)
For a more recent update to this question, javadocs are now available.
0.9.0
0.8.2
I asked myself the same question just some days ago, and it seems that the answer is still "no".
What worked for me was generating everything locally (as you mention) and then publish the html in an intranet server to address linking/conversational needs.
If you need something "public", I guess a Github project page will be good enough, until Kafka 1.0 ready.
In addition to #morgan09dev, you can find documentation for the scala code in the following links which consists of the old producer and consumer apis and the broker part:
0.9.0.1
0.8.2.2
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I would like to archive my old mails, so that I can keep them safe. I don't want them to be in my mail client, because they are too many, and they slow it down.
The best one would be an open-source solution where I can search when I have something old to retrieve.
Do you know something like that ?
Thanks
If you're using IMAP to access your messages (and you should be!), you can just put the archived messages into a folder and then unsubscribe from that folder (and, if necessary, mark it as non-indexed). Your mail client will then not consider it when checking for new mail, etc. It won't slow you down, but the messages will be there and accessible if you need them.
Another solution is to export the messages using something like the claws-mail archive plugin. It lets you dump the messages (even a whole folder at a time) to .tar.gz files or other archive types. It works very well.