I can see the following 4 channels by default in my channelGroup(satya-channelGroup):
private.admin
private.admin-meta
private.satya
private.satya-meta
What are these channels??
when I delete my channelGroup, these default channels also got deleted?
Is this behavior okay?
Channel Groups: adding & deleting channels
There is no such thing as a default channel in a channel group. They only get there because they get explicitly added to the channel group by your code (or someone else's code that has your keys).
Deleting a channel group just means you are getting rid of a the thing that kept a list of those channels. The channels can still exist if others are subscribed to them or they are in another channel group.
Related
I'm trying to figure out the poll records mechanism for Kafka over SCS in a K8s environment.
What is the recommended way to control max.poll.records?
How can I poll the defined value?
Is it possible to define it once for all channels and then override for a specific channel?
(referring to this comment form documentation):
To avoid repetition, Spring Cloud Stream supports setting values for
all channels, in the format of
spring.cloud.stream.kafka.default.consumer.=. The
following properties are available for Kafka consumers only and must
be prefixed with
spring.cloud.stream.kafka.bindings..consumer..")
Is this path supported: spring.cloud.stream.binding.<channel name>.consumer.configuration?
Is this: spring.cloud.stream.**kafka**.binding.<channel name>.consumer.configuration?
How are conflicts being resolved? Let's say in a case where both spring.cloud.stream.binding... and spring.cloud.stream.**kafka**.binding... are set?
I've tried all mentioned configurations, but couldn't see in the log what is the actual poll.records and frankly the documentation is not entirely clear on the subject.
These are the configurations:
spring.cloud.stream.kafka.default.consumer.configuration.max.poll.records - default if nothing else specified for given channel
spring.cloud.stream.kafka.bindings..consumer.configuration.max.poll.records
We are running Mirth 3.9.0 and noticed that some of our diet order messages are being stripped of segments. For example we create an order which may contain two ORC and ODS segments along with an NTE afterwards. Despite our interface adding the extra segments, once Mirth sends the message the extra segments have been removed. We can test this by calling our API (via Postman) and seeing the resulting message containing the extra segments but viewing the same message in Mirth Connect the segments are missing.
Why would Mirth remove the segments and how can we keep it from doing so?
I discovered that Mirth does not allow two ODC segments in a row. We added code to adjust for that by adding an extra ORC segment before the ODS. We didn't see any improvement and then found out our master branch does not seem to match production. The change allowed it to work in our test branch but not production. This is not a Mirth issue at this point but a deployment issue.
Sorry for wasting anyone's time on this.
DISCLAIMER: This question is specifically about the Erlang/OTP Kafka Client library Brod (No Tag available yet)
I am trying to establish three consumer groups, one should write messages just plain to the console, another one should update a state representing API with certain messages, and the third one should store every message into a long term database (crate). I use a supervisor to start three according brod_group_subscriber_v2 (see this GIST). If I also start three brod (kafka) clients first and attach each group subscriber its own client, everything works perfectly so that offsets are commited to Kafka for every group and reads start from the latest commited offset.
If I use only one client (as it should be possible and intended according to the Brod docs and issues, see Reference by zmstone), only the last group in my CHILD_SPEC works, both other do not receive handle_message calls.
At the moment starting a client for every group is not an issue for me, as there are only three. But as our project grows, we plan to establish a couple of consumer groups, and I don't really think that it might be a good idea to run 20 to 30 brod clients and blocking ressources for each of them.
I am receiving a large number of correlated HL7 messages in Mirth. They contain an ID which is always the same for all correlated messages and they always come within a minute. Multiple batches can be received at the same time. It's hard to say when the batch ends, but when there are no more messages for a minute, it's safe to assume that the batch has finished.
How could I implement an aggregator pattern in Mirth that would keep reading and completing correlated messages and send the completed message after it didn't receive any new messages with the same ID within a defined time interval?
You may drop all incoming message to a folder and store the message ID in a Global Map. Once new messages start to arrive with the message ID different than the one stored in the map (meaning that the next sequence is started), trigger another channel either by sending the message ID it needs to look for or in some other way. After that replace the message ID in the Global Map with the message ID of a new sequence.
If that sounds too complicated, you may do the same, but the second channel will constantly scan the folder (File Reader) and grab only files having the same message ID and older than a minute from a current time (which is in my mind is too vague qualifier).
I've implemented this by saving all messages in a folder using an ID inside the message (that identifies the sequence) as the file name. The file gets updated with each new message. Several sequences live together in the same folder.
The next channel is using a simple file reader that only fetches the files that are a minute or more old.
I am using activemq to pass requests between different processes. In some cases, I have multiple, duplicate message (which are requests) in the queue. I would like to have only one. Is there a way to send a message in a way that it will replace an older message with similar attributes? If there isn't, is there a way to inspect the queue and check for a message with specific attributes (in this case I will not send the new message if an older one exists).
Clarrification (based on Dave's answer): I am actually trying to make sure that there aren't any duplicate messages on the queue to reduce the amount of processing that is happening whenever the consumer gets the message. Hence I would like either to replace a message or not even put it on the queue.
Thanks.
This sounds like an ideal use case for the Idempotent Consumer which removes duplicates from a queue or topic.
The following example shows how to do this with Apache Camel which is the easiest way to implement any of the Enterprise Integration Patterns, particularly if you are using ActiveMQ which comes with Camel integrated out of the box
from("activemq:queueA").
idempotentConsumer(memoryMessageIdRepository(200)).
header("myHeader").
to("activemq:queueB");
The only trick to this is making sure there's an easy way to calculate a unique ID expression on each message - such as pulling out an XPath from the document or using as in the above example some unique message header
You could browse the queue and use selectors to identify the message. However, unless you have a small amount of messages this won't scale very well. Instead, you message should just be a pointer to a database-record (or set of records). That way you can update the record and whoever gets the message will then access the latest version of the record.