I have been looking at the org.hornetq.core.server package which seems to have the most interesting low level APIS relating to managing the server.
The server session has a few methods labelled something Queue but none include Topic ...
ServerSession
void createQueue(SimpleString address,
SimpleString name,
SimpleString filterString,
boolean temporary,
boolean durable) throws Exception;
void deleteQueue(SimpleString name) throws Exception
interface QueueFactory
Queue createQueue(long persistenceID,
final SimpleString address,
SimpleString name,
Filter filter,
boolean durable,
boolean temporary);
However i could not figure out how to create a topic. Am i missing something is a JMS topic implemented as a queue ?
The core API does not know the concept of a Topic as it is used in JMS, it only knows queues and adresses. In the documentation it says:
*For example, a JMS topic would be implemented by a single address to which many queues are bound. Each queue represents a subscription of the topic. A JMS Queue would be implemented as a single address to which one queue is bound - that queue represents the JMS queue.*
You could implement it with the core API the same way or just use JMS :-)
Related
I am working with Kafka cluster and using Transactional Producer for atomic streaming (read-process-write).
// Init Transactions
_transactionalProducer.InitTransactions(DefaultTimeout);
// Begin the transaction
_transactionalProducer.BeginTransaction();
// produce message to one or many topics
var topic = Topics.MyTopic;
_transactionalProducer.Produce(topic, consumeResult.Message);
I am using AvroSerializer since I publish messages with Schema.
Produce throws an exception:
"System.InvalidOperationException: Produce called with an IAsyncSerializer value serializer configured but an ISerializer is required.\r\n at Confluent.Kafka.Producer`2.Produce(TopicPartition topicPartition, Message`2 message, Action`1 deliveryHandler)"
All examples I've seen for transactional producer use Produce method rather than ProduceAsync so not sure I can simply switch to ProduceAsync and assume that transactional produce will function correctly. Correct me if I'm wrong or help me find documentation.
Otherwise, I am not able to find AvroSerializer that is not Async, inheriting from ISerializer.
public class AvroSerializer<T> : IAsyncSerializer<T>
I didn't realize that there is AsSyncOverAsync method which I can use when creating the Serializer. This exists because Kafka Consumer is also still Sync and not Async.
For example:
new AvroSerializer<TValue>(schemaRegistryClient, serializerConfig).AsSyncOverAsync();
Here is Confluent documentation of that method.
//
// Summary:
// Create a sync serializer by wrapping an async one. For more information on the
// potential pitfalls in doing this, refer to Confluent.Kafka.SyncOverAsync.SyncOverAsyncSerializer`1.
public static ISerializer<T> AsSyncOverAsync<T>(this IAsyncSerializer<T> asyncSerializer);
Two verticles need to send and receive data between them. Is it necessary to have two event bus addresses one for each direction. Or is it possible to have a single vert.x event bus address and send/receive both on the same address.
In vert.x, when you send a message through a eventBus, you can register a replayHandler to receive reply message.
<T> EventBus send(String address,
Object message,
Handler<AsyncResult<Message<T>>> replyHandler)
vert.x send with replyHandler
EventBus is unidirectional.
If you need to send independent messages from A->B and from B->A, you'll have to register two addresses.
I would still recommend against that, since circular dependencies are error prone. Essentially, that's a common recipe for a deadlock. A is waiting for a message from B, while B is waiting for a message from A.
What usually is done in such cases is introducing a broker. That broker will receive all messages, and decided where this messages goes from here. So, instead of A and B holding each an EventBus address, they'll use reply mechanism.
A customer wants to exchange data between his application and our application via ActiveMQ, so we want to create an Interface Specification Document which describes the settings and properties that both applications must use so that they can communicate. We don't know which programming language or API the customer will use; so if the specification is incomplete they might implicitly use settings that we don't expect.
So I'm wondering which settings must be the same on both sides, and which settings can be decided by each application on its own. This is what I have so far:
Must be specified in document:
connector type (openwire, stomp, ...)
connector settings (host name where broker runs, TCP port, user name, password)
message type (TextMessage, BytesMessage...)
payload details (XML with XSDs, JSON with schema, ...)
message encoding (UTF-8), for text payload
use queues, or topics, or durable topics
queue names
is any kind of request/response protocol being used
use single queue for requests and responses (with selectors being used to get correct messages), or use separate queues for requests and responses
how to transfer correlation ID used for correlating requests and responses
message expiration
Must not be specified in document:
ActiveMQ broker version (all versions are compatible, right?)
message compression (it should be transparent?)
What did I miss? Which things should be stated in such a document to ensure that two applications can communicate via ActiveMQ?
What did I miss?
You missed message headers. These can be broken into two categories:
Built-in (JMS) headers
Custom headers
Examples of the built-in headers are things such as JMSMessageID, JMSXGroupID, etc. In some cases, your interface definition will need to include details of whether and how these values will be set. For example, if messages need to be grouped, then any message producer or consumer using the definition will need to be aware of this.
Similarly, if there will any custom headers (common uses include bespoke ordering, source system identification, authorization tokens, etc.) attached to the messages need to be part of any interface definition.
In fact, I would argue that the interface definition only needs to include two things:
a schema definition for the message body, and
any headers + if they are required or optional
Everything else you have listed above is either a deployment or a management concern.
For example, whether a consumer or producer should connect to a queue or topic is a management concern, not an interface concern. The address of the queue/topic is a deployment concern, not an interface concern.
Can anybody explain the different types of queues provided by MSMQ.
Outgoing, private and system queues , what are the functions of them?
Thanks.
Yash
Private queues are registered on the local computer, not in the directory service, and typically cannot be located by other Message Queuing applications
Private queues are accessible only by Message Queuing applications that know the full path name, the direct format name, or the private format name of the queue
In a domain environment, public queues are queues that are published in Active Directory
A transactional queue is one that only contains transactional messages, which are messages sent within a transaction
http://technet.microsoft.com/en-us/library/cc778799(v=ws.10).aspx
I would like to translate the concept of JMS topics using HornetQ core API.
The problem i see from my brief examination it would appear the main class JMSServerManagerImpl (from hornetq-jms.jar) uses jndi to coordinate the various collaborators it requires. I would like to avoid jndi as it is not self contained and is a globally shared object which is a problem especially in an osgi environment. One alternative is to copy starting at JMSServerManagerImpl but that seems like a lot of work.
I would rather have a confirmation that my approach to emulating how topics are supported in hornetq is the right way to solve this problem. If anyone has sufficient knowledge perhaps they can comment on what i think is the approach to writing my own emulation of topics using the core api.
ASSUMPTION
if a message consumer fails (via rollback) the container will try deliverying the message to another different consumer for the same topic.
EMULATION
wrap each message that is added for the topic.
sender sends msg w/ an acknowledgement handler set.
the wrapper for (1) would rollback after the real listener returns.
the sender then acknowledges delivery
I am assuming after 4 the msg is delivered after being given to all msg receivers. If i have made any mistakes or my assumptions are wrong please comment. Im not sure exactly if this assumption of how acknowledgements work is correct so any pointers would be nice.
If you are trying to figure out how to send a message to multiple consumers using the core API; here is what I recommend
Create queue 1 and bind to address1
Create queue 2 and bind to address1
Make queue N and bind to address 1
Send a message on address1
Start N consumers where each consumer listens on queue 1-N
This way it basically works like a topic.
http://hornetq.sourceforge.net/docs/hornetq-2.0.0.BETA5/user-manual/en/html/using-jms.html
7.5. Directly instantiating JMS Resources without using JNDI