I am using .net framework to work with ws queues. Is it possible to remove/delete a message from a queue?
Retrieving it will remove it from the queue. You can then discard the retrieved message if you don't want to do anything with it.
// Hit up MSMQ for list of new messages
var queuedMessages = myMessageQueue.GetAllMessages();
// Receive actual message to remove from the message queue
var messages = new List<Message>();
foreach (var queuedMessage in queuedMessages)
{
// Receive the message, which removes it from the queue.
try
{
messages.Add(this.messageQueue.ReceiveById(queuedMessage.Id));
}
// If it's already been removed, we skip over it.
// TODO: Should try to catch only the "removed" exception
catch (Exception)
{
continue;
}
}
return messages;
Related
I want to process a topic in application startup using Confluent dotnet client. Assume following example:
while (true)
{
try
{
var cr = c.Consume();
Console.WriteLine($"Consumed message '{cr.Value}' at: '{cr.TopicPartitionOffset}'.");
}
catch (ConsumeException e)
{
Console.WriteLine($"Error occured: {e.Error.Reason}");
}
}
When there is no new message in Kafka, c.Consume will be blocked. Because I want to use it for application startup (Like cache warm up) I want to proceed my code when I found there is no new message.
I know there is an overload for setting timeout like c.Consume(timeout) but the problem with this approach is that if you have a message in your topic and the time duration of reading the message was more than your timeout, You receive null output which is not desirable.
The consumer(s) is not supposed to be aware of the producer(s).
Now if you want to know that you have read everything in the topic from the moment you start to consume, you can:
Load the newest offset before starting to consume.
Then start consuming messages.
If the message's offset is the same as the newest offset you loaded before, stop consuming.
I'm not a C# developper but from what I read in the dotnet confluent doc you can call QueryWatermarkOffsetson the consumer to get oldest and newest offset. https://docs.confluent.io/current/clients/confluent-kafka-dotnet/api/Confluent.Kafka.Consumer.html#Confluent_Kafka_Consumer_QueryWatermarkOffsets_Confluent_Kafka_TopicPartition_
And then, on the Messageclass you have an Offset accessor. So the whole thing should not be too hard to achieve.
https://docs.confluent.io/current/clients/confluent-kafka-dotnet/api/Confluent.Kafka.Message.html#Confluent_Kafka_Message_Offset
You can use OnPartitionEOF event that indicates you have reached the end of partition.
CancellationTokenSource source = new CancellationTokenSource();
bool isContinue = true;
c.OnPartitionEOF += (o, e) =>
{
Console.WriteLine($"You have reached end of partition");
isContinue = false;
source.Cancel();
};
while (isContinue)
{
try
{
var cr = c.Consume(source.Token);
Console.WriteLine($"Consumed message '{cr.Value}' at: '{cr.TopicPartitionOffset}'.");
}
catch (ConsumeException e)
{
Console.WriteLine($"Error occured: {e.Error.Reason}");
}
}
I found the Consumer.IsPartitionEOF useful.
I am having trouble subscribing to a socketcluster (http://socketcluster.io/) channel when using a redux-saga generator in my chat app. The socketcluster backend is setup in a way where any messages are saved in the database then published into the receiving user's personal channel, which is named after the user's id. For example, User A has an id '123abc' and would subscribe to the channel named '123abc' for their realtime messages.
The code below does receive new messages that are published to a channel but it throws a "TypeError: Converting circular structure to JSON" onload and breaks all of my other redux-saga generators in the app. I've done digging in Chrome Devtools and my theory is that it has something to do with queue created in the createChannel function. Also, I've tried returning a deferred promise in the subscribeToChannel function but that also caused a Circular Conversion Error, I can post that code on request.
I referred to this answer at first: https://stackoverflow.com/a/35288877/5068616 and it helped me get the below code in place but I cannot find any similar issues on the internet. Also something to note, I am utilizing redux-socket-cluster (https://github.com/mattkrick/redux-socket-cluster) to sync up the socket and state, but I don't think it is the root of the problem
sagas.js
export default function* root() {
yield [
fork(startSubscription),
]
}
function* startSubscription(getState) {
while (true) {
const {
userId
} = yield take(actions.SUBSCRIBE_TO_MY_CHANNEL);
yield call(monitorChangeEvents, subscribeToChannel(userId))
}
}
function* monitorChangeEvents(channel) {
while (true) {
const info = yield call(channel.take) // Blocks until the promise resolves
console.log(info)
}
}
function subscribeToChannel(channelName) {
const channel = createChannel();
const socket = socketCluster.connect(socketConfig);
const c = socket.subscribe(channelName);
c.watch(event => {
channel.put(event)
})
return channel;
}
function createChannel() {
const messageQueue = []
const resolveQueue = []
function put(msg) {
// anyone waiting for a message ?
if (resolveQueue.length) {
// deliver the message to the oldest one waiting (First In First Out)
const nextResolve = resolveQueue.shift()
nextResolve(msg)
} else {
// no one is waiting ? queue the event
messageQueue.push(msg)
}
}
// returns a Promise resolved with the next message
function take() {
// do we have queued messages ?
if (messageQueue.length) {
// deliver the oldest queued message
return Promise.resolve(messageQueue.shift())
} else {
// no queued messages ? queue the taker until a message arrives
return new Promise((resolve) => resolveQueue.push(resolve))
}
}
return {
take,
put
}
}
Thanks for the help!
I am using rabbitmq and I want to make sure that if I have a connection problem in the client, the messages that I posted won't be lost. I simulate it with eclipse: I do system.exit the program of fetching after 100 messages. I posted 1000 messages. The second run I don't limit the number of messages and it returns me 840 messages with 3 times. Can you help me?
the code of the producer is:
public void run() {
String json =SimpleQueueServiceSample.getFromList();
while (!(json.equals(""))){
json =SimpleQueueServiceSample.getFromList();
try {
c.basicPublish("", "test",
MessageProperties.PERSISTENT_TEXT_PLAIN, json.getBytes());
} catch (IOException e) {
e.printStackTrace();
}
}
try {
c.waitForConfirmsOrDie();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
the code of the consumber is:
QueueingConsumer consumer = new QueueingConsumer(channel);
channel.basicConsume(QUEUE_NAME, true, consumer);
while (true) {
System.out.println(count++);
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
String message = new String(delivery.getBody());
System.out.println(" [x] Received '" + message + "'");
}
So the challenge for your scenario is how you're handling the acknowledgements.
channel.basicConsume(QUEUE_NAME, true, consumer);
Is the problem. The second parameter of true is the auto-acknowledge field.
To fix that, use:
channel.basicConsume(QUEUE_NAME, false, consumer);
while (true) {
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
//...
channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
}
It looks like you're using RabbitMQ's tutorials, and your code snippet is from part one. If you look at part two, they start talking about acknowledgements and setting up quality of service to provide round-robin dispatch.
It's worth pointing out that the basicConsume() and nextDelivery() combination rely upon a hidden queue that lives within the consumer. So when you call basicConsume() several messages are pulled down to the client to local storage.
The benefit at that approach is that it avoids additional network overhead from calling for each individual message. The problem is that it can put more messages within your local consumer than you wish and you may lose messages if the consumer drops before processing all of the messages in the local hidden queue.
If you truly want your consumers only working on one message a time so that nothing is lost, you probably want to look at the basicGet() method instead of the basicConsume().
Whats the best way to receive MSMQ messages.
I used the following code, but after receiving MSMQ messages not getting removed in Queue
var msgEnumerator = myQueue.GetMessageEnumerator2();
var messages = new List<System.Messaging.Message>();
while (msgEnumerator.MoveNext(new TimeSpan(0, 0, 1)))
{
var msg = myQueue.ReceiveById(msgEnumerator.Current.Id, new TimeSpan(0, 0, 1));
messages.Add(msg);
for (int i = 0; i < messages.Capacity; i++)
{
String DataMessages = messages[i].ToString();
}
But i cant receive messages.
How to receive those messages.
I would use asynchronous MessageQueue.BeginPeek to start listening on the queue:
queue.BeginPeek();
Then register a handler with the MessageQueue.PeekCompleted event:
queue.PeekCompleted += new PeekCompletedEventHandler(MessageHasBeenReceived);
Then in your handler use MessageQueue.EndPeek to access the message. Remember to call BeginPeek again.
private void MessageHasBeenReceived(object sender, PeekCompletedEventArgs e)
{
// Get message
var msg = queue.EndPeek(e.AsyncResult);
// Do message processing here
....
// Remove message from queue
queue.ReceiveById(msg.Id);
// Listen for more messages
queue.BeginPeek();
}
See here for MSDN example.
I'm trying to send a message to a remote queue.
// Send a message to the queue.
if (myQueue.Transactional)
{
var myTransaction = new MessageQueueTransaction();
myTransaction.Begin();
Message objMessage = new Message();
objMessage.UseDeadLetterQueue = true;
objMessage.Body = message;
myQueue.Send(objMessage, myTransaction);
myTransaction.Commit();
}
else
{
Message objMessage = new Message();
objMessage.UseDeadLetterQueue = true;
objMessage.Body = message;
myQueue.Send(message);
}
but I get an exception
The specified format name does not support the requested operation. For example, a direct queue format name cannot be deleted.
I assume that my queue name is incorrect or I have a permission error so I enabled the dead letter queue but it's empty. My queue name is "FormatName:Direct=TCP:xx.xxx.xx.xx\private$\Test"
Thanks
You can't query information about a remote private queue.
Local queues, yes. Remote public queues, yes, but not with Formatname.