Flutter mqtt - Unicode(Korean) doesn't work - flutter

I am using mqtt_client: ^9.6.8 in my flutter project.
I want to subscribe to Korean topics, like topic: "test/안녕"
I want to get messages in Korean, like message: "안녕하세요"
currently if I try to subscribe to a korean topic, the client disconnects and the process just kills itself.
The log was a little bit different before, but this is the current log after I subscribe to a korean topic.
I/flutter (18972): 1-2022-08-09 15:38:40.135206 -- MqttConnectionBase::_onDone - calling disconnected callback
I/flutter (18972): 1-2022-08-09 15:38:40.135985 -- MqttConnectionKeepAlive::stop - stopping keep alive
I/flutter (18972): EXAMPLE::OnDisconnected client callback - Client disconnection
I/flutter (18972): EXAMPLE::OnDisconnected callback is unsolicited or none, this is incorrect - exiting
D/HostConnection(18972): createUnique: call
F/libc (18972): FORTIFY: pthread_mutex_lock called on a destroyed mutex (0xb400006dd36425a0)
D/HostConnection(18972): HostConnection::get() New Host Connection established 0xb400006de3607310, tid 19012
F/libc (18972): Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 19013 (1.raster), pid 18972
if I send a korean message, then it comes like HÅU± which is not what I want.
I tried to decode with utf8 but I can't seem to do it correctly.
Any helps on this? thanks for every suggestion!!
--edit--
this is the part where I print the received mqtt message
client.updates!.listen((List<MqttReceivedMessage<MqttMessage?>>? c) {
final recMess = c![0].payload as MqttPublishMessage;
final pt =
MqttPublishPayload.bytesToStringAsString(recMess.payload.message);
context.read<SubTopic>().newTopic(c[0].topic);
context.read<SubMessage>().newMessage(pt);
print('Notification: topic is <${c[0].topic}>, payload is <$pt>');
});
this is where I subscribe to the topic
try {
await client.connect();
} on NoConnectionException catch (e) {
print('EXAMPLE::client exception - $e');
client.disconnect();
} on SocketException catch (e) {
print('EXAMPLE::socket exception - $e');
client.disconnect();
}
if (client.connectionStatus!.state == MqttConnectionState.connected) {
print('EXAMPLE::Mosquitto client connected');
for (String topic in _subTopicList) {
print(topic);
client.subscribe(topic, MqttQos.exactlyOnce);
}
} else {
print(
'EXAMPLE::ERROR Mosquitto client connection failed - disconnecting, status is ${client.connectionStatus}');
client.disconnect();
// exit(-1);
}

Related

Flutter websocket connects to wrong port

In my app, I created a Route for communicating with a socket.
class _SocketRouteState extends State<SocketRoute> {
#override
void initState() {
super.initState();
try {
WebSocketChannel _channel = IOWebSocketChannel.connect(
Uri.parse('ws://192.168.1.90:9998'),
);
///
/// Start listening to new notifications / messages
///
_channel.stream.listen(
(data) {
debugPrint(data);
},
onDone: () {
debugPrint('ws channel closed');
},
onError: (error) {
debugPrint('ws error $error');
},
);
_channel.sink.add('testing');
} catch (e) {
///
/// General error handling
/// TODO handle connection failure
///
debugPrint('Connection exception $e');
}
}
}
When I run this code, it fails to connect to the socket, though. After waiting for ~2 minutes, Xcode shows me the following error:
flutter: ws error WebSocketChannelException: WebSocketChannelException: SocketException: OS Error: Operation timed out, errno = 60, address = 192.168.1.90, port = 52168
This clearly shows a different port. Could that be the issue? Anyone know why it's connecting on port 52168 instead of 9998?
The problem with the confusing port number is, that the error message is not that great, since it shows the local port being used on your own machine and not the remote port you are trying to connect to. TCP requires a port to be open on both the server and the client so they can communicate both ways. But normally, you are mostly interested in the remote port.
There are an old github issue here about this issue:
https://github.com/dart-lang/sdk/issues/12693

Sending a message to a Kafka Broker using TCP Sockets and Deno

We are trying to create a Kafka client using Deno and TCP sockets. As a first step, we are trying to ping a broker we have running on a docker instance using the below code:
import {
Client,
Packet,
Event,
} from 'https://deno.land/x/tcp_socket#0.0.1/mods.ts';
const client = new Client({
hostname: 'localhost',
port: 9092,
});
for (let i = 0; i < 2; i++) {
// Connection open
client.on(Event.connect, (client: Client) => {
console.log('Connect', client.conn?.remoteAddr);
});
// Receive message
client.on(Event.receive, (client: Client, data: Packet) => {
console.log('Receive', data.toString());
});
// Connection close
client.on(Event.close, (client: Client) => {
console.log('Close');
});
// Handle error
client.on(Event.error, (e) => {
console.error(e);
});
// Do
await client.connect(); // Start client connect
await client.write('Hello World'); // Send string data
await client.write(new Uint8Array()); // Send Uint8Array data
client.close();
}
We are successfully pinging the broker, but receive the following error:
[2021-08-13 00:20:36,472] WARN Unexpected error from /172.20.0.1; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 1214606444 larger than 104857600)
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:91)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:169)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:150)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:365)
at org.apache.kafka.common.network.Selector.poll(Selector.java:313)
at kafka.network.Processor.poll(SocketServer.scala:494)
at kafka.network.Processor.run(SocketServer.scala:432)
at java.lang.Thread.run(Thread.java:745)
[2021-08-13 00:28:19,708] INFO [Group Metadata Manager on Broker 0]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
We understand that this is due to a protocol issue and are trying to determine what the best way forward is to address this issue. Any help would be greatly appreciated

Dart client Socket connecting but not sending data to server?

I have a server running on a Raspberry Pi which is accessible from a browser.
http://192.168.1.67:55XXX/?email=a#b.com
yields:-
{Order=[{no=0, day_0=1, price=0, name=PPAC SUPERTHERM 20K, display_colour=blue, notice=1, special_order=false}, {no=1, day_0=1, price=0, name=SLACK 50KG , display_colour=blue, notice=1, special_order=false}, {no=0, day_0=1, price=0, name=PPAC SUPERTHERM 20K, display_colour=blue, day_5=1, notice=1, special_order=false, day_3=1}, {no=1, day_0=1, price=0, name=SLACK 50KG , display_colour=blue, day_5=1, notice=1, special_order=false, day_3=1}], Details={address=xx Farriers Lea, phone=0xxxx 606635, name=Fred Bloggs, mobile=, details=end shed on drive, email=a#b.com}}
I am using VSCODE to debug dart code.
my dart code:-
String remote_ip = '192.168.1.67'; //212.159.118.177';
var remote_port = 55XXX;
Socket socket;
String _dataToBeSent = "http://?email=a#b.com\n";
var reply;
// connect
main(List<String> arguments) async {
await _remoteServerConnect();
}
// REMOTE SERVER CONNECT
Future _remoteServerConnect() async {
// await Socket.connect(remote_ip, remote_port).then((Socket sock){
await Socket.connect(remote_ip, remote_port).then((Socket sock) {
socket = sock;
print('Got connected ${socket.remoteAddress}');
socket.listen(dataHandler,
onError: errorHandler, onDone: doneHandler, cancelOnError: false);
}).catchError((AsyncError e) {
print("Unable to connect: $e");
exit(1);
});
}
void dataHandler(data) async {
await print('"'+String.fromCharCodes(data).trim()+'"');
if (String.fromCharCodes(data).trim().endsWith('html')) {
print("Send Data = $_dataToBeSent");
socket.add(utf8.encode(_dataToBeSent));
// socket.writeln(_dataToBeSent);
socket.flush();
await Future.delayed(Duration(seconds: 5));
}
}
void errorHandler(error, StackTrace trace) {
print(error);
}
void doneHandler() {
// socket.destroy();
exit(0);
}
The dart debug consol yields:-
Connecting to VM Service at ws://127.0.0.1:54799/kE9Xa1JQclk=/ws
Got connected InternetAddress('192.168.1.67', IPv4)
"HTTP/1.1 200 OK" <sent by server
"ContentType: text/html" <sent by server
Send Data = http://?email=a#b.com
Exited
The server consol yields:-
Server is ready
WEB Client connected: /192.168.1.66
05.08/10:34:21.17 - Waiting for command..
05.08/10:34:31.80 - Socket Timeout
05.08/10:34:31.82 - Done -------------
05.08/10:34:31.82 - Waiting for command..
Connection has been closed
Server is ready
It would seem the
socket.add(utf8.encode(_dataToBeSent));
// socket.writeln(_dataToBeSent);
socket.flush();
did not send data to the server?? Why?? any ideas gratefully received!
Disabling my dev machines firewall did NOT improve answer!
Steve
The server used
final DataInputStream in = new DataInputStream(clientSocket.getInputStream());
while (estimatedTime < 60000) {
report("Waiting for command..");
try {
// in.readNBytes(command.data, 0, command.length);
in.readFully(command.data);
to receive the data command.data is 100bytes long so was waiting for the complete input. I padded to 100 bytes worked fine - will sort a better solution later.

Kafka transaction: Receiving CONCURRENT_TRANSACTIONS on AddPartitionsToTxnRequest

I am trying to publish in a transaction a message on 16 Kafka partitions on 7 brokers.
The flow is like this:
open transaction
write a message to 16 partitions
commit transaction
sleep 25 ms
repeat
Sometimes the transaction takes over 1 second to complete, with an average of 50 ms.
After enabling trace logging on producer's side, I noticed the following error:
TRACE internals.TransactionManager [kafka-producer-network-thread | producer-1] - [Producer clientId=producer-1, transactionalId=cma-2]
Received transactional response AddPartitionsToTxnResponse(errors={modelapp-ecb-0=CONCURRENT_TRANSACTIONS, modelapp-ecb-9=CONCURRENT_TRANSACTIONS, modelapp-ecb-10=CONCURRENT_TRANSACTIONS, modelapp-ecb-11=CONCURRENT_TRANSACTIONS, modelapp-ecb-12=CONCURRENT_TRANSACTIONS, modelapp-ecb-13=CONCURRENT_TRANSACTIONS, modelapp-ecb-14=CONCURRENT_TRANSACTIONS, modelapp-ecb-15=CONCURRENT_TRANSACTIONS, modelapp-ecb-1=CONCURRENT_TRANSACTIONS, modelapp-ecb-2=CONCURRENT_TRANSACTIONS, modelapp-ecb-3=CONCURRENT_TRANSACTIONS, modelapp-ecb-4=CONCURRENT_TRANSACTIONS, modelapp-ecb-5=CONCURRENT_TRANSACTIONS, modelapp-ecb-6=CONCURRENT_TRANSACTIONS, modelapp-ecb-=CONCURRENT_TRANSACTIONS, modelapp-ecb-8=CONCURRENT_TRANSACTIONS}, throttleTimeMs=0)
for request (type=AddPartitionsToTxnRequest, transactionalId=cma-2, producerId=59003, producerEpoch=0, partitions=[modelapp-ecb-0, modelapp-ecb-9, modelapp-ecb-10, modelapp-ecb-11, modelapp-ecb-12, modelapp-ecb-13, modelapp-ecb-14, modelapp-ecb-15, modelapp-ecb-1, modelapp-ecb-2, modelapp-ecb-3, modelapp-ecb-4, modelapp-ecb-5, modelapp-ecb-6, modelapp-ecb-7, modelapp-ecb-8])
The Kafka producer retries sending AddPartitionsToTxnRequest(s) several times until it succeeds, but this leads to delays.
The code looks like this:
Properties producerProperties = PropertiesUtil.readPropertyFile(_producerPropertiesFile);
_producer = new KafkaProducer<>(producerProperties);
_producer.initTransactions();
_producerService = Executors.newSingleThreadExecutor(new NamedThreadFactory(getClass().getSimpleName()));
_producerService.submit(() -> {
while (!Thread.currentThread().isInterrupted()) {
try {
_producer.beginTransaction();
for (int partition = 0; partition < _numberOfPartitions; partition++)
_producer.send(new ProducerRecord<>(_producerTopic, partition, KafkaRecordKeyFormatter.formatControlMessageKey(_messageNumber, token), EMPTY_BYTE_ARRAY));
_producer.commitTransaction();
_messageNumber++;
Thread.sleep(_timeBetweenProducedMessagesInMillis);
} catch (ProducerFencedException | OutOfOrderSequenceException | AuthorizationException | UnsupportedVersionException e) {
closeProducer();
break;
} catch (KafkaException e) {
_producer.abortTransaction();
} catch (InterruptedException e) {...}
}
});
Looking to broker's code, it seems there are 2 cases when this error is thrown, but I cannot tell why I get there
object TransactionCoordinator {
...
def handleAddPartitionsToTransaction(...): Unit = {
...
if (txnMetadata.pendingTransitionInProgress) {
// return a retriable exception to let the client backoff and retry
Left(Errors.CONCURRENT_TRANSACTIONS)
} else if (txnMetadata.state == PrepareCommit || txnMetadata.state == PrepareAbort) {
Left(Errors.CONCURRENT_TRANSACTIONS)
}
...
}
...
}
Thanks in advance for help!
Later edit:
Enabling trace logging on broker we were able to see that broker sends to the producer END_TXN response before transaction reaches state CompleteCommit. The producer is able to start a new transaction, which is rejected by the broker while it is still in the transition PrepareCommit -> CompleteCommit.

How to catch/Capture "javax.net.ssl.SSLHandshakeException: Failed to create SSL connection" while sending message over java vert.x Eventbus

I am trying to use SSL over eventbus. To test the failure case I tried sending message to the eventbus from another verticle in same cluster by passing some different keystore.
I am getting below exception on console but it is not failing the replyHandler hence my code is not able to detect the SSL exception.
my code:
eb.request("ping-address", "ping!", new DeliveryOptions(), reply -> {
try {
if (reply.succeeded()) {
System.out.println("Received reply " + reply.result().body());
} else {
System.out.println("An exception " + reply.cause().getMessage());
}
} catch (Exception e) {
System.out.println("An error occured" + e.getCause());
}
});
Exception on console:
**javax.net.ssl.SSLHandshakeException: Failed to create SSL connection**
at io.vertx.core.net.impl.ChannelProvider$1.userEventTriggered(ChannelProvider.java:109)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:341)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:327)
at io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:319)
at io.netty.handler.ssl.SslHandler.handleUnwrapThrowable(SslHandler.java:1249)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1230)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:813)
Caused by: javax.net.ssl.SSLException: Received fatal alert: bad_certificate
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615)
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781)
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:282)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1329)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
... 20 more
But handler is failing for timeout after 30 sec.
Timed out after waiting 30000(ms) for a reply. address: __vertx.reply.8419a431-d633-4ba8-a12e-c41fd5a4f37a, repliedAddress: ping-address
I want to capture the SSL exception immediately and handle it. Please guide me how can I Capture/catch this exception.
I tried with below code. Below one is able to handle the exception and I am not getting reply result from called event-bus. Reply result is always null. (value is always null)
MessageProducer<Object> ms = eb.sender("ping-address");
ms.write("ping!", reply -> {
if (reply.succeeded()) {
reply.map(value -> {
System.out.println("Received reply " + value);
return reply;
});
} else {
System.out.println("No reply");
System.out.println("An exception : " + reply.cause().getMessage());
}
});
You can't catch this exception because the Vert.x clustered EventBus implementation buffers messages when the nodes are not connected together. The message could be sent later if the problem is only temporary.
If you want to be notified earlier, you could set a lower timeout in DeliveryOptions.