I am using a Watson IOT Output (wiotp out) in a Node-RED flow on my Raspberry PI and am having issues with the connection repeatedly disconnecting and then re-connecting.
Here is a screenshot of my Credentials Node and one of my IOT Out Node.
The connection is configured so that I can send messages to the cloud and successfully have them trigger a flow in my cloud Node-RED instance.
The problem I'm having is that when I attempt to send a string array as my payload, very items in the array actually make it through before the service disconnects. I am limited to around 3-5 strings at a time, which is frustrating because I am losing 195-197 of the 200 items I am trying to send to the IOT platform.
How can I keep a persistent connection and make sure my entire payload makes it through to the IOT service?
If you're seeing very frequent disconnects, it can often mean that you're inadvertently performing clientId stealing (i.e., two MQTT clients are fighting over the same clientId). You can confirm this by looking at the device connection logs in the device drilldown panel: you'll see evidence of log messages such as "The client ID was reused."
Ben
Related
I am trying to connect Mavlink (installed on RPi) with QGroundControl via 3DR Radio Telemetry as to transmit telemetry data from RPi without dependency of Ardupilot/Px4 flightcontrollers. I want to achieve transmitting data such as GPS or Postion using the message format of mavlink without dependency of ardupilot/PX4.
So far, I have taken the example code from mavlink git and have built the code as per instructions. I start the QGroundControl and was able receive the System ID and component ID from QGC. I modified the code to transmit a heartbeat from RPI-mavlink to QGroundControl.
mavlink_msg_heartbeat_pack(system_id, 200, &msg, MAV_TYPE_HELICOPTER, MAV_AUTOPILOT_GENERIC, MAV_MODE_GUIDED_ARMED, 0, MAV_STATE_ACTIVE);
int len_hb = write_message(msg);
After building and executing this code, I was expecting the QGroundControl to connect/show received message, but nothing was available at the QGroundControl.
I need to understand if what I am doing is the right way to do or if there is predefined method to establish such kind of communication. Kindly, help me out with documentations (if any) available to accomplish message transmission without ardupilot/Px4. If someone could guide me on this issue, it would be really helpful.
Thanks
If you use RPi you not need the 3DR Radio Telemetry. You can connect RPi by WiFi or LTE to send/receive the Mavlink data. You need:
Wire the RPi to the flight controller;
Setup the Mavlink port in the QGroundControl, let the flight controller send the data to a telemetry port;
Install the mavlink-router to RPI;
Setup the mavlink-router send data to groundstation;
When you can receive mavlink data in the groundstation, you use pymavlink to send mavlink message. There is a sample code
If I have understood the question correctly, then what you are trying to do is to develop your own flight controller using raspberry pi. And, you want it to be detected as a vehicle in the QGC.
The example code that you are referring to from mavlink repository is used on the ground station to receive messages from a flight controller and send command messages to it. However, you are trying to send message from flight controller to the ground station.
What you will first have to do is to keep sending a heartbeat message at the rate of 1Hz as described in heartbeat microservice on mavlink documentation page. QGroundControl will detect a vehicle if it receives heartbeats from a vehicle I guess for about 5 seconds continuously. You have to keep sending the heartbeat at 1Hz after that as well, otherwise the QGC will state communication loss even if you are sending other mavlink messages to it.
On top of that, there are many mavlink messages which can have different interpretation for different flight controller. So, at some point you either will have to match your code to a certain extent to an existing flight controller or you will have to modify QGC to work with how you implement your flight controller.
Raspberry pi is acts as local host i'm trying to send data to raspbberry pi using mqtt with node mcu with two different topics.
eg:
if x>10 then i send 1 otherwise 0
same logic i have used in both node mcu.
if im communicate with only one nodemcu it getting good response but if i connect with both nodemcu's then sometimes not getting value in raspberry pi console.
This often depends on both the client and the broker used, and your configuration of each. The fact that two have problems where one does not suggests a client ID collision: every MQTT client device must have a different client ID. If a broker receives subscriptions from two clients with the same ID, the broker may disconnect one, usually the first. If each client is configured to reconnect, this can cause an endless series of disconnects for both, each of them connected half the time.
Any broker that does not disconnect duplicate clients could still fail to deliver to one, because it uses the client IDs to track which clients a message has been delivered to. The first client that pings for messages on its subscriptions will receive the latest message, and any later ones will miss that message because the message is already marked as delivered to that client ID.
Most clients avoid these problems with random IDs, yet let the developer set one manually. Does your identical logic set a client ID? You can verify what is actually set on each device through the broker's logs.
I am trying to do an IoT project where a node needs to receive some commands from cloud.
Previously, I made this with MQTT protocol. But searching about other protocols on internet I have found REST, which makes possible to communicate over HTTP. However I have seen that this protocol is more used for getting data (makeing a request to the node and receiving the data from it in the response).
I am very new with this protocol. So, I would like to know if it would also be possible to send commands to the node from the cloud, like in MQTT.
Thank you very much for your help.
The HTTP protocol and thus is based in a Request/Response model and using it for IoT device commands can have its drawbacks:
Your Devices will need to be accessible over the Internet and this can be a security compromise when you can have the Device with internet access using MQTT subscriptions but no incoming connections to it are allowed.
You will need to have some mechanisms for retrying and handling offline Devices in your cloud application sending the commands.
There is also more overhead on HTTP than MQTT given that MQTT has long lived connections. With HTTP you will waste more power on connecting and disconnection, also the network packets will be bigger than MQTT.
Can you use REST/HTTP for sending commands to Devices? Yes. Should you do it? Probably not, it all depends on your actual Device requirements and capabilities and why are you wanting to replace MQTT.
An IoT protocol that is similar to REST is CoAP. If you are thinking about using REST in the IoT context I recommend to have a look at it. If your nodes are always connected/reachable there is nothing that speaks against going in this direction. CoAP is a decentralized service protocol where each node might speak with another node.
MQTT is a publish/subscribe protocol with a central broker. As such your cloud could just send the commands to the broker and whenever the node connects to the broker it will receive the commands. So the node doesn't have to be online the whole time.
I am working on a legacy modbus program for an industrial SCADA system.
Currently, the c++ program acts as both a modbus TCP server and client.
Client behaviour:
It reads from a number of vendor PLCs (servers) on site, performs calculations and sends control commands back to the PLCs based on the data received across the site.
Server behaviour:
responds to a variety of TCP read and write requests from web interfaces and laptops on site.
Until now, this has worked fine, but we have recently installed a logging client on the network which polls our program very frequently (sub-second) and this has revealed timing issues: the program can potentially take a very long time in its client loop performing calculations and reading PLC values before acting as a server and responding to incoming requests.
Easy solution would be to split the programs into a modbus server and client instance, and keep them both running on the same embedded PC.
The issue I have is that the remote web interface (HMI) must be able to control the behaviour of vendor PLC 2 and Vendor PLC 2 will only allow one TCP connection from the embedded PC. In the past the program has handled writes requests from the HMI by forwarding them on to the PLC 2 via the open socket.
I'd be keen to gather thoughts on best practices here.
My thinking:
the modbus server program will need to respond to the HMI requests and somehow store the information required for vendor PLC 2, and it will also need to set a status register to inform the modbus client that there is data for vendor PLC 2.
The modbus client program will need to read the status register (and data) from the server and pass this on to vendor PLC 2.
Am I heading in the right direction?
Without having details on your implementation I can only guess the problem is that your program is single-threaded, and delays are caused by waiting responses from PLCs.
So, if my assumption is correct, you need to switch to 'select' function and redesign your software to be totally async. You have to put all sockets (both connected and accepted) in a FDs set and wait events on them.
win32:
https://learn.microsoft.com/en-us/windows/desktop/api/winsock2/nf-winsock2-select
linux:
https://www.opennet.ru/cgi-bin/opennet/man.cgi?topic=select&category=2
I've written the same app ages ago on win32 (but without calculations) and it easily processed about 200 PLCs, working on the same machine with SCADA.
I'm trying to build instant messaging app. Clients will not only send messages but also often send audios. And I've decided to use websocket connection to communicate with clients. It is fast and allows to send binary data.
The main idea is to receive from client1 message and notify about it client2. But here's the thing. My app will be running on GAE. And what if client1's socket is opened on server1 and client2's is opened on server2. This servers don't know about each others clients.
I have one idea how to solve it, but I am sure it is shitty way. I am going to use some sort of communication between servers(for example JMS or open another websocket connection between servers, doesn't matter right now).
But it surely will lead to a disaster. I can't even imagine how often those servers will speak to each other. For each message server1 should notify server2, server2 should notify client2. But things become even worse when serverN comes into play.
Another way I see this to work is Firebase. But it restricts message size to 4KB. So I can't send audios via it. As a solution I can notify client about new audio and he goes to my server for it.
Hope I clearly explained the problem. Does anyone know how to solve it? Or maybe there are another ways to build such apps?
If you are building a messaging cluster and expect communicating clients to connect to different instances of the server then server-server communication is inevitable. Usually it's not a problem though.
First, if you don't use any load balancing your clients will connect to the same server 50% of time on average (in case of 2 servers).
Second, intra-datacenter links are fast and free in all known public clouds.
Third, you can often do something smart on the frontend to make sure two likely to communicate clients connect to the same server. For instance direct all clients from the same country to the same server using DNS load balancing.
The second part of the question is about passing large media files. It's a common best practice to send it out of band - store on the server and only pass the reference to it. Like someone suggested in the comment, save the audio on the server and just send a message like "audio is available, fetch it from here ...". You don't need to poll the server for that. Just fetch it once when the receiving client requests it.
In general, it seems like you are trying to reinvent the wheel. Just use something off the shelf.
Let all client get connected to multiple servers and each server keeps this metadata
A centralized system like zookeeper stores active servers details
When a client c1 sends a message to client c2:
the message is received by a server (say s1, we can add a load balancer to distribute incoming requests)
s1 will broadcast this information to all other servers to get which server the client c2 is connected to OR a better approach to use consistent hashing which decides which server the client can connect to & in this approach message broadcast is not required
the corresponding server responses to server s1 (say s2)
now s1 sends the message m to s2 and server s2 to client c2
Cons of the above approach:
Each server will have a connection with the n-1 servers, creating a mesh topology
Centralized system (zookeeper) becomes a single point of failures (which is solvable)
Apps like Whatsapp, G-Talk uses XMPP and TCP/IP.