Prevent nginx from killing idle tcp sockets - sockets

I'm trying to use nginx as a reverse proxy for ssl/tcp sockets (so that I can write my server custom as raw tcp, but have nginx handle the ssl certificates). My use case requires the tcp connections remain alive, but to go idle (no packets back and forth) for extended periods of time (determined by the client, but as long as an hour). Unfortunately, nginx kills my socket connections after the first 10 minutes (timed to within a second) of inactivity, and I haven't been able to find either online or in the docs what actually controls this timeout.
I know that it has to be nginx doing it (not my raw server timing out, or my client's ssl socket), since I can directly connect to the server's raw tcp server without timeout issues, but if I run nginx as a raw tcp reverse proxy (no ssl) it does timeout.
Here's some code to reproduce the issue, note that I've commented out the ssl relevent pieces in nginx because the timeout occurs either way.
/etc/nginx/modules-enabled/test.conf:
stream {
upstream tcp-server {
server localhost:33445;
}
server {
listen 33446;
# listen 33446 ssl;
proxy_pass tcp-server;
# Certs
# ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
# ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
}
}
server.js;
const net = require("net");
const s = net.createServer();
s.on("connection", (sock) => {
console.log('Got connection from', sock.remoteAddress, sock.remotePort );
sock.on("error", (err) => {
console.error(err)
clearInterval(i);
});
sock.on("close", () => {
console.log('lost connection from', sock.remoteAddress, sock.remotePort );
clearInterval(i);
});
});
s.listen(33445);
client.js
const net = require('net');
const host = 'example.com';
let use_tls = false;
let client;
let start = Date.now()
// Use me to circumvent nginx, and no timeout occurs
// let port = 33445;
// Use me to use nginx, and no timeouts occur after 10 mins of no RX/TX
let port = 33446;
client = new net.Socket();
client.connect({ port, host }, function() {
console.log('Connected via TCP');
// Include me, and nginx doesn't kill the socket
// setInterval(() => { client.write("ping") }, 5000);
});
client.on('end', function() {
console.log('Disconnected: ' + ((Date.now() - start)/1000/60) + " mins");
});
I've tried various directives in the nginx stream block, but nothing seems to help. Thanks in advance!

Related

"EADDRINUSE: address already in use" On every change even when port is changes

I have a basic server with socket.io, but every time I do a change on the server, it throws the error:
Error: listen EADDRINUSE: address already in use :::3000
But, even if I change the port, it will throw the error anyways. I used sudo kill -9 and that allowed me to run the server with no problems the first time, but if I make a change, or if I reset it, it will throw the error again.
SERVER CODE
var express = require('express');
var app = express();
var http = require('http').createServer(app);
var io = require('socket.io')(http);
app.get("/", function(request, result) {
result.send("== SERVIDOR ==");
});
var port = 3000
http.listen(port, function() {
console.log("-- Servidor iniciado")
console.log("-- Escuchando puerto "+ port +"...")
})

MqttBrowserClient fails to connect due to missing conack package

I am trying to make webapp over flutter which will connect to HIVE broker. I took the broker name from the official website, set the port number to 8000 just like mentioned there and still get the error message as below:
error is mqtt-client::NoConnectionException: The maximum allowed connection attempts ({1}) were exceeded. The broker is not responding to the connection request message (Missing Connection Acknowledgement?
I really have no clue how to proceed. Can someone please help?
Below is my code:
MqttBrowserClient mq = MqttBrowserClient(
'wss://broker.mqttdashboard.com:8000', '',
maxConnectionAttempts: 1);
/*
MqttBrowserClient mq = MqttBrowserClient('ws://test.mosquitto.org', 'client-1',
maxConnectionAttempts: 1);
*/
class mqttService {
Future<MqttBrowserClient?> connectToServer() async {
try {
final connMess = MqttConnectMessage()
.withClientIdentifier('clientz5tWzoydVL')
.authenticateAs('a14guguliye', 'z5tWzoydVL')
.withWillTopic('willtopic')
.withWillMessage('My Will message')
.startClean() // Non persistent session for testing
.withWillQos(MqttQos.atLeastOnce);
mq.port = 1883;
mq.keepAlivePeriod = 50;
mq.connectionMessage = connMess;
mq.websocketProtocols = MqttClientConstants.protocolsSingleDefault;
mq.onConnected = onConnected;
var status = await mq.connect();
return mq;
} catch (e) {
print("error is " + e.toString());
mq.disconnect();
return null;
}
}
}
That port 8000 may be open but the HiveMQ broker may not be listening.
Make sure that the broker is fully booted and binds to that IP:Port combo.
In the HiveMQ broker startup output, you should see something similar to:
Started Websocket Listener on address 0.0.0.0 and on port 8000
If needed, the HiveMQ Broker configuration documentation is here.
You can use the public HiveMQ MQTT Websocket demo client to test your connection to make sure it's not a local code issue.
As a last option, use Wireshark to monitor MQTT traffic with a filter of tcp.port == 8000 and mqtt

Sending a message to a Kafka Broker using TCP Sockets and Deno

We are trying to create a Kafka client using Deno and TCP sockets. As a first step, we are trying to ping a broker we have running on a docker instance using the below code:
import {
Client,
Packet,
Event,
} from 'https://deno.land/x/tcp_socket#0.0.1/mods.ts';
const client = new Client({
hostname: 'localhost',
port: 9092,
});
for (let i = 0; i < 2; i++) {
// Connection open
client.on(Event.connect, (client: Client) => {
console.log('Connect', client.conn?.remoteAddr);
});
// Receive message
client.on(Event.receive, (client: Client, data: Packet) => {
console.log('Receive', data.toString());
});
// Connection close
client.on(Event.close, (client: Client) => {
console.log('Close');
});
// Handle error
client.on(Event.error, (e) => {
console.error(e);
});
// Do
await client.connect(); // Start client connect
await client.write('Hello World'); // Send string data
await client.write(new Uint8Array()); // Send Uint8Array data
client.close();
}
We are successfully pinging the broker, but receive the following error:
[2021-08-13 00:20:36,472] WARN Unexpected error from /172.20.0.1; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 1214606444 larger than 104857600)
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:91)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:169)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:150)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:365)
at org.apache.kafka.common.network.Selector.poll(Selector.java:313)
at kafka.network.Processor.poll(SocketServer.scala:494)
at kafka.network.Processor.run(SocketServer.scala:432)
at java.lang.Thread.run(Thread.java:745)
[2021-08-13 00:28:19,708] INFO [Group Metadata Manager on Broker 0]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
We understand that this is due to a protocol issue and are trying to determine what the best way forward is to address this issue. Any help would be greatly appreciated

Connecting two computers in different networks using socket and Port forwarding

I have set up a simple client-server communication code and it works well in my computer when my computer itself acts as a server and the client.
Now I am trying to run this same code on two different computers in different networks( different locations) where my computer will act as a server and my friend's computer as a client.
I have done port forwarding in my router as well as in my friend's router for the port which we are trying to communicate. We both have set up a static IP in our internal network behind the router. We both had shutdown the firewall while running the code.
I am running my code on Jupiter notebook and the same is my friend too.
here is my server code:
import socket
import threading
HEADER = 64
PORT = 5064
SERVER = '0.0.0.0'
ADDR = (SERVER, PORT)
FORMAT = 'utf-8'
DISCONNECT_MESSAGE = "quit"
Receive_from_client = "get info"
server = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
server.bind(ADDR)
def handle_client(conn, addr):
print("\n" + f"[NEW Connection] detected from IP: {addr[0]} & Port:{addr[1]} ")
conn.send(f"connected to server {ADDR}".encode(FORMAT))
connected =True
while connected:
msg_length = conn.recv(HEADER).decode(FORMAT) # decode the msg from byte to utf-8 format
if msg_length:
msg_length = int(msg_length)
msg = conn.recv(msg_length).decode(FORMAT)
if msg == DISCONNECT_MESSAGE:
connected = False
print(f" [Client][{addr}] {msg}")
print("Your session is disconnected")
break
if msg == Receive_from_client:
print("\n" + f"Send your msg to client with IP: {addr[0]}")
thread = threading.Thread(target = send2client, args = (conn, addr))
thread.start()
print(f" [Client][{addr}] {msg}")
conn.send(f"Msg received by server with IP:{addr[0]}".encode(FORMAT))
conn.close()
server.close()
def start():
server.listen()
print("\n"+ f"[LISTENING] Server is listening from IP: {SERVER} ")
while True:
conn, addr = server.accept()
thread = threading.Thread(target = handle_client, args = (conn, addr))
thread.start()
Here is the client code
import socket
import threading
HEADER = 64
PORT = 5064
FORMAT = 'utf-8'
DISCONNECT_MESSAGE = "quit"
SERVER = '103.192.207.250' # SERVERS public IP
ADDR = (SERVER, PORT)
client = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
client.connect(ADDR)
def send2server():
while True:
msg = input()
message = msg.encode(FORMAT)
msg_lenght = len(message)
send_length = str(msg_lenght).encode(FORMAT)
send_length += b' '*(HEADER - len(send_length))
client.send(send_length)
client.send(message)
print(client.recv(2048).decode(FORMAT))
if msg == DISCONNECT_MESSAGE:
print("session closed")
client.close()
def start():
print("\n"+ f"[LISTENING] client is listening from IP: {ADDR} ")
send2server()
I have opened the port by going on windows firewall defender and selecting new inbound and outbound rules to open 5064 TCP port.
but still, the code doesn't works..
my server keeps waiting for connection and the client-side after few seconds of running gives this error:
TCP error code 10060: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
Even while my server is listening from port 5064 when I scan this port to check if the port is open or not it says closed.
How do I check if the port I have forwarded is for sure open and also how do I get this thing work?
I have tried all of this and if there is any other thing I am missing please tell. I am struggling to get this work for the past 3 days.

How to setup HTTP and HTTPS on the same port with akka-http

I have a Scala app that runs an akka-http webserver on a custom port, let's say 8000.
Until a while ago, it would only handle http:// requests, but recently I switched to https://.
Some of the clients have the link bookmarked and keep getting the no connection error because they try the address with http:// instead of https:// and they keep forgetting why it happens.
I tried binding two services to the same port but failed because only the first one gets binded.
Http().bind(interface = "0.0.0.0", port = Global.settings.restPort, connectionContext = httpsContext)
Http().bind(interface = "0.0.0.0", port = Global.settings.restPort)
All I need from the http:// server is to return a 301 code and redirect to the same address, but with https protocol.
How can I achieve that?
As others have commented, you can't bind the HTTP and HTTPS servers to the same port. You can have both servers running on separate ports and redirect all HTTP traffic to the HTTPS server using Akka-http's scheme() and redirect():
val hostName = "www.example.com"
val portHttp = 8080
val portHttps = 8443
val route =
scheme("http") {
extract(_.request.uri) { uri =>
redirect( uri.withScheme("https").withAuthority(hostName, portHttps),
StatusCodes.MovedPermanently
)
}
} ~
pathSingleSlash {
get {
complete( HttpEntity( ContentTypes.`text/html(UTF-8)`,
"Welcome to Akka-HTTP!"
) )
}
}
Http().bindAndHandle(route, hostName, portHttp)
Http().bindAndHandle(route, hostName, portHttps, connectionContext = httpsContext)
Note that there is no need for applying withAuthority() if you're using standard HTTP and HTTPS ports (i.e. 80 and 443).