Elixir socket duplicate STX and ETX in response - sockets

Question

I have socket server with this method:
#impl true
def handle_call({:tcp, socket, packet}, state) do
Logger.info("Received packet: \x02#{packet}\x03 and send response")
{:reply, {:ok, packet}, state}
end
I wrote script in python that send "\x02Test\x03" to socket:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", port))
s.send("\x02Test\x03".encode())
print(s.recv(1024))
But the response printed by python is b'\x02\x02Test\x03\x03'

Answer 1

What does handle_call() have to do with gen_tcp?
gen_tcp can be configured to read from a socket and send() messages to whatever process called :gen_tcp.accept(), the so called controlling process, which can be a gen_server; but a gen_server handles messages sent to its mailbox with handle_info()--not handle_call(). handle_call() handles messages sent by :gen_server.call().
Here's an example:
defmodule TcpServer do
use GenServer
require Logger
def start_link() do
ip = Application.get_env :gen_tcp, :ip, {127,0,0,1}
port = Application.get_env :gen_tcp, :port, 6666
IO.puts "gen_tcp is listening on port: #{port}"
GenServer.start_link(__MODULE__, {ip, port},[])
end
def init({ip, port}) do
{:ok, listen_socket}= :gen_tcp.listen(
port,
[:binary, {:packet,0}, {:active,true}, {:ip,ip}]
)
{:ok, socket } = :gen_tcp.accept listen_socket
{:ok, %{ip: ip, port: port, socket: socket} }
end
def handle_call({:tcp, _socket, packet}, state) do
Logger.info("handle_call(): Received packet: #{inspect packet}")
{:reply, {:ok, packet}, state}
end
def handle_info({:tcp,socket,packet},state) do
Logger.info "handle_info(:tcp, ...): incoming packet: #{inspect packet}"
:gen_tcp.send(socket, "****#{packet}*****")
{:noreply,state}
end
def handle_info({:tcp_closed, _socket}, state) do
Logger.info("handle_info({:tcp_closed, ...): Client closed socket.")
{:noreply, state}
end
def handle_info({:tcp_error, socket, reason}, state) do
Logger.info("Connection closed due to #{reason}: #{socket}")
{:noreply,state}
end
end
To start the server:
~/elixir_programs/tcp_server$ iex -S mix
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]
Interactive Elixir (1.8.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> TcpServer.start_link()
gen_tcp is listening on port: 6666
In another terminal window:
~/python_programs$ cat 5.py
import socket
port = 6666
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", port))
s.send("\x02Test\x03".encode())
print(s.recv(1024))
~/python_programs$ p36 5.py
b'****\x02Test\x03*****'
~/python_programs$
As you can see, there was no duplication of the \x02 and \x03 characters.
Back in the server window:
{:ok, #PID<0.123.0>}
iex(2)>
21:54:18.363 [info] handle_info(:tcp, ...): incoming packet: <<2, 84, 101, 115, 116, 3>>
21:54:18.369 [info] handle_info({:tcp_closed, ...): Client closed socket.
Do you have another process that is the controlling process and is calling :gen_server.call({:tcp, socket, packet})?
By the way, in this code:
def handle_info({:tcp,socket,packet},state) do
packet may be the whole packet, 1/2 the packet or 1/10 of the packet. That's the way sockets work. The config option {packet, 0} tells gen_tcp that there is no length header (0 bytes) on the front of the packet, while {packet, 1|2|4} tells gen_tcp that the length of the packet is contained in the first byte, the first 2 bytes, or the first 4 bytes respectively. That way gen_tcp can read the first 1|2|4 bytes to get the packet length, say L, then keep reading from the socket until it has received L bytes. Then gen_tcp packages up the pieces into one message, and sends the whole message to the gen_server. On the other hand, when you specify {packet, 0}, you are telling gen_tcp that the packet has no length header; and because a socket splits up a single packet into an indeterminate number of chunks, gen_tcp has no idea where the end of the packet is, so gen_tcp's only option is to read a chunk from the socket and send the chunk to the gen_server; then read another chunk and send the chunk to the gen_server, etc., etc. That means the gen_server has to figure out where the end of the packet is.
Therefore, your server and your client have to agree on a protocol to signal the end of a packet; and handle_info(:tcp, ...) will have to store the pieces of the packet in the state (or in a db) until it has read all the chunks that make up a packet.
One protocol that you can use to signal the end of the packet is: the client closes the socket. In that case,
def handle_info({:tcp_closed, _socket}, state)
will be called, and inside that function clause you can assemble the chunks stored in state (or in a db) into a complete message, then do whatever is necessary, e.g. send the message back to the client.
If you use STX and ETX as your begin message, end message protocol, then handle_info(:tcp, ...) will still have to look for the STX character to signal that it should start storing chunks in state, and when handle_info(:tcp, ...) finds a chunk with an ETX character in it, then you have to assemble the entire message.