Related
I try to implement a CLI with 2 commands:
run that spawns an unix socket at /my/path/file.sock and produces output
watch that connects to the socket and forward the output to its own output
When I kill the watch command, I can't understand why the run command is also killed. They don't raise any exception nor sys signal. I noticed that the problems comes from the flush function. Once called, it acts like a exit 0. How come?
Here the sources: https://github.com/soywod/comodoro
The server has to listen for client disconnection and remove the connection from the conn list.
Something like this might work:
let client_thread conn =
let in_ch = in_channel_of_descr conn in
while true do
try input_line in_ch |> ignore
with End_of_file ->
(* Remove conn from !conn and close the conn *)
done
in
let add_conn () =
(* ... *)
Thread.create client_thread conn |> ignore;
(* ... *)
in
I have socket server with this method:
#impl true
def handle_call({:tcp, socket, packet}, state) do
Logger.info("Received packet: \x02#{packet}\x03 and send response")
{:reply, {:ok, packet}, state}
end
I wrote script in python that send "\x02Test\x03" to socket:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", port))
s.send("\x02Test\x03".encode())
print(s.recv(1024))
But the response printed by python is b'\x02\x02Test\x03\x03'
What does handle_call() have to do with gen_tcp?
gen_tcp can be configured to read from a socket and send() messages to whatever process called :gen_tcp.accept(), the so called controlling process, which can be a gen_server; but a gen_server handles messages sent to its mailbox with handle_info()--not handle_call(). handle_call() handles messages sent by :gen_server.call().
Here's an example:
defmodule TcpServer do
use GenServer
require Logger
def start_link() do
ip = Application.get_env :gen_tcp, :ip, {127,0,0,1}
port = Application.get_env :gen_tcp, :port, 6666
IO.puts "gen_tcp is listening on port: #{port}"
GenServer.start_link(__MODULE__, {ip, port},[])
end
def init({ip, port}) do
{:ok, listen_socket}= :gen_tcp.listen(
port,
[:binary, {:packet,0}, {:active,true}, {:ip,ip}]
)
{:ok, socket } = :gen_tcp.accept listen_socket
{:ok, %{ip: ip, port: port, socket: socket} }
end
def handle_call({:tcp, _socket, packet}, state) do
Logger.info("handle_call(): Received packet: #{inspect packet}")
{:reply, {:ok, packet}, state}
end
def handle_info({:tcp,socket,packet},state) do
Logger.info "handle_info(:tcp, ...): incoming packet: #{inspect packet}"
:gen_tcp.send(socket, "****#{packet}*****")
{:noreply,state}
end
def handle_info({:tcp_closed, _socket}, state) do
Logger.info("handle_info({:tcp_closed, ...): Client closed socket.")
{:noreply, state}
end
def handle_info({:tcp_error, socket, reason}, state) do
Logger.info("Connection closed due to #{reason}: #{socket}")
{:noreply,state}
end
end
To start the server:
~/elixir_programs/tcp_server$ iex -S mix
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]
Interactive Elixir (1.8.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> TcpServer.start_link()
gen_tcp is listening on port: 6666
In another terminal window:
~/python_programs$ cat 5.py
import socket
port = 6666
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", port))
s.send("\x02Test\x03".encode())
print(s.recv(1024))
~/python_programs$ p36 5.py
b'****\x02Test\x03*****'
~/python_programs$
As you can see, there was no duplication of the \x02 and \x03 characters.
Back in the server window:
{:ok, #PID<0.123.0>}
iex(2)>
21:54:18.363 [info] handle_info(:tcp, ...): incoming packet: <<2, 84, 101, 115, 116, 3>>
21:54:18.369 [info] handle_info({:tcp_closed, ...): Client closed socket.
Do you have another process that is the controlling process and is calling :gen_server.call({:tcp, socket, packet})?
By the way, in this code:
def handle_info({:tcp,socket,packet},state) do
packet may be the whole packet, 1/2 the packet or 1/10 of the packet. That's the way sockets work. The config option {packet, 0} tells gen_tcp that there is no length header (0 bytes) on the front of the packet, while {packet, 1|2|4} tells gen_tcp that the length of the packet is contained in the first byte, the first 2 bytes, or the first 4 bytes respectively. That way gen_tcp can read the first 1|2|4 bytes to get the packet length, say L, then keep reading from the socket until it has received L bytes. Then gen_tcp packages up the pieces into one message, and sends the whole message to the gen_server. On the other hand, when you specify {packet, 0}, you are telling gen_tcp that the packet has no length header; and because a socket splits up a single packet into an indeterminate number of chunks, gen_tcp has no idea where the end of the packet is, so gen_tcp's only option is to read a chunk from the socket and send the chunk to the gen_server; then read another chunk and send the chunk to the gen_server, etc., etc. That means the gen_server has to figure out where the end of the packet is.
Therefore, your server and your client have to agree on a protocol to signal the end of a packet; and handle_info(:tcp, ...) will have to store the pieces of the packet in the state (or in a db) until it has read all the chunks that make up a packet.
One protocol that you can use to signal the end of the packet is: the client closes the socket. In that case,
def handle_info({:tcp_closed, _socket}, state)
will be called, and inside that function clause you can assemble the chunks stored in state (or in a db) into a complete message, then do whatever is necessary, e.g. send the message back to the client.
If you use STX and ETX as your begin message, end message protocol, then handle_info(:tcp, ...) will still have to look for the STX character to signal that it should start storing chunks in state, and when handle_info(:tcp, ...) finds a chunk with an ETX character in it, then you have to assemble the entire message.
I'm new in Erlang.
My Problem is, that when I start the client for the 1st time everything seems okay, I get the sorted list: <<1,5,72,97,108,108,111>>.
But by the 2nd time it won't receive the sorted list, because I think the socket is closed. The output from the Client is "Connection closed".
Here is my code:
Client
-module(client).
-export([client/0]).
client() ->
case gen_tcp:connect("localhost", 6000, [{mode, binary}]) of
{ok, Sock} ->
Data = [1, "Hallo", 5],
gen_tcp:send(Sock, Data),
receive
{tcp, _, Bin} ->
io:fwrite("Received sorted list from server: ~w~n", [Bin]);
{tcp_closed, _} ->
io:fwrite("Connection closed"),
gen_tcp:close(Sock)
end;
{error,_} ->
io:fwrite("Connection error! Quitting...~n")
end.
Server
-module(server).
-export([server/0]).
-import(mergeSort,[do_recv/1]).
%creates a tcp socket on Port 6000
server() ->
{ok, Listen} = gen_tcp:listen(6000, [{keepalive, true}, %send keepalive packets
{reuseaddr, true}, %reuse address
{active, once}, %socket is active once
{mode, list}]), %binary traffic
spawn(fun() -> parallel_connection(Listen) end).
%server is listening
%accepts the connection
%starts MergeSort
parallel_connection(Listen) ->
io:fwrite("Listening connections..~n"),
{ok, Socket} = gen_tcp:accept(Listen),
io:fwrite("Connection accepted from ~w~n", [Socket]),
spawn(fun() -> parallel_connection(Listen) end),
do_recv(Socket).
MergeSort
-module(mergeSort).
-export([do_recv/1]).
merge_sort(List) -> m(List, erlang:system_info(schedulers)).
%break condition
m([L],_) ->
[L];
%for more than one scheduler
m(L, N) when N > 1 ->
{L1,L2} = lists:split(length(L) div 2, L),
%self () returns Pid, make_ref() returns almost unique reference
{Parent, Ref} = {self(), make_ref()},
%starts a new process for each half of the list
%and sends Message to Parent
spawn(fun()-> Parent ! {l1, Ref, m(L1, N-2)} end),
spawn(fun()-> Parent ! {l2, Ref, m(L2, N-2)} end),
{L1R, L2R} = receive_results(Ref, undefined, undefined),
lists:merge(L1R, L2R);
m(L, _) ->
{L1,L2} = lists:split(length(L) div 2, L),
lists:merge(m(L1, 0), m(L2, 0)).
receive_results(Ref, L1, L2) ->
receive
{l1, Ref, L1R} when L2 == undefined -> receive_results(Ref, L1R, L2);
{l2, Ref, L2R} when L1 == undefined -> receive_results(Ref, L1, L2R);
{l1, Ref, L1R} -> {L1R, L2};
{l2, Ref, L2R} -> {L1, L2R}
after 5000 -> receive_results(Ref, L1, L2)
end.
do_recv(Socket) ->
%{ok, {Address, _}} = inet:peername(Socket),
receive
{tcp, Socket, List} ->
try
Data = merge_sort(List),
gen_tcp:send(Socket, Data),
io:fwrite("Sent sorted list to ~w | Job was done! Goodbye :)~n", [Socket]),
gen_tcp:close(Socket)
catch
_:_ ->
io:fwrite("Something went wrong with ~w | Worker terminated and connection closed!~n", [Socket]),
gen_tcp:close(Socket)
end;
{tcp_closed, _} ->
io:fwrite("Connection closed ~n");
{error, _} ->
io:fwrite("Connection error from ~w | Worker terminated and connection closed!~n", [Socket]),
gen_tcp:close(Socket)
end.
Can anyone help me?
When you call client:client/0, it creates a connection, sends its data, receives the response, then returns. Meanwhile, the server closes the socket. When you call client:client/0 again, it again creates a connection and sends data, but then it receives the tcp_closed message for the previous socket, and then it returns.
You can fix this by specifying the client socket in your receive patterns:
receive
{tcp, Sock, Bin} ->
io:fwrite("Received sorted list from server: ~w~n", [Bin]);
{tcp_closed, Sock} ->
io:fwrite("Connection closed"),
gen_tcp:close(Sock)
end;
In this code, the variable Sock replaces both the underscores in the original code, in the {tcp, _, Bin} and {tcp_closed, _} tuples. This forces the messages to match only for the specified socket.
EDIT: I solved this question myself, look at the first answer if you have a similar issue
I am new to sockets and made a client-server test modifying one example, it sends and receives data ok but when closing the server get caught in a loop receiving blank messages and I can't find out why even though I use shutdown(socket.SHUT_RDWR) to close the connection immediately according to the documentation.
I had to use two cmd of windows for each one (server and client) because when running first the server in idle and then the client I got in client_example ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it here is the code. Anyways, here it is the sequence I tried:
(in the client console):
c:\python33\python client_example.py
SEND( TYPE q or Q to Quit):Hi
SEND( TYPE q or Q to Quit):q
(in the server console):
c:\python33\python server_example3.py
TCPServer Waiting for client on port 7000
I got a connection from ('127.0.0.1', 49263)
RECEIVED: b'Hi'
RECEIVED: b''
RECEIVED: b''
RECEIVED: b''
RECEIVED: b''
RECEIVED: b''
.... <---After I did CTRL+C
Traceback (most recent call last):
File "server_example.py", line 19, in <module>
print("RECEIVED:",data)
File "c:\python33\lib\encodings\cp850.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
KeyboardInterrupt
Here is the code of the server and client:
server_example.py:
#TCP server example
import socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(("", 7000))
server_socket.listen(5)
print("TCPServer Waiting for client on port 7000")
while 1:
client_socket, address = server_socket.accept()
print("I got a connection from ", address)
while 1:
data = client_socket.recv(32)
if (data == 'q' or data == 'Q'):
client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
break;
else:
print("RECEIVED:",data)
client_example.py:
# TCP client example
import socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(("localhost", 7000))
while 1:
data = input( "SEND( TYPE q or Q to Quit):" )
if (data != 'Q' and data != 'q'):
my_bytes = bytearray()
for c in data:
my_bytes.append(ord(c))
client_socket.send(my_bytes)
else:
client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
break;
Well, I feel dumb that I solved this thing after 11 minutes of posting the question when I was unable to solve it for 2 hours...
The problem was that I was comparing data with the string 'q' which was correct in python 2 but in python 3 what is being sent are bytes, b'q' <> 'q' ...
Here is the corrected code:
server_example.py
#TCP server example
import socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(("", 7000))
server_socket.listen(5)
print("TCPServer Waiting for client on port 7000")
while 1:
client_socket, address = server_socket.accept()
print("I got a connection from ", address)
while 1:
data = client_socket.recv(32)
print("RECEIVED:",data)
if (data == b'q' or data == b'Q'):
## client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
break;
break;
client_example.py
# TCP client example
import socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(("localhost", 7000))
while 1:
data = input( "SEND( TYPE q or Q to Quit):" )
if (data != 'Q' and data != 'q'):
my_bytes = bytearray()
for c in data:
my_bytes.append(ord(c))
client_socket.send(my_bytes)
else:
my_bytes = bytearray()
my_bytes.append(ord(data))
client_socket.send(my_bytes)
## client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
break;
You're assuming you're going to get the Q. You should also engage in the same processing when you get EOS from read, whatever form that takes in Python.
#EJP : By EOS you mean end of stream? If so you are right, this example was a very simple one to get started in which I overlooked details like these.
Anyway I found that using this check in the server it is solved:
data = input( "SEND( TYPE q or Q to Quit):" )
if (data == b''):
break;
I found an interesting problem when using gen_tcp behavior. I have a server and a client. The server accepts connections and the client creates many processes that all try to connect to the listening server.
If I try to start the client which spawns many processes that all try to connect to the socket at the same time then many fail. However if I put timer:sleep(x) then every socket is being accepted.
Does this mean that gen_tcp:accept() has a limit where it can accept some connection request?
Code for the server and the client follows:
accept(State = #state{lsocket = LSocket, num = Num}) ->
case gen_tcp:accept(LSocket) of
{ok, Socket} ->
io:format("Accepted ~p ~n", [Num]),
{sockets, List} = hd(ets:lookup(csockets, sockets)),
NewList = [Socket | List],
ets:insert(csockets, {sockets, NewList}),
Pid = spawn(fun() -> loop(Socket) end),
gen_tcp:controlling_process(Socket, Pid),
accept(State#state{num = Num + 1});
{error, closed} -> State
end.
loop(Socket) ->
case gen_tcp:recv(Socket, 0) of
{ok, Data} ->
gen_tcp:send(Socket, Data),
loop(Socket);
{error, closed} ->
io:format(" CLOSED ~n"),
ok
end.
Client:
send(State = #state{low = Low, high = Low}) ->
State;
send(State = #state{low = Low}) ->
N = Low rem 10,
Dest = lists:nth(N + 1, State#state.dest),
spawn(?MODULE, loop, [Dest, Low]),
%%timer:sleep(1),
NewState = State#state{low = Low + 1},
send(NewState).
loop({IP, Port}, Low) ->
case gen_tcp:connect(IP, Port, [binary]) of
{ok, Socket} ->
io:format("~p Connected ~n", [Low]),
gen_tcp:send(Socket, "Hi"),
receive
{tcp, RecPort, Data} ->
io:format("I have received ~p on port ~p ~p ~n", [Data, RecPort, Low])
end;
_Else ->
io:format("The connection failed ~n"),
loop({IP, Port}, Low)
end.
It is true that a single process can only gen_tcp:accept/1 so fast, though I'm not sure that that's the bottleneck you're running in to.
You might be interested in Ranch, the TCP library for the Cowboy webserver. The manual includes a section on internal features that talks about using multiple acceptors.
In your case, you should try to produce more debugging output for yourself. Printing the error when the client fails to connect would be a good start -- there are lots reasons why a TCP client might fail to connect.