I am following the simple TCP example found here: https://docs.julialang.org/en/v1/manual/networking-and-streams/#A-simple-TCP-example-1. The code can be seen below (I modified that from the link slightly):
using Sockets
using Printf
# listen will create a server waiting for incoming connections on the specified
# port (in this case it will be localhost::2000)
#async begin
server = listen(2000)
x :: Int = 1
while true
sock = accept(server)
#printf "Connection number %d\n" x
x += 1
end
end
for i = 1:10
connect(2000)
end
When I execute the commands within the REPL it works correctly producing the following output:
julia> #async begin
server = listen(2000)
x :: Int = 1
while true
sock = accept(server)
#printf "Connection number %d\n" x
x += 1
end
end
Task (runnable) #0x00000001169d06d0
julia> for i = 1:10
connect(2000)
end
Connection number 1
Connection number 2
Connection number 3
Connection number 4
Connection number 5
Connection number 6
Connection number 7
Connection number 9
Connection number 10
However, when I try to place these commands in a file:
using Sockets
using Printf
# listen will create a server waiting for incoming connections on the specified
# port (in this case it will be localhost::2000)
#async begin
server = listen(2000)
x :: Int = 1
while true
sock = accept(server)
#printf "Connection number %d\n" x
x += 1
end
end
for i = 1:10
connect(2000)
end
and run using julia <scriptName> or (from within the REPL) include("<scriptName>") I get the following error message:
ERROR: LoadError: IOError: connect: connection refused (ECONNREFUSED)
Stacktrace:
[1] wait_connected(::TCPSocket) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Sockets/src/Sockets.jl:520
[2] connect at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Sockets/src/Sockets.jl:555 [inlined]
[3] connect at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Sockets/src/Sockets.jl:541 [inlined]
[4] connect at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Sockets/src/Sockets.jl:537 [inlined]
[5] top-level scope at <pathToFile>:18
[6] include at ./boot.jl:328 [inlined]
[7] include_relative(::Module, ::String) at ./loading.jl:1105
[8] include(::Module, ::String) at ./Base.jl:31
[9] include(::String) at ./client.jl:424
[10] top-level scope at REPL[1]:1
in expression starting at <pathToFile>:17
How would I run this program from a script? I am fairly new to Julia and to sockets so apologies if this is an easy question!!
You're getting an error because connect will fail if listen has not yet had time to run. As you discovered, introducing a small pause between your server and client sides fixes the issue; let's try and explain this in more details.
In the REPL-based, interactive version, as soon as the server task is created using #async, control goes back to the REPL, which performs blocking operations (such as waiting for new commands to be entered in the command line). This gives the scheduler an opportunity to yield control to the newly created task. The call to listen appears early in it, ensuring that whenever control reaches the client side again - and calls to connect are executed - the server is listening.
In the script version, the server task gets created and scheduled, but the scheduler does not have any opportunity to actually run it, since the main task does not perform any blocking call before running connect. Therefore no-one is listening to the socket when the connection is performed, and the call fails. However, any blocking call put in this place will give an opportunity for the scheduler to run the server task. This includes sleep (of arbitrarily small amounts of time, as you noted), but also any function performing I/O: a simple println("hello") would have worked too.
I would think a cleaner way to fix things would be to ensure that the call to listen is performed first, by running it synchronously beforehand:
using Sockets
using Printf
# listen will create a server waiting for incoming connections on the specified
# port (in this case it will be localhost::2000)
server = listen(2000)
#async begin
x = 1
while true
sock = accept(server)
#printf "Connection number %d\n" x
x += 1
end
end
for i = 1:10
connect(2000)
end
Now you're left with another problem: when the client loop terminates, all calls to connect have been issued, but not necessarily all corresponding calls to accept have had time to run. This means that you're likely to get truncated output such as:
Connection number 1
Connection number 2
Connection number 3
Connection number
You'd need further coordination between your tasks to correct this, but I suspect this second problem might be only related to the MWE posted here, and might not appear in your real use case.
For example, presumably the server is meant to send something to the client. The read-write operations performed on sockets in this case naturally synchronizes both tasks:
using Sockets
using Printf
# listen will create a server waiting for incoming connections on the specified
# port (in this case it will be localhost::2000)
server = listen(2000)
#async begin
x = 1
while true
sock = accept(server)
msg = "connection number $x\n"
print("Server sent: $msg")
write(sock, msg)
x += 1
end
end
for i = 1:10
sock = connect(2000)
msg = readline(sock)
println("Client received: $msg")
end
The above example correctly yields a complete (untruncated) output:
Server sent: Connection number 1
Client received: Connection number 1
Server sent: Connection number 2
Client received: Connection number 2
...
Server sent: Connection number 10
Client received: Connection number 10
So I've figured this out I believe. The server and client either need to be separated between two files, OR, a pause command (seems to be okay with an arbitrarily small pause command) needs to be in between the server and client side:
#!/Applications/Julia-1.3.app/Contents/Resources/julia/bin/julia
using Sockets
using Printf
# listen will create a server waiting for incoming connections on the specified
# port (in this case it will be localhost::2000)
port = 2000
#async begin
server = listen(IPv6(0),port)
x::Int = 1
while true
sock = accept(server)
#printf "Connection number %d\n" x
x += 1
end
end
sleep(1E-10)
for i = 1:10
connect(port)
end
This is now working as expected!
Related
I want to simulate test cross-platform connection failures / timeouts, starting with blocking connect()s:
#!/usr/bin/python3
import socket
s = socket.socket()
endpoint = ('localhost', 28813)
s.bind((endpoint))
# listen for connections, accept 0 connections kept waiting (backlog)
# all other connect()s should block indefinitely
s.listen(0)
for i in range(1,1000):
c = socket.socket()
c.connect(endpoint)
# print number of successfully connected sockets
print(i)
On Linux, it prints "1" and hangs indefinitely (i.e. the behavior I want).
On Windows (Server 2012), it prints "1" and aborts with a ConnectionRefusedError.
On macOS, it prints all numbers from 1 to 128 and then hangs indefinitely.
Thus, I could accept the macOS ignores the backlog parameter and just connect enough sockets for clients to block on new connections.
How can I get Windows to also block connect() attempts?
On Windows, the SO_CONDITIONAL_ACCEPT socket option allows the application to have the incoming connections wait until it's accept()ed. The constant (SO_CONDITIONAL_ACCEPT=0x3002) isn't exposed in the Python module, but can be supplied manually:
s.bind(endpoint)
s.setsockopt(socket.SOL_SOCKET, 0x3002, 1)
s.listen(0)
It's so effective that even the first connect is kept waiting.
On macOS, backlog=0 is reset to backlog=SOMAXCONN, backlog=1 keeps all connections except the first waiting.
I've a bunch background tasks (Sidekiq workers) that update database, and I keep getting this failing thread exception.
Heroku Log: WARN: could not obtain a database connection within 5.000 seconds (waited 5.001 seconds)
Heroku
Heroku Postgres :: Olive - 20 connections limit.
Redis ToGo - 10 connections limit.
Sidekiq - 2 connections.
Each client request create ~50 threads - finally ~20 threads trying to update db.
Now I know this is too much threads trying to make connection (updating Active:Record..).
I don't mind them to wait in try again until success.
-config/unicorn.rb
worker_processes Integer(ENV["WEB_CONCURRENCY"] || 3)
timeout 30
preload_app true
before_fork do |server, worker|
Signal.trap 'TERM' do
puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
Process.kill 'QUIT', Process.pid
end
if defined?(ActiveRecord::Base)
ActiveRecord::Base.connection.disconnect!
end
end
after_fork do |server, worker|
Signal.trap 'TERM' do
puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to send QUIT'
end
if defined?(ActiveRecord::Base)
config = ActiveRecord::Base.configurations[Rails.env] ||
Rails.application.config.database_configuration[Rails.env]
config['pool'] = ENV['DB_POOL'] || 2
config['reaping_frequency'] = ENV['DB_REAP_FREQ'] || 10 # seconds
ActiveRecord::Base.establish_connection(config)
end
end
-config/initializers/sidekiq.rb
require 'sidekiq'
Sidekiq.configure_server do |config|
if(database_url = ENV['DATABASE_URL'])
p pool_size = Sidekiq.options[:concurrency] + 2
p ENV['DATABASE_URL'] = "#{database_url}?pool=#{pool_size}"
ActiveRecord::Base.establish_connection
end
end
--condig/sidekiq.yml
:concurrency: 2
Thanks a lot for all the help,
Eldar
I have created a TCP socket at one end of my application. Say the end is 1. This socket closes after about 10sec. Now the other side of my application (end 2) is allowed to connect to the above created socket. Im coding this socket app in python, so suppose my end 2 is trying to connect to the TCP socket, but the socket no longer exists, my program terminates because of some exception. I dont want that to happen. It's like there is a while loop in my end 2. So if a connection is not available it goes back and wait.
Are you handling the exception correctly ..
try:
s.connect((host,port))
except socket.error, (value,message):
if s:
s.close()
print "Could not open socket: " + message
"""Code to handle a retry"""
On getting an error .. you can retry by doing a bind and listen again.. Also you need to have retry count ..say 5 and then perhaps exit.
I have a standalone lua script that uses lua sockets to connect to a server via TCP IP. It uses receive call to receive data from that server. It works, however, when I try to stop it with Ctrl+C, one of the two scenarios is happening:
-If there is currently no traffic and receive is waiting, Ctrl+C will have no effect. The program will continue to run, and will have to be terminated by kill.
-If there is traffic, the program will exit with the below printout and with the socket still open and with the server not accepting another connection:
lua: luaSocketTest.lua:15: interrupted!
stack traceback:
[C]: in function 'receive'
luaSocketTest.lua:15: in function 'doWork'
luaSocketTest.lua:22: in main chunk
[C]: ?
I tried using pcall to solve the second scenario, without success. pcall doesn't return, the process still throws the error.
Sample of my program is below:
local socket = require ("socket")
local ip = "localhost"
local port = 5003
function doWork ()
print ("Starting socket: "..ip..":"..port)
client = assert(socket.connect(ip, port))
print ("Socket Accepted")
client:send("TEST TEST")
while 1 do
local byte, err = client:receive (1)
if not err then
print (byte)
end
end
end
while 1 do
local status = pcall(doWork())
print ("EXITED PCALL WITH STATUS: "..tostring(status))
if not status then client:close() end
end
This would be quite a change, but you could employ lua-ev. It allows to add Signal handlers, which is exactly what is required to react to ctrl-c.
local socket = require'socket'
-- make connect and send in blocking mode
local client = socket.connect(ip,port)
client:send('TEST TEST')
-- make client non-blocking
client:settimeout(0)
ev.IO.new(function()
repeat
local data,err,part = client:receive(10000)
print('received',data or part)
until err
end,client:getfd(),ev.READ):start(ev.Loop.default)
local ev = require'ev'
local SIGINT = 2
ev.Signal.new(function()
print('SIGINT received')
end,SIGINT):start(ev.Loop.default)
ev.Loop.default:loop()
I'm trying to send some data in bits from one computer to another one that are on the same network using TCP in Matlab.
Currently this is what I have setup to open the connections. I'm trying to simulate a peer-to-peer connection because they need to send and receive data from each other. It works fine on my local machine when I run it using my IPv4 and IPv6.
%code starts in one file
openRecieve('0.0.0.0', 3000); %accept all connections
openSend('10.32.41.235',3000);
Then I do the same in another file and I can run them i parallel on my machine:
%code starts in other file
openSend('10.32.41.235',3000); %IPv4 of PC
openRecieve('0.0.0.0', 3000); %accept all connections
The IPs are bogus...... This code works on my machine when run with 2 different instances of matlab open. However it doesn't work between 2 different computers.
Code for openReceive:
function connectionServer = openRecieve(client, port)
t = tcpip('0.0.0.0', port, 'NetworkRole', 'Server');
set(t, 'InputBufferSize', 3000000);
% Open connection to the client.
fopen(t);
fprintf('%s \n','Client Connected');
connectionServer = t;
set(connectionServer,'Timeout',.1);
end
Code for openSend:
function connectionSend = openSend(host, port)
d = tcpip(host, port, 'NetworkRole', 'Client');
set(d, 'OutputBufferSize', 3000000); % Set size of receiving buffer, if needed.
%Trying to open a connection to the server.
while(1)
try
fopen(d);
break;
catch
fprintf('%s \n','Cant find Server');
end
end
connectionSend = d;
end
Any help is appreciated.
It is now running although the only thing I changed was the port numbers from 3000 and 3000 to 3000 and 3001.......... Also using only IPv4 was pretty key as my network didn't allow for IPv6.
For anyone trying to write TCP code in Matlab just use '0.0.0.0' for the connection if you don't care who is connecting as it will accept all IPs trying to connect on that port #.
Current code for first file:
sConec = openSend('10.234.24.124', 3000); %IPv4 Address of comp your trying to connect to
rConec = openRecieve('0.0.0.0', 3001); %Accept all connections
Current code for second file:
rConec = openRecieve('0.0.0.0', 3000); %Accept all connections
sConec = openSend('10.109.22.142', 3001); %IPv4 Address of computer your trying to connect to