Haskell tcp server, fd is too big error - sockets

I've been trying to write a Haskell server for a Go client. For the Haskell TCP server I'm simply using Network.Socket. Whenever I try to run hWaitForInput, I am getting this error:
fdReady: fd is too big.
Here is the server code -
connHandler :: (Socket, SockAddr) -> IO()
connHandler (sock, _) = do
putStrLn "Starting Handler"
handle <- socketToHandle sock ReadWriteMode
hSetBuffering handle LineBuffering
hPutStrLn handle "Hello Client!"
putStrLn "Waiting for Input"
success <- hWaitForInput handle (1000*10)
putStrLn "Wait done"
if success
then do
putStrLn "Client timed out"
else do
msg <- hGetLine handle
putStrLn msg
hClose handle
The Go client is receiving and printing the server's message("Hello Client!") but the haskell server throws the error right after printing "Waiting for Input"

You aren't doing anything wrong. The specific error message you're seeing only shows up with GHC >=8.0.2 running on Windows and represents a bug/limitation in an internal GHC function fdReady that they've tried to address on non-Windows architectures but have left unfixed on Windows. (Don't feel too jealous, though -- the "fix" on non-Windows architectures is currently broken and crashes, too.) Trying an earlier version of GHC probably wouldn't help -- it would still cause an error, but the error message would be different.
Here's the problem: on Windows, the internal function fdReady uses the select() system call to poll file descriptors for sockets, and select is limited to a certain maximum numerical value for the file descriptors it can poll. It looks like the Windows default for this value is quite low (64) but can be increased at compile time (the time GHC is compiled, unfortunately, not the time when GHC compiles your program).
If you add the line:
hShow handle >>= putStrLn
just before your hWaitForInput, you should see some debug info printed for the socket, including something like loc=<socket: nnn> where nnn is the file descriptor. This may help you verify that you're seeing a file descriptor greater than 64 that's causing the problem.
If this is the case, I would suggest filing a GHC bug to see if you can get it fixed.

As an alternative/workaround, you could try reading a line in one thread and a timer in another thread.
putStrLn "Waiting for Input"
msgMVar <- newEmptyMVar
tid <- forkIO $ hGetLine handle >>= putMVar msgMVar
maybeExn <- waitTimeout tid (1000*10)
case maybeExn of
Nothing -> do
killThread tid
putStrLn "Client timed out"
Just (Just _) ->
putStrLn "Exception"
_ -> do
msg <- takeMVar msgMVar
putStrLn msg
hClose handle
This does have different behavior (can time out in the middle of reading a line) than your code (never times out if a single byte can be read, even if the line is not complete), though.

Related

No output from erlang tracer

I've got a module my_api with a function which is callback for cowboy's requests handle/2,
So when I make some http requests like this:
curl http://localhost/test
to my application this function is called and it's working correctly because I get a response in the terminal.
But in another terminal I attach to my application with remsh and try to trace calls to that function with a dbg module like this:
dbg:tracer().
dbg:tp(my_api, handle, 2, []).
dbg:p(all, c).
I expected that after in another terminal I make a http request to my api, the function my_api:handle/2 is called and I get some info about this call (at least function arguments) in the attached to the node terminal but I get nothing in there. What am I missing?
When you call dbg:tracer/0, a tracer of type process is started with a message handler that sends all trace messages to the user I/O device. Your remote shell's group leader is independent of the user I/O device, so your shell doesn't receive the output sent to user.
One approach to allow you to see trace output is to set up a trace port on the server and a trace client in a separate node. If you want traces from node foo, first remsh to it:
$ erl -sname bar -remsh foo
Then set up a trace port. Here, we set up a TCP/IP trace port on host port 50000 (use any port you like as long as it's available to you):
1> dbg:tracer(port, dbg:trace_port(ip, 50000)).
Next, set up the trace parameters as you did before:
2> dbg:tp(my_api, handle, 2, []).
{ok, ...}
3> dbg:p(all, c).
{ok, ...}
Then exit the remsh, and start a node without remsh:
$ erl -sname bar
On this node, start a TCP/IP trace client attached to host port 50000:
1> dbg:trace_client(ip, {"localhost", 50000}).
This shell will now receive dbg trace messages from foo. Here, we used "localhost" as the hostname since this node is running on the same host as the server node, but you'll need to use a different hostname if your client is running on a separate host.
Another approach, which is easier but relies on an undocumented function and so might break in the future, is to remsh to the node to be traced as you originally did but then use dbg:tracer/2 to send dbg output to your remote shell's group leader:
1> dbg:tracer(process, {fun dbg:dhandler/2, group_leader()}).
{ok, ...}
2> dbg:tp(my_api, handle, 2, []).
{ok, ...}
3> dbg:p(all, c).
{ok, ...}
Since this relies on the dbg:dhandler/2 function, which is exported but undocumented, there's no guarantee it will always work.
Lastly, since you're tracing all processes, please pay attention to the potential problems described in the dbg man page, and always be sure to call dbg:stop_clear(). when you're finished tracing.

Akka IO(Tcp) get reason of CommandFailed

I have the following example of Actor using IO(Tcp)
https://gist.github.com/flirtomania/a36c50bd5989efb69a5f
For the sake of experiment I've run it twice, so it was trying to bind to 803 port. Obviously I've got an error.
Question: How can I get the reason why "CommandFailed"? In application.conf I have enabled slf4j and debug level of logs, then I've got an error in my logs.
DEBUG akka.io.TcpListener - Bind failed for TCP channel on endpoint [localhost/127.0.0.1:803]: java.net.BindException: Address already in use: bind
But why is that only debug level? I do not want to enable all ActorSystem to log their events, I want to get the reason of CommandFailed event (like java.lang.Exception instance which I could make e.printStackTrace())
Something like:
case c # CommandFailed => val e:Exception = c.getReason()
Maybe it's not the Akka-way? How to get diagnostic info then?
Here's what you can do - find the PID that still keeps living and then kill it.
On a Mac -
lsof -i : portNumber
then
kill -9 PidNumber
I understood that you have 2 questions.
if you ran the same code simultaneously, bot actors are trying to bind to the same port (in your case, 803) which is not possible unless the bound one unbinds and closes the connection so that the other one can bind.
you can import akka.event.Logging and put val log = Logging(context.system, this) at the beginning of your actors, which will log all the activities of your actors and ...
it also shows the name of the actor,corresponding actor system and host+port (if you are using akka-cluster).
wish that helps

Haskell: Testing connection availability N times with a delay (scotty to mongodb)

I have a stupid problem with scotty web app and mongodb service starting in the right order.
I use systemd to start mongodb first and then the scotty web app. It does not work for some reason. The app errors out with connect: does not exist (Connection refused) from the mongodb driver meaning that the connection is not ready.
So my question. How can I test the connection availability say three times with 0.5s interval and only then error out?
This is the application main function
main :: IO ()
main = do
pool <- createPool (runIOE $ connect $ host "127.0.0.1") close 1 300 5
clearSessions pool
let r = \x -> runReaderT x pool
scottyT 3000 r r basal
basal :: ScottyD ()
basal = do
middleware $ staticPolicy (noDots >-> addBase "static")
notFound $ runSession
routes
Although the app service is ordered after mongodb service the connection to mongodb is still unavailable during the app start up. So I get the above mentioned error.
This is the systemd service file to avoid questions regarding the correct service ordering.
[Unit]
Description=Basal Web Application
Requires=mongodb.service
After=mongodb.service iptables.service network-online.target
[Service]
User=http
Group=http
WorkingDirectory=/srv/http/basal/
ExecStart=/srv/http/basal/bin/basal
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
I don't know why connection to mongodb is not available given the correct service order.
So I want to probe connection availability withing haskell code three times with 0.5s delay and then error out. How can I do it?
Thanks.
I guess from the functions you're using that you're using something like mongoDB 1.5.0.
Here, connect returns something in the IOE monad, which is an alias for ErrorTIOErrorIO.
So the best approach is to use the retrying mechanisms ErrorT offers. As it's an instance of MonadPlus, we can just use mplus if we don't care about checking for the specific error:
retryConnect :: Int -> Int -> Host -> IOE Pipe
retryConnect retries delayInMicroseconds host
| retries > 0 =
connect host `mplus`
(liftIO (threadDelay delayInMicroseconds) >>
retryConnect (retries - 1) delayInMicroseconds host)
| otherwise = connect host
(threadDelay comes from Control.Concurrent).
Then replace connect with retryConnect 2 500000 and it'll retry twice after the first failure with a 500,000 microsecond gap (i.e. 0.5s).
If you do want to check for a specific error, then use catchError instead and inspect the error to decide whether to swallow it or rethrow it.

Writing parallel TCP server in erlang

"Programming Erlang Software for a Concurrent World" says to write a parallel TCP server do like this:
start_parallel_server() ->
{ok, Listen} = gen_tcp:listen(...),
spawn(fun() -> par_connect(Listen) end).
par_connect(Listen) ->
{ok, Socket} = gen_tcp:accept(Listen),
spawn(fun() -> par_connect(Listen) end),
loop(Socket).
loop(...) -> %% handle request here
When start_parallel_server finishes its work it will close listen socket. Shouldn't we add something like timer:sleep(infinity) at the end of it?
If you run start_parallel_server() from the shell the shell process will own the listening socket, so it will stay alive as long as that shell process is alive. Note that the shell process dies on exceptions and a new shell process is respawned… Can cause confusion.
But if you e.g. spawn a new process that in turn calls the start_parallel_server() function you will need a sleep in that spawned process to keep it alive.
Moreover, for real world applications https://github.com/extend/ranch is more suitable.

How do I turn off the Scala Fast Compilation server's (FSC) timeout?

I am using a Scala compilation server. This is probably not related to my IDE IntelliJ IDEA, but I will just inform you that I start the Scala compilation server through a special run configuration in that IDE.
After some time that goes by without compiling anything, the compilation server terminates, without any message. Usually, I only notice this when I try to compile something and compilation fails. Then, I need to start the compilation server again, and of course the next compilation takes a long time, because it's once more the first compilation since starting the compilation server.
How do I turn off that timeout? I looked at the manpage for scalac, and there seems to be no option for it. I can add VM options for that run configuration.
Pass -max-idle 0 as parameter. It will work on a very (very!) recent nightly, and it should be available on Scala 2.9.0 when it comes out. However, there's no guarantee the name won't change until then.
I don't think you can. Here is a code snippet from the compilation server:
object SocketServer
{
// After 30 idle minutes, politely exit.
// Should the port file disappear, and the clients
// therefore unable to contact this server instance,
// the process will just eventually terminate by itself.
val IdleTimeout = 1800000
val BufferSize = 10240
def bufferedReader(s: Socket) = new BufferedReader(new InputStreamReader(s.getInputStream()))
def bufferedOutput(s: Socket) = new BufferedOutputStream(s.getOutputStream, BufferSize)
}
I think you should open a feature request in scala-lang.org