I'm trying to write a very basic webserver in Haskell. This is my code:
{-# LANGUAGE OverloadedStrings #-}
import Network (withSocketsDo, listenOn, PortID(..))
import Network.Socket (Socket, accept, close, setSocketOption, SocketOption(..))
import Network.Socket.ByteString (send, sendAll, recv)
import Control.Concurrent.Async (async)
import Control.Monad (forever)
import Data.ByteString.Char8 (unpack)
import Request
main = withSocketsDo $ do
sock <- listenOn $ PortNumber 3000
putStrLn "Listening on port 3000..."
forever $ do
(conn, _) <- accept sock
async $ handleAccept conn
handleAccept :: Socket -> IO ()
handleAccept sock = do
putStrLn $ "Connected!"
rawReq <- recv sock 4096
let req = parseRawRequest $ unpack rawReq -- returns Maybe Request
putStrLn $ show req
handleRequest sock req
handleRequest :: Socket -> Maybe Request -> IO ()
handleRequest sock Nothing = do
putStrLn "Closing..."
handleRequest sock req = do
sendAll sock "In handleRequest!" -- Doesn't appear until server is killed.
This is what I expected to happen:
Start server.
"Listening on port 3000..." is printed on server-side.
Do curl localhost:3000
"Connected!" is printed server-side.
The request is printed server-side.
"In handleRequest!" is printed.
What actually happens:
Start server.
"Listening on port 3000..." is printed on server-side.
Do curl localhost:3000
"Connected!" is printed server-side.
The request is printed server-side.
I wait patiently
I kill the server with CTRL+C
"In handleRequest!" prints client-side.
I suspect this has something to do with possible laziness in recv, although I use the value immediately afterwards (I parse the raw request into a Request type), so theoretically it should be evaluated.
If I put sendAll sock "Yadda yadda at the end of handleAccept, everything works fine. It's when I move this behaviour into a new function, handleRequest, that things go wonky.
Any thoughts? I'm new-ish to Haskell, so I'd appreciate any comments on the issue, or my code generally.
Cheers.
EDIT:
This is super weird! I "fixed" it, but I have no idea why this occurs.
This is the line that only appeared after I killed the server:
handleRequest sock req = do
sendAll sock "In handleRequest!" -- Doesn't appear until server is killed.
If I intentionally close the socket after sending, it works:
handleRequest sock req = do
sendAll sock "In handleRequest!" -- Now appears without killing the server
close sock
So it sends when the connection is closed. This is consistent with previous behaviour, since the connection automatically closes when the server is killed.
Now for the confusing bit. If I replace it with:
handleRequest sock req = do
sendAll sock "In handleRequest!\n" -- Works perfect
This works without closing the connection! It does what I expected, just by adding a newline. Why does this occur?
What on earth? Is it a printing problem with my terminal, not the code? (OSX iTerm2)
EDIT 2:
Was asked to provide the code for my Request module:
import Data.List (isInfixOf)
import Data.List.Split (splitOn)
data RequestType = GET | PUT
deriving Show
data Request =
Request {
reqType :: RequestType,
path :: String,
options :: [(String, String)]
} deriving Show
-- Turn a raw HTTP request into a request
-- object.
parseRawRequest :: String -> Maybe Request
parseRawRequest rawReq =
Request <$> parseRawRequestType rawReq
<*> parseRawRequestPath rawReq
<*> parseRawRequestOps rawReq
-- Turn an (entire) raw HTTP request into just
-- the request type.
parseRawRequestType :: String -> Maybe RequestType
parseRawRequestType rawReq =
case typ of
"GET" -> Just GET
"PUT" -> Just PUT
_ -> Nothing
where typ = (head . words . head . lines) rawReq
-- Turn an (entire) raw HTTP request into just
-- the path.
parseRawRequestPath :: String -> Maybe String
parseRawRequestPath = Just . (!! 1) . words . head . lines
-- Turn an (entire) raw HTTP request into just
-- a lookup table of their options.
parseRawRequestOps :: String -> Maybe [(String, String)]
parseRawRequestOps rawReq = Just [("One", "Two")] -- Test impl
I have one answer and one suggestion.
The suggestion is for you to turn off the naggle algorithm after accept:
setSocketOption conn NoDelay 1
The answer is that your sendAll is sending data but curl is not printing it. You can confirm this with netcat, for example. I commented out your Nothing case so that no matter what I typed in netcat I was sure to get the "In handleRequest!" message back:
server:
% ghc so.hs && ./so
Listening on port 3000...
Connected!
Nothing
client:
% nc localhost 3000
test ; My input, with a newline
In handleRequest! ; Printed out, no newline
Alternatively, you can use curl's -N option to disable buffering.
% curl -N localhost:3000
In handleRequest!
I found this when reading the warp source code.
-- | Run an 'Application' with the given 'Settings'.
runSettings :: Settings -> Application -> IO ()
runSettings set app = withSocketsDo $
bracket
(bindPortTCP (settingsPort set) (settingsHost set))
sClose
(\socket -> do
setSocketCloseOnExec socket
runSettingsSocket set socket app)
-- Copied from: https://github.com/mzero/plush/blob/master/src/Plush/Server/Warp.hs
setSocketCloseOnExec :: Socket -> IO ()
#if WINDOWS
setSocketCloseOnExec _ = return ()
#else
setSocketCloseOnExec socket =
setFdOption (fromIntegral $ fdSocket socket) CloseOnExec True
#endif
Sockets used by the web server need to be set so that they are not leaked into processes fork/exec'd by the shell.
However, acceptFork in Network.Simple.TCP module does not set CloseOnExec option. Shouldn't this option be the default?
Using Wireshark to debug, I receive the following error when sending UDP packets on localhost:
Destination Unreachable (Port Unreachable)
Checksum: 0x0000 (Illegal)
I am constructing my server first on a port between 10000 - 15000 using
startServer :: Port -> IO Server
startServer port = withSocketsDo $ do
-- Look up the server address and port information.
addrs <- getAddrInfo (Just $ defaultHints { addrFlags = [AI_PASSIVE] }) Nothing (Just port)
let serverAddress = head addrs
-- Bind to the socket.
sock <- socket (addrFamily serverAddress) Datagram defaultProtocol
bindSocket sock (addrAddress serverAddress)
-- Create the server and run the client send and receive threads.
clients <- newMVar $ createEmptyClients
let server = Server sock port clients
_ <- forkIO $ forever $ receiveClientJoin server
return server
I am listening for new clients connecting via UDP using
-- | Connected a client to the server.
receiveClientJoin :: Server -> IO ()
receiveClientJoin server = do
print "Receiving"
(msg, _, clSockAddr) <- recvFrom (sSocket server) 4096
print $ "Server received client join message: " ++ msg
And I am connecting to the server with clients using
connectToServer port = do
-- Get the server's address and port information.
addrInfo <- getAddrInfo Nothing (Just "localhost") (Just port)
let serverAddr = head addrInfo
sock <- socket (addrFamily serverAddr) Datagram defaultProtocol
sendTo sock "Hello from this client!" (addrAddress serverAddr)
Why are my clients' packets not finding the server?
The problem is you are listening on an IPv6 address and trying to connect to an IPv4 address. This is actually a slightly common problem. For example, I ran across this issue when working with commsec.
Consider the fragments where you discover your AddrInfo:
import Network.Socket
main :: IO ()
main = do
let port = "2474"
addrs <- getAddrInfo (Just $ defaultHints { addrFlags = [AI_PASSIVE] }) Nothing (Just port)
let serverAddress = head addrs
print serverAddress
addrInfo <- getAddrInfo Nothing (Just "localhost") (Just port)
let serverAddr = head addrInfo
print serverAddr
Now the output will vary by machine, but on one of my CentOS systems with both IPv4 and IPv6 addresses the output clearly shows the second (connect) address is IPv6 while the first (listen) address is IPv4:
AddrInfo {addrFlags = [AI_PASSIVE], addrFamily = AF_INET, addrSocketType = Stream, addrProtocol = 6, addrAddress = 0.0.0.0:2474, addrCanonName = Nothing}
AddrInfo {addrFlags = [AI_ADDRCONFIG,AI_V4MAPPED], addrFamily = AF_INET6, addrSocketType = Stream, addrProtocol = 6, addrAddress = [::1]:2474, addrCanonName = Nothing}
One solution is to force a particular version of IP via a hint or an address (ex. an IPv4 address as in my comment). The hint solution is probably more desirable:
-- For servers:
addrs <- getAddrInfo (Just defaultHints { addrFamily = AF_INET6
, addrFlags = [AI_PASSIVE] })
Nothing (Just port)
-- For clients:
addrInfo <- getAddrInfo (Just defaultHints { addrFamily = AF_INET6 })
(Just "localhost") (Just port)
I’m trying to make a simple web application using Snap. I want to have a global MongoDB connection which I can use across handlers.
I open this connection in appInit, which works.
data App = App { _mongoDB :: Pipe -- connection
}
appInit :: SnapletInit App App
appInit = makeSnaplet "site" "My Site" Nothing $ do
db <- liftIO $ do
pipe <- runIOE $ connect (host "127.0.0.1")
return pipe
return $ App db
However, I have no idea how to access this connection (_mongoDB) from a handler. I tried several things, including the following, but I get type errors every time.
watchHandler :: Handler App App ()
watchHandler = do
res <- liftIO $ do
pipe <- gets _mongoDB -- type error
results <- access pipe master "db" (find $ select [] "movies")
return results
writeBS "test"
It gives me the following error:
No instance for (MonadState App IO)
arising from a use of `gets'
Possible fix: add an instance declaration for (MonadState App IO)
In a stmt of a 'do' block: pipe <- gets _mongoDB
In the second argument of `($)', namely
`do { pipe <- gets _mongoDB;
results <- access pipe master "db" (find $ select [] "movies");
return results }'
In a stmt of a 'do' block:
res <- liftIO
$ do { pipe <- gets _mongoDB;
results <- access pipe master "db" (find $ select [] "movies");
return results }
It confuses the hell out of me. How can I access my MongoDB connection from a handler?
The liftIO block is just for IO operations, accessing the application state needs to be done in the Handler monad itself.
watchHandler :: Handler App App ()
watchHandler = do
pipe <- gets _mongoDB
res <- liftIO $ do
results <- access pipe master "db" (find $ select [] "movies")
return results
writeBS "test"
Also, binding a value and then immediately returning it is redundant. You can just write:
watchHandler :: Handler App App ()
watchHandler = do
pipe <- gets _mongoDB
res <- liftIO $ access pipe master "db" (find $ select [] "movies")
writeBS "test"
So I am working on an iPhone app that requires a socket to handle multiple clients for online gaming. I have tried Twisted, and with much effort, I have failed to get a bunch of info to be sent at once, which is why I am now going to attempt socket.
My question is, using the code below, how would you be able to have multiple clients connected? I've tried lists, but I just can't figure out the format for that. How can this be accomplished where multiple clients are connected at once and I am able to send a message to a specific client?
Thank you!
#!/usr/bin/python # This is server.py file
import socket # Import socket module
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 50000 # Reserve a port for your service.
print 'Server started!'
print 'Waiting for clients...'
s.bind((host, port)) # Bind to the port
s.listen(5) # Now wait for client connection.
c, addr = s.accept() # Establish connection with client.
print 'Got connection from', addr
while True:
msg = c.recv(1024)
print addr, ' >> ', msg
msg = raw_input('SERVER >> ')
c.send(msg);
#c.close() # Close the connection
Based on your question:
My question is, using the code below, how would you be able to have multiple clients connected? I've tried lists, but I just can't figure out the format for that. How can this be accomplished where multiple clients are connected at once and I am able to send a message to a specific client?
Using the code you gave, you can do this:
#!/usr/bin/python # This is server.py file
import socket # Import socket module
import thread
def on_new_client(clientsocket,addr):
while True:
msg = clientsocket.recv(1024)
#do some checks and if msg == someWeirdSignal: break:
print addr, ' >> ', msg
msg = raw_input('SERVER >> ')
#Maybe some code to compute the last digit of PI, play game or anything else can go here and when you are done.
clientsocket.send(msg)
clientsocket.close()
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 50000 # Reserve a port for your service.
print 'Server started!'
print 'Waiting for clients...'
s.bind((host, port)) # Bind to the port
s.listen(5) # Now wait for client connection.
print 'Got connection from', addr
while True:
c, addr = s.accept() # Establish connection with client.
thread.start_new_thread(on_new_client,(c,addr))
#Note it's (addr,) not (addr) because second parameter is a tuple
#Edit: (c,addr)
#that's how you pass arguments to functions when creating new threads using thread module.
s.close()
As Eli Bendersky mentioned, you can use processes instead of threads, you can also check python threading module or other async sockets framework. Note: checks are left for you to implement how you want and this is just a basic framework.
accept can continuously provide new client connections. However, note that it, and other socket calls are usually blocking. Therefore you have a few options at this point:
Open new threads to handle clients, while the main thread goes back to accepting new clients
As above but with processes, instead of threads
Use asynchronous socket frameworks like Twisted, or a plethora of others
Here is the example from the SocketServer documentation which would make an excellent starting point
import SocketServer
class MyTCPHandler(SocketServer.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print "{} wrote:".format(self.client_address[0])
print self.data
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()
Try it from a terminal like this
$ telnet localhost 9999
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Hello
HELLOConnection closed by foreign host.
$ telnet localhost 9999
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Sausage
SAUSAGEConnection closed by foreign host.
You'll probably need to use A Forking or Threading Mixin too
This program will open 26 sockets where you would be able to connect a lot of TCP clients to it.
#!usr/bin/python
from thread import *
import socket
import sys
def clientthread(conn):
buffer=""
while True:
data = conn.recv(8192)
buffer+=data
print buffer
#conn.sendall(reply)
conn.close()
def main():
try:
host = '192.168.1.3'
port = 6666
tot_socket = 26
list_sock = []
for i in range(tot_socket):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
s.bind((host, port+i))
s.listen(10)
list_sock.append(s)
print "[*] Server listening on %s %d" %(host, (port+i))
while 1:
for j in range(len(list_sock)):
conn, addr = list_sock[j].accept()
print '[*] Connected with ' + addr[0] + ':' + str(addr[1])
start_new_thread(clientthread ,(conn,))
s.close()
except KeyboardInterrupt as msg:
sys.exit(0)
if __name__ == "__main__":
main()
def get_clients():
first_run = True
startMainMenu = False
while True:
if first_run:
global done
done = False
Thread(target=animate, args=("Waiting For Connection",)).start()
Client, address = objSocket.accept()
global menuIsOn
if menuIsOn:
menuIsOn = False # will stop main menu
startMainMenu = True
done = True
# Get Current Directory in Client Machine
current_client_directory = Client.recv(1024).decode("utf-8", errors="ignore")
# beep on connection
beep()
print(f"{bcolors.OKBLUE}\n***** Incoming Connection *****{bcolors.OKGREEN}")
print('* Connected to: ' + address[0] + ':' + str(address[1]))
try:
get_client_info(Client, first_run)
except Exception as e:
print("Error data received is not a json!")
print(e)
now = datetime.now()
current_time = now.strftime("%D %H:%M:%S")
print("* Current Time =", current_time)
print("* Current Folder in Client: " + current_client_directory + bcolors.WARNING)
connections.append(Client)
addresses.append(address)
if first_run:
Thread(target=threaded_main_menu, daemon=True).start()
first_run = False
else:
print(f"{bcolors.OKBLUE}* Hit Enter To Continue.{bcolors.WARNING}\n#>", end="")
if startMainMenu == True:
Thread(target=threaded_main_menu, daemon=True).start()
startMainMenu = False
#!/usr/bin/python
import sys
import os
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
port = 50000
try:
s.bind((socket.gethostname() , port))
except socket.error as msg:
print(str(msg))
s.listen(10)
conn, addr = s.accept()
print 'Got connection from'+addr[0]+':'+str(addr[1]))
while 1:
msg = s.recv(1024)
print +addr[0]+, ' >> ', msg
msg = raw_input('SERVER >>'),host
s.send(msg)
s.close()