I use some cross-platform library (it's written in Haskell but it does not matter) and the library has a function (setInterface) which performs 2 C runtime calls:
gethostbyname()
setsockopt()
It's not a question about Haskell and its libraries but about these 2 C runtime calls.
The first function in my case gets a string which is IP address. Always. This is my use case. The second one - setsockopt() sets IP_MULTICAST_LOOP option.
And I have to support 3 OSes: Windows, Mac, Linux. So, the question includes:
can gethostbyname() be blocked in my case?
can setsockopt() be blocked in my case?
Or with different words, how fast can they be executed?
For me, it's more important the setsockopt() because I can rewrite this setInterface so to avoid gethostbyname() call at all, but I don't know how fast will be executed setsockopt() in my case.
For my part, I checked the documentation, from one point of view gethostbyname() calls DNS. But I don't know will it call DNS for IP? And I have not ideas about setsockopt() - I used to think that it is not blocked call, but from another point of view I checked both for Windows and I see that they both can return WSAEINPROGRESS:
WSAEINPROGRESS A blocking Windows Sockets 1.1 call is in progress, or the
service provide is still processing a callback function.
Maybe it would be more correct not to suggest about OSes implementations and to think about them that they always can be blocked (in Windows, Mac, Linux)?
Related
is using os.mknod(), os.rename(), os.mkdir(), etc. considered a system call?
I'm trying to understand exactly what a system call entails.
A system call is generally considered to be one that is dispatched through the system interrupt vector or an equivalent mechanism (such on Intel processors).
MSDN says closesocket() is the function to use. However, I couldn't help wondering if _close() will work also?
MSDN appears to say no in their description of SOCKET type: (http://msdn.microsoft.com/en-us/windows/ms740516(v=vs.80)):
In Winsock applications, a socket descriptor is not a file descriptor and must be used with the Winsock functions.
and more specifically from its note on renamed socket functions:
Sockets are represented by standard file descriptors in Berkeley Sockets, so the close function can be used to close sockets as well as regular files. While nothing in Windows Sockets prevents an implementation from using regular file handles to identify sockets, nothing requires it either. On Windows, sockets must be closed by using the closesocket routine. On Windows, using the close function to close a socket is incorrect and the effects of doing so are undefined by this specification.
However, even despite the above, some of the Windows file functions might work with sockets in practice:
Given that ReadFile and WriteFile work on sockets, I suspect _read and _write, for instance, might also work with sockets as well as file handles.
MSDN's overview of socket handles states:
A socket handle can optionally be a file handle in Windows Sockets 2. A socket handle from a Winsock provider can be used with other non-Winsock functions such as ReadFile, WriteFile, ReadFileEx, and WriteFileEx.
The short answer is no. Sockets handles on Windows are not file handles as they are on Unix. There's special support such that the low level Win32 APIs, ReadFile and WriteFile, can work with a socket handle. But that's likely where it ends.
With regards to _open_osfhandle, yes, that will possibly work in a very limited sense, but there's good reasons why you shouldn't do this. Most of the following I inferred just by browsing the sources of open, read,write, close, and open_osfhandle in the CRT sources (that comes with Visual Studio).
There's a lot of buffering that goes on with the CRT read/write calls. Any attempt to mix read/write with recv/send will be going into undefined behavior.
Performance. Just look at the source of read() and write() as seen in the CRT sources. A lot of wrapper code to eventually call ReadFile and WriteFile, which in turn have to forward to the actual socket API.
Socket error codes may not bubble to the file API as you think. Remember socket API errors get returned through WSAGetLastError. Win32 file IO calls get bubbled up through GetLastError. So if your call to write() hits a socket error, it might try to map the return value via GetLastError, which still returns success.
close() won't properly close the socket handle since it maps to CloseHandle, not closesocket.
More insightful than MSDN: [http://tangentsoft.net/wskfaq/articles/bsd-compatibility.html]
No they are definitely different in Windows 3.x, 9x:
'In Windows 3.1 and 95 Windows sockets and file handles are
completely distinct. In Windows NT, however, it appears they may usually be one and the same.'
And no, you can't use them together on Windows NT - at least not by default:
'The Visual C++ RTL emulates POSIX functions, except that they’re named
with a leading underscore: for example, _read() instead of read(). The
_read() function uses ReadFile() internally, so you’d think it would work with sockets. The problem is, the first argument is an
RTL-specific handle, not an operating system file handle. If you pass
a socket handle to _read() or _write(), the RTL will realize that it
isn’t an RTL handle and the call will fail.
'Fortunately, there is a bridge function in Visual C++’s RTL:
_open_osfhandle(). (If you’re not using Visual C++, you’ll have to check its RTL source for a similar function.) I’ve not tried it, but
it appears to take an operating system file handle (including socket
handles) and return a handle you can use with the POSIX emulation
functions in the RTL. I’m told that this will work with sanely-coded
non-Microsoft Winsock stacks, but since I haven’t tried it, you should
if you want to support these alternate stacks.'
I have 3rd party library that uses non-blocking send, recv (in Windows), so I can use libevent (or libev) with select backend to event loop. Is there easy way to use IOCP backend (libevent2, libuv)? Because in different manuals write, that i must use WSASend, WSARecv functions with some additional arguments (pointer to OVERLAPPED structure). How can I use IOCP backend event loop with this 3rd party library without rewriting its code? Or it is not impossible?
It's not possible.
At the very least:
you need to associate the socket that's used with the IOCP that you want to process the completions
you need to create the socket with different flags.
Either of these likely requires that you change the library code.
In current lua sockets implementation, I see that we have to install a timer that calls back periodically so that we check in a non blocking API to see if we have received anything.
This is all good and well however in UDP case, if the sender has a lot of info being sent, do we risk loosing the data. Say another device sends a 2MB photo via UDP and we check socket receive every 100msec. At 2MBps, the underlying system must store 200Kbits before our call queries the underlying TCP stack.
Is there a way to get an event fired when we receive the data on the particular socket instead of the polling we have to do now?
There are a various ways of handling this issue; which one you will select depends on how much work you want to do.*
But first, you should clarify (to yourself) whether you are dealing with UDP or TCP; there is no "underlying TCP stack" for UDP sockets. Also, UDP is the wrong protocol to use for sending whole data such as a text, or a photo; it is an unreliable protocol so you aren't guaranteed to receive every packet, unless you're using a managed socket library (such as ENet).
Lua51/LuaJIT + LuaSocket
Polling is the only method.
Blocking: call socket.select with no time argument and wait for the socket to be readable.
Non-blocking: call socket.select with a timeout argument of 0, and use sock:settimeout(0) on the socket you're reading from.
Then simply call these repeatedly.
I would suggest using a coroutine scheduler for the non-blocking version, to allow other parts of the program to continue executing without causing too much delay.
Lua51/LuaJIT + LuaSocket + Lua Lanes (Recommended)
Same as the above method, but the socket exists in another lane (a lightweight Lua state in another thread) made using Lua Lanes (latest source). This allows you to instantly read the data from the socket and into a buffer. Then, you use a linda to send the data to the main thread for processing.
This is probably the best solution to your problem.
I've made a simple example of this, available here. It relies on Lua Lanes 3.4.0 (GitHub repo) and a patched LuaSocket 2.0.2 (source, patch, blog post re' patch)
The results are promising, though you should definitely refactor my example code if you derive from it.
LuaJIT + OS-specific sockets
If you're a little masochistic, you can try implementing a socket library from scratch. LuaJIT's FFI library makes this possible from pure Lua. Lua Lanes would be useful for this as well.
For Windows, I suggest taking a look at William Adam's blog. He's had some very interesting adventures with LuaJIT and Windows development. As for Linux and the rest, look at tutorials for C or the source of LuaSocket and translate them to LuaJIT FFI operations.
(LuaJIT supports callbacks if the API requires it; however, there is a signficant performance cost compared to polling from Lua to C.)
LuaJIT + ENet
ENet is a great library. It provides the perfect mix between TCP and UDP: reliable when desired, unreliable otherwise. It also abstracts operating system specific details, much like LuaSocket does. You can use the Lua API to bind it, or directly access it via LuaJIT's FFI (recommended).
* Pun unintentional.
I use lua-ev https://github.com/brimworks/lua-ev for all IO-multiplexing stuff.
It is very easy to use fits into Lua (and its function) like a charm. It is either select/poll/epoll or kqueue based and performs very good too.
local ev = require'ev'
local loop = ev.Loop.default
local udp_sock -- your udp socket instance
udp_sock:settimeout(0) -- make non blocking
local udp_receive_io = ev.IO.new(function(io,loop)
local chunk,err = udp_sock:receive(4096)
if chunk and not err then
-- process data
end
end,udp_sock:getfd(),ev.READ)
udp_receive_io:start(loop)
loop:loop() -- blocks forever
In my opinion Lua+luasocket+lua-ev is just a dream team for building efficient and robust networking applications (for embedded devices/environments). There are more powerful tools out there! But if your resources are limited, Lua is a good choice!
Lua is inherently single-threaded; there is no such thing as an "event". There is no way to interrupt executing Lua code. So while you could rig something up that looked like an event, you'd only ever get one if you called a function that polled which events were available.
Generally, if you're trying to use Lua for this kind of low-level work, you're using the wrong tool. You should be using C or something to access this sort of data, then pass it along to Lua when it's ready.
You are probably using a non-blocking select() to "poll" sockets for any new data available. Luasocket doesn't provide any other interface to see if there is new data available (as far as I know), but if you are concerned that it's taking too much time when you are doing this 10 times per second, consider writing a simplified version that only checks one socket you need and avoids creating and throwing away Lua tables. If that's not an option, consider passing nil to select() instead of {} for those lists you don't need to read and pass static tables instead of temporary ones:
local rset = {socket}
... later
...select(rset, nil, 0)
instead of
...select({socket}, {}, 0)
Is there a way to pass a socket between processes (not same address space) in Windows?
I find this info Shared Sockets, therefore believe that it is possible.
"The WSADuplicateSocket function is introduced to enable socket sharing across processes"...
More info : at source code of Apache (httpd-2.2.11-win32-src.zip) have usage for api WSADuplicateSocket with comments.
[EDIT]
Recently I find this great sample about this question.
How duplication is done in the unmanaged world - Socket Duplication - Part 1
Is it possible to transfer sockets from unmanaged processes? - Socket Duplication - Part 2
See the Remarks section of WSADuplicateSocket. It effectively says you can use Your Favorite Interprocess Communication Scheme to send the WSAPROTOCOL_INFO structure (it's just data!) to the target.
There are lots of IPC schemes. I'd probably use shared memory with Boost::interprocess. But you could use SendMessage if the target has a window + message loop. Or the Clipboard API, for that matter (though somewhat weird). The mechanism is your choice.
If you're creating the child process there are some things that might do it for you. See
http://www.tangentsoft.net/wskfaq/articles/passing-sockets.html
(I know that this one worked in the ancient past; no idea if it works on current versions)
http://msdn.microsoft.com/en-us/library/ms682499.aspx
-- MarkusQ