I don't know exactly what socket means.
A server runs on a specific computer and has a socket that is bound to a specific port number. The server just waits, listening to the socket for a client to make a connection request.
When the server accepts the connection, it gets a new socket bound to the same local port and also has its remote endpoint set to the address and port of the client.
It needs a new socket so that it can continue to listen to the original socket for connection requests while tending to the needs of the connected client.
So, socket is some class created in memory? And for every client connection there is created new instance of this class in memory? Inside socket is written the local port and port and IP number of the client which is connected. Can someone explain me more in details the definition of socket?
Thanks
A socket is effectively a type of file handle, behind which can lie a network session.
You can read and write it (mostly) like any other file handle and have the data go to and come from the other end of the session.
The specific actions you're describing are for the server end of a socket. A server establishes (binds to) a socket which can be used to accept incoming connections. Upon acceptance, you get another socket for the established session so that the server can go back and listen on the original socket for more incoming connections.
How they're represented in memory varies depending on your abstraction level.
At the lowest level in C, they're just file descriptors, a small integer. However, you may have a higher-level Socket class which encapsulates the behaviour of the low-level socket.
According to "TCP/IP Sockets in C-Practical Guide for Programmers" by Michael J. Doonahoo & Kenneth L. Calvert (Chptr 1, Section 1.4, Pg 7):
A socket is an abstraction through which an application may send
and receive data,in much the same way as an open file allows an application to read and write data to stable storage.
A socket allows an application to "plug in" to the network and communicate
with other applications that are also plugged in to the same network.
Information written to the socket by an application on one machine can be
read by an application on a different machine, and vice versa.
Refer to this book to get clarity about sockets from a programmers point of view.
A network socket is one endpoint in a communication flow between two programs running over a network.
A socket is the combination of IP address plus port number
This is the typical sequence of sockets requests from a server application in the connectionless context of the Internet in which a server handles many client requests and does not maintain a connection longer than the serving of the immediate request:
Steps to implement
At Server side
initilize socket()
--
bind()
--
recvfrom()
--
(wait for a sendto request from some client)
--
(process the sendto request)
--
sendto (in reply to the request from the client...for example, send an HTML file)
A corresponding client sequence of sockets requests would be:
socket()
--
bind()
--
sendto()
--
recvfrom()
so that you can make a pipeline connection ..
for more http://www.steves-internet-guide.com/tcpip-ports-sockets
I found this article in online.
So to put it all back together, a socket is the combination of an IP
address and a port, and it acts as an endpoint for receiving or
sending information over the internet, which is kept organized by TCP.
These building blocks (in conjunction with various other protocols and
technologies) work in the background to make every google search,
facebook post, or introductory technical blog post possible.
https://medium.com/swlh/understanding-socket-connections-in-computer-networking-bac304812b5c
Socket definition
A communication between two processes running on two computer systems can be completely specified by the association: {protocol, local-address, local-process, remote-address, remote-process} We also define a half association as either {protocol, local-address, local-process} or {protocol, remote-address, remote-process}, which specify half of a connection. This half association is also called socket, or transport address. The term socket has been popularized by the Berkeley Unix networking system, where it is "an end point of communication", which corresponds to the definition of half association.
Related
I am trying to establish a TCP/IP connection between a controller (client) and a program in my PC (server) using C++, I used a sniffer to see how client’s requests are being sent and I found out that each connect request from the controller is sent from a different port and known IP, it starts with random port number and increment by 1 with each request till I restart the controller or the server receives the request, I have some questions.
1- Is that a standard behaviour and what is the idea behind this knowing that the controller is a Mitsubishi controller?
2- Is there any way I can get the new port of the controller without using accept?
This is not so much the behaviour of the controller as it is the network stack running on top of the controller and may be integrated into the controller hardware (Search keyword: TCP offload).
This is expected behaviour. To prevent all sorts of nasty side effects, a simple example is late packets from a previous connection trying to sneak in as legitimate packets for a later connection, a port is not recycled for reuse for a lengthy period after the socket using the port is closed. Your port may not be available for use. A simple solution is to do exactly what OP's network stack did: sequentially assign the next port number.
Not with BSD-style sockets. accept accepts a connection with the client. If you do not accept, you don't get a socket to handle the connection and with the socket, you should not care what the port is. It's all abstracted away and hidden out of sight.
If this is a problem, consider using a connectionless protocol like UDP. You don't get automatic re-transmission when packet loss is detected and all of the other nice things TCP does for you, but there is no connection overhead.
The following is my recent interview experience with a reputed network software company. I was asked questions about interconnecting TCP level and web requests and that confused me a lot. I really would like to know expert opinions on the answers. It is not just about the interview but also about a fundamental understanding of how networking work (or how application layer and transport layer cross-talk, if at all they do).
Interviewer: Tell me the process that happens behind the scenes when
I open a browser and type google.com in it.
Me: The first thing that happens is a socket is created which is
identified by {SRC-IP, SRC-PORT, DEST-IP, DEST-PORT, PROTOCOL}. The
SRC-PORT number is a random number given by the browser. Usually the TCP/IP
connection protocol (three-way handshake is established). Now
both the client (my browser) and the server (Google) are ready to handle
requests. (TCP connection is established).
Interviewer: Wait, when does the name resolution happen?
Me: Yep, I am sorry. It should have happened before even the socket is created.
DNS name resolve happens first to get the IP address of Google to
reach at.
Interviewer: Is a socket created for DNS name resolution?
Me: hmm, I actually do not know. But all I know DNS name resolution is
connectionless. That is, it's not TCP but UDP. Only a single
request-response cycle happens. (So there is a new socket created for DNS
name resolution).
Interviewer: google.com is open for other requests from other
clients. So is establishing your connection with Google blocking
other users?
Me: I am not sure how Google handles this. But in a typical socket
communication, it is blocking to a minimal extent.
Interviewer: How do you think this can be handled?
Me: I guess the process forks a new thread and creates a socket to handle my
request. From now on, my socket endpoint of communication with
Google is this child socket.
Interviewer: If that is the case, is this child socket’s port number
different than the parent one?
Me: The parent socket is listening at 80 for new requests from
clients. The child must be listening at a different port number.
Interviewer: How is your TCP connection maintained since your destination port number has changed. (That is the src-port number sent on Google's packet) ?
Me: The dest-port that I see as a client is always 80. When
a response is sent back, it also comes from port 80. I guess the OS/the
parent process sets the source port back to 80 before it sends back the
post.
Interviewer: How long is your socket connection established with
Google?
Me: If I don’t make any requests for a period of time, the
main thread closes its child socket and any subsequent requests from
me will be like I am a new client.
Interviewer: No, Google will not keep a dedicated child socket for
you. It handles your request and discards/recycles the sockets right
away.
Interviewer: Although Google may have many servers to serve
requests, each server can have only one parent socket opened at port 80. The number of clients to access Google's webpage must be larger than the number of servers they have. How is this usually handled?
Me: I am not sure how this is handled. I see the only way it could
work is spawn a thread for each request it receives.
Interviewer: Do you think the way Google handles this is different from
any bank website?
Me: At the TCP-IP socket level, it should be
similar. At the request level, slightly different because a session
is maintained to keep state between requests for banks' websites.
If someone can give an explanation of each of the points, it will be very helpful for many beginners in networking.
How many sockets do Google open for every request it receives?
This question doesn't actually appear in the interview, but it's in your title so I'll answer it. Google doesn't open any sockets at all. Your browser does that. Google accepts connections, in the form of new sockets, but I wouldn't describe that as 'opening' them.
Interviewer : Tell me the process that happens behind the scene when I open a browser and type google.com in it.
Me : The first thing that happens is a socket is created which is identified by {SRC-IP, SRC-PORT, DEST-IP, DEST-PORT, PROTOCOL}.
No. The connection is identified by the tuple. The socket is an endpoint to the connection.
The SRC-PORT number is a random number given by browser.
No. By the operating system.
Usually TCP/IP connection protocol (three way handshake is established). Now the both client (my browser) and server (google) are ready to handle requests. (TCP connection is established)
Interviewer: wait, when does the name resolution happens?
Me: yep, I am sorry. It should have happened before even the socket is created. DNS name resolve happens first to get the IP address of google to reach at.
Interviewer : Is a socket created for DNS name resolution?
Me : hmm, Iactually do not know. But all I know DNS name resolution is a connection-less. That is it not TCP but UDP. only a single request-response cycle happens. (so is a new socket created for DNS name resolution).
Any rationally implemented browser would delegate the entire thing to the operating system's Sockets library, whose internal functioning depends on the OS. It might look at an in-memory cache, a file, a database, an LDAP server, several things, before going out to a DNS server, which it can do via either TCP or UDP. It's not a great question.
Interviewer: google.com is open for other requests from other clients. so is establishing you connection with google is blocking other users?
Me: I am not sure how google handles this. But in a typical socket communication, it is blocking to a minimal extent.
Wrong again. It has very little to do with Google specifically. A TCP server has a separate socket per accepted connection, and any rationally constructed TCP server handles them completely independently, either via multithreading, muliplexed/non-blocking I/O, or asynchronous I/O. They don't block each other.
Interviewer : How do you think this can be handled?
Me : I guess the process forks a new thread and creates a socket to handle my request. From now on, my socket endpoint of communication with google is this this child socket.
Threads are created, not 'forked'. Forking a process creates another process, not another thread. The socket isn't 'created' so much as accepted, and this would normally precede thread creation. It isn't a 'child' socket.
Interviewer: If that is the case, is this child socket’s port number different than the parent one.?
Me: The parent socket is listening at 80 for new requests from clients. The child must be listening at a different port number.
Wrong again. The accepted socket uses the same port number as the listening socket, and the accepted socket isn't 'listening' at all, it is receiving and sending.
Interviewer: how is your TCP connection maintained since your Dest-port number has changed. (That is the src-port number sent on google's packet) ?
Me: The dest-port as what I see as client is always 80. when request is sent back, it also comes from port 80. I guess the OS/the parent process sets the src port back to 80 before it sends back the post.
This question was designed to explore your previous wrong answer. Your continuation of your wrong answer is still wrong.
Interviewer : how long is your socket connection established with google?
Me : If I don’t make any requests for a period of time, the main thread closes its child socket and any subsequent requests from me will be like am a new client.
Wrong again. You don't know anything about threads at Google, let alone which thread closes the socket. Either end can close the connection at any time. Most probably the server end will beat you to it, but it isn't set in stone, and neither is which if any thread will do it.
Interviewer : No, google will not keep a dedicated child socket for you. It handles your request and discards/recycles the sockets right away.
Here the interviewer is wrong. He doesn't seem to have heard of HTTP keep-alive, or the fact that it is the default in HTTP 1.1.
Interviewer: Although google may have many servers to serve requests, each server can have only one parent socket opened at port 80. The number of clients to access google webpage must be exceeding larger than the number of servers they have. How is this usually handled?
Me : I am not sure how this is handled. I see the only way it could work is spawn a thread for each request it receives.
Here you haven't answered the question at all. He is fishing for an answer about load-balancers or round-robin DNS or something in front of all those servers. However his sentence "the number of clients to access google webpage must be exceeding larger than the number of servers they have" has already been answered by the existence of what you are both incorrectly calling 'child sockets'. Again, not a great question, unless you've reported it inaccurately.
Interviewer : Do you think the way Google handles is different from any bank website?
Me: At the TCP-IP socket level, it should be similar. At the request level, slightly different because a session is maintained to keep state between requests in Banks websites.
You almost got this one right. HTTP sessions to Google exist, as well as to bank websites. It isn't much of a question. He should be asking for facts, not your opinion.
Overall, (a) you failed the interview completely, and (b) you indulged in far too much guesswork. You should have simply stated 'I don't know' and let the interview proceed to things that you do know about.
For point #6, here is how I understand it: if both ends of an end to end connection were the same as that of another socket, there would indeed be no way to distinguish both socket, but if a single end is not the same as that of the other socket, then it's easy to distinguish both. So there is not need to turn destination port 80 (the default) forth and back, since the source ports differ.
I've noticed that if you want to write an application that utilizes listening sockets, you need to create port forwarding rules on your router. If I want to connect two computers without either one of the the computers messing about with router settings, is there a way that I can get the two clients to connect to each other without either of them using listening sockets? There would need to be another server somewhere else telling them to connect but is it possible?
Some clarifications, and an answer:
Routers don't care about, or handle ports, that is the role of a firewall, which do port forwarding. The router/firewall combined device most of us have at home adds to the common misunderstanding.
Can you connect two computers without ServerSocket? No. You can use UDP (a stateless, connectionless communication protocol), but the role of a ServerSocket is to "listen" for incoming connection requests, and generate a Socket from those requests, which creates a communications channel between two endpoints. A Socket has both an InputStream and an OutputStream, so it can both read at write at either end. At that point (once the connection is made), the distinction between client/server is arbitrary, since a Socket is a two-way connection object, which allows both sides to send/receive.
What about proxying? Doesn't that allow connections between two computers without a ServerSocket? Well, no, because the server that's doing the proxying still has to be using a ServerSocket. Depending on what application you're trying to implement, this might be the way to go, or or might just add overhead. Even if there were "another server somewhere else telling them to connect", somebody has to listen for a connection request, which is the job of the ServerSocket.
If connections are happening over already open ports (most publicly accessible servers have ports <1024 not blocked by firewalls, but exceptions exist), then you shouldn't need to change firewall settings to get the connection to work.
So, to reiterate, the ONLY role of a ServerSocket (as far as your question is concerned) is to listen for incoming connection requests, and from those requests, create a Socket, which is a two-way communications channel between the two end points.
To answer the question, "How do I design a peer-to-peer app that avoids using listening sockets?", you don't. In the case of something like Vuze, the software acts as both client and server simultaneously, hence the term "peer", vs. "client" or "server" alone. In Vuze every client is a server, and every server (except for the tracker) is a client.
If you need a TCP connection between the 2 computers and both of them are behind routers (and you don't want to set up port forwarding) I think the only other possibility you have is having a third server somewhere that isn't behind a firewall running a ServerSocket and accepting connections between your 2 other computers and proxying communications between the 2. You can't establish a TCP Connection between the 2 without one listening to a socket and the other connecting to it.
Q: If I want to connect two computers without either one of the the
computers messing about with router settings, is there a way that I
can get the two clients to connect to each other
Yes: have the server listen on an open port :)
I've been reading up on basic network programming, but am having a difficult time finding a straight-forward explanation for what exactly and socket is, and how it relates to either the OSI or TCP/IP stack.
Can someone explain to me what a socket is? Is it a programmer- or API-defined data structure, or is it a hardware device on a network card?
What layers of the mentioned network models deal with "raw" sockets? Transport layer? Network layer?
In terms of the data they pass between them, are socket text-based or binary?
Is there an alternative to sockets-based network programming? Or do all networked applications use some form of socket?
If I can get this much I should have a pretty clear understanding of everything else I'm reading. Thanks for any help!
Short answers:
Socket is an abstraction of an IP connection endpoint - so if you think of it as an API structure, you are not very far off. Please do read http://en.wikipedia.org/wiki/Internet_socket
Internet layer i.e. IP Protocol. In practice you usually use explicitly sockets that bind to a certain transport layer parameters (datagram/UDP or stream/TCP)
Sockets send data, in network byte order - whether it is text or binary, depends on the upper layer protocol.
Theoretically, probably yes - but in practice all IP traffic is done using 'sockets'
Socket is a software mechanism provided by the operating system. Like its name implies, you can think of it like an "electrical outlet" or some electrical connector, even though socket is not a physical device, but a software mechanism. In real world when you have two electrical connectors, you can connect them with a wire. In the same way in network programming you can create one socket on one computer and another socket on another computer and then connect those sockets. And when you write data to one of them, you receive it on the other one. There are also a few different kinds of sockets. For example if you are programming a server software, you want to have a listening socket which never sends or receives actual data but only listens for and accepts incoming connections and creates a new socket for each new connection.
A socket, in C parlance, is a data structure in kernel space, corresponding to one end-point of a UDP or TCP session (I am using session very loosely when talking about UDP). It's normally associated with one single port number on the local side and seldom more than one "well-known" number on either side of the session.
A "raw socket" is an end-point on, more or less, the physical transport. They're seldom used in applications programming, but sometimes used for various diagnostic things (traceroute, ping, possibly others) and may required elevated privileges to open.
Sockets are, in their nature, a binary octet-transport. It is not uncommon to treat sockets (TCP sockets, at least) as being text-based streams.
I have not yet seen a programming model that doesn't involve something like sockets, if you dig deep enough, but there have certainly been other models of doing networking. The "/net/" pseudo-filesystem, where opening "/net/127.0.0.0.1/tcp/80" (or "tcp/www") would give you a file handle where writes end up on a web server on localhost is but one.
Suppose your PC at home, and you have two browser windows open.
One looking at the facebook website, and the other at the Yahoo website.
The connection to facebook would be:
Your PC – IP1+port 30200 ——– facebook IP2 +port 80 (standard port)
The combination IP1+30200 = the socket on the client computer and IP2 + port 80 = destination socket on the facebook server.
The connection to Yahoo would be:
your PC – IP1+port 60401 ——–Yahoo IP3 +port 80 (standard port)
The combination IP1+60401 = the socket on the client computer andIP3 + port 80 = destination socket on the Yahoo server.
For an UDP server architecture that will have long-lived connections, one architecture is to have one socket that listens to all incoming UDP traffic, and then create separate sockets for each connection using connect() to set the remote address. My question is whether it is possible to do this atomically similar to what accept() does for TCP.
The reason for creating a separate socket and using connect() is that this makes it easy to spread the packet-processing across multiple threads, and also make it easier to have the socket directly associated with the data structures that are needed for processing.
The demultiplexing logic in the networking stack will route the incoming packets to the most specific socket.
Now my question is basically what happens when one wants to emulate accept() for UDP like this:
Use select() with a fd-set that includes the UDP server-socket.
Then read a packet from the UDP server-socket.
Then create a new UDP socket which is then connect()ed to the remote address
I call select() with a fd-set that includes both sockets.
What is returned?
given that a packet arrives to the OS somewhere between 1 and 3.
Will the packet be demultiplexed to the UDP server-socket, or will it be demultiplexed to the more specific socket created in 3. That is, at what point does demultiplexing take place? When the packet arrives, or must it happen "as if" it arrived at point 4?
Follow-up question in case the above does not work: What's the best way to do this?
I see that this discussion is from 2009, but since it keeps popping up when I search, I thought I should share my approach. Both to get some feedback and because I am curios about how the author of the question solved the problem.
The way I chose emulate UDP-accept was a combination of number one and two in nik's answer. I have a root thread which listens on a given socket. I have chosen to use TCP for simplicity, but changing this socket to UDP is not very hard. When a client wants to "connect" to my server using UDP, it first connects to the TCP socket and requests a new connection.
The root thread then proceeds by creating a UDP socket, binds it to a local interface, does connect and sets up data structures. This file descriptor is then passed to the thread that will be responsible for the connection. The IP/port information of the new UDP socket is passed back to the client, which creates a new UDP socket and sends data to the provided IP/port.
This approach works well for my use, but the additional steps for setting up a flow introduces an overhead. In some cases, this overhead might not be acceptable.
I found this question after asking it myself here...
UDP server and connected sockets
Since connect() is available for UDP to specify the peer address, I wonder why accept() wasn't made available to effectively complete the connected UDP session from the server side. It could even move the datagram (and any others from the same client) that triggered the accept() over to the new descriptor.
This would enable better server scalability (see the rationale behind SO_REUSEPORT for more background), as well as reliable DTLS authentication.
This will not work.
You have two simple options.
Create a multi-threaded program that has a 'root' thread listening on the UDP socket and 'dispatching' received packets to the correct thread based on the source. This is because you want to segregate processing by source.
Extend your protocol so the the sources accept an incoming connection on some fixed port and then continue with the protocol communication. In this case you would let the source request on the standard UDP port (of your choice), then your end will respond from a new UDP socket to the sources' UDP port. This way you have initiated a new UDP path from your end backwards to the known UDP port of each source. That way you have different UDP sockets at your end.