I'm trying to implement an HTTP proxy for learning and debug purpose.
The support of plain HTTP transactions was pretty straightforward to implement and now I'm looking to implement support for SSL/TLS tunnels.
From RFC 7230:
A "tunnel" acts as a blind relay between two connections without
changing the messages. Once active, a tunnel is not considered a party
to the HTTP communication, though the tunnel might have been initiated
by an HTTP request.
It's not very clear whether I shall build the TLS socket from the socket on which the HTTP CONNECT transaction took place. I assume it is the case, since HTTP is stateless, but I just want to be sure.
When a client connects to an HTTP proxy, CONNECT is used to have the proxy establish a persistent TCP connection with the target TCP server. Then the proxy blindly passes data as-is back and forth between the two TCP connections until either the client or server disconnects, then the proxy disconnects the other party. This allows the client to send data to the server and vice versa, such as TLS packets. This is important so the TLS server can verify the client's identity during the TLS handshake.
So, to answer your question - yes, the client must establish a TLS session with the target server using the same TCP socket that it used to issue the CONNECT request on. Once the CONNECT request has succeeded, the client can treat the existing TCP connection as if it had connected to the server directly. The proxy is transparent at that point, neither party needs to care that it is present.
Related
I'm building a message broker which communicates with clients over ZeroMQ PUSH/PULL sockets and has the ability to exclude clients from messages they're not subscribed to from the server side (unlike ZeroMQ pub/sub which excludes messages on the client side).
Currently, I implement it in the following way:
Server: Binds ZeroMQ PULL socket on a fixed port
Client: Binds a ZeroMQ PULL socket on a random or fixed port
Client: Connects to the server's PULL socket and sends a handshake message containing the new client's address and port.
Server: Recieves handshake from client and connects a PUSH socket to the client's PULL server. Sends handshake response to the client's socket.
Client: Recieves handshake. Connected!
Now the client and server can communicate bidirectionally and the server can send messages to only a certain subset of clients. It works great!
However, this model doesn't work if the clients binding PULL sockets are unable to open a port in their firewall so the server can connect to them. How can I resolve this with minimal re-architecting (as the current model works very well when the firewall can be configured correctly)
I've considered the following:
Router/dealer pattern? I'm fairly ignorant on this and documentation I found was sparse.
Some sort of transport bridging? The linked example provides an example for PUB/SUB.
I was hoping to get some advice from someone who knows more about ZeroMQ than me.
tl;dr: I implemented a message broker that communicates with clients via bidirectional push/pull sockets. Each client binds a PULL socket and the server keeps a map of PUSH sockets so that it can address specific subscribers. How do I deal with a firewall blocking the client ports?
You can use the router/dealer to do this like you say. By default the ROUTER socket tracks every connection it has. The way it does this is by having the caller stick the connection identity information in front of each message it recieves. This makes things like pub/sub fairly trivial as all you need to do is handle a few messages server side that the DEALER socket sends it. In the past I have done something like
1.) Server side is a ROUTER socket. The ROUTER handles 2 messages from DEALER sockets SUB/UNSUB. This alongside the identity info sent as the first part of a frame allows the router to know the messages that a client is interested in.
2.) The server checks the mapping to see which clients should be sent a particular type of data using the map and then forwards the message to the correct client by appending the identity again to the start of the message.
This is nice in that it allows a single port to be exposed on the server. Client side we do not need to expose ports, simply just connect to the server ROUTER socket.
See https://zguide.zeromq.org/docs/chapter3/ for more info.
Just some concept about TCP Socket, let's say there are 100 clients simultaneously communicating with a traditional HTTP/TCP web server. How many sockets are respectively at the server and at each client? Do all of the sockets at the server
have the same server-side port number?
The question is generic, so the answer is going to be as well.
For traditional TCP-based HTTP server, there will be 100 sockets on the server (one for each client), and one socket on every client. All server sockets will be bound to the same server port.
This answer doesn't take into account the fact that in modern HTTP model a client usually opens more than one socket to serve a single request.
I'm trying to create multiple tunnels out of a single TCP connection to a SOCKSv4a proxy server in order to keep from closing and reopening a bunch of sockets to the SOCKS proxy.
Something like "Connection: keep-alive". How can I do that? Does the protocol allow it?
No, this is not supported by any version of SOCKS (nor is it supported by HTTP/1.1 CONNECT method — keep-alive is ignored for CONNECT). Once a tunnel is established, it is a straight pass-through of raw data until either the client or server disconnects. You need to open a separate client-proxy connection for each new connection to the server.
In RFC 3920 (XMPP core), it define there must be one TCP connection for each direction for server-to-server communication, but for client-to-server it doesn't force it. So any reason or benefit to force two TCP connections for server-to-server?
In the context of client-to-server communications, a server MUST allow
a client to share a single TCP connection for XML stanzas sent from
client to server and from server to client. In the context of
server-to-server communications, a server MUST use one TCP
connection for XML stanzas sent from the server to the peer and
another TCP connection (initiated by the peer) for stanzas from the
peer to the server, for a total of two TCP connections.
Your reference is outdated. RFC 3920 has been replaced by RFC 6120. The new RFC removes the requirement for 2 TCP connection between server.
Historically, those two s2s connections are needed because of s2s dialback. Dialback is DNS based authentication of the third party server. If you do TLS cert based authentication for your s2s connections, you can rely on a single connection.
Reference: http://xmpp.org/rfcs/rfc6120.html#tcp
Is it possible this Scenario without using webrtc?
Client A connects to Server
Client B connects to Server
Server send information to B and to A with each others information
A and B start talking directly using the same connection (by transforming their sockets info with the new port/ip address for example)
note: Client A and B doesn't have any socket server. They are only clients
Thank you very much.
It's definitely possible to set up a direct TCP connection, if at least one of the clients can accept an incoming TCP connection. In that case the server can just tell the other client what IP address and port to connect to, and then the two clients can communicate directly over the new TCP connection.
If both clients are behind firewalls or NAT, on the other hand, things get a lot more iffy -- in some cases you can use TCP hole punching techniques to get a direct TCP connection started; in other cases you're just out of luck.
In no case (AFAIK) can you modify an existing TCP connection to have new endpoints; a TCP connection's endpoints are fixed, and to get new endpoints you have to set up a new TCP connection.