I need to explore the traffic from one program.
The program makes something like a connection through the WebSockets.
Fiddler displays this:
Request Headers: CONNECT 144.***:443 HTTP/1.0
Response: HTTP/1.0 200 Connection Established
End empty body.
But the HTTP analyzer displays full information after that response, and that information continues flowing. Very likely like WebSockets (one connection and receive more answers).
And fiddler display zero traffic.
How can I explore such traffic through the fiddler?
A CONNECT call is always the first command a client sends if it uses a Proxy. Translated CONNECT just means: Please start a connection to the following server and that port. Through that connection the real HTP calls are then transmitted. Therefore CONNECT is not a real HTTP
request.
Fiddler does not show the content of CONNECT requests/responses to port 443 endpoints because those connections are HTTPS/TLS protected (hence the shown data would be useless). You need to enable HTTPS decryption and install the Fiddler root CA certificate into the client app/OS to see the decrypted content of those connections.
Related
Given a URL that ends with .svc and that is supposed to run a SOAP web service, how can I get some data from it?
I tried:
to access it via a web browser
to access it via the Python's library Zeep
to access it via the Microsoft utilitary svcutil.exe
In all cases, I get a timeout error:
Failed to establish a new connection: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time.
Does it mean that the web service does not work, or that I do things the wrong way?
Importantly - there is a big distinction between "service not active" (and by that I mean no listener on port 80), and "port not open in firewall".
If the problem were simply that you didn't have a service listening on port 80, you would have gotten something like "connection reset" or "connection rejected" as an error.
Instead, you appear to have gotten a timeout, which implies that either the SYN from the client doesn't reach the server, or the SYN/ACK from the server doesn't reach the client. [ You could verify this by doing a packet capture for port 80 on both client and server ]
I would be tempted to check any firewall in front of the server to see that it's letting port 80 traffic through from your client.
Diagnosing Connectivity Issues
Without more details it is difficult to say, but given your timeout error:
Failed to establish a new connection: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time.
This indicates a network connectivity error at the TCP level, so it is likely web service is not active on the port your are using (default of 80 for http, 443 for https).
In a comment you said you pinged the URL and it responded normally - I assume this means you pinged the hostname. If this is responding normally it means the server is active, but that doesn't tell you anything about the availability of the web service on that server.
telnet %hostname% %port%
where %port% is 80 for http or 443 for https, or something else if there is a port number in the URL you are using (e.g. http://somehost.somewhere.com:port/path.scv)
If ping works and telnet does not connect, then the service is not active.
I suspect this is the case. If the service was active and it was simply that you requesting the data incorrectly, I believe you'd get a different error message - e.g. a valid HTTP response with status code 500 or 404 or similar.
Getting Data from a Web Service
As to your original question as to how to get data from it - once you verify that the service is active, the method to get the data will depend on the specification of the service - i.e.:
which HTTP methods (GET, POST, etc.) does it support
what parameters it requires
what format it requires the parameters in
are the parameters in the query string or POST body.
To interact with a web service there are many command line tools that can be used, as well as the options you have tried, including:
POSTMan Google Chrome Plugin
curl
wget
In windows Powershell, the Invoke-WebRequest
Getting Data from a SOAP Web Service
As you have said it is a SOAP web service, if you have the URL for the wsdl, you can often interract with it using Powershell SOAP WebService Proxies.
The wsdl location varies, but is often at a URL that looks something like.
http://host/path.svc?wsdl
http://host/path.svc/?wsdl
http://host/path/?wsdl
Also if it's configured correctly, just loading the URL in a browser will present a page with a link to the wsdl.
The general idea is:
$URI="http://hostname/path.svc?wsdl"
$Proxy = New-WebserviceProxy $URI –Namespace X
$Proxy | get-member -MemberType Method
This will return a list of methods on the proxy that you can invoke as powershell methods. Any types defined in the wsdl that are needed for arguments, or returned from methods will be available within the namespace X. Invoking the methods will proxy the request to the service, taking care of serializing parameters and serializing results into powershell objects.
Breif Description of what I am trying to accomplish. So I am working with Crestrons Simpl+ software. My job is to create a module for a sound masking system called QT Pro. Now, QT Pro has an API where you can control it via HTTP. I need a way to establish a connection with the QT Pro via HTTP( I have everything I need, IP, Username, Password).
Whats the problem? I have just started working with this language. Unfortunately there isn't as much documentation as I would like, otherwise I wouldn't be here. I know I need to create a socket connection via TCP on port 80. I just don't know what I'm supposed to send through it.
Here is an example:
http://username:password#address/cmd.htm?cmd=setOneZoneData&ZN=Value&mD=Value
&mN=Value&auxA=Value&auxB=Value&autoR=Value
If I were to put this into the URL box, and fill it in correctly. then it would change the values that I specify. Am I supposed to send the entire thing? Or just after cmd.htm? Or is there some other way I'm supposed to send data? I'd like to stay away from the TCP/IP Module so I can keep this all within the same module.
Thanks.
You send
GET /cmd.htm?cmd=setOneZoneData&ZN=Value&mD=Value&mN=Value&auxA=Value&auxB=Value&autoR=Value HTTP/1.1
Host: address
Connection: close
(End with a couple of newlines.)
If you need to use HTTP basic authentication, then also include a header like
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
where the gibberish is the base64-encoded version of username:password.
But surely there is some mechanism for opening HTTP connections already there for you? Just blindly throwing out headers like this and hoping the response is what you expect is not robust, to say the least.
To see what is going on with your requests and responses, a great tool is netcat (or telnet, for that matter.)
Do nc address 80 to connect to server address on port 80, then paste your HTTP request:
GET /cmd.htm HTTP/1.1
Host: address
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
Connection: close
and see what comes back. SOMETHING should come back. (Remember to terminate with two newlines.)
To see what requests your browser is sending when you do something that works, you can listen like this: nc -l -p 8080.
Then direct your browser to localhost:8080 with the rest of the URL as before, and you'll see the request that was sent. (Then you can type back to see how the browser handles the response.)
Since HTTP is an application layer protocol using TCP, if I request to download a big file via HTTP here is what happens:
My HTTP request is going to be fragmented into TCP packets, and TCP is going to do a 3-way handshake and send my request packets to server. My question is the response from server ( the file) going to pass through old TCP connection, or server initiates another Transport layer connection with my browser and another 3-way handshake in order to send me the file?
The file transfer will use the existing connection. that will however make the connection busy until the file is transferred.
So if the user clicks on a link while the file is downloaded the connection is then busy. The web browser will therefore have to open an additional connection to be able to request the clicked url.
In HTTP/1.1 existing connections will be used if idle (idle connections will be closed when a period of time have passed).
I'm trying to implement an HTTP proxy for learning and debug purpose.
The support of plain HTTP transactions was pretty straightforward to implement and now I'm looking to implement support for SSL/TLS tunnels.
From RFC 7230:
A "tunnel" acts as a blind relay between two connections without
changing the messages. Once active, a tunnel is not considered a party
to the HTTP communication, though the tunnel might have been initiated
by an HTTP request.
It's not very clear whether I shall build the TLS socket from the socket on which the HTTP CONNECT transaction took place. I assume it is the case, since HTTP is stateless, but I just want to be sure.
When a client connects to an HTTP proxy, CONNECT is used to have the proxy establish a persistent TCP connection with the target TCP server. Then the proxy blindly passes data as-is back and forth between the two TCP connections until either the client or server disconnects, then the proxy disconnects the other party. This allows the client to send data to the server and vice versa, such as TLS packets. This is important so the TLS server can verify the client's identity during the TLS handshake.
So, to answer your question - yes, the client must establish a TLS session with the target server using the same TCP socket that it used to issue the CONNECT request on. Once the CONNECT request has succeeded, the client can treat the existing TCP connection as if it had connected to the server directly. The proxy is transparent at that point, neither party needs to care that it is present.
An external application will send HTTP POST request to multiple HTTP/HTTPS servers (e.g. 10 HTTP servers). These HTTP servers may get almost same HTTP Post request. These HTTP servers will analyze the data and send 200 OK response if data validation pass.
I am having all these HTTP servers listening on single host with different ports.
Please suggest me some way to achieve it.
FYI - This request response between Application and HTTP Server(s) will happen only once and then HTTP server will be closed.
I am thinking to implement it using forking the HTTP:Daemon 10 times but looking forward for some light solution.
Also I am thinking to capture these data through a single interface rather then checking the data from all 10 individual 10 HTTP server.
for PORT in `seq 11111 11121` ; do plackup -Ilib --listen :$PORT app.psgi & done