I'm writing an HTTP server and client in python. When I run my scripts for client and server in terminal everything works fine. However, when I go to my browser and type "localhost:12000" in the searchbar, I get an error saying "The page isn't working. localhost didn't send any data. ERR_EMPTY_RESPONSE". What I expect to see instead, is the content of the html file contained in the response message.
This is the code for my HTTP client.
from socket import *
clientSocket = socket(AF_INET,SOCK_STREAM)
serverPort = 18000
clientSocket.connect(("localhost",serverPort))
request = "GET www.somepage/index.html HTTP/1.0\r\nHost: www.somepage.com\r\nConnection: close\r\nUser-Agent: Chrome/86.0.4240.183\r\nAccept: text/html, application/xhtml+xml\r\nAccept-Language: it-IT, en-US\r\nAccept-Encoding: gzip, deflate\r\nAccept-Charset: ISO-8859-1, utf-8\r\n"
print(request)
clientSocket.send(request.encode())
response = (clientSocket.recv(1024)).decode()
print(response)
clientSocket.close()
This is the code for my server.
from socket import *
from datetime import date
from time import gmtime, strftime
import calendar
serverSocket = socket(AF_INET,SOCK_STREAM)
serverPort = 12000
serverSocket.bind(("localhost",serverPort))
serverSocket.listen(1)
current_date = calendar.day_abbr[date.today().weekday()]+", "+date.today().strftime("%d %b %Y")+strftime(" %H:%M:%S", gmtime())+ " GMT"
while True:
connection , addr = serverSocket.accept()
request = (connection.recv(1024)).decode()
request = request.split()
method = request[0]
URL = request[1]
version = request[2]
if method == "GET" and URL == "www.somepage/index.html" and version == "HTTP/1.0":
response = "HTTP/1.0 200 OK\r\nConnection: close\r\nDate: {}\r\nServer: Apache\r\nLast-Modified: Tue, 10 Nov 2020, 6:31:00 GMT\r\nContent-Length: 72 bytes\r\nContent-Type: text/html\r\n<html>\r\n<title>PAGE TITLE</title>\r\n<body>\rThis is the body\r\n</body></html>".format(current_date)
connection.send(response.encode())
connection.close()
So the server is checking if the request line is correct and then sending the HTTP response, which I do see in the terminal, but when I try in the browser I get an error instead. I've also tried checking Wireshark and I do see the HTTP messages there, so I don't understand why my browsers says no data has been sent.
Thank you all for your help.
Edit:
I couldn't post my code in the comment so I'll try here. What I'm trying to do is create an HTTP client and server than don't implement the entire HTTP protocol, but just a few request methods and a few replies. For now I was starting with the GET method and the 200 OK reply.
This is the code for my client. I have added an extra \r\n at the end of the header in the request.
from socket import *
clientSocket = socket(AF_INET,SOCK_STREAM)
serverPort = 12000
clientSocket.connect(("localhost",serverPort))
request = "GET /index.html HTTP/1.0 \r\nHost: www.somepage.com\r\nConnection: close\r\nUser-Agent: Chrome/86.0.4240.183\r\nAccept: text/html, application/xhtml+xml\r\nAccept-Language: it-IT, en-US\r\nAccept-Encoding: gzip, deflate\r\nAccept-Charset: ISO-8859-1, utf-8\r\n\r\n"
print(request)
clientSocket.send(request.encode())
response = (clientSocket.recv(1024)).decode()
print(response)
clientSocket.close()
This is the code for my server, with an added \r\n at the end of header as well.
from socket import *
from datetime import date
from time import gmtime, strftime
import calendar
serverSocket = socket(AF_INET,SOCK_STREAM)
serverPort = 12000
serverSocket.bind(("localhost",serverPort))
serverSocket.listen(1)
current_date = calendar.day_abbr[date.today().weekday()]+", "+date.today().strftime("%d %b %Y")+strftime(" %H:%M:%S", gmtime())+ " GMT"
while True:
connection , addr = serverSocket.accept()
request = (connection.recv(1024)).decode()
request = request.split()
method = request[0]
URI = request[1]
version = request[2]
host = request[4]
if method == "GET" and URI == "/index.html" and version == "HTTP/1.0" and host == "www.somepage.com":
response = "HTTP/1.0 200 OK \r\nConnection: close\r\nDate: {}\r\nServer: Apache\r\nLast-Modified: Tue, 10 Nov 2020, 6:31:00 GMT\r\nContent-Length: 83 bytes\r\nContent-Type: text/html\r\n\r\n<html>\r\n<title>PAGE TITLE</title>\r\n<body>\rThis is the body\r\n</body></html>".format(current_date)
connection.send(response.encode())
connection.close()
I've studied the standard and I'm trying to write my code according to the specifications. What I see in my browser is this error:
error
I've also noticed that if I change my server code to this:
from socket import *
from datetime import date
from time import gmtime, strftime
import calendar
serverSocket = socket(AF_INET,SOCK_STREAM)
serverPort = 12000
serverSocket.bind(("localhost",serverPort))
serverSocket.listen(1)
current_date = calendar.day_abbr[date.today().weekday()]+", "+date.today().strftime("%d %b %Y")+strftime(" %H:%M:%S", gmtime())+ " GMT"
while True:
connection , addr = serverSocket.accept()
request = (connection.recv(1024)).decode()
request = request.split()
method = request[0]
URI = request[1]
version = request[2]
host = request[4]
if "GET" in request:
response = "HTTP/1.0 200 OK \r\nConnection: close\r\nDate: {}\r\nServer: Apache\r\nLast-Modified: Tue, 10 Nov 2020, 6:31:00 GMT\r\nContent-Length: 83 bytes\r\nContent-Type: text/html\r\n\r\n<html>\r\n<title>PAGE TITLE</title>\r\n<body>\rThis is the body\r\n</body></html>".format(current_date)
connection.send(response.encode())
connection.close()
where basically the only difference is the way the if statement is written, then my browser will display correctly the html, that is, I see this:
page
So it seems the problem lies in the syntax I used for my python code, and not the way the standard is implemented?
Thank you again so very much for your help.
I couldn't post my code in the comment so I'll try here. What I'm trying to do is create an HTTP client and server than don't implement the entire HTTP protocol, but just a few request methods and a few replies. For now I was starting with the GET method and the 200 OK reply.
This is the code for my client. I have added an extra \r\n at the end of the header in the request.
from socket import *
clientSocket = socket(AF_INET,SOCK_STREAM)
serverPort = 12000
clientSocket.connect(("localhost",serverPort))
request = "GET /index.html HTTP/1.0 \r\nHost: www.somepage.com\r\nConnection: close\r\nUser-Agent: Chrome/86.0.4240.183\r\nAccept: text/html, application/xhtml+xml\r\nAccept-Language: it-IT, en-US\r\nAccept-Encoding: gzip, deflate\r\nAccept-Charset: ISO-8859-1, utf-8\r\n\r\n"
print(request)
clientSocket.send(request.encode())
response = (clientSocket.recv(1024)).decode()
print(response)
clientSocket.close()
This is the code for my server, with an added \r\n at the end of header as well.
from socket import *
from datetime import date
from time import gmtime, strftime
import calendar
serverSocket = socket(AF_INET,SOCK_STREAM)
serverPort = 12000
serverSocket.bind(("localhost",serverPort))
serverSocket.listen(1)
current_date = calendar.day_abbr[date.today().weekday()]+", "+date.today().strftime("%d %b %Y")+strftime(" %H:%M:%S", gmtime())+ " GMT"
while True:
connection , addr = serverSocket.accept()
request = (connection.recv(1024)).decode()
request = request.split()
method = request[0]
URI = request[1]
version = request[2]
host = request[4]
if method == "GET" and URI == "/index.html" and version == "HTTP/1.0" and host == "www.somepage.com":
response = "HTTP/1.0 200 OK \r\nConnection: close\r\nDate: {}\r\nServer: Apache\r\nLast-Modified: Tue, 10 Nov 2020, 6:31:00 GMT\r\nContent-Length: 83 bytes\r\nContent-Type: text/html\r\n\r\n<html>\r\n<title>PAGE TITLE</title>\r\n<body>\rThis is the body\r\n</body></html>".format(current_date)
connection.send(response.encode())
connection.close()
I've studied the standard and I'm trying to write my code according to the specifications. What I see in my browser is this error:
error
I've also noticed that if I change my server code to this:
from socket import *
from datetime import date
from time import gmtime, strftime
import calendar
serverSocket = socket(AF_INET,SOCK_STREAM)
serverPort = 12000
serverSocket.bind(("localhost",serverPort))
serverSocket.listen(1)
current_date = calendar.day_abbr[date.today().weekday()]+", "+date.today().strftime("%d %b %Y")+strftime(" %H:%M:%S", gmtime())+ " GMT"
while True:
connection , addr = serverSocket.accept()
request = (connection.recv(1024)).decode()
request = request.split()
method = request[0]
URI = request[1]
version = request[2]
host = request[4]
if "GET" in request:
response = "HTTP/1.0 200 OK \r\nConnection: close\r\nDate: {}\r\nServer: Apache\r\nLast-Modified: Tue, 10 Nov 2020, 6:31:00 GMT\r\nContent-Length: 83 bytes\r\nContent-Type: text/html\r\n\r\n<html>\r\n<title>PAGE TITLE</title>\r\n<body>\rThis is the body\r\n</body></html>".format(current_date)
connection.send(response.encode())
connection.close()
where basically the only difference is the way the if statement is written, then my browser will display correctly the html, that is, I see this:
page
So it seems the problem lies in the syntax I used for my python code, and not the way the standard is implemented?
Thank you again so very much for your help.
request = "GET www.somepage/index.html HTTP/1.0\r\nHost: www.somepage.com\r\nConnection: close\r\nUser-Agent: Chrome/86.0.4240.183\r\nAccept: text/html, application/xhtml+xml\r\nAccept-Language: it-IT, en-US\r\nAccept-Encoding: gzip, deflate\r\nAccept-Charset: ISO-8859-1, utf-8\r\n"
This is not a valid HTTP request. First, it should only contain the path /index.html and not domain/path as you currently do. It is also missing the final \r\n at the end which signals the end of the HTTP header.
In the same way the expectations of the server wrong too, which explains why it causes problems when faced with a client correctly implementing HTTP (the browser). Additionally the HTTP response is also missing the final \r\n after the HTTP header and the Content-length: 72 does not match the actual length of the content.
Please don't implement HTTP by (wrongly) second-guessing how it works. There is an actual standard for this and implementations are expected to follow this standard.
After the edit the code looks like this:
request = request.split()
...
version = request[2]
host = request[4]
if method == "GET" and URI == "/index.html" and version == "HTTP/1.0" and host == "www.somepage.com":
... send response ...
There are multiple problems here: the first one is that the browser will not use HTTP/1.0 as version but HTTP/1.1.
The next problem is that the domain might not be in the variable host since it is might not be in request[4]. It is blindly assumed that the Host header is in the second line of the request since it is implemented like this in the client. But the HTTP standard does in now way require this. And while it might be the case with some clients it is not the case with others. Instead of blindly assuming that something is in a specific place in the HTTP header the header should actually be parsed properly to extract the Host header.
Currently running tests for my REST API which:
takes an endpoint from the user
using that endpoint, grabs info from a server
sends it to another server to be translated
then proceeds to jsonify the data.
I've written a series of automated tests running and I cannot get one to pass - the test that actually identifies the content of the response. I've tried including several variations of what the test is expecting but I feel it's the actual implementation that's the issue. Here's the expected API response from the client request:
{ "name": "random_character", "description": "Translated description of requested character is output here" }
Here is the testing class inside my test_main.py:
class Test_functions(unittest.TestCase):
# checking if response of 200 is returned
def test_healthcheck_PokeAPI(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/")
status_code = response.status_code
self.assertEqual(status_code, 200)
# the status code should be a redirect i.e. 308; so I made a separate test for this
def test_healthcheck_ShakesprAPI(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/charizard")
self.assertEqual(response.status_code, 308)
def test_response_content(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/charizard")
self.assertEqual(response.content_type,
'application/json') <<<< this test is failing
def test_trans_shakespeare_response(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/charizard")
self.assertFalse(b"doth" in response.data)
Traceback:
AssertionError: 'text/html; charset=utf-8' != 'application/json' - text/html; charset=utf-8 + application/json
Any help would be greatly appreciated
I have this feature file and i get a response correctly. I want to print obtained value from response but somehow I am not able to do that. Tried to research some stuff but I couldnt help myself.
Can anyone help please? Thanks in advance
Feature:
test of soap
Background:
* url 'http://www.dataaccess.com/webservicesserver/numberconversion.wso'
Scenario: soap 1.1
Given request
"""
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:web="http://www.dataaccess.com/webservicesserver/">
<soapenv:Header/>
<soapenv:Body>
<web:NumberToDollars>
<web:dNum>10</web:dNum>
</web:NumberToDollars>
</soapenv:Body>
</soapenv:Envelope>
"""
When soap action 'Conversion'
Then status 200
* print '\n', response
#working
* match response /Envelope/Body/NumberToDollarsResponse/NumberToDollarsResult == 'ten dollars'
#not working
* print response.Envelope.Body.NumberToDollarsResponse.NumberToDollarsResult
#not working
* print response /Envelope/Body/NumberToDollarsResponse/NumberToDollarsResult
#not working
* def x = response /Envelope/Body/NumberToDollarsResponse/NumberToDollarsResult
* print x
If you read the docs, print only handles JS on the right-hand-side, not XPath.
For what you want, please do in 2 steps:
* def temp = /Envelope/Body/NumberToDollarsResponse/NumberToDollarsResult
* print temp
I have built a sample Citrus testcase to simulate a SOAP server that responds with an MTOM attachment.
runner.soap(action -> action.server("simulationServer")
.receive()
...[validation etc]
);
runner.soap(action -> action.server("simulationServer")
.send()
.name("get-response")
.mtomEnabled(Boolean.TRUE)
.attachment("myAttachment", "application/octet-stream", new ClassPathResource("testfiles/myAttachment.pdf"))
.payload("<getResponse xmlns:xmime=\"http://www.w3.org/2005/05/xmlmime\">\n" +
" <document>\n" +
" <contentElements>\n" +
" <contentElement xmime:contentType=\"application/pdf\">cid:myAttachment</contentElement>\n" +
" </contentElements>\n" +
" <id>Test</id>\n" +
" </document>\n" +
"</getResponse>\n")
);
When I run this test and call the Citrus simulation with SoapUI, I see the contents of myAttachment.pdf in the debug logs. So at least it looks like Citrus tries to send the attachment.
However, in SoapUI I do not get an attachment. There is a XOP element in the SOAP response, but no attachment. The RAW view of SoapUI of the Citrus response looks like this.
HTTP/1.1 200 OK
Date: Tue, 16 Jan 2018 15:30:36 GMT
Accept: text/xml, text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
SOAPAction: ""
Content-Type: Multipart/Related; boundary="----=_Part_0_382348859.1516116636524"; type="application/xop+xml"; start-info="text/xml"
Transfer-Encoding: chunked
Server: Jetty(9.4.6.v20170531)
------=_Part_0_382348859.1516116636524
Content-Type: application/xop+xml; charset=utf-8; type="text/xml"
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header/><SOAP-ENV:Body><getResponse xmlns:xmime="http://www.w3.org/2005/05/xmlmime">
<document>
<contentElements>
<contentElement xmime:contentType="application/pdf"><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:myAttachment"/></contentElement>
</contentElements>
<id>Test</id>
</document>
</getResponse></SOAP-ENV:Body></SOAP-ENV:Envelope>
------=_Part_0_382348859.1516116636524--
In an MTOM response with attachment the attachment starts where this RAW view ends. It should continue like this
------=_Part_0_382348859.1516116636524-- [last line from above]
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID: <myAttachment>
%PDF-1.4... [PDF content]
I am using Citrus 2.7.2 release.
Update
Still no success on this. Wireshark shows the same picture as SoapUI: the attachment is missing in the response.
However, when I debug into the code on the server (Citrus) side, I see the attachment in the response message until I get lost somewhere in a MessageSendingTemplate. Same on the console log. The message has the attachment.
There is an inline MTOM variant in the Citrus documentation, but I can't find a way to set this mtom-inline on the attachment in Java config.
Any hints, where to set a breakpoint to find where the attachment get lost? Or any other suggestions/examples on the Citrus side?
The setMtomInline field sits on the SoapAttachment interface. I am not sure if I got the setup right - but seems to work for inlined attachements - fails for soap attachements / multipart. The SoapUI Mock does not show any attachements when receiving requests from following testcase.
SoapAttachment soapAttachment = new SoapAttachment();
soapAttachment.setMtomInline(false);
soapAttachment.setContentResourcePath("log4j.xml");
soapAttachment.setContentType("application/octet-stream");
soapAttachment.setContentId("FILE");
SoapMessage soapMessage = new SoapMessage();
soapMessage.mtomEnabled(true);
soapMessage.soapAction("/HelloService/sayHello");
soapMessage.setPayload(
"<ht:HelloRequest " +
"xmlns:ht=\"http://citrusframework.org/schemas/samples/HelloMtomService\" " +
"xmlns:xop=\"http://www.w3.org/2004/08/xop/include\" >\n" +
" <ht:Message>Hei .. citrus does stream mtom</ht:Message>\n" +
" <ht:Data><xop:Include href=\"cid:FILE\"/></ht:Data>\n" +
"</ht:HelloRequest>");
soapMessage.addAttachment(soapAttachment);
runner.soap(action -> {
action.client("helloMtomSoapuiClient")
.send()
.soapAction("/HelloService/sayHello")
.message(soapMessage);
});
If I do the same for MtomInline set to true, I see the attachement as base64 encoded content text in the ht:Data node.
SoapAttachment soapAttachment = new SoapAttachment();
soapAttachment.setContentResourcePath("log4j.xml");
soapAttachment.setMtomInline(true);
soapAttachment.setContentType("application/xml");
soapAttachment.setContentId("MyAttachement");
soapAttachment.setEncodingType("base64Binary");
runner.soap(action -> {
action.client("helloMtomSoapuiClient")
.send()
.soapAction("/HelloService/sayHello")
.mtomEnabled(true)
.payload("<ht:HelloRequest xmlns:ht=\"http://citrusframework.org/schemas/samples/HelloMtomService\">\n" +
" <ht:Message>Hei .. citrus does mtom</ht:Message>\n" +
" <ht:Data>cid:MyAttachement</ht:Data>\n" +
"</ht:HelloRequest>")
.attachment(soapAttachment);
});
Either soapUI or citrus swallows the attachement. Some help or working JavaDSL sample would be nice.
It was actually a bug that will be fixed in Citrus 2.7.4 release. See https://github.com/christophd/citrus/issues/328
The inline MTOM variant with XML config works for me in the current release.
<ws:send endpoint="simulationServer" mtom-enabled="true">
<message>
<resource file="testfiles/simulation/get-response.xml" />
</message>
<ws:attachment content-id="myAttachment" content-type="application/octet-stream" mtom-inline="true" encoding-type="base64Binary">
<ws:resource file="classpath:testfiles/myAttachment.pdf"/>
</ws:attachment>
</ws:send>
Trying to consume Azure media service rest api. (following the tutorial : https://learn.microsoft.com/en-us/azure/media-services/media-services-rest-get-started)
Everything works fine until the point I try to create a Job. Sending the same request as in example (except asset id and token) and getting response :
Parsing request content failed due to: Make sure to only use property names that are defined by the type
Request:
POST https://wamsdubclus001rest-hs.cloudapp.net/api/Jobs HTTP/1.1
Connection: Keep-Alive
Content-Type: application/json
Accept: application/json; odata=verbose
Accept-Charset: UTF-8
Authorization: Bearer token -> here i send real token
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 3.0;NetFx
x-ms-version: 2.11
Content-Length: 458
Host: wamsdubclus001rest-hs.cloudapp.net
{
"Name":"TestJob",
"InputMediaAssets":[
{
"__metadata":{
"uri":"https://wamsdubclus001rest-hs.cloudapp.net/api/Assets('nb%3Acid%3AUUID%3A5168b52a-68ed-4df1-bac8-0648ce734ff6')"
}
}
],
"Tasks":[
{
"Configuration":"Adaptive Streaming",
"MediaProcessorId":"nb:mpid:UUID:ff4df607-d419-42f0-bc17-a481b1331e56",
"TaskBody":"<?xml version=\"1.0\" encoding=\"utf-8\"?><taskBody><inputAsset>JobInputAsset(0)</inputAsset> <outputAsset>JobOutputAsset(0)</outputAsset></taskBody>"
}
]
}
Response:
{
"error":{
"code":"",
"message":{
"lang":"en-US",
"value":"Parsing request content failed due to: Make sure to only use property names that are defined by the type"
}
}
}
It seems to be related with __metadata property. when I follow instruction from here : Creating Job from REST API returns a request property name error, the error changes:
"error":{
"code":"",
"message":{
"lang":"en-US",
"value":"Invalid input asset reference in TaskBody - "
}
}
}
Cant figure out whats wrong, thanks
Let me check on this, but it could be a couple issues that I have run into in the past.
First. Set both the Accept and Content-Type headers to:
"application/json; odata=verbose"
Next, double check that you are actually using the long underscore character on the metadata property. I've had issues where that was sending the wrong underscore character and it didn't match the property name.
Let me know if either of those helps.
It seems the issue was about "Content-Type". As I am using .net Core it was not easy to set the Conent-type as "application/json; odata=verbose".
1) Tried with RestSharp - dosnt support it, it cuts "odata=verbose" part out
2) Tried with Systsem.Net.Http.HttpClient -> Possible but difficult.
To add it as "Accept" :
MediaTypeWithQualityHeaderValue mtqhv;
MediaTypeWithQualityHeaderValue.TryParse("application/json;odata=verbose", out mtqhv);
client.DefaultRequestHeaders.Accept.Add(mtqhv);//ACCEPT header
To add it as "Content-Type" :
request.Content = new StringContent(content,
System.Text.Encoding.UTF8); //CONTENT-TYPE header -> default type will be text/html
request.Content.Headers.Clear(); // need to clear it - it will fail otherwise
request.Content.Headers.TryAddWithoutValidation("Content-Type","application/json;odata=verbose");