First off thanks for reading!
Second off YES I have tried to find the answer! :) Perhaps I haven't found it because I'm not using the right words to describe my problem, but it's been about 4 hours that I've been trying to figure it out now and I'm getting a little loopy trying to piece it together on my own.
I am very new to programming. Python is my first language. I am on my third Python course. I have an assignment to use the socket library (not urllib library - I know how to do that) to make a socket and use GET to receive information. The problem is that the program needs to take raw input for the URL in question.
I have everything else the way I want it, but I need to know the syntax that I'm supposed to be using INSIDE my "GET" request in order for the HTTP message to include the requested document path.
I have tried (obviously not all together lol):
mysock.send('GET (url) HTTP/1.0\n\n')
mysock.send( ('GET (url) HTTP:/1.0\n\n'))
mysock.send(('GET (url) HTTP:/1.0\n\n'))
mysock.send("GET (url) HTTP/1.0\n\n")
mysock.send( ("'GET' (url) HTTP:/1.0\n\n"))
mysock.send(("'GET' (url) 'HTTP:/1.0\n\n'"))
and:
basically every other configuration of the above (, ((, ( (, ', '' combinations listed above.
I have also tried:
-Creating a string using the 'url' variable first, and then including it inside mysock.send(string)
-Again with the "string-first" theory, but this time I used %r to refer to my user input (so 'GET %r HTTP/1.0\n\n' % url basically)
I've read questions here, other programming websites, the whole chapter in the book and the whole lectures/notes online, I've read articles on the socket library and the .send(), and of course articles on GET requests... but I'm clearly missing something. It seems most don't use socket library when they can use urllib and I don't blame them!!
Thank you again...
Someone from the university posted back to me that the url variable can concatenated with the GET syntax and assigned to a string variable which can then be called with .send(concatenatedvariable) - I had mentioned trying that but had missed that GET requires a space after the word 'GET' so of course concatenating didn't include a space and that blew it. In case anyone else wants to know :)
FYI: A fully quallified URL is only allowed in HTTP/1.1 requests. It is not the norm, though, as HTTP/1.1 requires setting the Host header. The relevant piece of reading would've been RFC 7230, sec. 3.1.1 and possibly RFC 3986. The syntax of the parameters is largely borrowed from the CGI format. It is in no way enforced, however. In a nutshell, everything put together would look like this on the wire:
GET /path?param1=value1¶m2=value2 HTTP/1.1
Host: example.com
As a final note: The line delimiter in HTTP is CRLF (\r\n). For robustness, a simple linefeed is acceptable as well but not recommended.
Related
I am trying to call the resource /matchroute via a GET request.
However, I can't figure out how to encode the GPX file so that the resource accepts my request: I always receive HTTP error 400 as a response from the Here server.
As exemplary data I used the following file:
<?xml version="1.0"?>
<gpx version="1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.topografix.com/GPX/1/0"
xsi:schemaLocation="http://www.topografix.com/GPX/1/0
http://www.topografix.com/GPX/1/0/gpx.xsd">
<trk>
<trkseg>
<trkpt lat="51.10177" lon="0.39349"/>
<trkpt lat="51.10181" lon="0.39335"/>
<trkpt lat="51.10255" lon="0.39366"/>
<trkpt lat="51.10398" lon="0.39466"/>
<trkpt lat="51.10501" lon="0.39533"/>
</trkseg>
</trk>
</gpx>
that I got from the this example.
I encoded this file using MATLAB's function matlab.net.base64encode which yielded the following base64-encoded string:
PD94bWwgdmVyc2lvbj0iMS4wIj8+PGdweCB2ZXJzaW9uPSIxLjAieG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8y
MDAxL1hNTFNjaGVtYS1pbnN0YW5jZSJ4bWxucz0iaHR0cDovL3d3dy50b3BvZ3JhZml4LmNvbS9HUFgvMS8wInhzaTpz
Y2hlbWFMb2NhdGlvbj0iaHR0cDovL3d3dy50b3BvZ3JhZml4LmNvbS9HUFgvMS8wIGh0dHA6Ly93d3cudG9wb2dyYWZp
eC5jb20vR1BYLzEvMC9ncHgueHNkIj48dHJrPjx0cmtzZWc+PHRya3B0IGxhdD0iNTEuMTAxNzciIGxvbj0iMC4zOTM0
OSIvPjx0cmtwdCBsYXQ9IjUxLjEwMTgxIiBsb249IjAuMzkzMzUiLz48dHJrcHQgbGF0PSI1MS4xMDI1NSIgbG9uPSIw
LjM5MzY2Ii8+PHRya3B0IGxhdD0iNTEuMTAzOTgiIGxvbj0iMC4zOTQ2NiIvPjx0cmtwdCBsYXQ9IjUxLjEwNTAxIiBs
b249IjAuMzk1MzMiLz48L3Rya3NlZz48L3Ryaz48L2dweD4=
However, as stated before, the HERE server consistently responds with HTTP-error 400 to my request
https://rme.api.here.com/2/matchroute.json?app_id={app_id}&app_code={app_code}&routemode=car&file=...
where "..." equals the above mentioned base64-encoded string.
Question: Could anyone please provide a code sample showing how to encode the above mentioned GPX file correctly (ideally in MATLAB language) so that the /matchroute resource is able to respond?
Remarks:
If I use the base64 string
UEsDBBQAAAAIANmztEQSwaeZzwAAAM8BAAAQAAAAc2FtcGxlLXRyYWNlLmdweIXPTQuCMBwG8HufQnZv%2F605S0k9dj
EIungdZjpSJ27kPn6%2BRBgYXcYYv2cPzzG2deU8805L1YSIYoLiaHMsWvv9uBlYowOrZYhKY9oAoO973DOsugJ2hFBI
z8k1K%2FNabGWjjWiy%2FJ36ShjVqqITd2lxpmo4XVKgMP6vZaCneKIyYabivzHnr4BhCbb6hoZRpnvMp86L%2BdIapx
ImRJxiSuh%2Bj5xq7CWY%2Bcz1EaypA10qxlfVjvOl8rxVxfzDQrk%2FFCfLRs7YpOCzA%2BZd49LoBVBLAQIUABQAAA
AIANmztEQSwaeZzwAAAM8BAAAQAAAAAAAAAAEAIAAAAAAAAABzYW1wbGUtdHJhY2UuZ3B4UEsFBgAAAAABAAEAPgAAAP
0AAAAAAA%3D%3D
from this example the GET request works. However, I couldn't figure out how to reproduce this encoding myself so that I am able to encode my own data accordingly.
Link to the Here API definition: https://developer.here.com/documentation/route-match/topics/resource-matchroute-request.html
Looking at the two base64 strings I can tell you the fundamental difference between them - the first one (which doesn't work) is unescaped whereas the second one (which works) is.
You can convert between the two formats manually using various online tools like this one. The escaped version of the non-working base64 string, in case you want to test it, is:
PD94bWwgdmVyc2lvbj0iMS4wIj8+PGdweCB2ZXJzaW9uPSIxLjAieG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8y
%0AMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSJ4bWxucz0iaHR0cDovL3d3dy50b3BvZ3JhZml4LmNvbS9HUFgvMS8wInhza
Tpz%0AY2hlbWFMb2NhdGlvbj0iaHR0cDovL3d3dy50b3BvZ3JhZml4LmNvbS9HUFgvMS8wIGh0dHA6Ly93d3cudG9wb2
dyYWZp%0AeC5jb20vR1BYLzEvMC9ncHgueHNkIj48dHJrPjx0cmtzZWc+PHRya3B0IGxhdD0iNTEuMTAxNzciIGxvbj0
iMC4zOTM0%0AOSIvPjx0cmtwdCBsYXQ9IjUxLjEwMTgxIiBsb249IjAuMzkzMzUiLz48dHJrcHQgbGF0PSI1MS4xMDI1
NSIgbG9uPSIw%0ALjM5MzY2Ii8+PHRya3B0IGxhdD0iNTEuMTAzOTgiIGxvbj0iMC4zOTQ2NiIvPjx0cmtwdCBsYXQ9I
jUxLjEwNTAxIiBs%0Ab249IjAuMzk1MzMiLz48L3Rya3NlZz48L3Ryaz48L2dweD4%3D
I'm not an expert on this, but as I understand, you need to URL-encode strings only when you want to paste them as-is into the web path of your browser (read about "URL Params"). If you construct your HTTP requests the right way™ (by this I mean specify the headers of the request and the key-value pairs correctly), you shouldn't have to worry about URL-encoding at all, since the tool that you're using (in this case, MATLAB) should take care of the conversion for you.
Unfortunately, I cannot test this theory, as I have no access to the discussed API - but I am fairly certain that this would solve your problem.
I had the exact same problem.
The documentation seems to be incomplete. You can check here for additional information. Several ways I solved this:
Use filetype='CSV' or filtetype='GPX' in parameter. It says the filetype is guessed if passed, that is actually not true. After passing an XML file the API told me my file didn't look like a 'CSV'
Compression is OPTIONAL, I suggest to avoid it completely I could not find a suitable compression either. It works fine with plain base64 encoding.
I suggest to actually use CSV because the XML actually returns parsing errors.
In python
data='''latitude,longitude
51.10177,0.39349
'''
r = requests.get('https://rme.api.here.com/2/matchroute.json?app_id={APP_ID}&app_code={APP_CODE}&routemode=car&file={file}&filetype={filetype}'.format(
APP_ID=os.getenv('HERE_APP_ID'),
APP_CODE=os.getenv('HERE_APP_CODE'),
filetype='CSV',
file=base64.b64encode(data.encode()).decode()
))
I need to access the Date: header when I handle the request, but this seems to be "swallowed" by the framework; any other header (even made up FooBar ones) show up and I can get them, but this gives me None (I'm using Postman to send a simple GET request - everything else works just fine):
println("Date: " + request.headers.get("Date").getOrElse("no date!"))
returns "no date!" no matter how I try to send something sensible.
I'm wondering whether this gets processed before the request object reaches my Action.
I need the actual string value sent, as this should be part of the request's signature - so an equivalent Date object representing the same value would not be of much use (as it needs to be part of the hash, to avoid replay attacks).
Just as a test, I replaced the Date header with a Date-Auth one, and this one shows up just fine:
ArrayBuffer((Date-Auth, ArrayBuffer(Wed, 15 Nov 2014 06:25:24 GMT))
Any ideas or suggestions greatly appreciated!
Are you sure there is a Date Header in your request (tested with tools like firebug or wireshark)?
Browsers do not need to send a Date header.
RFC 2616 (HTTP 1.1) from the Date section (14.18)
Clients SHOULD only send a Date header field in messages that include an entity-body, as in the case of the PUT and POST requests, and even then it is optional. A client without a clock MUST NOT send a Date header field in a request.
I stand corrected - it turns out that Chrome blocks a whole bunch of headers:
http://www.getpostman.com/docs/requests
I wrote a Python Flask test server and, in fact, the Date header is not there.
That page has also a fix, which works just fine with Postman Version 0.10.4.3 and Interceptor(1).
sorry for wasting everyone's time!
1 Incidentally, IMO Postman is the best REST client and has now also some awesome looks, beyond incredible functionality. If you're working with REST APIs, I highly recommend it.
I need to update an order which is done via PUT method passing the order id as part of the https url string and a single parameter, the status_id.
https://mystore.mybigcommerce.com/orders/12345.json
I have tried several methods to pass the status_id value but no matter what I try "status_id=12" or formatted as JSON "{"status_id": 12,}" I always get the same response:
[{"status":415,"message":"The specified input content type is not valid."}]
I have also tried as a POST request passing the JSON or XML code as raw data but that method is not supported.
How am I supposed to pass that field=value pair? can I embed it in the url string?
I also tried it but it wouldn't work for me.
Any ideas?
In case you are wondering I am doing it within FileMaker with TROIUrl plugIn, not a very popular technology, but the GET method retrieving orders works like a charm
TURL_Put( ""; $url ;"status_id=12") (I have also tried other FM plugIns to no avail)
Don't get too caught up in the Filemaker part, I don't expect many people out there to be familiar with BigCommerce and Filemaker. I just need a generic answer.
Thanks
Commandline tool curl is worth a try. It supports put and https.
Mac OS X: curl already installed, call from FileMaker via AppleScript do shell script.
Windows: must be installed, call via Powershell.
It works for me using { "status_id": "3" } which means you probably need to put quotes around the actual number.
Also, it is a PUT operation and application/json which is part of the request content.
The error message received by the OP:
[{"status":415,"message":"The specified input content type is not valid."}]
Is saying that he did not supply the 'Content-Type' header in his request or that the header supplied is for a content type that is not allowed. For the OP's case using JSON he would need to include the header:
Content-Type: application/json
in his HTTPS request. This description can be found along with those of the other status codes you may see here:
https://developer.bigcommerce.com/api/status-codes
I'm pretty new to Lift, and one of the things I've been trying to find is how to, in the context of a snippet, find the '#' in the current page's URL. So if a user visits http://www.example.com/some/path/page#stuff then I would like to extract "stuff" from that. I've been googling and searching the API docs and have yet to find anything for this.
I don't think the part behind the # ever gets sent to the server in the first place.
That's what wikipedia has to say about it:
In URIs a hashmark # introduces the
optional fragment near the end of the
URL. The generic RFC 3986 syntax for
URIs also allows an optional query
part introduced by a question mark ?.
In URIs with a query and a fragment
the fragment follows the query. Query
parts depend on the URI scheme and are
evaluated by the server — e.g., http:
supports queries unlike ftp:.
Fragments depend on the document MIME
type and are evaluated by the client
(Web-browser). Clients are not
supposed to send URI-fragments to
servers when they retrieve a document,
and without help from a local
application (see below) fragments do
not participate in HTTP redirections.
I don't think the part behind the #
ever gets sent to the server in the
first place.
You are correct, sir. That is the entire point of the hash.
Dylan, you could do something from the Javascript side:
$.ajax( { data : { fragment : window.location.hash ...
I'm using Jon Crosby's open source Objective-C OAuth library http://code.google.com/p/oauthconsumer/ for some basic http authentication that does not deal with tokens, only consumer key and consumer secret. My code works great for GET, GET with parameters in the URL, and POST. When I issue a POST request that has parameters in the URL, though, the request fails authorization. I'm trying to figure out why.
The server is using Apache Commons OAuth, so I'd like to compare my base string with that library. Here's a contrived example and the base string and signature produced by my library. Can anyone see what the problem is?
consumer key: abcdef
consumer secret: ghijkl
POST request: http://emptyrandomhost.com/a/uriwith/params?interesting=foo&prolific=bar
my base string: POST&http%3A%2F%2Femptyrandomhost.com%2Fa%2Furiwith%2Fparams&interesting%3Dfoo%26oauth_consumer_key%3Dabcdef%26oauth_nonce%3D1%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D2%26oauth_version%3D1.0%26prolific%3Dbar
This data produces the following OAuth header authorization:
Authorization: OAuth oauth_consumer_key="abcdef",
oauth_version="1.0",
oauth_signature_method="HMAC-SHA1",
oauth_timestamp="2",
oauth_nonce="1",
oauth_signature="Z0PVIz5Lo4eB7aZFT8FE3%2FFlbz0%3D"
And apparently my signature is wrong. The problem has to either be in the construction of the base string, in the way that the HMAC-SHA1 function is implemented (using Apple's CCHmac from CommonHMAC.h, so hopefully this isn't it), or with my Base64Transcoder, which is open source c. 2003 by Jonathan Wight/Toxic Software. I primarily suspect the base string, since the requests work for GET and POST and only fail with POST with URL parameters as above.
Can someone with lots of OAuth experience spot the problem above? Something else that would be very useful is the base string that is produced by Apache Commons OAuth in their authentication. Thanks.
As per RFC 5849 section 3.4.1.2, the OAuth base string URI does not include the query string or fragment. If either the client or the server does not remove the query parameters from the base string URI and add them to the normalized OAuth parameter list, the signatures won't match. Unfortunately, it's hard to tell which side is making this mistake. But it's easy to determine this is the problem: If it always works without query parameters but always fails with query parameters, you can be pretty sure that one side or the other is generating the wrong base string. (Be sure that it always happens though... intermittent errors would be something else. Similarly, if it never works with or without a query string, that would also be something else.) The other possibility is that normalization was done incorrectly — the parameter list must be sorted and percent encoded sequences must be upper-cased. If it's not normalized correctly on both sides, that will also cause a base string mismatch, and thus a signature mismatch.
you can build and check visually your request at this URL:
http://hueniverse.com/2008/10/beginners-guide-to-oauth-part-iv-signing-requests/
Open the boxes denoted by [+] signs and fill in your values, that way you may be able to see if the problem is at your code, or at the provider side.