just start to use fiddler in my project for debug purpose, but haven't figure out how to handle following case with autoresponder.
I want to extract the timestamp in url and respond to the timestamp value itself.
if url is
https://example.com/20220816/hello/
I would like to respond by text as follows.
20220816
I wrote the rule as follows, but it didn't work. The fiddler recognizes $1 as a file path.
regex:(?insx)^https://example\.com/(.*)/hello/
$1
I don't think there's a way to respond to text other than url right away in the fiddler autoresponder, so can't you solve it?
Related
I have a CSV of ~2000 URLs that, when queried, do a 301 or 302 redirect, and I'm trying to figure out if OpenRefine is able to export to a new column the destination url that it retrieves HTML from when I fetch the html from it (or some other way).
e.g.
https://www-istp.gsfc.nasa.gov/stargaze/Ssolsys.htm
redirects to
https://pwg.gsfc.nasa.gov/stargaze/Ssolsys.htm
And I know that from clicking the link in my browser of choice. I've found a few answers suggesting that this can be done in various coding languages, but nothing so far suggesting how to do so in OpenRefine, even though I'm like 80% sure that it can be.
Does anyone out there know what I might be able to do to make this happen?
In OpenRefine you can write expressions in GREL, Jython (Java Implementation of Python 2) and Clojure.
As far as I know GREL does not support analyzing the target of a redirection URL, so I would use Python for that.
In your OpenRefine Project go to your column containing the urls and use "Edit column" > "Add column based on this column..."
In the corresponding dialog window (see Screenshot below) you change the expression language to "Python / Jython" and use the following code snippet to retrieve the "real" URL of the request.
import urllib2
response = urllib2.urlopen(value)
return response.geturl()
I am trying to communicate with kdb+ via HTTP. I succeeded to get the the response from the DB when I did it from the browser, but for some reason, the response is neither JSON nor anything that looks machine readable.
What I did:
Opened a port on q console (8080)
Defined a function getData that gives me the data I want.
The above steps can be presented in this .q file I created:
\p 8080
system["l db"]
getData: {[a;b] ?[table;((>=;`start;$[`long;a]);(<=;`end;$[`long;b]));0b;()]}
h:hopen `:localhost:8080:user:pass
I then typed http://localhost:8080/?getData[1;2] in the browser to get the results
Got the results as text with spaces inside <pre> tag.
The Problem
I have no I idea how to parse it to JSON. How can I solve this? is there a way to tell kdb+ to send response in JSON format?
The solution I found:
If you add .json before the ? in the url (i.e. http://localhost:8080/.json?getData[1;2]) the response will be in JSON.
It looks like, based on your code, the following may work to return data:
http://localhost:8080/?getData[a;b]
Where a and b are start and end times (or dates, this will depend on your data).
e.g.
http://localhost:8080/?getData[2021.07.23;2021.07.30]
In order to convert a KDB+ object to json you can make use of the .j.j operator.
For example you can change your code to something like:
getData:{[a;b] t:?[table;((>=;`start;$[`long;a]);(<=;`end;$[`long;b]));0b;()];.j.j t}
If you want to see what the incoming request looks like on your Q process you could add some logging to .z.ph (the default http message handler), e.g. something like:
.z.ph:{[zph;x]zph 0N!x}[.z.ph]
This will log any incoming requests out to your q process using 0N! and then run the usual .z.ph message handing code on it
First off thanks for reading!
Second off YES I have tried to find the answer! :) Perhaps I haven't found it because I'm not using the right words to describe my problem, but it's been about 4 hours that I've been trying to figure it out now and I'm getting a little loopy trying to piece it together on my own.
I am very new to programming. Python is my first language. I am on my third Python course. I have an assignment to use the socket library (not urllib library - I know how to do that) to make a socket and use GET to receive information. The problem is that the program needs to take raw input for the URL in question.
I have everything else the way I want it, but I need to know the syntax that I'm supposed to be using INSIDE my "GET" request in order for the HTTP message to include the requested document path.
I have tried (obviously not all together lol):
mysock.send('GET (url) HTTP/1.0\n\n')
mysock.send( ('GET (url) HTTP:/1.0\n\n'))
mysock.send(('GET (url) HTTP:/1.0\n\n'))
mysock.send("GET (url) HTTP/1.0\n\n")
mysock.send( ("'GET' (url) HTTP:/1.0\n\n"))
mysock.send(("'GET' (url) 'HTTP:/1.0\n\n'"))
and:
basically every other configuration of the above (, ((, ( (, ', '' combinations listed above.
I have also tried:
-Creating a string using the 'url' variable first, and then including it inside mysock.send(string)
-Again with the "string-first" theory, but this time I used %r to refer to my user input (so 'GET %r HTTP/1.0\n\n' % url basically)
I've read questions here, other programming websites, the whole chapter in the book and the whole lectures/notes online, I've read articles on the socket library and the .send(), and of course articles on GET requests... but I'm clearly missing something. It seems most don't use socket library when they can use urllib and I don't blame them!!
Thank you again...
Someone from the university posted back to me that the url variable can concatenated with the GET syntax and assigned to a string variable which can then be called with .send(concatenatedvariable) - I had mentioned trying that but had missed that GET requires a space after the word 'GET' so of course concatenating didn't include a space and that blew it. In case anyone else wants to know :)
FYI: A fully quallified URL is only allowed in HTTP/1.1 requests. It is not the norm, though, as HTTP/1.1 requires setting the Host header. The relevant piece of reading would've been RFC 7230, sec. 3.1.1 and possibly RFC 3986. The syntax of the parameters is largely borrowed from the CGI format. It is in no way enforced, however. In a nutshell, everything put together would look like this on the wire:
GET /path?param1=value1¶m2=value2 HTTP/1.1
Host: example.com
As a final note: The line delimiter in HTTP is CRLF (\r\n). For robustness, a simple linefeed is acceptable as well but not recommended.
I'm trying to make an auto response for a website, let me take an exemple
Website : http://mywebsite.com/1.0/?appID=blah_blah&appVersion=RandomNumber&getAppicationSettings=blah_blah
I'd like that Fiddler match the autoresponse after, for exemple, "appVersion=" I've tried to put "REGEX:http://mywebsite.com/1.0/?appID=blah_blah&appVersion=.*" but it doesn't work.
I don't know if you know what I mean, I have trouble to explain it. :(
Thanks for the reply.
The simplest thing would be to just change the rule to
http://mywebsite.com/1.0/?appID=blah_blah&appVersion=
If you need to use a regular expression, the expression needs to be valid, with escaping of characters that have a meaning in regular expressions. E.g. more like:
REGEX:mywebsite\.com\/1\.0\/\?appID=blah_blah&appVersion=[\d]+&getAppicationSettings=blah_blah
I need to update an order which is done via PUT method passing the order id as part of the https url string and a single parameter, the status_id.
https://mystore.mybigcommerce.com/orders/12345.json
I have tried several methods to pass the status_id value but no matter what I try "status_id=12" or formatted as JSON "{"status_id": 12,}" I always get the same response:
[{"status":415,"message":"The specified input content type is not valid."}]
I have also tried as a POST request passing the JSON or XML code as raw data but that method is not supported.
How am I supposed to pass that field=value pair? can I embed it in the url string?
I also tried it but it wouldn't work for me.
Any ideas?
In case you are wondering I am doing it within FileMaker with TROIUrl plugIn, not a very popular technology, but the GET method retrieving orders works like a charm
TURL_Put( ""; $url ;"status_id=12") (I have also tried other FM plugIns to no avail)
Don't get too caught up in the Filemaker part, I don't expect many people out there to be familiar with BigCommerce and Filemaker. I just need a generic answer.
Thanks
Commandline tool curl is worth a try. It supports put and https.
Mac OS X: curl already installed, call from FileMaker via AppleScript do shell script.
Windows: must be installed, call via Powershell.
It works for me using { "status_id": "3" } which means you probably need to put quotes around the actual number.
Also, it is a PUT operation and application/json which is part of the request content.
The error message received by the OP:
[{"status":415,"message":"The specified input content type is not valid."}]
Is saying that he did not supply the 'Content-Type' header in his request or that the header supplied is for a content type that is not allowed. For the OP's case using JSON he would need to include the header:
Content-Type: application/json
in his HTTPS request. This description can be found along with those of the other status codes you may see here:
https://developer.bigcommerce.com/api/status-codes