I'm a newbie in Flutter who don't know a lot about HMAC code. I want to get datas from an online opendata interface, and I wonder how to use Flutter's functions to encode the parameters and complete my authorization, such as my key, my ID, and x-date. Showed below is the example format provided by the opendata interface website. Would somebody teach me how to encode those parameters in the form of it? Grateful for everyone who spends time trying to answer my question!!
Key Value
Authorization hmac username="APP ID", algorithm="hmac-sha1", headers="x-date", signature="Base64(HMAC-SHA1("x-date: " + x-date , APP Key))"
x-date
Wed, 19 Apr 2017 08:37:50 GMT
enter image description here
Related
I was quite frustrated with IPN testing. Although in the end I was able to pin point the issue in Validate step timestamp field, I need help with how to handle the + sign in time stamp.
I noticed when I decode and encode, the space from Paypal request became + sign. So I did a replacement of + with %20. This was tested okay. However it would be an issue if there is timezone info inside the payment date.
E.g. Fri Jul 08 2016 10:22:01 GMT+0800 (Malay Peninsula Standard Time)
parameter came in as:
Fri%20Jul%2008%202016%2010%3A22%3A01%20GMT+0800%20%28Malay%20Peninsula%20Standard%20Time%29
after decoding:
Fri Jul 08 2016 10:22:01 GMT 0800 (Malay Peninsula Standard Time) <=====the plus sign is missing.....
encode again:
Fri+Jul+08+2016+10%3A22%3A01+GMT+0800+%28Malay+Peninsula+Standard+Time%29
What I did was: replay the + sign before decoding with some temp placeholder. then once decode / encode, revert back the replacement.
Some how this could not be verified by Paypal.
Okay, I got it working....not sure whether it's the best way, but works now.
basically I patch the incoming parameter value by replacing + with a placeholder.
patchedValue = value.replace("+", "TEMPXXX");
....
In the end, after encoding, replace the placeholder with %2B, which is + sign.
....
URLEncoder.encode(decodedValue, encoding).replace("+", "%20").replace("TEMPXXX", "%2B")
Is it possible to use Kdb+ http client to access pages protected by login? I am using https://github.com/KxSystems/cookbook/blob/master/yahoo.q as example of basic GET/POST. Does anyone have an example how to extract a cookie and use it in the following requests?
It is probably a bit crude, but the following will extract headers from an http, then cookies, parse and return as a dictionary:
x:"HTTP/1.0 200 OK\r\nContent-type: text/html\r\nSet-Cookie: theme=light\r\nSet-Cookie: sessionToken=abc123; Expires=Wed, 09 Jun 2021 10:18:14 GMT\r\n\r\n";
left:{(first y ss x)#y};
vs1:{{(y#x;(count[z]+y)_x)}[y;;x](first y ss x)};
headers:{{(`$x[0];x[1])} flip vs1[": "] each 1_"\r\n" vs left["\r\n\r\n"]x};
cookies:{(!). {(`$x[0];x[1])} flip vs1["="] each {x[1]#where x[0]=`$"Set-Cookie"} x};
cookies headers[x]
Whilst you might be able to various bits and bobs from an http response, the fact that you won't be able to manipulate http methods means that q can't be your tool to do this - well, not without some vigorous effort.
I would use something like Beautiful Soup in conjunction with q. Soup has some great tools for handling this kind of thing (e.g. cookies etc). There are various other similar projects too.
System call for Beautiful Soup that make relevant get/post/put calls and download required data
system"/path/to/code.py"
Where the code dumps the result somewhere or puts it into kdb directly. Then do whatever you like with it.
I have noticed the various email clients prepend/append text to the text written by the user. For example, Gmail seems to prepend the following text to all email bodies:
"On Tue, Jul 14, 2015 at 11:41 AM, Jonny Bravo wrote: >"
The added text differs based on the client. I am not interested in this information. I would like to be able to extract the message body from the text with an approach that is relatively cross-platform. Does anything like this exist? Is the best solution to clean the text on a case-by-case basis?
We had the same problem at mailparser.io when we developed our "last reply" filter. We get very decent results by just checking against a set of regular expression.
The regular expressions we use are:
'/^(--)$/ms', // -- Signature break
'/^(-----(.+))$/ms', // ----- reply above
'/^(From:(.+))$/ms', // From:
'/^(On\s(.+)wrote:)$/ms', // On DATE, NAME <EMAIL> wrote:
'/^(Sent from(.+))$/ms', // Sent from (iPhone / iPad / Windows Mail ...)
With those you should actually catch most cases produced by e-mail clients which have their language set to English.
Somebody in my company is being subject to phishing. My first suggestion was just to change the password. However after awhile I received a fake mail from her address again.
Looking at the raw source of the email I found that there is another person's email in X-Sender-ID and I'm wondering who that might be. Is that the person who sent the email or can it be an account that has been hijacked? (I replaced the email with "somebody#host.com")
X-Virus-Scanned: OK
Received: by smtp5.relay.iad3a.emailsrvr.com (Authenticated sender: somebody-AT-host.com) with ESMTPA id DF2788019C;
Fri, 21 Nov 2014 07:54:42 -0500 (EST)
X-Sender-Id: somebody#host.com
Received: from smtp.emailsrvr.com ([UNAVAILABLE]. [2.133.148.211])
by 0.0.0.0:587 (trex/5.3.2);
Fri, 21 Nov 2014 12:54:46 GMT
What is X-Sender-ID? And what is the email it contains?
My deliberations are based on this RFC which describes the Privacy Enhancement for Emails which you are obviously using.
Basically it says about the X-Sender-ID:
[...] encapsulated header field, required for all
privacy-enhanced messages, identifies a message's sender and provides
the sender's IK identification component.
What does this mean?
First of all you have to check if the mail is properly signed. If thats the case you can be sure that somebody#host.com has a certificate. And you can be sure that the mail you received has been sent from this mail address.
I can't tell you the consequences which result out of this fact as I don't know how your company is deploying the certificates etc. ... the mail address/certificate could also have been hacked and thereby abused.
I hope this helps you for your further research.
While #LMF's answer is useful technical information, I'd like to offer a possible alternative explanation.
Spammers who are not familiar with e-mail (and PHP programmers with no other malicious intent) tend to succumb to cargo cult programming when it comes to email headers. In other words, if there is something they don't understand, they might think it does something useful, and include it in their message template.
Without knowledge about your email infrastructure, or other messages of yours to compare to, I would simply assume everything below the top-most Received: header is forged, and basically without meaning.
If you have a system which runs something called trex (maybe this one?) and it really manages to write a Received: header like that, I might be wrong. The format needlessly deviates from the de-facto standard Sendmail template in a few places, but it's not technically wrong (the format is basically free-form, but introducing ad-hoc syntax makes it harder to guess what the fields mean).
Again, more information about what your typical email (and your correspondent's typical mail) looks like, this is heavy on speculation.
The x-sender-id, along with the x-recipient-id are used to specify which interchange key was used in the broadcast of the message.
X-Sender-ID entity_id : issuing_authority : version
X-Recipient-ID entity_id : issuing_authority : version
The first field contains the identity of the sender or receiver. The first field is mandatory, must be unique, and must be formatted as user#host whereas the host is a fully qualified host address.
The second identifies the name of the authority which issued the interchange key.
The third field specifies the specific type of interchange key which was used. This is represented by an alphanumeric string defined by the issuing authority to label and organize the numerous interchange keys issued by that authority. It is recommmended that they use a timestamp but is not always the case.
If the field values of the x-sender-id second and third field are identical to that of the x-recipient-id they may be only listed in the field which is defined last.
Further Reading
"Distributed Computing & Cryptography: Proceedings of a DIMACS Workshop"
I am wondering how to determine the age of a web site (not the age of the host / domain registration) in a robust and universal way.
Take this site as an example:
Most of the times, the age / date (December 21, 2011, in this case) appears on the site, but AFAIK there is no universal way of getting this information from the page (could be on the page, in the META-tag, header...)
If you google the headline, Google will show the age (first result; gray; so Google extracted this information somehow):
http://i.stack.imgur.com/BcXwo.png [I don't have privileges to embedd this as an image]
Alongside, there are other sites with the same news (i guess it's from a press agency) and Google shows the age for those as well, but not for the last one, despite its occurrence in the text (First line; Wednesday, December 21, 2011).
Q1) How to determine the age in a universal way?
Q2) How does Google do it? Is it just the time the URL showed up in the index? Why isn't there a date then for the last result?
Q3) If there is no other way then actually getting it from Google, how can that be done automatically for a couple of domains? After a number of automated requests, Google will block / prevent you from sending more requests. I had a look in the Google Custom Search API, but the data does not show up in the results there.
Thanks!
If the server supports it you can use the Last-Modified Header part of the HTTP-Request.
try: curl -I http://online.wsj.com/article/SB10001424052970204058404577110380555673036.html
to get only the HTTP-Header of the Reply and have a look at the output
HTTP/1.1 200 OK
Date: Wed, 09 May 2012 12:40:10 GMT
Server: Apache/2.2.15 (CentOS)
...
FastDynaPage-ServerInfo: secj2kentwap07 - Wed 05/09/12 - 08:40:10 EDT
Last-Modified: Wed, 09 May 2012 12:40:10 GMT
Content-Type: text/html; charset=UTF-8
Actually I haven't found a proper way to get the date from the URL. So I took another approach: I try to find a feed (either from the site itself or through Google) that contains that URL as an item.
Then there is a good chance that I'll either get a pubDate or dc:date which contains the date of publication. This is then usable.
Thanks for all the input.