We're building a web app which contains a form of what documents to request from a client. So
A user ticks for example driver license, passport and application letter on that form and click "Send document request".
The application sends an email to the client which says "Please send us your 1) driver license 2) passport 3) application letter". It sends the email using it's own robot account like "noreply#ourcompany.com"
The client gets the email asking him to reply back with requested documents. He replies on that email with text like "Ok, great, thanks! I have attached my driver license, passport and letter to this letter" and attaches the files.
The user from the first step should get the letter client sends on the third step.
How can I achieve the behavior on the fourth step? I guess there is something like 'Reply-To' header I should set to the real user's email.
I've created an example for you to confirm the logic. Please add feedback to comments and I'll edit accordingly. The idea here is that with your feedback, we can create a Narrative of emails to form a complete and successful conversation.
Does the following logic matches your intent?
An email is ACCEPTED via HTML form FROM "User1" <user1#example.com>
This email is ORIGINALLY SENT FROM "No-Reply" <noreply#ourcompany.com>
This email is SENT TO "User2" <user2#example.com>
ANY replies to This email ARE in REPLY-TO "User1" <user1#example.com>
...is this correct?
If so, here's a working example of a text/plain email including headers + content:
From: "No-Reply" <noreply#ourcompany.com>
To: "User2" <user2#example.com>
Reply-To: "User1" <user1#example.com>
Subject: Document Request
MIME-Version: 1.0 (Created with SublimeText 3)
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Good morning, User2.
Please send us your 1) driver license 2) passport 3) application letter.
Thank you,
Ourcompany Inc.
Related
If you google "How to check an email address for existence" question, you will find, basically, only solutions using SMTP protocol what is not reliable. I tried this approach and found that Gmail SMTP server says "Yes, this email is registered here" on each and every email address I ask about. I suspect such strategy is used on the majority of popular email servers.
The method I would like to share is used in Gmail registration form to ensure you are going to register a brand new email. It uses AJAX request to ask Gmail server if given email exists or not
Request URL:https://accounts.google.com/InputValidator?resource=SignUp
Request Method:POST
Status Code:200
Remote Address:173.194.222.84:443
Response Headers
alt-svc:quic=":443"; ma=2592000; v="37,36,35"
cache-control:private, max-age=0
content-encoding:gzip
content-type:application/json; charset=utf-8
date:Wed, 29 Mar 2017 21:06:06 GMT
expires:Wed, 29 Mar 2017 21:06:06 GMT
server:GSE
set-cookie:GAPS=1:<redacted>;Path=/;Expires=Fri, 29-Mar-2019 21:06:06 GMT;Secure;HttpOnly;Priority=HIGH
status:200
strict-transport-security:max-age=10893354; includeSubDomains
x-content-type-options:nosniff
x-frame-options:DENY
x-xss-protection:1; mode=block
Request Headers
Provisional headers are shown
Content-type:application/json
Origin:https://accounts.google.com
Referer:https://accounts.google.com/SignUp?hl=en-GB
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
Query String Parameters
resource=SignUp
Request Payload
{"input01":{"Input":"GmailAddress","GmailAddress":"andy.v.che","FirstName":"","LastName":""},"Locale":"en-GB"}
Response
{"input01":{"Valid":"false","ErrorMessage":"Someone already has that username. Note that we ignore full stops and capitalisation in usernames. Try another?","Errors":{"GmailAddress":"Someone already has that username. Note that we ignore full stops and capitalisation in usernames. Try another?"},"ErrorData":["andyvche959"]},"Locale":"en_GB"}
As you can see, there is "Valid":"false" in the response if such an email does exist, and (spoilers) "Valid":"true" if it doesn't.
Throttling queries down
Guys from Gmail do understand this method could be used by spammers to look for existing emails. That's why they don't allow massive scans using it. I was doing such a scan for some time and could scan only 200 emails a day approximately.
More details
I was scanning 1 email a minute, and if I was getting response "No, this email doesn't exist", I also asked if my own email exists. If I got "No, your email doesn't exist as well" answer, I could clearly understand that I got ban from Gmail server by my IP address. Then, I took a break for 45 minutes to get unbanned, then continued the loop. The number af emails scanned a day was fluctuating around 200.
You may ask: you did a scan like a spammer would perform, for what purpose did you do that scan then?
My answer is: I was trying to find a guy who wrote his email unclearly (bad cursive). There was no other option to find him.
There were 3 unclear letters in his written email but it was clear the domain of it is gmail.com, so I came up with an idea to find a way to check an email address for existence on Gmail, generate a list of all possible emails (trying to substitute unknown symbols with all possible English letters) and check them all for existence. Then, send a letter to all existing ones.
The right of this information to be published is discussed in this question. I understand this article will be very useful for spammers so I'm open to deleting it partially or even completely for the sake of security.
I tried to install ReadTheDocs on my linux server following the documentation: http://read-the-docs.readthedocs.io/en/latest/install.html
However I do not receive the activation email.
I was able to see the output of the email on the server
Hello from example.com!
You're receiving this e-mail because you or someone else has requested a password for your user account at example.com.
It can be safely ignored if you did not request a password reset. Click the link below to reset your password.
http://xxxxx:8080/accounts/password/reset/key/xxxxxxxxxx/
In case you forgot, your username is xxxxxxxxxxxx.
Thank you for using example.com!
example.com
note: This email is a password reset email, it has the same problem as the activation email. (I didn't receive any email from my ReadTheDocs installation)
There are two things that got my attention.
1. Where does read the docs configure the website name, I would like to change example.com to th real website name.
2. The sending adres currently is no-reply#readthedocs.org, where can I change it? I think its also a part of the issue.
Here are the other headers of the email
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Subject: [example.com] Password Reset E-mail
From: no-reply#readthedocs.org
To: xxx#xxxxx.nl
Date: Tue, 12 Jul 2016 11:30:49 -0000
So actually I have two questions
Where can I change the content of the email and things like the sender adres.
What happend with the email (why won't I receive it)
Struggling to find any useful information on this, so I'm hoping some light may be shed here.
I received an email to which I don't appear to have been addressed.
It's a legitimate email to our organisation, however there appears to be no reason for it to be delivered to me. Digging through the headers DOES reveal my email address, in a header field x-newtrunsreceiver.
ie.
x-sender: sender#sendersemail
x-newtrunsreceiver: legitimaterecipient1#ourorg
x-newtrunsreceiver: myemail#ourorg
x-newtrunsreceiver: legitimaterecipient2#ourorg
The recipients legitimaterecipient1 and legitimaterecipient2 are in the To: and Cc: fields respectively.
I am kind of assuming I've been BCc'd, but can't find any info on what the x-newtrunsreceiver header field is / does.
Any help welcomed.
MIME headers have absolutely nothing to do with who actually receives the message.
When a message is submitted to an SMTP server, the recipients are specified in the "RCPT TO" SMTP command. The MIME headers that follow the DATA command can contain anything. The SMTP server can in theory take a look at the To and CC headers, but there is absolutely no requirement or reason to do that.
Now the receiving SMTP server can add an extra header (e.g. Apparently-To), but, again, it has no obligation to do that.
im thinking what is the best RESTful way how confirm email and request reseting password. Im only aiming to find correct URI...
confirm email
PUT /users/{userId}/confirmEmail?code=xyz - does not seem much RESTful because of confirmEmail
PUT /users/{userId}/email?confirmedBy=xyz - maybe better? dunno
reset password (similar problem)
PUT /users/{userId}/resetPassword --DATA {email:xyz#xyz.xy} - same thinkin as before
PUT /users/{userId}/password --DATA {state:reseted,resent:xyz#xyz.xy} - hmmm... again Im not sure
are there any better ways in your mind?:-)
If you want your URIs to refer to resources, then call the resource confirmation and POST confirmations to user accounts.
POST /users/{userid}/confirmation
The true RESTful answer is the URL does not matter, you put it in the confirmation e-mail anyway for the recipient to follow. Use whatever is most convenient for your load balancer, reverse proxy, servers, etc.
For convenience you'll end up accepting the confirmation even if it comes in a GET request, because that's what the browsers of flesh-and-bones humans oblivious to Dr Roy T. Fielding et al. send when clicking on a link in an e-mail :-)
Having established it is completely academic, I'd argue you were right to think of PUT, as the client idempotently places evidence of having access to the e-mail. Repeating the request has no further effect.
Considering that he said a reset service for someone who forgot her password, and not a change password service for someone already logged in...
I would use 2 services. 1st to request the reset password mail, and 2nd to set the new password with the token received in the received mail.
For the 1st:
POST baseUrl/passwordReset
Request body
{
"email" : "my#self.com"
}
This could be POST or PUT, but since a mail delivery is not a resource subject to CRUD anyway, let's not be pedantic and use the old POST that was always used in html forms.
Obviously I would control that the same client (ip? browser? ...) doesn't make me send 20K mails in a minute.
Sending the mail to the user doesn't imply that the old password is not valid. That will only happen later in the second request when the new one updates it.
Response 204 (perhaps you should do it even if you don't know that email, because if you return error that means that when you don't return error you are confirming to a stranger that the given email is registered)
For the 2nd:
POST baseUrl/password
Request body
{
"token" : "3D21BA...4F",
"newPassword" : "m%4pW1!O"
}
Where the token is received in the mail. So the mail could have a link to a page including the token, when the page is loaded, the form is filled and submitted, being the token a hidden field that some javascript reads from the URL and puts here.
This is really a resource that you update, so POST. And I don't think it makes sense to have the same URI with 2 verbs for both, because they are not the same resource/entity at all.
Add
By the way, I would make both HTTPS only, and that's why I put all the sensitive information in the body, not URL parameters.
Firstly, I don't think that PUT is the right method for this. PUT broadly means "put this here", where the URL is identifying where the content should be located. You're really asking an existing resource to perform some action, which makes POST more correct.
To answer your direct question, a RESTful URL should identify the resource you want to handle your request. In this case, the resource is either the user, or some password-resetting resource within the user.
My preference would be for a password-resetting resource:
POST /users/{userid}/password-reset
This makes sense from a HTTP point of view, since you could issue a GET on the resource and receive something which indicates how to action a password reset (e.g. a HTML form prompting for the email address of the associated account).
EDIT:
For the purposes of email validation, there are two obvious options. You could either POST to a "confirm email" resource with the email address and confirmation data, to ask the server to process the confirmation, or you can execute a PUT to put the confirmation information on the server:
POST /users/{userid}/confirm-email
or
PUT /users/{userid}/email-confirmation
Here is a RESTful way.
Request
PUT /{userid}/email HTTP/1.1
Content-Type: text/json+confirmation-code
{"activateCode": "23sfgsg3twt3rgsdhgs"}
Response
HTTP/1.1 200 OK
Content-Type: text/json+email-status
{"email": "my-email#address.com", "active": "true"}
No verbs in the URI needed :)
I don't really see anything wrong with having confirmEmail like the 1st example. In the URL you have the key to the user, confirmEmail is the action, and the data with that action is in the query string.
I've recently worked on this, my take was
POST /{base_url}/password
because I was actually creating a new random password and sending it over to the user
and
PUT /{base_url}/confirmation?token=...
Because I am updating the confirmation that was already sent out when the user registered.
What's the difference between an email Sender, From and Return-Path value?
Example: I have a contact form where the user can input their email, would this be assigned to sender, from or return-path?
I had a quick search on the StackOverflow and couldn't find anything useful.
So, over SMTP when a message is submitted, the SMTP envelope (sender, recipients, etc.) is different from the actual data of the message.
The Sender header is used to identify in the message who submitted it. This is usually the same as the From header, which is who the message is from. However, it can differ in some cases where a mail agent is sending messages on behalf of someone else.
The Return-Path header is used to indicate to the recipient (or receiving MTA) where non-delivery receipts are to be sent.
For example, take a server that allows users to send mail from a web page. So, sender#yourcompany.com types in a message and submits it. The server then sends the message to its recipient with From set to sender#yourcompany.com. The actual SMTP submission uses different credentials, something like mailagent#mywebmail.com. So, the sender header is set to mailagent#mywebmail.com, to indicate the From header doesn't indicate who actually submitted the message.
In this case, if the message cannot be sent, it's probably better for the agent to receive the non-delivery report, and so Return-Path would also be set to mailagent#mywebmail.com so that any delivery reports go to it instead of the sender.
If you are doing just that, a form submission to send e-mail, then this is probably a direct parallel with how you'd set the headers.
The official RFC which defines this specification could be found here:
https://www.rfc-editor.org/rfc/rfc4021#section-2.1.2 (look at paragraph 2.1.2. and the following)
2.1.2. Header Field: From
Description:
Mailbox of message author
[...]
Related information:
Specifies the author(s) of the message; that is, the mailbox(es)
of the person(s) or system(s) responsible for the writing of the
message. Defined as standard by RFC 822.
2.1.3. Header Field: Sender
Description:
Mailbox of message sender
[...]
Related information:
Specifies the mailbox of the agent responsible for the actual
transmission of the message. Defined as standard by RFC 822.
2.1.22. Header Field: Return-Path
Description:
Message return path
[...]
Related information:
Return path for message response diagnostics. See also RFC 2821
[17]. Defined as standard by RFC 822.
A minor update to this: a sender should never set the Return-Path: header. There's no such thing as a Return-Path: header for a message in transit. That header is set by the MTA that makes final delivery, and is generally set to the value of the 5321.From unless the local system needs some kind of quirky routing.
It's a common misunderstanding because users rarely see an email without a Return-Path: header in their mailboxes. This is because they always see delivered messages, but an MTA should never see a Return-Path: header on a message in transit. See https://www.rfc-editor.org/rfc/rfc5321#section-4.4