In our express (with typescript) app, we are calling an external api and normally we would send a http GET using axios like
foo/api/search/hello world where hello world is entered by the user of a web app. We call encodeURIComponent on the parameter so we will send foo/api/search/hello%20world to the external api. And this works fine until someone tried to enter something that is non-ascii, s.a. hello tânia; this causes us to send foo/api/search/hello%20t%C3%A2nia to the external api and we got a http 400 back.
Is this something that the external api should handle or we shall handle it on our side?
Either way, how to handle this?
So in general the encodeURIComponent() function encodes all special characters into their matching UTF-8 encoding, see here. So your client is doing everything right, but it seems like your server which is receiving the HTTP GET is for some reason not capable of correctly handling the UTF-8 encoded URI.
I am testing some HTML email layout, I am not sure to understand which is the HTML I should consider:
use the HTML as it is sent from my server to mailgun (an email service provider)?
use the HTML as I receive it in raw format in the email I receive?
The difference is that in the raw source I receive there are =3D and =20 characters because of the quoted printable format. I'm using testi# but I suppose any rendering software will do, just it's not clear which version of the HTML should be used (needless to say I see =20 everywhere if it's raw, whereas it's perfect if I use the "pure" HTML)
You should use the HTML as it is before being sent. I'm unsure of how testi# works but with other services such as Litmus & Email on Acid you can send a test to them directly from your ESP and they will process that for the previews.
[Added from comment on question to allow close]
im thinking what is the best RESTful way how confirm email and request reseting password. Im only aiming to find correct URI...
confirm email
PUT /users/{userId}/confirmEmail?code=xyz - does not seem much RESTful because of confirmEmail
PUT /users/{userId}/email?confirmedBy=xyz - maybe better? dunno
reset password (similar problem)
PUT /users/{userId}/resetPassword --DATA {email:xyz#xyz.xy} - same thinkin as before
PUT /users/{userId}/password --DATA {state:reseted,resent:xyz#xyz.xy} - hmmm... again Im not sure
are there any better ways in your mind?:-)
If you want your URIs to refer to resources, then call the resource confirmation and POST confirmations to user accounts.
POST /users/{userid}/confirmation
The true RESTful answer is the URL does not matter, you put it in the confirmation e-mail anyway for the recipient to follow. Use whatever is most convenient for your load balancer, reverse proxy, servers, etc.
For convenience you'll end up accepting the confirmation even if it comes in a GET request, because that's what the browsers of flesh-and-bones humans oblivious to Dr Roy T. Fielding et al. send when clicking on a link in an e-mail :-)
Having established it is completely academic, I'd argue you were right to think of PUT, as the client idempotently places evidence of having access to the e-mail. Repeating the request has no further effect.
Considering that he said a reset service for someone who forgot her password, and not a change password service for someone already logged in...
I would use 2 services. 1st to request the reset password mail, and 2nd to set the new password with the token received in the received mail.
For the 1st:
POST baseUrl/passwordReset
Request body
{
"email" : "my#self.com"
}
This could be POST or PUT, but since a mail delivery is not a resource subject to CRUD anyway, let's not be pedantic and use the old POST that was always used in html forms.
Obviously I would control that the same client (ip? browser? ...) doesn't make me send 20K mails in a minute.
Sending the mail to the user doesn't imply that the old password is not valid. That will only happen later in the second request when the new one updates it.
Response 204 (perhaps you should do it even if you don't know that email, because if you return error that means that when you don't return error you are confirming to a stranger that the given email is registered)
For the 2nd:
POST baseUrl/password
Request body
{
"token" : "3D21BA...4F",
"newPassword" : "m%4pW1!O"
}
Where the token is received in the mail. So the mail could have a link to a page including the token, when the page is loaded, the form is filled and submitted, being the token a hidden field that some javascript reads from the URL and puts here.
This is really a resource that you update, so POST. And I don't think it makes sense to have the same URI with 2 verbs for both, because they are not the same resource/entity at all.
Add
By the way, I would make both HTTPS only, and that's why I put all the sensitive information in the body, not URL parameters.
Firstly, I don't think that PUT is the right method for this. PUT broadly means "put this here", where the URL is identifying where the content should be located. You're really asking an existing resource to perform some action, which makes POST more correct.
To answer your direct question, a RESTful URL should identify the resource you want to handle your request. In this case, the resource is either the user, or some password-resetting resource within the user.
My preference would be for a password-resetting resource:
POST /users/{userid}/password-reset
This makes sense from a HTTP point of view, since you could issue a GET on the resource and receive something which indicates how to action a password reset (e.g. a HTML form prompting for the email address of the associated account).
EDIT:
For the purposes of email validation, there are two obvious options. You could either POST to a "confirm email" resource with the email address and confirmation data, to ask the server to process the confirmation, or you can execute a PUT to put the confirmation information on the server:
POST /users/{userid}/confirm-email
or
PUT /users/{userid}/email-confirmation
Here is a RESTful way.
Request
PUT /{userid}/email HTTP/1.1
Content-Type: text/json+confirmation-code
{"activateCode": "23sfgsg3twt3rgsdhgs"}
Response
HTTP/1.1 200 OK
Content-Type: text/json+email-status
{"email": "my-email#address.com", "active": "true"}
No verbs in the URI needed :)
I don't really see anything wrong with having confirmEmail like the 1st example. In the URL you have the key to the user, confirmEmail is the action, and the data with that action is in the query string.
I've recently worked on this, my take was
POST /{base_url}/password
because I was actually creating a new random password and sending it over to the user
and
PUT /{base_url}/confirmation?token=...
Because I am updating the confirmation that was already sent out when the user registered.
We need to pass some data in emails that can be used for tracking and correlation, we were looking at using a custom header or a token in the subject/body.
Does anyone know of an email header that can be used that will not be removed by email servers? I know you can add X- headers to specify custom content but it seems you can never be sure they won't survive either the server stripping them or being removed when the email is forwarded.
The token option also doesn't seem ideal as it can easily be removed by the user, are there any other better techniques?
Thanks for your time
A server should not be stripping X- headers (as long as they are correctly formatted, anyway), unless perhaps you happen to pick a name for that X- header that the server uses for its own purposes. But there is no header that cannot be stripped or altered by a server or user. A proper forward operation should preserve the headers, and an improper forward isn't guaranteed to retain any headers.
A header is probably the best method, both because most users won't be aware that headers besides "From", "To", and "Subject" even exist and because that's really the place for such metadata. Encoding it in the Message-Id header might be slightly more robust than in an X- header. Or if it's very short, you could encode it in the subject line as a "ticket number" or the like.
I have used referrer before in foo.php to decide whether the page iframing foo.php is of a particular URL. (using $_SERVER['HTTP_REFERER'])
It turned out that most of the time, it worked (about 98% of the time), but it also seemed like some users arrived the page and $_SERVER['HTTP_REFERER'] was not set in foo.php and therefore broke the code. [update: These user claimed that they followed the usual page flow and didn't use the URL of foo.php all by itself on the browser (that they let it be an iframe) and the users never altered their browser settings.]
I wonder what the reasons are that it could happen?
The HTTP/1.1 RFC does not make it mandatory to send an HTTP referer header. You can't make any assumptions about its presence when writing robust code; perfectly conforment browsers may not include it.
Moreoever, the RFC advises that "The Referer field MUST NOT be sent if the Request-URI was obtained from a source that does not have its own URI, such as input from the user keyboard", and "We suggest, though do not require, that a convenient toggle interface be provided for the user to enable or disable the sending of From and Referer information".
The later is not very common (though some browsers have a "Private" mode that fulfils the requirements). More likely for your 2% is that people Bookmarked the URL, which fulfils the first criteria (URI obtained from a source without a URI), and so the browser sends no referer.
Not by default AFAIK, but it's easy to turn it off (for privacy) e.g. in Firefox via about:config, and surely some users could be using browsers distributed to them (e.g. by their IT department) with such kinds of setting. So you should try to avoid relying on REFERER for any important functionality (also because it's mis-spelled, of course;-).