I'm wondering if the OrientDB element key "#:" format can take another format, for instance changing the colon to dash.
the problem with hash and colon that both are reserved for URI use (RFC 3986), and they cant be transferred by HTTP requests (Not Safe).
any idea?
thanks
OrientDB HTTP Server allows to ignore # for RIDs when are in URL.
Related
I wrote a gateway application using Spring cloud Greenwich binaries. I'm seeing issues when special characters are present in URL. The request fails with below exception in Spring gateway when request URI contains special characters.
localhost:8080/myresource/WG_splchar_%26%5E%26%25%5E%26%23%25%24%5E%26%25%26*%25%2B)!%24%23%24%25%26%5E_new
When I hit above url, Spring fails with below exception. I'm not able to figure out why it's an invalid sequence and how things like these can be handled.
java.lang.IllegalArgumentException: Invalid encoded sequence "%^&#%$^&%&*%+)!$#$%&^_new"
at org.springframework.util.StringUtils.uriDecode(StringUtils.java:741) ~[spring-core-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at org.springframework.http.server.DefaultPathContainer.parsePathSegment(DefaultPathContainer.java:126) ~[spring-web-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at org.springframework.http.server.DefaultPathContainer.createFromUrlPath(DefaultPathContainer.java:111) ~[spring-web-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at org.springframework.http.server.PathContainer.parsePath(PathContainer.java:76) ~[spring-web-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at org.springframework.cloud.gateway.handler.predicate.PathRoutePredicateFactory.lambda$apply$2(PathRoutePredicateFactory.java:79) ~[spring-cloud-gateway-core-2.1.0.RC3.jar:2.1.0.RC3]
at org.springframework.cloud.gateway.support.ServerWebExchangeUtils.lambda$toAsyncPredicate$1(ServerWebExchangeUtils.java:128) ~[spring-cloud-gateway-core-2.1.0.RC3.jar:2.1.0.RC3]
at org.springframework.cloud.gateway.handler.AsyncPredicate.lambda$and$1(AsyncPredicate.java:35) ~[spring-cloud-gateway-core-2.1.0.RC3.jar:2.1.0.RC3]
at org.springframework.cloud.gateway.handler.RoutePredicateHandlerMapping.lambda$null$2(RoutePredicateHandlerMapping.java:112) ~[spring-cloud-gateway-core-2.1.0.RC3.jar:2.1.0.RC3]
at reactor.core.publisher.MonoFilterWhen$MonoFilterWhenMain.onNext(MonoFilterWhen.java:116) [reactor-core-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2070) [reactor-core-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at reactor.core.publisher.MonoFilterWhen$MonoFilterWhenMain.onSubscribe(MonoFilterWhen.java:103) [reactor-core-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:54) [reactor-core-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at reactor.core.publisher.MonoFilterWhen.subscribe(MonoFilterWhen.java:56) [reactor-core-3.2.5.RELEASE.jar:3.2.5.RELEASE]
I answered the other question already and don't feel like retyping. The spirit of the answer is the exact same.
Write a unit test exercising this method off of the Spring cloud utils. This is what's breaking. You can try passing in more or less of the string you're concerned about to find where the breakage is. Use a binary search to figure out what's broken. Make sure you don't split the string in the middle of an encoded character or else you'll give yourself a false positive. When it says you have an invalid sequence I would expect you have something like %99 where 99 is does not map to any valid character (I'm just making one up)
https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/util/StringUtils.html#uriDecode-java.lang.String-java.nio.charset.Charset-
As an aside
Where is this encoded string coming from? Did someone at your company create their own solution to encode this string to begin with? Are you accepting user data? It's VERY POSSIBLE that whomever is responsible for producing this string encoded it incorrectly by homerolling their own encoder.
ALTERNATIVELY
spring.cloud.gateway.routes[7].predicates[0]=Path=/test/{testId}/test1/test_%26%5E%26%25%5E%26%25%26*%25%2B)!
When I look at this I see a path that is already encoded. For example, you've taken your ampersand & character and replaced it with %26
Have you tried inputting a path that is NOT already encoded?
For example
spring.cloud.gateway.routes[7].predicates[0]=Path=/test/{testId}/test1/test_&^&%^ < I only partially decoded it by hand using this chart. https://www.w3schools.com/tags/ref_urlencode.asp
I am using JWT token to authorize my APIs, during implementation I found header and payload in token always start with eyJ. What does this indicate?
JWTs consist of base64url encoded JSON, and a JSON structure just starts with {"..., which becomes ey...when encoded with a base64 encoder.
The JWT header starts with {"alg":..., which then becomes eyJ...
You can try on this online encoder and enter {"alg" and click on encode. The result will be eyJhbGciPSA=
I'm afraid the question, and answer above is a little too extensive/certain.
The best you can check for is (only) 'ey', as the first JSON member could be something else such as "typ" (rather than "alg"); I wouldn't recommend assuming the order of JSON members (even if they are supposed to follow a prescribed order - i.e. allowing for the possibility of real-world anomalies/a small amount of flex).
Also, as much as it is probably unlikely - as far as what is been produced by a particular implementation, there could be a (/some) whitespace following the opening (JSON object) brace character (and maybe even before it!) - I'm not sure if the standard/RFCs forbid this, but even if it's only a temporal instance of a bug (within the JWT generation process) this could in-theory occur; so you're better in only checking for 'ey' - as a quick smoke-test, before then proceeding on to a fuller/complete validation of the JWT.
(F.Y.I. I believe it may have been Microsoft 'Identity Platform' whereby "typ" preceded "alg" - if memory serves me correctly (?), but I can't swear to it / as to where I've seen this being the case - at least at one point in time.)
In a HTTP request, I am adding a token as a query-param.
It seems that mule is encoding the value.
- If I add the parameter, mule will encode it in the way it is wrong.
- If I add already encoded parameter, mule will double encode it and therefore won't be usable anymore.
So the question is: Is there a way or a workaround to prevent mule from encoding the URL query-param?
Example of the parameter: {AES}ZEoksxIg484magPtWwNUUQ==;iT0kI2HsqGkh%2Bdc2baW2B4dNR2vouKkWQsDTdbMP8us=
My colleague found a workaround for this, so I'm sharing it here.
Apparently, you can set a variable before the HTTP request and add the manually encoded value. Let's call it ourTokenVariable In my example above that would be %7BAES%7DZEoksxIg484magPtWwNUUQ%3D%3D%3BiT0kI2HsqGkh%252Bdc2baW2B4dNR2vouKkWQsDTdbMP8us%3D
After that, you can use this newly created variable directly in the url path. For example: /example/someapi?someToken=[#flowvars.ourTokenVariable]
This way you don't need to use uri-param or query-param anymore (where mule is double encoding the value). The value will be taken 'as is'.
Should path parts be encoded or not encoded when it comes to Google Cloud Storage?
Encoding URI path parts says they should be encoded, but Object names talks about the possibility of naming GCS objects in a seemingly-hierarchical manner...
So if I name an object abc/xyz, is the path to my object https://www.googleapis.com/storage/v1/b/example-bucket/o/abc%2fxyz or https://www.googleapis.com/storage/v1/b/example-bucket/o/abc/xyz?
Which is it!? Somebody please help me with this confusion.
TL;DR
You can use nested folders when working through a GCS client library but sending GET requests to the URL itself will need you to understand how to map the folder names appropriately.
Let's all pretend that folders are real
Yes, you need to encode the object names. There's a useful description here which I partially quote below (with my emphasis) for reference:
Object names reside in a flat namespace within a bucket, [...] means
that objects do not reside within subdirectories in a bucket. For
example, you can name an object
/europe/france/paris.jpg
to make it appear that paris.jpg resides in the subdirectory /europe/france, but to Cloud Storage, the object simply exists in the bucket and has the
name /europe/france/paris.jpg.
So there are no subdirectories but appropriate naming and the use of a knowledgeable UI or API will make it appear as if there is some hierarchy.
All GCS client libraries will know to encode the names correctly but if you are running raw GETs on them (with appropriate authentication), you will have to do this yourself. The relevant section is here and I quote the most relevant part here:
For example, if you send a GET request for the object named foo/?bar
in the bucket example-bucket, then your request URI should be:
GET https://www.googleapis.com/storage/v1/b/example-bucket/o/foo%2f%3fbar
So you can see that the object name part as been encoded with %2f for the slash (/) character. There's a more complete description of the naming convention here.
Metadata v Content using GCS JSON API
I was slightly surprised that the default behaviour for the API was to return metadata about the object in the bucket. To get the actual content I had to append '?alt=media' as described at the end of this section:
By default, this responds with an object resource in the response
body. If you provide the URL parameter alt=media, then it will respond
with the object data in the response body.
I am building a RESTful API for retrieving and storing comments in threads.
A comment thread is identified by an arbitrary URI -- usually this is the URL of the web page where the comment thread is related to. This design is very similar to what Disqus uses with their system.
This way, on every web page, querying the related comment thread doesn't require storing any additional data at the client -- all that is needed is the canonical URL to the page in question.
My current implementation attempts to make an URI work as a resource by encoding the URI as a string as follows:
/comments/https%3A%2F%2Fexample.org%2Ffoo%2F2345%3Ffoo%3Dbar%26baz%3Dxyz
However, before dispatching it to my application, the request URI always gets decoded by my server to
/comments/https://example.org/foo/2345?foo=bar&baz=xyz
This isn't working because the decoded resource name now has path delimiters and a query string in it causing routing in my API to get confused (my routing configuration assumes the request path contains /comments/ followed by a string).
I could double-encode them or using some other encoding scheme than URI encode, but then that would add complexity to the clients, which I'm trying to avoid.
I have two specific questions:
Is my URI design something I should continue working with or is there a better (best?) practice for doing what I'm trying to do?
I'm serving API requests with a Go process implemented using Martini 'microframework'. Is there something Go or Martini specific that I should do to make the URI-encoded resource names to stay encoded?
Perhaps a way to hint to the routing subsystem that the resource name is not just a string but a URL-encoded string?
I don't know about your url scheme for your application, but single % encoded values are valid in a url in place of the chars they represent, and should be decoded by the server, what you are seeing is what I would expect. If you need to pass url reserved characters as a value and not have them decoded as part of the url, you will need to double % encode them. It's a fairly common practice, the complexity added to the client & server will not be that much, and a short comment will do rightly.
In short, If you need to pass url chars, double % encode them, it's fine.