The framerork validates the input parameters based on YAML and emits an error if needed.
I would like to incercept this error and reformat it to our app's standard format.
What is the best way to do this?
Related
I am developing a small application to test the change stream functionality in MongoDB.
I have found that if one uses a client session, that information is included in the change stream output (change event)
For instance, here is the output when I insert a document:
{"txnNumber"=>1, "lsid"=>{"id"=><BSON::Binary:0x70310118878160 type=uuid data=0x05d30a0fa4db4f24...>, "uid"=><BSON::Binary:0x70310118878040 type=generic data=0x08e97261f57b1617...>}, "_id"=>{"_data"=>"8262D407C4000000022B022C0100296E5A100483BECD0AF46146E4A271EDAC0922356946645F6964006462D407C48C187B092534BD050004"}, "operationType"=>"insert", "clusterTime"=>#<BSON::Timestamp:0x00007fe4b351d4f8 #seconds=1658062788, #increment=2>, "fullDocument"=>{"_id"=>BSON::ObjectId('62d407c48c187b092534bd05'), "one"=>"one"}, "ns"=>{"db"=>"change_stream_testing", "coll"=>"testing"}, "documentKey"=>{"_id"=>BSON::ObjectId('62d407c48c187b092534bd05')}}
The "lsid"-field contains information about the session from which the write originated. After taking a closer look at this i found that it contains base64 encoded data (just doing a json.parse() on the id and uid fields)
ID IS
{"$binary":{"base64":"BdMKD6TbTySICrHNHE6GBA==","subType":"04"}}
UID IS
{"$binary":{"base64":"COlyYfV7FhdDV8hhDrSY7+10/NVCs/fLwkGrKMztex4=","subType":"00"}}
Now, the problem/question is that i can't decode that base64 string to something readable. Using an online decoder i get
У
¤ЫO$€
±НN†**
and **éraõ{CWÈa´˜ïítüÕB³÷ËÂA«(Ìí{
respectively - when using the "auto detect" feature or UTF-8 which MongoDB uses internally (according to a quick google search)
The reason I ask is that I have a use case where, in some cases, I would like to be able to identify where an event in the change stream originated from ie. and what client issued the write. The only way I've been able to sort of accomplishing that without using the mutateFields operator to actually change the documents themselves and adding some kind of marker I could inspect in the change stream code (which I ideally don't want to do) is to use explicit client sessions which at least lets me know that whatever was writing the document was using an explicit session. But I would like to be able to go further and actually decipher this session information to, if possible, get some kind of unique identifier.
I am trying to communicate with kdb+ via HTTP. I succeeded to get the the response from the DB when I did it from the browser, but for some reason, the response is neither JSON nor anything that looks machine readable.
What I did:
Opened a port on q console (8080)
Defined a function getData that gives me the data I want.
The above steps can be presented in this .q file I created:
\p 8080
system["l db"]
getData: {[a;b] ?[table;((>=;`start;$[`long;a]);(<=;`end;$[`long;b]));0b;()]}
h:hopen `:localhost:8080:user:pass
I then typed http://localhost:8080/?getData[1;2] in the browser to get the results
Got the results as text with spaces inside <pre> tag.
The Problem
I have no I idea how to parse it to JSON. How can I solve this? is there a way to tell kdb+ to send response in JSON format?
The solution I found:
If you add .json before the ? in the url (i.e. http://localhost:8080/.json?getData[1;2]) the response will be in JSON.
It looks like, based on your code, the following may work to return data:
http://localhost:8080/?getData[a;b]
Where a and b are start and end times (or dates, this will depend on your data).
e.g.
http://localhost:8080/?getData[2021.07.23;2021.07.30]
In order to convert a KDB+ object to json you can make use of the .j.j operator.
For example you can change your code to something like:
getData:{[a;b] t:?[table;((>=;`start;$[`long;a]);(<=;`end;$[`long;b]));0b;()];.j.j t}
If you want to see what the incoming request looks like on your Q process you could add some logging to .z.ph (the default http message handler), e.g. something like:
.z.ph:{[zph;x]zph 0N!x}[.z.ph]
This will log any incoming requests out to your q process using 0N! and then run the usual .z.ph message handing code on it
I am trying to call the resource /matchroute via a GET request.
However, I can't figure out how to encode the GPX file so that the resource accepts my request: I always receive HTTP error 400 as a response from the Here server.
As exemplary data I used the following file:
<?xml version="1.0"?>
<gpx version="1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.topografix.com/GPX/1/0"
xsi:schemaLocation="http://www.topografix.com/GPX/1/0
http://www.topografix.com/GPX/1/0/gpx.xsd">
<trk>
<trkseg>
<trkpt lat="51.10177" lon="0.39349"/>
<trkpt lat="51.10181" lon="0.39335"/>
<trkpt lat="51.10255" lon="0.39366"/>
<trkpt lat="51.10398" lon="0.39466"/>
<trkpt lat="51.10501" lon="0.39533"/>
</trkseg>
</trk>
</gpx>
that I got from the this example.
I encoded this file using MATLAB's function matlab.net.base64encode which yielded the following base64-encoded string:
PD94bWwgdmVyc2lvbj0iMS4wIj8+PGdweCB2ZXJzaW9uPSIxLjAieG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8y
MDAxL1hNTFNjaGVtYS1pbnN0YW5jZSJ4bWxucz0iaHR0cDovL3d3dy50b3BvZ3JhZml4LmNvbS9HUFgvMS8wInhzaTpz
Y2hlbWFMb2NhdGlvbj0iaHR0cDovL3d3dy50b3BvZ3JhZml4LmNvbS9HUFgvMS8wIGh0dHA6Ly93d3cudG9wb2dyYWZp
eC5jb20vR1BYLzEvMC9ncHgueHNkIj48dHJrPjx0cmtzZWc+PHRya3B0IGxhdD0iNTEuMTAxNzciIGxvbj0iMC4zOTM0
OSIvPjx0cmtwdCBsYXQ9IjUxLjEwMTgxIiBsb249IjAuMzkzMzUiLz48dHJrcHQgbGF0PSI1MS4xMDI1NSIgbG9uPSIw
LjM5MzY2Ii8+PHRya3B0IGxhdD0iNTEuMTAzOTgiIGxvbj0iMC4zOTQ2NiIvPjx0cmtwdCBsYXQ9IjUxLjEwNTAxIiBs
b249IjAuMzk1MzMiLz48L3Rya3NlZz48L3Ryaz48L2dweD4=
However, as stated before, the HERE server consistently responds with HTTP-error 400 to my request
https://rme.api.here.com/2/matchroute.json?app_id={app_id}&app_code={app_code}&routemode=car&file=...
where "..." equals the above mentioned base64-encoded string.
Question: Could anyone please provide a code sample showing how to encode the above mentioned GPX file correctly (ideally in MATLAB language) so that the /matchroute resource is able to respond?
Remarks:
If I use the base64 string
UEsDBBQAAAAIANmztEQSwaeZzwAAAM8BAAAQAAAAc2FtcGxlLXRyYWNlLmdweIXPTQuCMBwG8HufQnZv%2F605S0k9dj
EIungdZjpSJ27kPn6%2BRBgYXcYYv2cPzzG2deU8805L1YSIYoLiaHMsWvv9uBlYowOrZYhKY9oAoO973DOsugJ2hFBI
z8k1K%2FNabGWjjWiy%2FJ36ShjVqqITd2lxpmo4XVKgMP6vZaCneKIyYabivzHnr4BhCbb6hoZRpnvMp86L%2BdIapx
ImRJxiSuh%2Bj5xq7CWY%2Bcz1EaypA10qxlfVjvOl8rxVxfzDQrk%2FFCfLRs7YpOCzA%2BZd49LoBVBLAQIUABQAAA
AIANmztEQSwaeZzwAAAM8BAAAQAAAAAAAAAAEAIAAAAAAAAABzYW1wbGUtdHJhY2UuZ3B4UEsFBgAAAAABAAEAPgAAAP
0AAAAAAA%3D%3D
from this example the GET request works. However, I couldn't figure out how to reproduce this encoding myself so that I am able to encode my own data accordingly.
Link to the Here API definition: https://developer.here.com/documentation/route-match/topics/resource-matchroute-request.html
Looking at the two base64 strings I can tell you the fundamental difference between them - the first one (which doesn't work) is unescaped whereas the second one (which works) is.
You can convert between the two formats manually using various online tools like this one. The escaped version of the non-working base64 string, in case you want to test it, is:
PD94bWwgdmVyc2lvbj0iMS4wIj8+PGdweCB2ZXJzaW9uPSIxLjAieG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8y
%0AMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSJ4bWxucz0iaHR0cDovL3d3dy50b3BvZ3JhZml4LmNvbS9HUFgvMS8wInhza
Tpz%0AY2hlbWFMb2NhdGlvbj0iaHR0cDovL3d3dy50b3BvZ3JhZml4LmNvbS9HUFgvMS8wIGh0dHA6Ly93d3cudG9wb2
dyYWZp%0AeC5jb20vR1BYLzEvMC9ncHgueHNkIj48dHJrPjx0cmtzZWc+PHRya3B0IGxhdD0iNTEuMTAxNzciIGxvbj0
iMC4zOTM0%0AOSIvPjx0cmtwdCBsYXQ9IjUxLjEwMTgxIiBsb249IjAuMzkzMzUiLz48dHJrcHQgbGF0PSI1MS4xMDI1
NSIgbG9uPSIw%0ALjM5MzY2Ii8+PHRya3B0IGxhdD0iNTEuMTAzOTgiIGxvbj0iMC4zOTQ2NiIvPjx0cmtwdCBsYXQ9I
jUxLjEwNTAxIiBs%0Ab249IjAuMzk1MzMiLz48L3Rya3NlZz48L3Ryaz48L2dweD4%3D
I'm not an expert on this, but as I understand, you need to URL-encode strings only when you want to paste them as-is into the web path of your browser (read about "URL Params"). If you construct your HTTP requests the right way™ (by this I mean specify the headers of the request and the key-value pairs correctly), you shouldn't have to worry about URL-encoding at all, since the tool that you're using (in this case, MATLAB) should take care of the conversion for you.
Unfortunately, I cannot test this theory, as I have no access to the discussed API - but I am fairly certain that this would solve your problem.
I had the exact same problem.
The documentation seems to be incomplete. You can check here for additional information. Several ways I solved this:
Use filetype='CSV' or filtetype='GPX' in parameter. It says the filetype is guessed if passed, that is actually not true. After passing an XML file the API told me my file didn't look like a 'CSV'
Compression is OPTIONAL, I suggest to avoid it completely I could not find a suitable compression either. It works fine with plain base64 encoding.
I suggest to actually use CSV because the XML actually returns parsing errors.
In python
data='''latitude,longitude
51.10177,0.39349
'''
r = requests.get('https://rme.api.here.com/2/matchroute.json?app_id={APP_ID}&app_code={APP_CODE}&routemode=car&file={file}&filetype={filetype}'.format(
APP_ID=os.getenv('HERE_APP_ID'),
APP_CODE=os.getenv('HERE_APP_CODE'),
filetype='CSV',
file=base64.b64encode(data.encode()).decode()
))
I'm trying to add a new content type to a REST endpoint. Currently it only returns json but I now need to be able to return also a CSV file.
As far as I know, the best way to do this is by using the Accept header with value text/csv and then add a converter that is able to react to this and convert the returned body to the proper CSV representation.
I've been able to do this but then I have a problem handling exceptions. Up until know, all the errors returned are in json. The frontend expects any 500 status code to contain a specific body with the error. But now, by adding the option to return either application/json or text/csv to my endpoint, in case of an error, the converter to be used to transform the body is going to be either the jackson converter or my custom one depending on the Accept header passed. Moreover, my frontend is going to need to read the content-type returned and parse the value based on the type of representation returned.
Is this the normal approach to handle this situation?
A faster workaround would be to forget about the Accept header and include a url parameter indicating the format expected. Doing it this way, I'd be able to change the content-type of the response and the parsing of the data directly in the controller as the GET request won't include any Accept header and it will be able to accept anything. There are some parts of the code already doing this where the only expected response format is CSV so I'm going to have a difficult time defending the use of the Accept header unless there is a better way of handling this.
my frontend is going to need to read the content-type returned and parse the value based on the type of representation returned.
Is this the normal approach to handle this situation?
Yes.
For example, RFC 7807 describes a common format for describing problems. So the server would send an application/problem+json or an application/problem+xml representation of the issue in the response, along with the usual meta data in the headers.
Consumers that understand application/problem+json can parse the data with in, and forward a useful description of the problem to the user/logs whatever. Consumers that don't understand that representation are limited to acting on the information in the headers.
A faster workaround would be to forget about the Accept header and include a url parameter indicating the format expected.
That's also fine -- more precisely, you can have a different resource responsible for the each of the different media-types that you support.
It may be useful to review section 3.4 of RFC 7231, which describes the semantics of content negotiation.
I'm using Filemaker 11 to manage content over Custom Web publishing, with a json xslt sheet to convert the XML to a json format is there any way to add a parameter to the url, and have it come back down without modifying it?
I thought about globals, but from what I can tell if two requests were sent within a short enough amount of time, there could be a race condition, one overwriting the others global..
Parameters are passed to the stylesheet in <xsl:param name="request-query" />, whose structure must be specified in the documentation, but as far as I remember it's
<query action="..."> <-- the form action, i.e. the url -->
<parameter name="...">...</parameter>
</query>
The query namespace is http://www.filemaker.com/xml/query. FileMaker's own idea of passing custom parameters is to name them like token.1, etc., but it's possible to pass just any set of parameters to a XSLT that uses a process action (i.e. doesn't hit the database), sort the parameters out into what goes into the db and what stays, make an inline call, and then process the result and add the saved parameters back.
Note that you also can access the cookies and the headers, if you need. Also note that XSLT API is deprecated and won't be supported in future FileMaker version.