I've developed a method that does the following steps, in this order:
1) Get a report's metadata via /gdc/md//obj/
2) From that, get the report definition and use that as payload for a call to /gdc/xtab2/executor3
3) Use the result from that call as payload for a call to /gdc/exporter/executor
4) Perform a GET on the returned URI to download the generated CSV
So this all works fine, but the problem is that I often get back a blank CSV or an incomplete CSV. My workaround has been to put a sleep() in between getting the URI back and actually calling a GET on the URI. However, as our data grows, I have to keep increasing the delay on this, and even then it is no guarantee that I got complete data. Is there a way to make sure that the report has finished exporting data to the file before calling the URI?
The problem is that export runs as asynchronous task - result on the URL returned in payload of POST to /gdc/exporter/executor (in form of /gdc/exporter/result/{project-id}/{result-id}) is available after exporter task finishes its job.
If the task has not been done yet, GET to /gdc/exporter/result/{project-id}/{result-id} should return status code 202 which means "we are still exporting, please wait".
So you should periodically poll on the result URL until it returns status 200 which will contain a payload (or 40x/50x if something wrong happened).
Related
I am working on a program than launches Jenkins jobs using the REST API. After the job has completed, I'd like to get its log, so I call http://jenkins.domain.com/job/my_job_name/#/consoleText in my code.
In 75% of the cases that works and I get the text in return. But there a some cases where it comes back with HttpStatus 100 and no text. (Opening the URL with the browser then shows the text, so clearly there is something to return.) (I haven't found any pattern that would explain it, like "exceptionally large log" or so.)
I found no documentation about calls returning 100 and have no idea how to proceed. Simple repeating the call gives the same result. So how can I get the expected result?
Surprisingly "exceptionally large" was the answer. This caused a timeout (followed by some inappropriate handling) in the library that I used to handle the HttpGet. (Fortunately it was fixed very quickly.)
CKFinder being used is 3.4 for ASP.NET. It is operating on S3 file system (custom driver) with the following structure.
When content of the "Forms" folder is requested, CKFinder never completes and keeps displaying "Please wait. Loading..." even though GetFiles request has completed and returned JSON result. Here is the request.
And here is the response.
According to the docs (http://docs.cksource.com/ckfinder3-net/commands.html#command_getfiles) GetFiles response should be a JSON object structured like { files:[...], currentFolder:{...}, resourceType:'...' }.
But for the "Forms" folder CKFinder only returns "files" data, there is no "currentFolder" and "recourceType" properties. This leads to the following JS error and makes CKFinder break and show "Please wait. Loading..." forever.
So, for some folders CKFinder returns incomplete JSON response which leads to JS error and frozen UI.
Does anyone have an idea why CKFinder would generate incomplete GetFiles response?
There was inaccurate folder detection logic in the IFileSystem.FolderExistsAsync() method of custom IFileSystem implementation.
I am trying to synchronize an Outlook folder (say the Inbox) using the beta version of the Outlook Rest Api see doc here
I need only to retrieve the property IsRead and the PR_INTERNET_MESSAGE_ID
So following documentation, for the first synchronization my requests look like:
The following Http headers are always added:
request.Headers.Add("Prefer", "odata.track-changes");
request.Headers.Add("Prefer", "odata.maxpagesize=5"); //Use a small page size easier for debugging
The first initial full synchronization request
https://outlook.office365.com/api/beta/Me/MailFolders('inbox')/messages?$select=IsRead&$expand=SingleValueExtendedProperties($filter=(PropertyId eq 'String 0x1035'))
Good results the value array contain what I need.
The second request after the first request uses the deltatoken
https://outlook.office365.com/api/beta/Me/MailFolders('inbox')/messages?$select=IsRead,Subject&$expand=SingleValueExtendedProperties($filter=(PropertyId eq 'String 0x1035'))&$deltatoken=a758b90491954a61ad463ef3a0e690a2
Bad results, no SingleValueExtendedProperties entries
Next requests for paginations with skiptoken...
https://outlook.office365.com/api/beta/Me/MailFolders('inbox')/messages?$select=IsRead,Subject&$expand=SingleValueExtendedProperties($filter=(PropertyId eq 'String 0x1035'))&$skiptoken=e99ad10324464488b6b219ca5ed6be1c
Bad results again, same as 2.
It looks like a bug to me. Can you provide a workaround? From a list of ItemId is possible to retrieve easily the list of corresponding PR_InternetMessage_Id efficiently (not item per item)?
Note also that in the documentation it is written that:
The response will include a Preference-Applied: odata.track-changes
header. If you attempt to sync a resource that is not supported, this
header will not be returned in the response. Check for this header
before processing the response to avoid errors.
It seems that for 2. and 3. calls this response header "Preference-Applied" is not set.
The sync functionality today doesn't support extended properties. However, we are working to enable this and it should start working in a few weeks.
EDIT:
For a workaround for the very special case of the PR_INTERNETMESSAGE_ID look at the comment below.
I have seen several posts addressing this issue or similar to this issue for requests or GETs. I am not having this problem getting the data from the server, its solely on the POST.
The Errors I get are
The JSON request was too large to be deserialized.
or either
Error during serialization or deserialization using the JSON JavaScriptSerializer. The length of the string exceeds the value set on the maxJsonLength property. Parameter name: input
I haven't been able to consistently determine which actions result in which error, but it is predominately the latter one.
In an effort to get the value of the MaxJsonSize value, on the Index method of the controller, I get this data and dump it into a viewbag to write to console on the client side. Every time it comes back at 10k (102400).
If I reduce the data package size, and still serialize as previously, I get no errors.
In fiddler I can inspect the package and all the JSON is deserializable in fiddler, so I don't see an issue in my JSON. Additionally if I console.log(data) chrome sees no problems with it either.
The VM in the controller is the same for both POST and GET. With the exception there is more data with the POST than the GET. To test this I got a huge data set from the server;
GeoJSON data for all 50 states. Following was the result.
GET Content-Length: 3229309 return 200
POST Content-Length: 2975244 return 500
The POST failed in this scenario and returned the second error listed previously.
I only changed the data minimally (one string) and don't know why when sent back its smaller, but the JSON for both the GET and the POST is virtually identical.
I've tried changing the web.config file:
<system.web.extensions>
<scripting>
<webServices>
<jsonSerialization maxJsonLength="2147483644"/>
</webServices>
</scripting>
</system.web.extensions>
I just added this to the end of my config file just prior to
I've also added a parameter in Settings.config
<add key="aspnet:MaxJsonDeserializerMembers" value="2147483644" />
I have also verified that this param loads as part of the application settings in IIS.
Is there something else I can try to change to allow for these large data sets to be sent in a POST.
As a last resort, I was going to pull all of the GeoJSON data out of the POST. However when a user navigates back and they haven't changed what they were mapping, we'd have to find all the GeoJSON data again, causing undue work on the server etc. I thought if I only had to fetch it once that would be best from an efficiency perspective.
I struggled with this too, nothing I changed in web.config helped, despite several SO answers looking relevant. They helped with returning large JSON data, but the large JSON post kept failing. In the end I found this:
increase maxJsonLength for JSON POST and used the solution there, and it worked for me.
Quoting from there :
the MVC json serializer does not look at the webconfig to get the max length (thats for asp.net web services). you need to use your own serializer. you override ExecuteResult and supply you own json serializer. to override the input, create a new JsonValueProviderFactory, then override ValueProvider in the controller to return your new json factory when its a json request.
Please show me how can get the result of REST sampler in Jmeter. Because I need to to it for checking my sampler is right or wrong
Thanks,
You can use View results tree and uncheck Errors and Success check boxes in order to show all responses you receive. The Response data tab will show response data :) which you can format using drop down list in the lower left part of the View Results Tree component, bellow the request tree.
When you run test with large number of requests I suggest you check Errors (to record only requests that failed) to avoid filling up the RAM.
Or even better (for advanced test verification), you can use Assertions to mark the requests/responses that failed (which doesn't have to mean only "response code != 200", you may want to include your business logic and check arbitrary response header/param).
Add a 'View Results Tree View' to your testplan to see how the tests run.
When clicking on a test you can view the request/response data.
Next you might want to validate that response; add a 'Response Assertion' to your test
where you can match the response to anything you want.