Loading paginated data into Grafana - grafana

I'm using one of the two datasources:
marcusolsson-json-datasource
yesoreyeram-infinity-datasource
Both can call HTTP API endpoints.
The problem is the endpoint returns paginated response (JSON)..
My panel will be a very simple table that just shows the endpoint's response to the query (HTTP Request).
What I want to achieve is making Grafana grab the paginated data..
Example: When I click on page 1 in table it make a request with page_size=X&page_number=1 and so on.. Is that possible ?
If not possible.. what is the best thing to do here?
Should I use some kind of caching ? What cache suits best with my case and how to implement it?
I've been digging but I didn't really come to something useful.

The workaround I could come up with is by using a dashboard variable..
We create a variable that fills its value from a request to the endpoint and refreshes with the dashboard reload.
We then use that as a dropdown variable starting from 1 to number of pages.. When a page number is selected, it makes a new request to the endpoint to load the data. (By including variable value as a query parameter.)
The thing with this approach is that it doesn't look nice like Grafana's table pagination pane. Instead, the user will have to choose a page number from a dropdown list.

Related

Sub routing best practices

I am building an api which has a route: /market/items/{category}, it returns items from a category. If a user clicks on an item I have a route /item/{id}/ which returns information about the item. I'm wondering if this is a bad practice when creating a restful api. because often I would see routes like /market/items/{category}/{id}. What do you think?
If only the id is needed in order to retrieve the item (in other words, the id by itself uniquely identifies the item without any other information), then there's no reason to require a category in the URL also so this:
/item/{id}/
would be just fine for a restful API.
If, on the other hand, there are multiple types of items, each with overlapping item identifiers, then you may need something else in the URL to uniquely identify which type of item and thus which pool of item identifiers to look in.
One reason you may see some web sites doing something like this in their web page URLs:
/market/items/{category}/{id}
is for search indexing where they want the category name to be associated with the item for purposes of search engine indexing. But, if this is just a restful API, not visible web pages, then you probably aren't trying to optimize that for search hits.

Azure data factory pagination doesn't work

I am working on a pipeline which executes oAuth2 flow in order to access REST API json data. Once I have the bearer token I am executing a request which returns the following structure:
As you can see, since the response is quite large, there's paging enabled and as part of the response I get a link to the next page. In order to get to that resource I need to also present MS-ContinuationToken in the headers. So, this is how I basically do it in the config of the Copy activity that I use to get the data from the REST endpoint:
and the issue here is that I only get the first 2000 rows and the next page(s) don't seem to be visited at all. Pipeline executes successfully and only the first 2000 items are fetched.
NOTE: continuationToken and links.next.headers.value have the exact same values from the initial response.
Even if you fix the other issue you’ll have an issue with the “next” URL not including “v1”. This is a known issue in the partner center api team. I’ve escalated it pretty high. But they don’t want to break backwards compatibility by changing the “next” URI to include the v1 or to be relative. They are considering other options but I wouldn’t hold your breath.
I would ditch the idea of using data factory and instead write a .NET console app using the partner center SDK
(You might think to paginate manually with loops etc but the Copy activity doesn’t return eg the http headers, so you will need a complex set up to somehow store the data in a data store and be able to look up the last page in order to get the continuation token. I couldn’t figure it out)

Web API resources for global counts - what Restful API to create for that?

Suppose I have a website with the following menu items on the left site of my page:
Jobs (12)
Users (3)
News (5)
Documents (4)
Please note the labels on the right side of the menu. They indicate how much items there are for each resource. So for example, the are currently 3 users and 5 News items etc.
When I create a web API for this, then I treat each item as a separate resource. So in my Web API I would have the basic end points like:
(GET) /api/job
(GET) /api/job/{id}
(GET) /api/job/count
(POST) /api/job
(GET) /api/user
(GET) /api/user/{id}
(GET) /api/user/count
(POST) /api/user
etc. etc.
Of course also end points for UPDATE and DELETE.
The point here is that I have a count endpoint for each resource. So what I could do is call the count API for each API resource so I can show that number next to each menu item.
But in this case that means I would have to do 4 separate API calls to get the count for each item. And this becomes more when I have more menu items. I don't want to end up with 15 separate calls just to show a label next to each menu item showing how much items each resource has.
What is a good practice to solve this in a good Web API manner? Can I for example create a new end point : api/MenuCount that returns all the separate counts in one call?
But then again, having such an API end point isn't really a resource. It only has one purpose, counting the total rows for all other resources.
Or is it still better to call each count end point for the API's separately? Eventhough that could mean I have to do 15 separate calls for that?
PS. I am talking about a SPA website.

Ember CLI - Custom routing for very basic dropdown

New to ember and ember cli, and not having any JS based framework experience apart from jQuery (which is not a framework)
I find my self stuck at the very beginning compared to work done in Angular,
I have a static list of groups which are on REST api `http://localhost:8000/api/groups' and it is only 8 items there, I needed to show them as dropdown for a search criteria. Nothing fancy
I started with creating a route and model with the name of groups but app stopped working and I had to create a model for group which is identical to groups model, only having 2 items for dropdown
Now i have a url in my ember app http://localhost:4200/groups which I dont need and i dont want it to be there,
But I ignored it and had to create a dropdown of the cities, api is
http://localhost:8000/api/cities <-- All cities list, needed for admin
http://localhost:8000/api/cities/active <-- For clients to show active cities so they can add their records
http://localhost:8000/api/cities/filled <-- For users, to show them where clients are available
So in ember, I created a route and model for cities, and same model is copied as city just to show the list of cities in dropdown, I needed to show cities which are filled, I had created ember g route city/filled and it created folders, but its model is also same like other two models.
Ajax call is not being sent to city/filled url and now I ended up having
http://localhost:4200/cities // useless for me
http://localhost:4200/cities/filled //useless for me
and in filled i see that ajax call is made but to the http://localhost:8000/api/cities two times, loading same data. I tried adding a dropdown on my application and opened http://localhost:4200/cities/filled in browswer, and woosh, it send ajax call to the same url 3 times, one for application.hbs and one for cities.hbs and one for city/filled. Why load the same data 3 times if it is already fetched from same url within single request?
in angular I just call a custom url and I can get the data, but for ember its really hard to get grip on these small things and there is no help around
active and filled are filters for your cities resource and these are not standalone resources, so you should try to use them as query parameters. like
http://localhost:8000/api/cities?type=filled
http://localhost:8000/api/cities?type=active
Facebook uses this style for query params. You can also use query params for getting paginated data of a resource, like for 1st page and 25 records per page, the request will look like:
http://localhost:8000/api/cities?page=1&records=25

MVC 2.0 Post Form to action instead of redirect to action

I am using T4MVC to redirect to another action return RedirectToAction(MVC.MyController.MyAction());.
In result it is doing get request.
Is there any way to make post request from controller. I want to keep all the same but only make post instead get. I cant find any methods for that. I found one post helper here http://geekswithblogs.net/rakker/archive/2006/04/21/76044.aspx but i cant pass any values i need using this post helper. I was trying to pass values through TempData but they are not coming when i using this helper. May be some one have any ideas?
The reason i want to do this because when user come from one controller to another and then if user click update or just click enter in browser address bar, page will break.
Should i use session for that reason?
A RedirectToAction will always perform a GET, never a POST (it returns a HTTP 302 to the browser, which will then issue a GET request).
To persist data across the redirect, if it is data that can be easily represented as a string and stored in the query string, then you can just add it to the route values of the redirect.
e.g.
return RedirectToAction("Search", new { searchString = "whatever" });
If it is a complex type, then you will need to store it in TempData. A number of other questions on StackOverflow (such as this one) give details on how.
If repeatedly storing to and reading from TempData across your application offends your code-sense, then you can encapsulate this by using the PassParametersDuringRedirect attribute and generic RedirectToAction available in the MvcContrib project. Some details on this technique are available here.
only way of doing post is by having a form and doing submit on that form, either with a submit button or with javascript, any info you want passed to that action must be in that form and you will find everything posted in FormCollection(hope I spelled it right).