I am using a SQL Server query which would return the last 3 months since a customer last purchased a product. For instance, There's a customer 100 that last made a purchase in August 2022. The SQL query will return June, July, August. Which would be in the format 062022, 072022, 082022. Now I need to be able to pass these values to the Copy data activity REST api dataset Relative URL (/salemonyr/062022) in the ForEach activity.
So during the first iteration the Relative URL should be set to /salemonyr/062022 the second would be /salemonyr/072022 and third /salemonyr/082022.
Error: The expression 'length(activity('MonYear').output.value)' cannot be evaluated because property 'value' doesn't exist, available properties are 'resultSetCount, recordsAffected, resultSets, outputParameters, outputLogs, outputLogsLocation, outputTruncated, effectiveIntegrationRuntime, executionDuration, durationInQueue, billingReference
Script activity json:
{
"resultSetCount": 1,
"recordsAffected": 0,
"resultSets": [
{
"rowCount": 3,
"rows": [
{
"MonYear": 062022
},
{
"MonYear": 072022
},
{
"MonYear": 082022
}
]
}
],
"outputParameters": {},
"outputLogs": "",
"outputLogsLocation": "",
"outputTruncated": false,
"effectiveIntegrationRuntime": "",
"executionDuration": 0,
"durationInQueue": {
"integrationRuntimeQueue": 3
},
"billingReference": {
"activityType": "PipelineActivity",
"billableDuration": [
{
"meterType": "",
"duration": 0.016666666666666666,
"unit": "Hours"
}
]
}
}
How would I accomplish this to read the values dynamically from the SQL query.
You can use #split(item().colname,',')[0] , split(item().colname,',')[1] and split(item().colname,',')[2] in the relative URL path.
Check the below video for details:
You can use REST Dataset parameter and use it in the Relative URL.
Relative URL:
Give lookup output to ForEach. use your query in lookup.
Give this to ForEach and inside ForEach, in copy sink(REST DATASET) use the below expression for the dataset parameter.
/salemonyr/#{item().sample_date}
In source, you can give your source.
By this, you can copy the data to the respective Relative URL.
When providing the incorrect format of a field for a request to my application if the type throws an error then the error message returned by micronaut is vague.
E.G two scenarios
public class fakeClass {
#NotNull
private String fakeName;
}
if my request is {"fakeName": ""}
then the response, correctly, would be something like
{
"violations": [
{
"field": "create.fakeClass.fakeName",
"message": "must not be blank"
}
],
"type": "https://zalando.github.io/problem/constraint-violation",
"title": "Constraint Violation",
"status": 400 }
But lets say my class looks like this:
public class fakeClass {
#Format("yyyy-MM-dd")
private LocalDate exampeDate;
}
With an invalid date or incorrect format of {"exampleDate": 202222--01-01} or {"exampleDate": 2022/01/01}
Then the error message is
{
"type": "about:blank",
"parameters": {
"path": "/project"
},
"status": 400,
"detail": "Required argument [fakeClass fakeClass] not specified"
}
Is there a simple way to provide more information to the error message to make it clear why the request failed for an invalid format or type like #NotNull or #NotBlank?
The problem here is not Micronaut but your payloads. The examples you mentioned are invalid JSON documents.
For example this on here is invalid, since the value is not a number nor a string.
{
"exampleDate": 202222--01-01
}
this would be the a valid variant
{
"exampleDate": "202222--01-01"
}
Make sure you send the date as a String. In your case this is expected to be valid.
{
"exampleDate": "2022-11-01"
}
In general it is recommended to send date using the ISO-8601 format, which you did (yyyy-MM-dd). Furthermore I recommend to apply a global configuration rather than using on each POJO a #Format("yyyy-MM-dd") annotation.
jackson:
dateFormat: yyyyMMdd
timeZone: UTC
serializationInclusion: NON_NULL
serialization:
writeDatesAsTimestamps: false
#Format("yyyy-MM-dd") is a formatter not a Constraint.
You can use #Pattern(<regex>). There is also date specific ones like #Past, #PastOrPresent, #Futureand #FutureOrPresent.
I am creating an api using hapi framework and need to take in time as one of the data types in the payload that I receive. I have defined the validation as
payload: {
startTime: Joi.date().timestamp().required(),
endTime: Joi.date().timestamp().required()
}
But when I bring up the swagger documentation page for this validation I see the inputs to be received a below
{
"startTime": 0,
"endTime": 0
}
I was expecting a more user-friendly approach where it would display the timestamp format in swagger like below.
{
"startTime": HH:MM:SS,
"endTime": HH:MM:SS
}
How do I make this possible?
I'm trying out JSON views, not on top of domain class using #Resource, but by creating a RestfulController and trying to render that using JSON views. I've added all the relevant dependencies in build config.
I have a domain Post class like this (which I didn't want to directly expose)
class Post implements Serializable {
Map comments
User user
Venue venue
String description
Rating rating //should this be an enum?
LocalDateTime dateCreated
LocalDateTime lastUpdated
static belongsTo = [user:User]
static hasOne = [rating:Rating]
static constraints = {
venue nullable:true
comments nullable:true
description nullable:true
rating nullable:true, lazy:false
}
static mapping = {
//set the sort order for Posts - default using newest post first
sort dateCreated :"desc"
}
}
So I then created a default RestfulController like this:
class PostRestController extends RestfulController {
static responseFormats = ["json", "xml"]
//constructor - tells rest controller which domain class to scaffold
PostRestController() {
super (Post)
}
}
I'm not overriding any of the default scaffolding methods here.
When I used a rest client to access the default (I've mapped /api/posts (resources: postRest in the UrlMappings). When I access the URL with my REST client I got the full dump of the Post (including comments field persisted in a map) - this looks like this in my rest client - all OK:
[
{
"id": 1,
"comments": {
"view": "lovely"
},
"dateCreated": {
"class": "java.time.LocalDateTime",
"dayOfMonth": 7,
"dayOfWeek": {
"enumType": "java.time.DayOfWeek",
"name": "TUESDAY"
},
"dayOfYear": 66,
"hour": 19,
"minute": 15,
"month": {
"enumType": "java.time.Month",
"name": "MARCH"
},
"monthValue": 3,
"nano": 263000000,
"second": 10,
"year": 2017,
"chronology": {
"calendarType": "iso8601",
"class": "java.time.chrono.IsoChronology",
"id": "ISO"
}
},
"description": null,
"lastUpdated": {
"class": "java.time.LocalDateTime",
"dayOfMonth": 7,
"dayOfWeek": {
"enumType": "java.time.DayOfWeek",
"name": "TUESDAY"
},
"dayOfYear": 66,
"hour": 19,
"minute": 15,
"month": {
"enumType": "java.time.Month",
"name": "MARCH"
},
"monthValue": 3,
"nano": 263000000,
"second": 10,
"year": 2017,
"chronology": {
"calendarType": "iso8601",
"class": "java.time.chrono.IsoChronology",
"id": "ISO"
}
},
"rating": null,
"user": {
"id": 1
},
"venue": null
}
],
I then tried to add JSON views on top of this in the grails-app/views/postRest folder.
I did a really simple template _post.gson like this:
model {
Post post
}
json {
comments post.comments
description post.description
//rating post.rating
userWhoPosted "${post?.user}"
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MMM-dd")
def when = post.dateCreated.format(formatter)
created when
}
I then added an index.gson to render the template:
model {
List<Post> postList
}
//call the template to iterate over the postList to produce the output
json g.render(postList)
This breaks the server with this stacktrace and a 500 error to the REST client. If I comment out the line in_post.gson relating to user it all works. Leave it in and it fails:
Caused by: grails.views.ViewRenderException: Error rendering view: null
at grails.views.AbstractWritableScript.writeTo(AbstractWritableScript.groovy:43)
at grails.views.mvc.GenericGroovyTemplateView.renderMergedOutputModel(GenericGroovyTemplateView.groovy:73)
at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:303)
at grails.views.mvc.renderer.DefaultViewRenderer.render(DefaultViewRenderer.groovy:111)
at grails.artefact.controller.RestResponder$Trait$Helper.internalRespond(RestResponder.groovy:188)
at grails.artefact.controller.RestResponder$Trait$Helper.respond(RestResponder.groovy:62)
at grails.rest.RestfulController.index(RestfulController.groovy:64)
at grails.transaction.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:96)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at grails.transaction.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:93)
... 4
If I comment out the post.user (and ratings reference) it works OK, but when I try and post the post.user it fails with the above. There was a note in the docs about ensuring that your query pulled the refs with a fetch join - so I tried to provide a override to ensure I returned the fetch join - all I get is empty returned to the client:
class PostRestController extends RestfulController {
static responseFormats = ["json", "xml"]
//constructor - tells rest controller which domain class to scaffold
PostRestController() {
super (Post)
}
def index() {
Collection<Post> res = Post.list([fetch:[user:"join",rating:"join"]])
res
}
}
Why when I do it without the JSON view it works fine and when I use the JSON view I can't get the output including references? I checked the list request and it returns the list successfully in the debugger - but breaks in the rendering.
If I can get this to work, JSON views on Grails 3.2.6 looks pretty nice.
Aaargh - think this issue is with jsonViews 1.1.5 - its not ready for java 8 LocalDateTime.
i saw a trace on stackoverflow see topic
hibernate will now take localdateTime in your domain classes - that works. But the json template rendering wont, even if you add the java8 plugin.
So i went back into domain class changed my LocalDateTime back to Date, also changed the json template to use the older SimpleDateTime format (instead of DateTimeFormatter) and re ran - low and behold it worked.
I'll be really glad when we can say grails is properly java8 ready.
apparently json views requires feature enablement which is due in json views 2 (think this is an M2 right now) - so i've had to revert to java 7 Date until then.
blimey another lost day in the weeds.
I want to support pagination in my RESTful API.
My API method should return a JSON list of product via /products/index. However, there are potentially thousands of products, and I want to page through them, so my request should look something like this:
/products/index?page_number=5&page_size=20
But what does my JSON response need to look like? Would API consumers typically expect pagination meta data in the response? Or is only an array of products necessary? Why?
It looks like Twitter's API includes meta data: https://dev.twitter.com/docs/api/1/get/lists/members (see Example Request).
With meta data:
{
"page_number": 5,
"page_size": 20,
"total_record_count": 521,
"records": [
{
"id": 1,
"name": "Widget #1"
},
{
"id": 2,
"name": "Widget #2"
},
{
"id": 3,
"name": "Widget #3"
}
]
}
Just an array of products (no meta data):
[
{
"id": 1,
"name": "Widget #1"
},
{
"id": 2,
"name": "Widget #2"
},
{
"id": 3,
"name": "Widget #3"
}
]
ReSTful APIs are consumed primarily by other systems, which is why I put paging data in the response headers. However, some API consumers may not have direct access to the response headers, or may be building a UX over your API, so providing a way to retrieve (on demand) the metadata in the JSON response is a plus.
I believe your implementation should include machine-readable metadata as a default, and human-readable metadata when requested. The human-readable metadata could be returned with every request if you like or, preferably, on-demand via a query parameter, such as include=metadata or include_metadata=true.
In your particular scenario, I would include the URI for each product with the record. This makes it easy for the API consumer to create links to the individual products. I would also set some reasonable expectations as per the limits of my paging requests. Implementing and documenting default settings for page size is an acceptable practice. For example, GitHub's API sets the default page size to 30 records with a maximum of 100, plus sets a rate limit on the number of times you can query the API. If your API has a default page size, then the query string can just specify the page index.
In the human-readable scenario, when navigating to /products?page=5&per_page=20&include=metadata, the response could be:
{
"_metadata":
{
"page": 5,
"per_page": 20,
"page_count": 20,
"total_count": 521,
"Links": [
{"self": "/products?page=5&per_page=20"},
{"first": "/products?page=0&per_page=20"},
{"previous": "/products?page=4&per_page=20"},
{"next": "/products?page=6&per_page=20"},
{"last": "/products?page=26&per_page=20"},
]
},
"records": [
{
"id": 1,
"name": "Widget #1",
"uri": "/products/1"
},
{
"id": 2,
"name": "Widget #2",
"uri": "/products/2"
},
{
"id": 3,
"name": "Widget #3",
"uri": "/products/3"
}
]
}
For machine-readable metadata, I would add Link headers to the response:
Link: </products?page=5&perPage=20>;rel=self,</products?page=0&perPage=20>;rel=first,</products?page=4&perPage=20>;rel=previous,</products?page=6&perPage=20>;rel=next,</products?page=26&perPage=20>;rel=last
(the Link header value should be urlencoded)
...and possibly a custom total-count response header, if you so choose:
total-count: 521
The other paging data revealed in the human-centric metadata might be superfluous for machine-centric metadata, as the link headers let me know which page I am on and the number per page, and I can quickly retrieve the number of records in the array. Therefore, I would probably only create a header for the total count. You can always change your mind later and add more metadata.
As an aside, you may notice I removed /index from your URI. A generally accepted convention is to have your ReST endpoint expose collections. Having /index at the end muddies that up slightly.
These are just a few things I like to have when consuming/creating an API.
I would recommend adding headers for the same. Moving metadata to headers helps in getting rid of envelops like result , data or records and response body only contains the data we need. You can use Link header if you generate pagination links too.
HTTP/1.1 200
Pagination-Count: 100
Pagination-Page: 5
Pagination-Limit: 20
Content-Type: application/json
[
{
"id": 10,
"name": "shirt",
"color": "red",
"price": "$23"
},
{
"id": 11,
"name": "shirt",
"color": "blue",
"price": "$25"
}
]
For details refer to:
https://github.com/adnan-kamili/rest-api-response-format
For swagger file:
https://github.com/adnan-kamili/swagger-response-template
As someone who has written several libraries for consuming REST services, let me give you the client perspective on why I think wrapping the result in metadata is the way to go:
Without the total count, how can the client know that it has not yet received everything there is and should continue paging through the result set? In a UI that didn't perform look ahead to the next page, in the worst case this might be represented as a Next/More link that didn't actually fetch any more data.
Including metadata in the response allows the client to track less state. Now I don't have to match up my REST request with the response, as the response contains the metadata necessary to reconstruct the request state (in this case the cursor into the dataset).
If the state is part of the response, I can perform multiple requests into the same dataset simultaneously, and I can handle the requests in any order they happen to arrive in which is not necessarily the order I made the requests in.
And a suggestion: Like the Twitter API, you should replace the page_number with a straight index/cursor. The reason is, the API allows the client to set the page size per-request. Is the returned page_number the number of pages the client has requested so far, or the number of the page given the last used page_size (almost certainly the later, but why not avoid such ambiguity altogether)?
just add in your backend API new property's into response body.
from example .net core:
[Authorize]
[HttpGet]
public async Task<IActionResult> GetUsers([FromQuery]UserParams userParams)
{
var users = await _repo.GetUsers(userParams);
var usersToReturn = _mapper.Map<IEnumerable<UserForListDto>>(users);
// create new object and add into it total count param etc
var UsersListResult = new
{
usersToReturn,
currentPage = users.CurrentPage,
pageSize = users.PageSize,
totalCount = users.TotalCount,
totalPages = users.TotalPages
};
return Ok(UsersListResult);
}
In body response it look like this
{
"usersToReturn": [
{
"userId": 1,
"username": "nancycaldwell#conjurica.com",
"firstName": "Joann",
"lastName": "Wilson",
"city": "Armstrong",
"phoneNumber": "+1 (893) 515-2172"
},
{
"userId": 2,
"username": "zelmasheppard#conjurica.com",
"firstName": "Booth",
"lastName": "Drake",
"city": "Franks",
"phoneNumber": "+1 (800) 493-2168"
}
],
// metadata to pars in client side
"currentPage": 1,
"pageSize": 2,
"totalCount": 87,
"totalPages": 44
}
This is an interessting question and may be perceived with different arguments. As per the general standard meta related data should be communicated in the response headers e.g. MIME type and HTTP codes. However, the tendency I seem to have observed is that information related to counts and pagination typically are communicated at the top of the response body. Just to provide an example of this The New York Times REST API communicate the count at the top of the response body (https://developer.nytimes.com/apis).
The question for me is wheter or not to comply with the general norms or adopt and do a response message construction that "fits the purpose" so to speak. You can argue for both and providers do this differently, so I believe it comes down to what makes sense in your particular context.
As a general recommendation ALL meta data should be communicated in the headers. For the same reason I have upvoted the suggested answer from #adnan kamili.
However, it is not "wrong" to included some sort of meta related information such as counts or pagination in the body.
generally, I make by simple way, whatever, I create a restAPI endpoint for example "localhost/api/method/:lastIdObtained/:countDateToReturn"
with theses parameters, you can do it a simple request.
in the service, eg. .net
jsonData function(lastIdObtained,countDatetoReturn){
'... write your code as you wish..'
and into select query make a filter
select top countDatetoreturn tt.id,tt.desc
from tbANyThing tt
where id > lastIdObtained
order by id
}
In Ionic, when I scroll from bottom to top, I pass the zero value, when I get the answer, I set the value of the last id obtained, and when I slide from top to bottom, I pass the last registration id I got