I'm trying to create a CloudFront distribution (using Serverless.js) that has two origins (both S3 buckets). There's a default origin that serves the public website, and an origin for paths starting with /attachments. That second origin has a viewer-response Lambda#Edge function that handles some authentication.
The current implementation is deployed using some homegrown scripts and a JSON CloudFormation template, which works. I'm trying to get rid of those idiosyncratic scripts, however, and standardize on Serverless.js (which has at least has the benefit of being a standard tool).
However, I'm having difficulty bending Serverless.js to my will here. Here's the resoruces section of my serverless.yml file (the relevant bits, at any rate):
resources:
Resources:
CloudFrontDistribution:
Type: AWS::CloudFront::Distribution
Properties:
DistributionConfig:
// all the usual properties...PriceClass, Enabled, etc omitted for brevity
DefaultCacheBehavior:
TargetOriginId: AppBucket
Origins:
- DomainName: foo-attachments.s3.amazonaws.com
Id: AttachmentsBucket
- DomainName: foo-app.s3.amazonaws.com
Id: AppBucket
That part works fine, and the origins are created correctly. Attaching the viewer-response function is where things start to go sideways:
functions:
viewerRequest:
handler: viewerRequest.authorize
events:
- cloudFront:
eventType: viewer-response
pathPattern: '*-resources/*'
origin: ????????
The official documentation seems to indicates that you specify origins with a URI, like s3://foo-attachments.s3.amazonaws.com. When I try that, I get CF errors, and it's clear why. If I look at the actual generated update template (./serverless/foo-template-update-stack.json), this is what I see (only the relevant bits):
"CloudFrontDistribution": {
"Type": "AWS::CloudFront::Distribution",
"Properties": {
"Origins": [
{
"Id": "AttachmentsBucket",
"DomainName": "foo-attachments.s3.amazonaws.com"
}
],
"CacheBehaviors": [
"TargetOriginId": "s3/foo-attachments.s3.amazonaws.com"
Note the mangled TargetOriginId. What it should be is AttachmentsBucket (at least that's what would work in the old homegrown scripts). Note that it also mangles the URI, which is weird because that's what it seems to expect.
I've tried:
Specifying the entire second origin in the function (got both Serverless.js and CloudFormation errors)
Using AttachmentsBucket; Serverless.js complains, and what gets written into the update template is custom/NullAttachmentsBucket (clearly it wants a URI)
Half a dozen other things I can't remember.
Serverless.js would make this deployment a lot more idiomatic -- and I like that it handles the fussy aspects of updating Lambda#Edge functions -- but I can't make it do what I want!
Related
I have the following resources in my system 1. Services 2. Features where a feature has the following JSON structure,
{
id: "featureName",
state: "active",
allowList: [serviceID1, serviceID2],
denyList: [serviceID3, serviceID4]
}
I am trying to update the allowList or denyList which consists of serviceIDs and thinking of using PATCH method to do it like below,
/features/{featureId}/allowlist
/features/{featureId}/denylist
/features/{featureName}/state/{state}
My first question is should I even include allowlist, state, denylist in the url as my resources are services and features, not the allowlist or denylist.
How should the rest endpoint look like?
After reading thread mentioned below I was thinking about restructuring urls as below,
/features/{featureId}
[
{ "op": "add", "path": "/allowList", "value": [ "serviceA", "serviceB"]},
{ "op": "update", "path": "/state", "value": false}
]
Lastly, the use of PATCH even justified here? or there is any better way to design the api.
Note: I have got some help from the thread REST design for update/add/delete item from a list of subresources but have not used patch often.
How should the rest endpoint look like?
The URI that you use to edit (PUT, PATCH) a resource should look the same as the URI that you use to read (GET) the resource. The motivation for this design is cache-invalidation; your successful writes automatically invalidate previously cached reads of the same resource (same URI).
Lastly, the use of PATCH even justified here? or there is any better way to design the api.
In this example, the representation of the document is small compared to the HTTP headers, and the size of your patch document is close to the size of the resource representation. If that's the typical case, I'd be inclined to use PUT rather than PATCH. PUT has idempotent semantics, which general purpose components can take advantage of (for example, automatically resending requests when the response to an earlier request has been lost on the network).
GET /features/1 by user1
PUT /features/1 //done by user 2
PUT /features/1 //done by user1
the PUT by user2 will not be visible for user1 and user1 will make an update on the old object's state (with id=1) what can be done in this situation?
Conditional Requests.
You arrange things such that (a) the GET request from the server includes validators that identify the representation (b) the server responds 428 Precondition Required when the request lacks conditional headers (c) the clients know to read the validators from the resource metadata, and use the correct condition headers when submitting the PUT request (d) the server knows to compare the validator to the current representation before accepting the new representation.
I'm looking to incorporate Test-AzureRmResourceGroupDeployment into a build pipeline so I know before deployment that the template / parameters has got any major problems.
However I'm finding if I used nested deployments it provides no validation to the nested deployment whatsoever, I can have a bad templateLink -> uri with incorrect variables even in the URI and it's still validating as successful.
I have tried with a local template, a template uri, with/without hashed parameters and parameters file just in case.
I assume underneath the AzureRM powershell is using the Resource Manager API, it doesn't hint to what the validate actually does with nested templates: https://learn.microsoft.com/en-us/rest/api/resources/deployments/validate
Anything I've missed? Any suggestions on how to validate the entire template, do I need to parse the nested templates and some how re-construct the parameters from json and do the sub-deployments by hand (which would be a shame)?
Reading a forum post from a Microsoft Employee in the Resource Manager team (a private forum so unfortunately cannot provide a link), it appears Test-AzureRmResourceGroupDeployment does "template expansion" which as 4c74356b41 has also kindly pointed out - surely the nested template validation should work...
So further experimentation has led to finding a limitation in the validation, see below for an example. If there is a variable missing entirely in a nested deployment it doesn't appear to be picked up as a validation warning in the parent template, and also appears to interfere with the template expansion leading to the nested template to be ignored also.
If "parameters": { "missing" : "[variables('PURPOSEFULLY_MISSING')]" } is removed then the template is validated as normal and the nested template also.
Snippet of the overall template for just the nested resources:
"resources": [
{
"name": "[variables('deploymentName')]",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2018-05-01",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[variables('deploymentUri')]",
"contentVersion": "1.0.0.0"
},
"parameters": { "missing" : "[variables('PURPOSEFULLY_MISSING')]" }
}
}
],
that is not true, it will validate nested deployment even if you gate it with condition: false, so you are doing something wrong, we would need to look at the template and how you are calling the cmdlet to understand whats going on
as to the validation: there is no real way to validate the deployment works (test-azurermresourcegroupdeployment is just garbage, extremely low value). the only way to validate it - deploy it.
I've been taking several hours fiddling this Resource Mapping for no luck. I have put a new node in /etc/map/http named GoogleAnalyticsMap. It had the following properties:
Name: jcr:primaryType
Type: Name
Value: sling:Mapping
Name: sling:internalRedirect
Type: String
Value: /content/dam/website/Common/my_google_analytics_key.html
Name: sling:match
Type: String
Value: [^/]+/[^/]+/my_google_analytics_key.html
Funnily, when i checked to /system/console/jcrresolver or Adobe CQ Web Console Sling Resource Resolver. It resolves into the following
Pattern: ^http/[^/]+/[^/]+/my_google_analytics_key.html
Replacement: /content/dam/website/Common/my_google_analytics_key.html
Redirect: internal
I have no idea how they could register ^[^/]+/[^/]+/welcome$ as shown in the example and did not have the 'http' there. Nevertheless, whenever i try to hit the pattern locally (i.e. http://localhost:4502/my_google_analytics_key.html) it only shows resource not found. Also checked through the Configuration test section that available in Adobe CQ Web Console Sling Resource Resolver it says that it can't resolve. Did i miss something?
, type=sling:nonexisting, path=/my_google_analytics_key.html, resource=[NonExistingResource, path=/my_google_analytics_key.html]
Thank you for the help.
p.s. i've checked the Sling documentation also, however, it baffles me that i can't add sling:alias, perhaps its also something to do with the sling version. Nevertheless, i should ask this as second question later, once the sling:match is working properly.
Edit: fiddling a bit, changing the sling:match to localhost.4502/google.html does let it to redirect localhost:4502/google.html to the desired DAM item. Well, i'm still asking for 'domain-free' matching though.. so i have no trouble when using the mapping for all environment i have..
Edit, Updates:
Apparently due to me put the new sling:Mapping under http folder it will always prefix the path with http. Perhaps the example is meant to be put not under http folder.
Next item is, it will always assume the matching item will be protocols, domain, and port separated by dots (yes, even \. is not a valid input and CQ or rather sling will sanitize it somehow). Something like *.(4502|4503|80|443)/my_google_analytics_key.html will accept any domains and those designated ports. I do wonder if it really the case, since they says regex but it does not feel so much regex.
Some says that sling:internalRedirect should be a String[] - yes i know the docs says String. I have not test this further and take it as it is..
Let's say I have a REST API adhering to basic HATEOAS principles. Items belong to a User.
GET /item/13
{
id: 13,
name: 'someItem',
type: 'someType',
_links: [
{
rel: 'user',
href: '/user/42'
}
]
}
Now I need a way to change the user for a given item. Using either a PUT or a PATCH, which is the preferable way of performing that modification?
Establish the new relation by setting the id of the new linked resource as a simple property in the JSON body
PATCH /item/13
{
userId: 43
}
Establish the new relation by having the client pass the link itself as the input
PATCH /item/13
{
_links: [
rel: 'user',
href: '/user/43'
]
}
I usually think of links as read-only representations of relations stored in other formats (such as id:s to other resources), returned from GET calls. It doesn't feel very natural to me to have links as input to POST/PUT/PATCH calls, and the fact that links is an array makes it even stranger (should you be able to update all links? One single link?), but I have seen it suggested in various articles. Is there a best practice? What would be the benefits of using the links approach?
The point of REST is (at least one of them) is to make everything visible through a standard interface. In other words, if the 'relations' are a thing, than it too should have its own resource.
The API should also be more descriptive. This might be subjective, and I don't know all the details of your model/design, but 'items' don't have 'links'. 'Items' might instead have a single 'owner'. If this is the case, it might look something like:
GET /item/123/owner
So POSTing or PUTing an URL of a user (or some simple representation) would 'change' the owner of the item. It might be not allowed to DELETE the owner, depending on if the model allows unowned items.
Note, that the representation under "/item/123" would in this case have to link to "/item/123/owner", since the client only follows links it gets from the server.
So, think about what are important 'things', all of those should have a resource. Also, try to add as much 'meaning'/semantics as you can. The relation should not be called 'user', it should be called 'owner' (or whatever the meaning should be in your model).
I'm working a REST-ful API in which resources which are fairly interrelated. Resources reference each other, and these references may be created or deleted. I'm a little uncertain how to support associating resources together when they reference each other using hyperlinks.
A simple example follows with two resources, A and B.
Resource A:
name: integer
list_b: [list of resource B]
Resource B:
id: integer
description: String
Now, A does not include B in its document, but rather links to it. When using hypermedia, it might look something like this:
Resource A:
{
id: 1,
list_b: [
{ id: 1, href: "https://server/api/b/1" },
{ id: 2, href: "https://server/api/b/2" }
]
}
If a user wants to add or delete one of the B references in A's list, how do they do so, taking into account the presence of the hyperlink? I want the user to be able to update the entire A resource in one PUT operation, but nothing in the output indicates which value for B is required. It make sense to me for the user to perform PUT with content like this:
Resource A:
{
id: 1,
list_b: [
{ id: 1, href: "https://server/api/b/1" },
{ id: 2, href: "https://server/api/b/2" },
{ id: 3 },
]
}
and receive the updated resource (in the response) like this:
Resource A:
{
id: 1,
list_b: [
{ id: 1, href: "https://server/api/b/1" },
{ id: 2, href: "https://server/api/b/2" },
{ id: 3, href: "https://server/api/b/3" }
]
}
My concern is that the user won't necessarily know what to include in the resource when updating resource A's list_b.
When dealing with hyperlinks from one resource to another, how should creates and updates work? Should clients be allowed to update part of the link (the id), or should they be required to update both parts of the link?
Note: I know another approach might be exposing a sub-url for resource A. It could expose list_b as a resource which is operable via HTTP (allowing clients to use POST, PUT, and DELETE on the list resource itself). But this seems less reasonable when A contains multiple references to other resource types. Each field which references another would potentially require a sub-url, which, if there are 10+ fields, is unwieldy, and requires multiple HTTP requests to update the resource.
HATEOAS connects resources together in a RESTful interface, and it's not clear here whether or not the subsidiary objects you're describing really make sense as independent resources. The "AS" part of HATEOAS reminds us of the role that Web pages play as "resources" in a Web application. Each Web page is really an interactive representation of application state (the "application" in this case being a classical, multiple-page Web application), and the hyperlinks to other resources provide the user with transitions to other application states.
A RESTful Web API, having JavaScript code rather than human beings as its client, is naturally data-access-oriented, so few if any of its resources take the form of "application state," per se. In a tradition Web application, you can draw a state transition diagram and clearly see the connections among states, and thus among resources. In a RESTful API, the boundaries among passive data resources are motivated more by the efficiencies of client/server interactions and other subtle forces.
So do your subsidiary objects ("B") here really need to be represent as first-class resources? Are there instances where the front end will enumerate or otherwise access them independent of the aggregates in which they participate ("A")?
If the answer is "no," then they obviously shouldn't be represented hyptertextually in the "A" structure. I presume that the answer is "yes," however, and that you also have good reason to offer all of the other subsidiary objects to which you refer as independent resources. In this case, there's some amount of interface work in the form of routes and controllers that is necessary to support all of those resources no matter what, because your application is presumably providing a means to manipulate them each on their own, or at least query them (through hyperlinks such as those in your example).
This being the case, a POST to the path representing your collection of "B" objects (e.g., "server/api/b") can return a URL in the response's "location" header value as POSTs that create new resources are supposed to do. When your user interactively adds a new "B" to a list belonging to an "A" on your Web page, your front end can first POST the new "B," getting its URL back through the location header on success. It can then incorporate that link into the list representation inside its "A" object before PUTting the updated "A."
The ID value is a bit of a wrinkle, as you'll be tempted to break the encapsulation of the back end by extracting the ID value from the text of the URL. True HATEOAS zealots make their RESTful APIs produce obfuscated, hashed or otherwise unintelligible URLs specifically to frustrate such encapsulation-breaking on the part of clients. Better that the POST of the new "B" object returns a complete representation of the new "B" object, including its ID, in its response body, so that the client can reconstitute the full object and extract the ID from it, thus narrowing the coupling to the resource itself and not the details of the RESTful interface through which it is obtained.
You should also look at the LINK method:
LINK /ResourceA/1 HTTP/1.1
Link: <http://example.com/ResourceB/3>; rel="list_b"
...
204 Yeah Fine, Whatever
This tells /ResourceA/1 to link to /ResourceB/3 using the relationship "list_b".