Have a collection of cloudformation templates in a parent-child relationship and want to pass an AWS::IAM::Role into the parameters of a child stack and use GetAttr to get the Arn.
This fails validation because can only call GetAttr on resources, not on parameters.
Anyone know/guess why this is designed in this way?
It's not a problem as it can be worked around by just passing the Arn into the stack, I'm just curious really
What Fn::GetAttr and Parameters are trying to do in AWS CloudFormation is fundamentally different. As per AWS docs:
The intrinsic function Fn::GetAtt returns the value of an attribute
from a resource in the template.
[1] http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html
You can use the optional Parameters section to pass values into your
template when you create a stack. With parameters, you can create
templates that are customized each time you create a stack.
[2] http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html
I believe your confusion is stemming from the fact that you're trying to think of this in terms of the object-oriented/some other programming paradigm, where Resources and Parameters are some kind of objects and Fn::GetAttr is a generic function which retrieves the value of a reference that's passed in as an argument.
In my case, I wanted to access the resource attributes like ARN and Name in the nested stack. If I pass resource string and use GetAtt to get these, I dont need to pass them as two parameters. With this limitation, I had to pass them as two parameters.
Related
Using Data Fusion Argument Setter, I've defined all parameters in it for a reusable pipeline. While executing it, I provide runtime arguments for some parameters which are different from default arguments provided in the JSON URL embedded in Argument Setter.
But a number of times, the pipeline ends up taking the default values from Argument Setter URL instead of Runtime Arguments causing failures.
This behavior is not consistent in every pipeline I create - which confirms that Runtime arguments are supposed to supersede any prior value defined for an argument.
The workarounds I use is by deleting the plugin and re-adding it for every new pipeline. But that defeats the purpose of creating a re-usable pipeline.
Has anyone experienced this issue ?
Current Runtime Options
This wiki https://cloud.google.com/data-fusion/docs/tutorials/reusable-pipeline provides the sample of how to create re-usable pipeline using Argument Setter. From there, it seems like the runtime arguments was used to notify the data fusion pipeline to use the macro from Argument Setter URL. Argument Setter is a type of Action plugin that allows one to create reusable pipelines by dynamically substituting the configurations that can be served by an HTTP Server. It looks like no matter how you change the runtime arguments, as long as long the same marco can be read when pipeline is running, the arguments will be override.
Using Argument setter by supplying the parameter value I want to make the Datafusion pipeline as resuable. As said by many other answer's have tried implementing using the cloud reusable pipeline example given in Google guide.I was not able to pass the parameter Json file.So how to create the API to that parameter Json file stored in Google storage.Please explain the values to be passed to Argument setter like URL,Request response etc., If any one of you had implemented in your projects.
Thank you.
ArgumentSetter plugin reads from a HTTP endpoint and it must be publicly accessible as is depicted within the GCP documentation. Currently, there is not a way to read from a non-public file stored in GCS. This behavior has been reported in order to be improved to CDAP through this ticket.
Can you please provide what you've tried so far and where you're stuck?
The URL field in argument setter would contain the API endpoint you're making a call to. Make sure you include any headers your call would need like Authorization, Accept etc.
If you're having issues with argument setter a good check is to use Curl or any other tool to make sure you're able to talk to the endpoint you're trying to use.
Here's some documentation about Argument setter: https://github.com/data-integrations/argument-setter
Define a JSON file with appropriate name/value pairs. Upload it in a GCS bucket - make it public by changing permissions (add "allUsers" in permissions list). When you save it, the file will say "Public to Internet"
Copy the https path to the file and use it in Arguments Setter. If you're able to access this path from curl/ your browser, Argument Setter will be able to do too..
There are other problems I've encountered while using Argument Setter though - the pipe doesn't supersede runtime arguments over default values provided in the URL many a times, specially when the pipe is duplicated.
To make file public
You have to make your bucket public, currently there is no other way.
gsutil iam ch allUsers:objectViewer gs://BUCKET_NAME
I have a Cloudformation template that uses a custom Macro to generate part of the template. The lambda for the Macro uses template parameters (via the templateParameterValues field in the incoming event) to generate the template fragment.
When I change the Cloudformation Stack's parameters, I get an error:
The submitted information didn't contain changes. Submit different information to create a change set.
If I use the CLI I get a similar error:
An error occurred (ValidationError) when calling the UpdateStack operation: No updates are to be performed.
The parameters I am changing are only used by the Macro, not the rest of the template.
How can I make Cloudformation reprocess the template with the macro when I update these parameters?
After working with AWS Support I learned that you must supply the template again in order for the macro to be re-processed.
Even if it is the same exact template it will cause the macros to be reprocessed.
You can do this via the Console UI (by uploading the template file again) or the CLI (by passing the template / template URL again).
I recently came across this when using the count macro to create instances.
I found i was able to modify just the parameters used just by the macro by moving that part of the template to a nested stack and passing the parameters through.
It does involve a bit more work to setup by having the separate stacks, but it did allow me to modify just the parameters of the parent stack how i wanted.
I'm trying to figure out the differences between GetAtt vs Ref vs ${}.
From what I understand you use GetAtt to refer to objects within the same template only and Ref can be used to refer to anything? Plus when would I use ${}?
Ref can be used for two things:
To return the value of a parameter that you passed in via the parameters section of the template.
When you ref the logical ID of another resource in your template, Ref returns what you could consider as a default attribute for that type of resource. So using ref for an EC2 instance will return the instance ID, and Ref'ing an s3 bucket resource will return the bucket name. You can look at the bottom of each CloudFormation resources page in the AWS docs to see what this value will be.
GetAtt is essentially the same as the 2nd function of Ref above, it also returns an attribute of the resource that you created within your resource, but while ref returns only a default attribute, GetAtt allows you to choose from different attributes to return.
For example, GetAtt for an EC2 instance gives you the option to return the AvailabilityZone, PrivateDnsName, PublicDNSName, etc of an instance - whereas Ref will only return the InstanceID. The different attributes you can return are different per resource type. You can also look at the bottom of each CloudFormation resources page in the AWS docs to see what attributes you can all return.
${} is another way to reference parameters passed in through the parameters section of the template.
All of this is in the AWS documentation though.
CloudFormation Ref and GetAtt cheatsheet is a very handy webpage that can be used to quickly reference what you can get via a Ref and GetAtt for most CloudFormation resources. You can use a Ref for a logical resource's default value (including inside a Sub using the ${NAME} method) as well as for a parameter of the stack. GetAtt is only useful for logical resources of the stack.
This drawing may help clarify the uses:
I'm attempting to dynamically use the Microsoft.Web.Administration namespace within a powershell cmdlet.
Is there a way to add a variable into a Namespace.Class declaration.
I'm using the namespace [Microsoft.Web.Administration] and it's class [Microsoft.Web.Administration.ApplicationPool], it has a property under it 'Recycling' that you can access using a GetProperty method.
[Microsoft.Web.Administration.ApplicationPool].GetProperty("Recycling")
If you spit out the PropertyType of the above using this
[Microsoft.Web.Administration.ApplicationPool].GetProperty("Recycling").PropertyType.FullName
You get this result,
Microsoft.Web.Administration.ApplicationPoolRecycling
which is another class. I now want to access this class dynamically within the cmdlet. How do access this class from within the code, I want the code to dynamically discover the new class of the object and then access that class. But I can't find a way to accomplish this.
Psuedocode for what I'm trying
[System.Reflection.Assembly]::LoadFrom( "C:\windows\system32\inetsrv\Microsoft.Web.Administration.dll" )
$appPoolProperty = [Microsoft.Web.Administration.ApplicationPool].GetProperty($Property)
$subClassName = $appPoolProperty.PropertyType.FullName
#This is the step I'm lost on
$subClass = New-Object $subClassName
So I'm not sure if there's a way to have a TypeName for an object put in as a string value and I can't seem to find a way to cast the variable as anything else. Sorry if this is remedial, I'm a basement grown coder and just learn things as I go.
EDIT: As requested by Mathias below in the comments, an overview of what I'm trying to achieve.
I have a script that automates the configuration of many IIS components. At this time I'm attempting to add validation to the cmdlet Set-AppPoolConfiguration that I've created that allows a configuration to be fed into the cmdlet that configures an AppPool (this is used to deploy websites/weapplications throughout a distributed web environment). Utilizing the details inside the Microsoft.Web.Administration I'm able to get enum values, as well as types of the many configuration components of an AppPool. This is saving me time to where I don't have to hard code in the data types and can just dynamically discover them and do validation on the values when the specific configuration element is chosen.
For example. If an AppPool Recycle schedule is need to be configured, I need to validate it against a TimeSpan data type. If it is not a TimeSpan data type issues will arise when being added to the ScheduleCollection. So I'm looking to validate the value provided before attempting to add it.
Since there are many AppPool configuration elements, I don't want to have to create a massive switch or if/elseif chain that checks each configuration element and statically dictate what data type it is for validation. I want the class itself to dynamically provide this information to simplify the code.
I can get the majority of these data types by simply accessing the property chain within the namespace. For example, if you want to know what type is required for the QueueLength, use this:
[Microsoft.Web.Administration.ApplicationPool].GetProperty("QueueLength").PropertyType.Name
And you'll get Int64.
It's also extremely useful for getting enums.
[Microsoft.Web.Administration.ApplicationPool].GetProperty("ManagedPipelineMode").PropertyType.GetEnumNames()
However attempting this with Schedule and you run into a small issue as it returns ScheduleCollection. This is true of any of the configuration elements that are part of a collection.
[Microsoft.Web.Administration.ApplicationPool].GetProperty("Recycling").PropertyType.GetProperty('PeriodicRestart').PropertyType.GetProperty('Schedule').PropertyType.Name
However the knowledge that the schedule item inside the ScheduleCollection is only accessible from within the Schedule class. My code currently checks to see if it is a collection, and then if it is, it is attempting to access that collection/class and get the information that is required. To find out that schedule is a TimeSpan you have to access it's specific class instance:
[Microsoft.Web.Administration.Schedule].GetProperty('Time').PropertyType.Name
Now AppPools are simple, there's only a single collection group that is normally edited, so hard coding in that if you're attempting to set a new recycle schedule it will be a TimeSpan isn't that big of a deal, however when we move over to WebSite/WebApplication configurations, it becomes more tedious to statically declare data types for each configuration element that is part of a collection, and becomes more useful to try and discover these dynamically based on the configuration element selected.
This was my initial approach, I just included the above for clarity. I'm going to step back and take another look at how to attack this as this does not appear to be as easy as I had hoped, I'll post my solution here.
You can retrieve the constructor from the type literal and invoke it like so:
$type = [Microsoft.Web.Administration.ApplicationPoolRecycling]
$ctor = $type.GetConstructor('NonPublic,Instance',$null,#(),$null)
$appPoolRecyclingInstance = $ctor.Invoke($null)
Though there may be a way to do the above, in order to complete the updates to my cmdlet and proceed forward with my project I went a hybrid route.
The reason why I started exploring the [Microsoft.Web.Administration] namespace was that it provided information on the data types where the typical way I was manipulating IIS settings failed using Get/Set/Add-WebConfigurationProperty.
The specific failure is in reporting back valid data types for Enums. Take for instance ProcessModel.IdentityTypes. There is a set of valid entries for an IdentityType. However the following doesn't provide you with those valid types, so you either have to create static instances of them inside your cmdlet, or some other external data source, whereas I wanted Microsoft to provide them either through IIS itself, or through the classes attached to these configuration elements so the cmdlet would need minimal updating as IIS versions/configuration elements change.
This code returns Boolean
(Get-WebConfigurationProperty -pspath 'MACHINE/WEBROOT/APPHOST' -filter "system.applicationHost/applicationPools/add[#name='$AppPool']" -name "AutoStart").Value.GetType().Name
However, this code returns string, which is true, but I needed to know that it is an enum and I needed to know the proper values.
(Get-WebConfigurationProperty -pspath 'MACHINE/WEBROOT/APPHOST' -filter "system.applicationHost/applicationPools/add[#name='$AppPool']" -name "processModel").identityType.GetType().Name
Using a mix of both the Namespace and the traditional Get-WebConfigurationProperty commands I can successfully now test for data types, as well as gather proper values for enums dynamically in code. If there is interest I can post the entire validation code here that I'm using for this cmdlet.