I'm trying to construct a variable by inserting values from another variable environment variable, a bit like a template.
Similar to this example from Octopus...
The ConnectionString variable is a template which uses Server and Database variables.
Above the examples are because Octopus is using different values per environment. In my case I'd like to keep the template as an unprotected variable so I can see it and have the inserted variable protected because it contains sensitive information.
I've tried using macro syntax ($(Server)) and runtime expression syntax ($[Server]), neither of which seem to replace the values at build time.
Expression syntax ${{Server}} gives me an error "bad substitution" which implies that there's a good substitution but I'm missing something.
This is not supported. We could not use dynamic password when use a service endpoint. If you want to change the password, you need update the endpoint directly.
Besides service endpoint is independent, should not dependent on other variable or variable group.
No need to use a protected SECRET_REPLACED_AT_BUILD variable with password to protect it. Password in the service endpoint is also protected.
You could refer the format of Environment Variables with Credential Provider in our official link.
VSS_NUGET_EXTERNAL_FEED_ENDPOINTS: Json that contains an array of service endpoints, usernames and access tokens to authenticate
endpoints in nuget.config. Example:
{"endpointCredentials": [{"endpoint":"http://example.index.json", "username":"optional", "password":"accesstoken"}]}
Related
I have a rest API with symfony and two local environments:
dev (env.local)
test (env.test.local)
My Behat tests run on the test environment with a dedicated database.
But when I call a resource with Postman, it always uses the dev environment (all new resources are created on the dev database).
I tried to create a new environment on Postman with all the variables and values of my dev.test.local (included the test database access).
But it doesn't work. What am I missing or doing wrong?
Thanks for any help :)
In order to use the different variables for each environment selected, you need to make references to the variable names in your requests using the {{..}} syntax.
I don't know what value you have for the APP_DEV variable but that would be used in the URL like this:
GET http://{{APP_DEV}}/api/users
The way to make these the most effective is to create the variables with the same name in the environment files and then make each value different under the different environment files.
For example:
GET http://{{DOMAIN}}/api/users
If you have a DOMAIN key in each environment but change the value of that variable. When you switch between the two environments, it's going to use the variable key but use a different value for dev and test.
https://learning.postman.com/docs/postman/variables-and-environments/variables/
I am having problems using SSM valueForStringParameter method in CDK. It's working the first time I deploy the stack, but it is not picking up updates to the parameter value when I redeploy the stack because CloudFormation template hasn't changed and so CloudFormation thinks there were no updates, even if SSM parameter has changed.
For the context, I am deploying stack via CodePipeline, where I run cdk synth first, and then use CloudFormationCreateUpdateStackAction action to deploy template.
Anyone knows how to work around that? The only other option that I know will work is to switch to a custom resource lambda that calls SSM and returns value using aws-sdk, but that feels like a overly complicated option.
Update 1
I cannot use ValueFromLookupbecause value is only updated at runtime as part of cloudformation deployment by another stack (I deploy both stacks in CodePipeline, in 2 different regions), so synthesis time lookup would result in stale value.
All the valueOf* and from* methods work by adding a CloudFormation parameter. As you figured out already, changing the parameter value does not change the template and no change will be triggered.
What you probably want to use instead is the method valueFromLookup. Lookups are executed during synth and the result is put into the generated CFN template.
ssm.StringParameter.valueFromLookup(this, 'param-name');
But be aware, lookups are stored in the cdk.context.json. If you have commited that file to your repo, you need to erase that key via cdk context -e ... before synth/diff/deploy.
Since you cannot use lookup functions and the most common way to pass config to cdk is through context variables, I can only suggest dirty workarounds.
For example, you could create a dummy parameter in your stack to bump every time there's deployment.
var deploymentId = new CfnParameter(this, "deploymentId", new CfnParameterProps() { Type = "String", Description = "Deployment Id" });
SetParameterValue(deploymentId, this.Node.GetContext("deploymentId").ToString());
and when you synthesize the CF, you could generate an ID:
cdk synth -c deploymentId=$(uuidgen)
If you can avoid the "environment agnostic" syth and you really need an immutable artifact to deploy across multiple environments, you could use the built package from your cdk, for example, the npm package containing your cdk. Therefore, you could deploy it in each environment by overwriting the context parameters instead of using ssm parameters store.
See https://docs.aws.amazon.com/cdk/latest/guide/get_ssm_value.html, you can use method valueFromLookup which gets you parameter store value at synthesis time, when value is different from previous one, this shall trigger CF stack update.
However, I was under impression that valueForStringParameter should work on updated ssm parameter values as well, based on https://aws.amazon.com/blogs/mt/integrating-aws-cloudformation-with-aws-systems-manager-parameter-store/ Example 2:
Using Argument setter by supplying the parameter value I want to make the Datafusion pipeline as resuable. As said by many other answer's have tried implementing using the cloud reusable pipeline example given in Google guide.I was not able to pass the parameter Json file.So how to create the API to that parameter Json file stored in Google storage.Please explain the values to be passed to Argument setter like URL,Request response etc., If any one of you had implemented in your projects.
Thank you.
ArgumentSetter plugin reads from a HTTP endpoint and it must be publicly accessible as is depicted within the GCP documentation. Currently, there is not a way to read from a non-public file stored in GCS. This behavior has been reported in order to be improved to CDAP through this ticket.
Can you please provide what you've tried so far and where you're stuck?
The URL field in argument setter would contain the API endpoint you're making a call to. Make sure you include any headers your call would need like Authorization, Accept etc.
If you're having issues with argument setter a good check is to use Curl or any other tool to make sure you're able to talk to the endpoint you're trying to use.
Here's some documentation about Argument setter: https://github.com/data-integrations/argument-setter
Define a JSON file with appropriate name/value pairs. Upload it in a GCS bucket - make it public by changing permissions (add "allUsers" in permissions list). When you save it, the file will say "Public to Internet"
Copy the https path to the file and use it in Arguments Setter. If you're able to access this path from curl/ your browser, Argument Setter will be able to do too..
There are other problems I've encountered while using Argument Setter though - the pipe doesn't supersede runtime arguments over default values provided in the URL many a times, specially when the pipe is duplicated.
To make file public
You have to make your bucket public, currently there is no other way.
gsutil iam ch allUsers:objectViewer gs://BUCKET_NAME
In my Octopus project, the first step launches a bunch of nested stacks implemented with cloudformation.
I need to share the outputs of the master stack launched from Octopus, how can I do that?
Thanks.
The output variables from the CloudFormation template will be available to later steps the same as any other Octopus output variable, this is mentioned in the first paragraph of the documentation page.
Output variables can be accessed a number of different ways, depending on where you are accessing them, for example, in Powershell they can be accessed via the parameters dictionary $OctopusParameters["Octopus.Action[Step Name].Output.VariableName"].
You can also access them using the Variable Binding syntax, #{Octopus.Action[Step Name].Output.VariableName}
More information about output variables is available in the docs.
I'm moving my configuration from using web.config transforms to being based on VSTS variables. I get process variables, you define a variable, pick an environment, and you're good to go. I also see "Variable Groups", these seem great, have KeyVault integration, and overall seem like a much better option.
But...I don't see a way to bind a Variable Group to a specific environment in my VSTS release process. I can't honestly see how these would be any use to me without this feature.
I've experimented with one workaround, but it didn't work. I tried:
Naming my variable group & variables with an environment prefix e.g.
Variable Group Name="Production ConnectionStrings"
Variable name="Production_LoggingConnectionString"
I thought once I linked the "Production_ConnectionStrings" variable, I could reference $(Production_LoggingConnectionString) from within a standard Process variable, but this didn't work.
I think I could come up with some powershell that would do something like the above and set variables, but this seems a bit too custom for me.
Does anyone else have an idea that I can use variable groups per environment, easily, without waiting around for VSTS to build this feature (if ever). Btw, if you want this feature, there is a suggestion here you can upvote: Make it possible to link a variable group to a specific environment in a release definition
This has now been implemented in VSTS variable groups as scopes. Go to your release definition -> Variables -> Variable Groups -> Link variable group, and you get the link window as below, where you can choose the scope to be either release or one or more of your environments!
I did not manage to find any release information on this feature, I just stumbled upon it as I was tweaking my releases.
I ended up using a powershell script to define my process variable based on the variable groups, it works great.
Let's say I want a variable named "LoggingConnectionString" and this has different values per environment
Steps:
Define a Variable group, e.g. "SharedLoggingVariables"
Inside this Variable group, define a variable/value for each environment, e.g. "LoggingConnectionStringDev", "LoggingConnectionStringProduction"
Back in your Process Variables for the Build/Release, make SURE you don't have a variable named "LoggingConnectionString", otherwise this will overwrite the value coming from the variable group
In your Release process, create a Powershell inline script at the beginning of the Agent with the following code
Param(
[string]$LoggingConnectionString
)
Write-Host "##vso[task.setvariable variable=LoggingConnectionString]$LoggingConnectionString"
Pass your variable group as an argument to this inline powershell, e.g.
-LoggingConnectionString "$(LoggingConnectionStringDev)"
The final powershell step should look something like this:
During release, the powershell will set your process variable from the variable groups. If powershell isn't an option for you, there are other options
No, there is no way to use variable Groups per environment.
As the user voice you linked, you can vote and follow up for the suggested feature.
The work around for now is using environment variables to overwrite the variables in variable Group.
Assume the variable LoggingConnectionString with the value Server=myDB in variable group need to be used both for Dev environment and staging environment. But for staging environment, it needs to use another value (such as Server=stageDB) from the variable LoggingConnectionString. So you can add the an environment variable LoggingConnectionString with the value Server=stageDB for staging environment.
When the variable $(LoggingConnectionString) is used in Dev environment, it will use the value (Server=myDB) defined in variable group.
When the variable $(LoggingConnectionString) is used in staging environment, since the variables both defined in environment variable and variable group, it will use the value (Server=stageDB) defined in environment variable.