I'm trying to import an existing azure-native StorageAccount into Pulumi.
I've used the command given in the docs and it works, but it creates a ton of properties with their default values. I'm trying to get rid of them but I can't seem to do it for these ones:
accessTier, allowBlobPublicAccess, allowSharedKeyAccess, enableHttpsTrafficOnly, enableNfsV3, encryption, identity, isHnsEnabled, networkRuleSet
They are all set to their default values, but when I remove them from my C# code Pulumi wants to update them.
I tried to remove them manually from the input in state file, but Pulumi still says it want to remove their <null> value.
What makes this worse is that this is not an in-place update, Pulumi wants to replace the StorageAccount with a new one, which is not ok for me.
To test, I created a new storage account from scratch (without those properties) and everything is just the same (C# code, actual properties of account, Pulumi JSON state) and it works fine... but nope, I can't remove those default from the imported account.
Anyone knows why / how to remove those default imported values?
Related
I am referencing an environment value from a Realm function as instructed here: context.values.get("appTwilioNumber")
I confirmed appTwilioNumber exists in our project: and that our project is assigned an environment:
Yet, when I call console.log('twilioNumberForEnv:', context.values.get("appTwilioNumber")); in our Realm function, I get twilioNumberForEnv: undefined.
EDIT 1: I have more info now--I logged out and logged back in (in case of multi-user sync issues), then exported my app from the Realm UI and the values folder is empty. Not sure if that is a separate bug, but updating in case this info is useful.
EDIT 2: the environment values are stored under environment, not under values. Edit 1 was a red herring. I do see appTwilioNumber in the exported app, but it still returns undefined in the Realm functions.
Wow... Mongo's documentation might be off.
In another function, I found this: context.environment.values.anotherEnvValue instead of context.values.get("appTwilioNumber") . So I updated my code to context.environment.values.appTwilioNumber, and it worked.
I did a CMD-f on both https://docs.mongodb.com/realm/values-and-secrets/define-environment-values/ and https://docs.mongodb.com/realm/values-and-secrets/access-a-value/ for ".environment", and it isn't on either page.
I'll follow up with Mongo, but please use context.environment.values.YOURENVVALUE instead of context.values.get("YOURENVVALUE") if you encounter this issue.
TL;DR: Search and replace placeholders in a text file with the decrypted values of secrets in a variable group.
I would like to use a PowerShell script to receive a variable group in a release pipeline and then iterate through the list, performing search-and-replace on a file being released.
The variables in the variable list are secrets so I want to overwrite the placeholders in the file with the decrypted value of the variables.
The values of the variables are environment specific, so I do not want to provide the values at build time and do not want to include the decrypted values in a stored artifact.
The file being search-replaced will be used in an execution at release time but will not be deployed to a host, so will be destroyed upon completion of the pipeline execution.
The Tokenization task from the Visual Studio Marketplace does this job well. You'll need to install it into your Azure DevOps organisation, it's available at https://marketplace.visualstudio.com/items?itemName=TotalALM.totalalm-tokenization
By default the Tokenization task uses double underscores to identify the placeholders. It will replace any text that that matches the name of a variable in your release definition as long as it's surrounded by double underscores.
So if you want to write the value of a variable called MySecretVariable into your file you'll need to add a place holder like __MySecretVariable__ into the file where ever you want that value to be written.
The Tokenization task will write any encrypted values into the file in plain text but in the release logs they will be obfuscated.
If you're storing your variables in a variable group just link that to the release definition and set the scope to the appropriate environment.
So, I've found that, as long as I have a Base64 token configured for Variable Groups (click 'Show all scopes' when creating a new PAT) then I can call GET https://dev.azure.com/{organization}/{project}/_apis/distributedtask/variablegroups/{groupId}?api-version=5.1-preview.1 to get the variable group I need.
The above, however, will not return a value for secrets, although there is a hack/workaround for this (involving multiple pipeline steps).
The advisable route is to create a Key Vault in Azure and perform the processing either in Powershell or code.
As I need the decrypted key values passed into my application via a repeated find-replace, I have implemented a Powershell script in one Release pipeline step and consume the output in the next step.
I have lots of URL values and their keys. But there is no way to batch import the variables and the "value" controls are also not text boxes in the Variables Group page to perform chrome browser extensions assisted find and replace.
If this is possible, what is the syntax to refer to the key?
As in, I have a variable App.URL : www.contoso.com.
I am using the key to substitute value in my next variable like this Login.URL : $(App.URL)\Login and this doesn't work.
GitHub link : https://github.com/MicrosoftDocs/vsts-docs/issues/3902#issuecomment-489694654
This isn't currently available, not sure if it will be. Can you create a task early in your pipeline that sets the variables you need in subsequent tasks/steps? This gives you more control as you can store the script along with your source. You could then use a pipeline variable for the environment you're in and let your script use that to set values appropriately.
See Set variables in scripts in the MS docs.
If it's not possible to re-architect your app to concatenate the url strings in the application, what the previous commenter said about creating a simple script to do that for you would be the way to go. Ie:
#!/bin/bash
#full login url
fullLoginUrl=$APP.URL\$LOGINSUFFIX
echo "##vso[task.setvariable variable=Login.URL]$fullLoginUrl
Otherwise, perhaps playing around with the run time vs compile time variables in YAML pipelines might be worth trying.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#understand-variable-syntax
In many of the calls described in the Azure DevOps REST API documentation, I need to supply the name of the organization, e.g.:
https://vsrm.dev.azure.com/{organization}/{project}/_apis/release/releases?api-version=5.0-preview.8
The project I can get from System.TeamProject. I would have expected something similar for organization name, something like:
System.TeamFoundationCollectionName
This does not seem to be available. I've even printed out all of my environment variables on the agent and don't see anything that fits the need exactly. Sure, I can parse it out of one of the other values, but this seems fragile since MS seems to like to change the format of URLs.
I also can't hard code the organization name because this release definition will live in multiple organizations and we don't want to have to manually update it for each. How are others solving this problem?
Try using System.TeamFoundationServerUri and System.TeamFoundationCollectionUri to build your API requests. They have the organization included in them.
https://learn.microsoft.com/en-us/azure/devops/pipelines/release/variables?view=vsts&tabs=batch
edit: SYSTEM_TEAMFOUNDATIONSERVERURI/BUILD_PROJECTNAME/_apis/release/releases?api-version=5.0-preview.8
It looks like currently there is no such variable for the organization, also, the variables return the old URL (xxx.visualstudio.com) and not the new URL (dev.azure.com/xxx) so if you use the System.TeamFoundationCollectionName the API should work without the {organization}:
https://System.TeamFoundationCollectionName/{project}/_apis/release/releases?api-version=5.0-preview.8.
In Powershell, do this:
# Where SYSTEM_TEAMFOUNDATIONCOLLECTIONURI=https://some_org_name.visualstudio.com/
([System.Uri]$Env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI).Host.split('.')[-3] # returns 'some_org_name'
Now, just assign that to a variable and use it anywhere you like. "SYSTEM_TEAMPROJECT" is the Project Name, so no need to do any parsing there. It is already available.
My simple experiment reads from an Azure Storage Table, Selects a few columns and writes to another Azure Storage Table. This experiment runs fine on the Workspace (Let's call it workspace1).
Now I need to move this experiment as is to another workspace(Call it WorkSpace2) using Powershell and need to be able to run the experiment.
I am currently using this Library - https://github.com/hning86/azuremlps
Problem :
When I Copy the experiment using 'Copy-AmlExperiment' from WorkSpace 1 to WorkSpace 2, the experiment and all it's properties get copied except the Azure Table Account Key.
Now, this experiment runs fine if I manually enter the account Key for the Import/Export Modules on studio.azureml.net
But I am unable to perform this via powershell. If I Export(Export-AmlExperimentGraph) the copied experiment from WorkSpace2 as a JSON and insert the AccountKey into the JSON file and Import(Import-AmlExperiment) it into WorkSpace 2. The experiment fails to run.
On PowerShell I get an "Internal Server Error : 500".
While running on studio.azureml.net, I get the notification as "Your experiment cannot be run because it has been updated in another session. Please re-open this experiment to see the latest version."
Is there anyway to move an experiment with external dependencies to another workspace and run it?
Edit : I think the problem is something to do with how the experiment handles the AccountKey. When I enter it manually, it's converted into a JSON array comprising of RecordKey and IndexInRecord. But when I upload the JSON experiment with the accountKey, it continues to remain the same and does not get resolved into RecordKey and IndexInRecord.
For me publishing the experiment as a private experiment for the cortana gallery is one of the most useful options. Only the people with the link can see and add the experiment for the gallery. On the below link I've explained the steps I followed.
https://naadispeaks.wordpress.com/2017/08/14/copying-migrating-azureml-experiments/
When the experiment is copied, the pwd is wiped for security reasons. If you want to programmatically inject it back, you have to set another metadata field to signal that this is a plain-text password, not an encrypted password that you are setting. If you export the experiment in JSON format, you can easily figure this out.
I think I found the issue why you are unable to export the credentials back.
Export the JSON graph into your local disk, then update whatever parameter has to be updated.
Also, you will notice that the credentials are stored as 'Placeholders' instead of 'Literals'. Hence it makes sense to change them to Literals instead of placeholders.
This you can do by traversing through the JSON to find the relevant parameters you need to update.
Here is a brief illustration.
Changing the Placeholder to a Literal: