How to perform substring like function in jmespath on a string - substring

I would like to know if there is substring function one can leverage in JMESPATH (supported by az cli).
I have the below az cli request and I want to just extract the name of the linked subnet with a security group, but unlike other cloud providers azure doesn't store associated resources names the same way.
The name can be extracted in the subnet.id node which looks like below
$ az network nsg show -g my_group -n My_NSG --query "subnets[].id" -o json
[
"/subscriptions/xxxxxx2/resourceGroups/my_group/providers/Microsoft.Network/virtualNetworks/MY-VNET/subnets/My_SUBNET"
]
I want to only extract "MY_SUBNET" from the the result.
I know there is something called search that is supposed to mimic
substring (explained here
https://github.com/jmespath/jmespath.jep/issues/5) but it didn't
work for me .
$ az network nsg show -g my_group -n My_NSG --query "subnets[].search(id,'#[120:-1]')" -o json
InvalidArgumentValueError: argument --query: invalid jmespath_type value: "subnets[].search(id,'#[120:-1]')"
CLIInternalError: The command failed with an unexpected error. Here is the traceback:
Unknown function: search()
Thank you
Edit :
I actually run the request including other elements that's why using substring with bash in a new line is not what I want .
here's an example of the full query :
az network nsg show -g "$rg_name" -n "$sg_name" --query "{Name:name,Combo_rule_Ports:to_string(securityRules[?direction==\`Inbound\`].destinationPortRanges[]),single_rule_Ports:to_string(securityRules[?direction==\`Inbound\`].destinationPortRange),sub:subnets[].id,resourceGroup:resourceGroup}" -o json
output
{
"Combo_rule_Ports": "[]",
"Name": "sg_Sub_demo_SSH",
"resourceGroup": "brokedba",
"single_rule_Ports": "[\"22\",\"80\",\"443\"]",
"sub": [
"/subscriptions/xxxxxxx/resourceGroups/brokedba/providers/Microsoft.Network/virtualNetworks/CLI-VNET/subnets/Sub_demo"
]
}

I had a similar problem with EventGrid subscriptions and used jq to transform JSON returned by the az command. As a result, you get an JSON array.
az eventgrid event-subscription list -l $location -g $resourceGroup --query "[].{
Name:name,
Container:deadLetterDestination.blobContainerName,
Account:deadLetterDestination.resourceId
}" \
| jq '[.[] | { Name, Container, Account: (.Account | capture("storageAccounts/(?<name>.+)").name) }]'
The expression Account: (.Account | capture("storageAccounts/(?<name>.+)").name) transforms the original resourceId from the Azure CLI.
# From Azure resourceId...
"Account": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/my-resource-group/providers/Microsoft.Storage/storageAccounts/mystorageaccount"
# .. to Azure Storage Account name
"Account": "mystorageaccount"
I've adapted the approach from How to extract a json value substring with jq.

cut can be used to extract desired values:
az network nsg show -g my_group -n My_NSG --query "subnets[].id|[0]" -o json | cut -d"/" -f11

If you run Azure CLI in bash, here are string manipulation operations you can do:
Following syntax deletes the longest match of $substring from the front of $string
${string##substring}
In this case, you can retrieve the subnet like this.
var=$(az network nsg show -g nsg-rg -n nsg-name --query "subnets[].id" -o tsv)
echo ${var##*/}
For more information, you could refer to https://www.thegeekstuff.com/2010/07/bash-string-manipulation/

Related

How can I get Azure pipeline log file from command line

I have an Azure pipeline. I can start and see the logs in Chrome. But I would like to do these steps from command line (actually Cygwin, but IMHO this is not relevant).
az pipelines run --name $pipeline --branch $branch
This command gives back a json file format text on the stdout. This json has a entry, called logs:
{
...
"logs": {
"id": 0,
"type": "Container",
"url": "https://dev.azure.com/myazure/a56234f9-0101-5183-b422-db6f8cb55857/_apis/mybuild/mybuilds/1822266/logs"
},
...
}
If I copy the "url" into Chrome I get another json format homepage, like:
{"count":27,"value":[{"lineCount":371,"createdOn":"2022-10-17T13:38:14.013Z","lastChangedOn":"2022-10-17T13:38:14.183Z","id":1,"type":"Container","url":"https://dev.azure.com/myazure/...
But I cannot get back this json data from command line. I tried to get by curl or wget. I got back a HTML page (full with with JavaScript), not the json answer.
I also tried az:
az rest --method get --url "$logs_url"
But the response is:
Can't derive appropriate Azure AD resource from --url to acquire an access token. If access token is required, use --resource to specify the resource
Not a json response, outputting to stdout. For binary data suggest use "--output-file" to write to a file
Then I tried to do:
az account get-access-token --query accessToken --output tsv > access_token.tsv
az rest --method get --resource access_token.tsv --url "$logs_url"
So, I assume I should get an access token to the URL. But how can I acquire it?
You can use curl to call rest api: https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/logs/list?view=azure-devops-rest-6.0
E.g.
curl https://dev.azure.com/{organization}/{project}/_apis/pipelines/{pipelineId}/runs/{runId}/logs?api-version=6.0-preview.1 ' -H 'Authorization: Basic YourPAT'

Iterate on all the secret from Group of ADO library

I would like to iterate over all the secrets from group and would like to pass those to another script. I am able to get all the keys by running az command
for allkeys in `az pipelines variable-group list --organization https://dev.azure.com/myorg --project myproj --group-name mygroup | jq -r '.[].variables | keys | .[]' `;do
echo $allkeys
done
OUTPUT
key1
key2
key3
Now I would like to iterate over this and would like to use key=value for which a bash solution would be
for allkeys in `az pipelines variable-group list --organization https://dev.azure.com/myorg --project myproj --group-name mygroup | jq -r '.[].variables | keys | .[]' `;do
echo $allkeys=`eval echo \${${allkeys}}`
done
above loop will output
key1=val1
key2=val2
key3=val3
But it wont work in ADO pipeline as we refer variable in ADO pipeline as $(key1) which will output the secret.
So, looking for a solution to iterate over all the secret instead of reading those one by one with hardcoded value.
I would really appreciate any help.

How to store Databricks token created from CLI in Yaml Bash step

I have the following Yaml script. I am looking for how to grab the token created and store into a variable:
- bash: |
echo {} > ~/.databricks-connect#
source py37-venv/bin/activate
pip3 install wheel
pip3 install databricks-cli
displayName: Install Databricks CLI
- bash: |
source py37-venv/bin/activate
databricks configure --token <<EOF
${DATABRICKS_HOST}
${DATABRICKS_AAD_TOKEN}
EOF
databricks tokens create --lifetime-seconds 129600 --comment "My comment."
The response that the above command returns is this json:
{
"token_value": "dapi1a23b45678901cd2e3fa4bcde56f7890",
"token_info": {
"token_id": "1ab23cd45678e90123f4567abc8d9e012345fa67890123b45678cde90fa123b4",
"creation_time": 1621287738473,
"expiry_time": 1621417338473,
"comment": "My comment."
}
}
I want to store the value of token_value above so I can use it in another task below.
You can use jq to parse the response json to get token value, for example:
token=$(databricks tokens create --lifetime-seconds 129600 --comment "My comment." | jq .token_value --raw-output)
Set $token as variable with logging command(you can set it as secret or not,click the link to check the usage), then use it in next job($(setvar.databrickstoken)).
echo "##vso[task.setvariable variable=databrickstoken;issecret=true;isoutput=true]$token"

STDIN is getting listed while listing Azure VMs, hence getting "invalid resource id" error

I used the below command to list and stop all the VMs in my account.
VMs are listing, but an additional STDIN is getting listed.
This STDIN is causing the error "invalid resource id"
What can I do to ignore the STDIN??. Your help greatly appreciated.
az vm stop --ids $(az vm list --query "[].id" -o tsv) | grep -v "ABDK"
thanks
If you mean the VM name by the STDIN and want to stop all the VMs that the name does not contain the string ABDK, then you can use the CLI command only like this:
az vm stop --ids $(az vm list --query "[?contains(#.name, 'ABDK')==\`false\`].id" -o tsv)
Update:
If you run the CLI command in the Windows PowerShell, then you need to change the command like this:
az vm stop --ids $(az vm list --query "[?!contains(#.name, 'ABDK')].id" -o tsv)

SumoLogic dashboards - how do I automate?

I am getting some experience with SumoLogic dashboards and alerting. I would like to have all possible configuration in code. Does anyone have experience with automation of SumoLogic configuration? At the moment I am using Ansible for general server and infra provisioning.
Thanks for all info!
Best Regards,
Rafal.
(The dashboards, alerts, etc. are referred to as Content in Sumo Logic parlance)
You can use the Content Management API, especially the content-import-job. I am not an expert in Ansible, but I am not aware of any way to plug that API into Ansible.
Also there's a community Terraform provider for Sumo Logic and it supports content:
resource "sumologic_content" "test" {
parent_id = "%s"
config =
{
"type": "SavedSearchWithScheduleSyncDefinition",
"name": "test-333",
"search": {
"queryText": "\"warn\"",
"defaultTimeRange": "-15m",
[...]
Disclaimer: I am currently employed by Sumo Logic
Below is the shell script to import the dashboard. Here it is SumoLogic AU instance. eg: https://api.au.sumologic.com/api. This will be changed based on your country.
Note: You can export all of your dashboard as json files.
#!/usr/bin/env bash
set -e
# if you are using AWS parameter store
# accessKey=$(aws ssm get-parameter --name path_to_your_key --with-decryption --query 'Parameter.Value' --region=ap-southeast-2 | tr -d \")
# accessSecret=$(aws ssm get-parameter --name name path_to_your_secret --with-decryption --query 'Parameter.Value' --region=ap-southeast-2 | tr -d \")
# yourDashboardFolderName="xxxxx" # this is the folder id in the sumologic where you want to create dashboards
# if you are using just key and secreat
accessKey= "your_sumologic_key"
accessSecret= "your_sumologic_secret"
yourDashboardFolderName="xxxxx" # this is the folder id in the sumologic
# you can place all the json files of dashboard in ./Sumologic/Dashboards folder.
for f in $(find ./Sumologic/Dashboards -name '*.json'); \
do \
curl -X POST https://api.au.sumologic.com/api/v2/content/folders/$yourDashboardFolderName/import \
-H "Content-Type: application/json" \
-u "$accessKey:$accessSecret" \
-d #$f \
;done