how to split a string in powershell based on output from azure command - powershell

I have a powershell script that i'm trying to write. I need to take some input / output from the console and pipe it into a split command but I don't know how.
I'm running an azure cli command... to list a bunch of resources. and I need to extract the name of the storage account.
Here's some sample output:
Name ResourceGroup Location Type
------------------ ------------- ------------ ----------
asdf1234-insights jjResourceGrp eastus microsoft.insights/components
asdf1234-storage jjResourceGrp eastus Microsoft.Storage/storageAccounts
asdf1234 jjResourceGrp eastus Microsoft.Web/serverFarms
asdf1234 jjResourceGrp eastus Microsoft.Web/sites
This is the powershell command I'm using right now to find just the storage Account:
az resource list -g jjResourceGrp -o table | Select-String -Pattern "storageAccounts"
But what I really need is to extract the "asdf1234-storage" from that line.
Any help would be appreciated.

As Ash has pointed out:
It is always preferable to use PowerShell commands that output objects whose properties you can operate on, which in this case requires installing the Az PowerShell module (Install-Module Az), which then allows you to call Get-AzStorageAccount.
If you're interacting with external programs, such as the az CLI, you're of necessity dealing with text (string) output, which complicates subsequent processing:
The next best option is to deal with an external program's structured text output format, if available, such as CSV or JSON; indeed, as Ash also points out, the az CLI's default output format is JSON, so you could omit -o table and process the output further with ConvertFrom-Json
Absent that, text parsing based, typically based on regexes, such as via the -replace and -split operators, are required.
To answer the question as asked (because text parsing is so much fun):
The switch statement combined with its -Regex switch offers a concise solution:
switch -Regex (az resource list -g jjResourceGrp -o table) {
'^(\w+).*\bstorageAccounts\b' {
$Matches[1]; break
}
}

Related

Powershell: convert stdin to UTF-8

I'm trying to automate some Terraform work on Windows. I want to export my Terraform state as JSON then use the Windows version of jq.exe to pull out relevant bits of information.
Ideally, my command line would look like:
terraform show -json | jq '<my-jq-query>'
Unfortunately, by default Windows appears to use UTF-16 LE (so that's what the Terraform JSON output is encoded with) and jq.exe only supports UTF-8.
I found that the PowerShell command Set-Content has an -Encoding parameter that can be used to specify output encoding, but I can't figure out how to get Set-Content to read from stdin instead of from a file. I mean, I'd like to do:
terraform show -json | Set-Content -Encoding utf8 | jq '<my-jq-query'>
but I can't figure out how to get it to work.
How can I coax PowerShell into allow me to convert character encoding in the pipeline without reading/writing to a file?

Multi parameters in Powershell Bash/Zsh command

Unable to run the following Bash/Zsh command in Powershell:
$KeyPath = Join-Path -Path $this.Plate -ChildPath "install/tekton.key"
kubectl create secret docker-registry regcred `
--docker-server="https://gcr.io" `
--docker-username=_json_key `
--docker-email="name#org.iam.gserviceaccount.com" `
--docker-password="$(cat $KeyPath)"
I get error:
error: exactly one NAME is required, got 5
See 'kubectl create secret docker-registry -h' for help and examples
If I run this command directly in bash it works:
kubectl create secret docker-registry regcred --docker-server="https://gcr.io" --docker-username=_json_key --docker-email="name#org.iam.gserviceaccount.com" --docker-password="$(cat ./tekton.key)"
I don't know if it's the cause of your problem, but there are two potential problems:
An expanded value that contains spaces causes PowerShell to double-quote the argument as a whole when it rebuilds the command line behind the scenes (on Windows):
For instance, if $(cat $KeyPath) ($(Get-Content $KeyPath)) expands to one two, PowerShell passes "--docker-password=one two" behind the scenes, not --docker-password="one two".
Whether this changes the meaning of the argument depends on how the target program parses its command line - I don't know what kubectl does.
If you do need to address this, escape the enclosing " (double quotes) with `` ``` (the backtick, PowerShell's escape character to make PowerShell pass your argument in the original syntax form:
--docker-password=`"$(cat ./tekton.key)`"
Note that - unlike in POSIX-like shells such as Bash and Zsh - you normally do not enclose a variable reference or subexpression in "..." in order to pass it through safely; e.g., --foo=$someVar or --foo=$(Get-Date) work fine, even if $someVar or the output from Get-Date contains spaces or wildcard characters.
If file $KeyPath contains multiple lines, the lines are concatenated with spaces in the argument:
For instance, if the file contains "a`nb`n" ("`n" being a newline), PowerShell will pass
"--docker-password=a b".
By contrast, POSIX-like shells such as Bash or Zsh will preserve the interior newlines, while trimming (any number of) trailing ones.
On a side note: PowerShell's handling of embedded double-quoting in arguments passed to external programs is broken - see this answer.

Running executable with command line parameters in PowerShell

This has to be possible. I am able to open a command prompt in windows and do the following:
<some exe> <some exe command line parameters>
There must be an equivalent way to do this in PowerShell or even a standard windows batch file. For example, from the windows command prompt I can start a docker container with:
docker run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0
however if I try something like this with PowerShell
& "docker" run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0
it fails with an generic error:
invalid reference format.
Perhaps PowerShell is not suited for this type of advanced use case. Any suggestions would be appreciated!
Is there a better scripting language for advanced usages like this?
I think Start-Process cmdlet will be useful. ArgumentList can be single or double quoted.
Start-Process docker -ArgumentList "run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0"
By and large, external programs in PowerShell are called the same way as from cmd.exe - there are differences, due to PowerShell having additional metacharacters such as $ and #, but they do not come into play in your specific case.
(Your & "docker" ... variant would work in principle too, but the use of & is only necessary if you must use a quoted or variable-based command name or path).
The problem is that your original command line contains two instances of – (EN DASH, U+2013) instead of the expected ASCII-range - dash (hyphen), which docker doesn't recognize.
A quick way to discover the problem:
# Print the code points of characters outside the ASCII range.
PS> [int[]] [char[]] '& "docker" run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0' -gt 127
8211
8211
Decimal 8211 is hex. 0x2013, the code point of en-dash, whereas the code point of the regular - is 45 (0x2d).
All that is needed is to replace these – instances with - (and, since docker needn't be quoted, there is no need for &):
docker run --net=kafka -d --name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0
Your own answer shows a variable-based implementation of the command that is effectively the same as the command above - if all the arguments are known in advance, there is never a need to use variables.
If you do want to use variables, it is much simpler to use a single array variable for all the arguments and pass that:
$dockerExe = 'docker'
$dockerArgs = 'run',
'--net=kafka',
'-d',
'--name=zookeeper',
'-e',
'ZOOKEEPER_CLIENT_PORT=2181',
'confluentinc/cp-zookeeper:4.1.0'
& $dockerExe $dockerArgs
Note:
The executable name/path must always be specified separately, and if it is quoted or involves variable references (as in this case), &, the call operator must be used for invocation, for syntactic reasons.
Passing the arguments as an array this way works with external programs; for PowerShell commands, you'd create a hashtable variable that you pass with sigil # instead of $, a feature known as splatting.
There is a lot of complexities in powershell escaping. I wrote this module to assist with running of external commands:
https://github.com/choovick/ps-invoke-externalcommand
Demo:
https://terminalizer.com/view/49acb54a979
Install-Module -Name ExternalCommand -Scope CurrentUser
Invoke-ExternalCommand -Command "docker" -Arguments #("run","-d","--name=zookeeper","--net=kafka","-e","ZOOKEEPER_CLIENT_PORT=2181", "confluentinc/cp-zookeeper:4.1.0")
Here's how to do it.
$app = 'docker'
$a1 = 'run'
$a2 = '--net=kafka'
$a3 = '-d'
$a4 = '--name=zookeeper'
$a5 = '-e'
$a6 = 'ZOOKEEPER_CLIENT_PORT=2181'
$a7 = 'confluentinc/cp-zookeeper:4.1.0'
& $app $a1 $a2 $a3 $a4 $a5 $a6 $a7

Configuration of Owasp Zap on Azure Container Instances

I am trying to create an owasp zap instance using azure container instances using the following code:
$containerGroupName = "EW-owaspzap"
$containerDnsName = "EW-owaspzap"
$imageName = "owasp/zap2docker-stable"
$myIpAddress = (Invoke-WebRequest ifconfig.me/ip).Content.Trim()
$environmentVars = #{"api.key"="myreallysecureapikey";"api.addrs.addr.name"=$myIpAddress}
$containerGroup = Get-AzureRmContainerGroup -ResourceGroupName $resourceGroupName -Name $containerGroupName -ErrorAction SilentlyContinue
if (!$containerGroup) {
New-AzureRmContainerGroup -ResourceGroupName $resourceGroupName -Name $containerGroupName -Image $imageName -Command zap-webswing.sh -Port 8080,8090 `
-IpAddressType Public -DnsNameLabel $containerDnsName -RestartPolicy OnFailure -Location WestEurope -AzureFileVolumeShareName $storageShareName `
-AzureFileVolumeMountPath '/output' -AzureFileVolumeAccountCredential $storageCredentials -EnvironmentVariable $environmentVars
}
However I get the error:
The environment variable name in container 'EW-owaspzap' of container group 'EW-owaspzap' is invalid. A valid environment variable
name must start with alphabetic character or '', followed by a string of alphanumeric characters or '' (e.g. 'my_name', or 'MY_NAME', or 'MyName')
according to this https://github.com/zaproxy/zaproxy/wiki/Docker I have the format of the environment variables correct. Is there anything else I have missed?
This is ACI limitation - see here for naming limitation for env vars:
| Environment variable | 1-63 |Case insensitive |Alphanumeric, and
underscore (_) anywhere except the first or last character
This is not an issue with Zap, but with ACI.
This can be solved with a script that gets the env vars in Azure format and converts them to Zap's format (e.g. api_key to api.key). This is a pseudo-code (I did not test it), just to give you an idea:
export set api.key=$API_KEY
./zap
Create a new docker image based on Zap's official image, copy the script and use it to start Zap instead of the regular Zap's command.
For your issue, I think there is something you have misunderstood. The command in the link you posted docker run -u zap -p 8080:8080 -i owasp/zap2docker-stable zap-x.sh -daemon -host 0.0.0.0 -port 8080 -config api.addrs.addr.name=.* -config api.addrs.addr.regex=true, you should take a look at docker run, there is no parameter like -config.
So, I think the command from zap-x.sh to the end is a whole bash command with the script zap-x.sh. You can check the parameter definition in the script zap-x.sh.
And the environment in PowerShell command is a Hashtable, you can get more details here. Also, there are some limitations about Naming conventions in Azure Container Instances.
Not sure if you got his working, but I used your powershell script & was able to create Zap container by replacing "." with "_" in $environmentVars array.

Creating CSV file as an object

I have got a robust script which gets, parse and uses some data from .csv file. To run the script I can use
.\script.ps1 -d data_file.csv
The thing is I cannot interfere into script itself that is why i need to create some kind of a wrapper which will create new csv file and use script.ps1 with a new made file. I am wondering if there is a possibility to create a csv file as an object which will be passed directly to the command like this
.\script.ps1 -d csv_file_as_object.csv
without creating file in some path directory.
What you'd need in this case is the equivalent of Bash's process substitution (<(...)), which, in a nutshell, would allow you to present a command's output as the content of a temporary file whose path is output:
.\scripts.ps1 -d <(... | ConvertTo-Csv) # !! does NOT work in PowerShell
Note: ... | ConverTo-Csv stands for whatever command is needed to transform the original CSV in-memory.
No such feature exists in PowerShell as of Windows PowerShell v5.1 / PowerShell Core v6.1, but it has been proposed.
If .\scripts.ps1 happens to also accept stdin input (via pseudo-path - indicating stdin input), you could try:
... | ConvertTo-Csv | .\script.ps1 -d -
Otherwise, your only option is to:
save your modified CSV data to a temporary file
pass that temporary file's path to .\script.ps1
remove the temporary file.