I am trying to create an owasp zap instance using azure container instances using the following code:
$containerGroupName = "EW-owaspzap"
$containerDnsName = "EW-owaspzap"
$imageName = "owasp/zap2docker-stable"
$myIpAddress = (Invoke-WebRequest ifconfig.me/ip).Content.Trim()
$environmentVars = #{"api.key"="myreallysecureapikey";"api.addrs.addr.name"=$myIpAddress}
$containerGroup = Get-AzureRmContainerGroup -ResourceGroupName $resourceGroupName -Name $containerGroupName -ErrorAction SilentlyContinue
if (!$containerGroup) {
New-AzureRmContainerGroup -ResourceGroupName $resourceGroupName -Name $containerGroupName -Image $imageName -Command zap-webswing.sh -Port 8080,8090 `
-IpAddressType Public -DnsNameLabel $containerDnsName -RestartPolicy OnFailure -Location WestEurope -AzureFileVolumeShareName $storageShareName `
-AzureFileVolumeMountPath '/output' -AzureFileVolumeAccountCredential $storageCredentials -EnvironmentVariable $environmentVars
}
However I get the error:
The environment variable name in container 'EW-owaspzap' of container group 'EW-owaspzap' is invalid. A valid environment variable
name must start with alphabetic character or '', followed by a string of alphanumeric characters or '' (e.g. 'my_name', or 'MY_NAME', or 'MyName')
according to this https://github.com/zaproxy/zaproxy/wiki/Docker I have the format of the environment variables correct. Is there anything else I have missed?
This is ACI limitation - see here for naming limitation for env vars:
| Environment variable | 1-63 |Case insensitive |Alphanumeric, and
underscore (_) anywhere except the first or last character
This is not an issue with Zap, but with ACI.
This can be solved with a script that gets the env vars in Azure format and converts them to Zap's format (e.g. api_key to api.key). This is a pseudo-code (I did not test it), just to give you an idea:
export set api.key=$API_KEY
./zap
Create a new docker image based on Zap's official image, copy the script and use it to start Zap instead of the regular Zap's command.
For your issue, I think there is something you have misunderstood. The command in the link you posted docker run -u zap -p 8080:8080 -i owasp/zap2docker-stable zap-x.sh -daemon -host 0.0.0.0 -port 8080 -config api.addrs.addr.name=.* -config api.addrs.addr.regex=true, you should take a look at docker run, there is no parameter like -config.
So, I think the command from zap-x.sh to the end is a whole bash command with the script zap-x.sh. You can check the parameter definition in the script zap-x.sh.
And the environment in PowerShell command is a Hashtable, you can get more details here. Also, there are some limitations about Naming conventions in Azure Container Instances.
Not sure if you got his working, but I used your powershell script & was able to create Zap container by replacing "." with "_" in $environmentVars array.
Related
I have a powershell script that i'm trying to write. I need to take some input / output from the console and pipe it into a split command but I don't know how.
I'm running an azure cli command... to list a bunch of resources. and I need to extract the name of the storage account.
Here's some sample output:
Name ResourceGroup Location Type
------------------ ------------- ------------ ----------
asdf1234-insights jjResourceGrp eastus microsoft.insights/components
asdf1234-storage jjResourceGrp eastus Microsoft.Storage/storageAccounts
asdf1234 jjResourceGrp eastus Microsoft.Web/serverFarms
asdf1234 jjResourceGrp eastus Microsoft.Web/sites
This is the powershell command I'm using right now to find just the storage Account:
az resource list -g jjResourceGrp -o table | Select-String -Pattern "storageAccounts"
But what I really need is to extract the "asdf1234-storage" from that line.
Any help would be appreciated.
As Ash has pointed out:
It is always preferable to use PowerShell commands that output objects whose properties you can operate on, which in this case requires installing the Az PowerShell module (Install-Module Az), which then allows you to call Get-AzStorageAccount.
If you're interacting with external programs, such as the az CLI, you're of necessity dealing with text (string) output, which complicates subsequent processing:
The next best option is to deal with an external program's structured text output format, if available, such as CSV or JSON; indeed, as Ash also points out, the az CLI's default output format is JSON, so you could omit -o table and process the output further with ConvertFrom-Json
Absent that, text parsing based, typically based on regexes, such as via the -replace and -split operators, are required.
To answer the question as asked (because text parsing is so much fun):
The switch statement combined with its -Regex switch offers a concise solution:
switch -Regex (az resource list -g jjResourceGrp -o table) {
'^(\w+).*\bstorageAccounts\b' {
$Matches[1]; break
}
}
This question already has answers here:
Using --% in PowerShell
(3 answers)
Closed 2 years ago.
I have a small Powershell script that uploads a file to blob storage. The idea is to run this as an inline script inside Devops once a build has been completed instead of having the path/file hardcoded like now.
If I run this in a Powershell command prompt on my computer, it works fine.
az storage blob upload --% --container-name mycontainer --account-name myaccount --name "File.zip" --file "c:\Projects\File.zip" --sas-token "{my token}"
However, I want to exchange the hardcoded path+file with a variable that my build pipleline can set. But here is where I haven't figured out how to actually use the variable.
The following does not work when I try to run it locally. To test I created a variable and then made a call.
// Create variable
New-Variable -Name "TestFile" -Value "c:\projects\File.zip"
(enter)
// Try to upload it
az storage blob upload --% --container-name mycontainer --account-name myaccount --name "File.zip" --file $TestFile --sas-token "{my token}"
Results in:
FileOperationError: [WinError 2] The system cannot find the file
specified: '$TestFile'
I am assuming that I need to declare it in a different way or pipe it to make it work, but how?
Use a speech mark at the beginning, and at the end of your variable, for example,"$TestFile"
I am using Azure Pipelines to build a Rakudo binary for Raku (previously aka Perl 6) in Windows.
This is my azure-pipelines.yml file:
jobs:
- job: Windows
pool:
vmImage: 'vs2017-win2016'
steps:
- bash: |
mkdir -p $(Build.SourcesDirectory)/rakudo-win
curl -L https://github.com/rakudo/rakudo/releases/download/2019.07.1/rakudo-2019.07.1.tar.gz | tar xz
mv rakudo-2019.07.1 rakudo
cd rakudo
C:/Strawberry/perl/bin/perl Configure.pl --gen-moar --gen-nqp --backends=moar --prefix=$(Build.SourcesDirectory)/rakudo-win
make
make install
- bash: |
echo "##vso[task.prependpath]$(Build.SourcesDirectory)/rakudo-win/bin"
- bash: |
perl6 -v
The pipeline script builds perl6 binary fine inside $(Build.SourcesDirectory)/rakudo-win/bin folder. There is indeed perl6.exe inside $(Build.SourcesDirectory)/rakudo-win/bin. To make it available, I set the path by prepending it in the bash script. But when I try to run command perl6 -v, the build fails at this step.
I searched for similar issues in SO here, here, here.
Still I could not solve my issue. Any help how to make perl6 binary available at PATH?
EDITED
Next thing I did was create another .yml script as follows:
jobs:
- job: Windows
pool:
vmImage: 'vs2017-win2016'
steps:
- script: |
call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
- pwsh: |
mkdir -p C:\rakudo-win
Invoke-WebRequest -Uri "https://github.com/rakudo/rakudo/releases/download/2019.07.1/rakudo-2019.07.1.tar.gz" -OutFile "rakudo.tar.gz"
tar -xvf .\rakudo.tar.gz
cd rakudo-2019.07.1
C:\Strawberry\perl\bin\perl Configure.pl --gen-moar --gen-nqp --backends=moar --prefix=C:\rakudo-win
make
make install
- pwsh: |
$oldpath = (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH).path
$newpath = "C:\rakudo-win\bin;$oldpath"
Set-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH -Value $newpath
- script: |
SET PATH=C:\rakudo-win\bin;%PATH%
- script: |
perl6 -v
and tried changing PATH twice once in powershell and another in cmdline. But still it throws following error:
'perl6' is not recognized as an internal or external command,
operable program or batch file.
Any help?
The best method I have found for setting the PATH for subsequent tasks in Azure Pipelines is by using the logging command syntax mentioned in the first of the three SO links you looked at. Since you are using PowerShell in your updated yaml pipeline, the command would be:
Write-Host "##vso[task.prependpath]$(Build.SourcesDirectory)/rakudo-win/bin"
Note that this only applies to subsequent tasks, if you try outputting the PATH variable in the current task it will not have updated.
In fact, you are very close to the correct solution. Your second powershell task has set the PATH successfully. You can add another separate task to print out the system PATH value to verify this.
- pwsh: |
$NewPathInRegistry = (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH).path
Write-Host $NewPathInRegistry
To set the PATH programmatically, you can not use set command, this command can indeed set the environment variable PATH, but the disadvantage of set is the new PATH value is only available in the current command line terminal, it does not actually been added into the System Variable. That's why you were getting an unrecognized error in the next script step.
To permanently add the directory to environment variable PATH so that it can work for next others steps, you need use setx or add them into Registry by using reg add. But the usage of setx has limitation that the PATH value max to 1024 characters. So, here the best solution is updating PATH by modifying the Registry value.
BUT, updating Registry still has another issue, you must kill current process and run one new process to run perl6 so that it can read the new available Registry setting.
If run stop-process in Azure devops pipeline, it will make the task failed with exit code -1. This is the expected exit code, so you can set the continueOnError: true to step so that the next steps can continue.
why not just do this:
- script: |
PATH=$BUILD_SOURCESDIRECTORY/rakudo-win/bin:$PATH perl6 -v
This has to be possible. I am able to open a command prompt in windows and do the following:
<some exe> <some exe command line parameters>
There must be an equivalent way to do this in PowerShell or even a standard windows batch file. For example, from the windows command prompt I can start a docker container with:
docker run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0
however if I try something like this with PowerShell
& "docker" run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0
it fails with an generic error:
invalid reference format.
Perhaps PowerShell is not suited for this type of advanced use case. Any suggestions would be appreciated!
Is there a better scripting language for advanced usages like this?
I think Start-Process cmdlet will be useful. ArgumentList can be single or double quoted.
Start-Process docker -ArgumentList "run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0"
By and large, external programs in PowerShell are called the same way as from cmd.exe - there are differences, due to PowerShell having additional metacharacters such as $ and #, but they do not come into play in your specific case.
(Your & "docker" ... variant would work in principle too, but the use of & is only necessary if you must use a quoted or variable-based command name or path).
The problem is that your original command line contains two instances of – (EN DASH, U+2013) instead of the expected ASCII-range - dash (hyphen), which docker doesn't recognize.
A quick way to discover the problem:
# Print the code points of characters outside the ASCII range.
PS> [int[]] [char[]] '& "docker" run –-net=kafka -d –-name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0' -gt 127
8211
8211
Decimal 8211 is hex. 0x2013, the code point of en-dash, whereas the code point of the regular - is 45 (0x2d).
All that is needed is to replace these – instances with - (and, since docker needn't be quoted, there is no need for &):
docker run --net=kafka -d --name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:4.1.0
Your own answer shows a variable-based implementation of the command that is effectively the same as the command above - if all the arguments are known in advance, there is never a need to use variables.
If you do want to use variables, it is much simpler to use a single array variable for all the arguments and pass that:
$dockerExe = 'docker'
$dockerArgs = 'run',
'--net=kafka',
'-d',
'--name=zookeeper',
'-e',
'ZOOKEEPER_CLIENT_PORT=2181',
'confluentinc/cp-zookeeper:4.1.0'
& $dockerExe $dockerArgs
Note:
The executable name/path must always be specified separately, and if it is quoted or involves variable references (as in this case), &, the call operator must be used for invocation, for syntactic reasons.
Passing the arguments as an array this way works with external programs; for PowerShell commands, you'd create a hashtable variable that you pass with sigil # instead of $, a feature known as splatting.
There is a lot of complexities in powershell escaping. I wrote this module to assist with running of external commands:
https://github.com/choovick/ps-invoke-externalcommand
Demo:
https://terminalizer.com/view/49acb54a979
Install-Module -Name ExternalCommand -Scope CurrentUser
Invoke-ExternalCommand -Command "docker" -Arguments #("run","-d","--name=zookeeper","--net=kafka","-e","ZOOKEEPER_CLIENT_PORT=2181", "confluentinc/cp-zookeeper:4.1.0")
Here's how to do it.
$app = 'docker'
$a1 = 'run'
$a2 = '--net=kafka'
$a3 = '-d'
$a4 = '--name=zookeeper'
$a5 = '-e'
$a6 = 'ZOOKEEPER_CLIENT_PORT=2181'
$a7 = 'confluentinc/cp-zookeeper:4.1.0'
& $app $a1 $a2 $a3 $a4 $a5 $a6 $a7
I am trying to connect to ec2 (AWS) from my powershell (windows7)
I added the following lines to the knife.rb file:
knife[:aws_access_key_id] = ENV['XXXXXX']
knife[:aws_secret_access_key] = ENV['xxxxxxxxxxxxxxxxxxxxxxxxxx']
I run for example
knife ec2 server list --region eu-west-1
but get the following:
knife : ERROR: You did not provide a valid 'AWS Access Key Id' value. At line:1 char:1
ERROR: You did ... Key Id' value.:String) [], RemoteException
ERROR: You did not provide a valid 'AWS Secret Access Key' value.
do I need to upload the knife.rb file to the server after I saved it? (how?)
where should I save my pem file and how should I use it in the commands? i tried for example:
knife ec2 server create -I ami-6e7bd919 -N MyEc2Instance -x ec2-user -r "role[webserver]" -i C:\Users\MyName\Documents\openvoip.pem --region eu-west-1
Thanks!
The knife.rb file should have the following and not as described in the question:
knife[:aws_access_key_id] = "XXXXXXXXXXXXXX"
knife[:aws_secret_access_key] = "XXXXXXXXXXXXXXXXXXXX"
I had no such ENV variables, so I set the credentials directly.
To use environment variables you have to put them first in you ~/.bashrc file:
vi ~/.bashrc
export AWS_ACCESS_KEY_ID=/home/yourname/.ec2/prodaccess
export AWS_SECRET_ACCESS_KEY=/home/yourname/.ec2/prodsecret
Save. then Source .bashrc file:
. ~/.bashrc
Run: env # to see if your new variables are propagated to environment. Then you may run your knife ec2 command
If you plan on sharing your knife.rb, you might want to keep the environment variables and setup as described under 'Many Users, Same Repo' in the docs, https://docs.getchef.com/config_rb_knife.html.
In order to use the ENV vars, use something like below as opposed to the key values that you appear to use:
knife[:aws_access_key_id] = ENV['AWS_ACCESS_KEY_ID']
knife[:aws_secret_access_key] = ENV['AWS_SECRET_ACCESS_KEY']
Then, when running knife, ensure your local environment variables are set.