Convert sed command from cUrl to Powershell - powershell

I'm trying to convert the commands from How to redeploy instance using API in rancher to powershell.
pod_upgrade_body=$(curl -u "token-[use your token here]" \
-s 'https://rancher.mydomain.com/v3/project/c-zqpm5:p-h884r/workloads/deployment:development:api' \
-X GET \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Pragma: no-cache' \
-H 'Cache-Control: no-cache' 2>&1 | sed "s/\"cattle\.io\/timestamp\"\:\"[0-9T:Z-]*\"/\"cattle\.io\/timestamp\":\"$(date -u +"%Y-%m-%dT%H:%M:%SZ")\"/g")
This is my attempt but I can't seem to make it work
Invoke-WebRequest -Method GET -Uri https://rancher... -Header #{Accept = 'application/json'; Authorization = 'Bearer [my-token]'; 'Content-type' = 'application/json'; Pragma = 'no-cache'; 'Cache-Control' = 'no-cache'} |cat %{$_ -replace "s/\`"cattle.io/timestamp\`":\`"[0-9T:Z-]*\`"/\`"cattle.io/timestamp","$(date -u +"%Y-%m-%dT%H:%M:%SZ")\`"/g"}
I'm getting the following error
Get-Content : A positional parameter cannot be found that accepts argument '$_ -replace "s/\`"cattle.io/timestamp\`":\`"[0-9T:Z-]*\`"/\`"cattle.io/timestamp","$(date -u
+"%Y-%m-%dT%H:%M:%SZ")\`"/g"'.

Since you're receiving JSON data, it is better to use Invoke-RestMethod than Invoke-WebRequest.
Doing so means that the JSON response is automatically parsed into a [pscustomobject] object graph that allows direct OO access to the data returned for easy, dot-notation-based querying and updating.
You can can re-convert to JSON using ConvertTo-Json (beware the default serialization depth of just 2: use -Depth as needed).
To put it all together:
$fromJson =
Invoke-RestMethod -Method GET -Uri https://rancher... -Headers #{Accept = 'application/json'; Authorization = 'Bearer [my-token]'; 'Content-type' = 'application/json'; Pragma = 'no-cache'; 'Cache-Control' = 'no-cache' } |
ForEach-Object {
$_.annotations.'cattle.io/timestamp' = Get-Date ([datetime]::UtcNow) -UFormat '%Y-%m-%dT%H:%M:%SZ'
}
# Re-convert to JSON, if needed.
# Adjust -Depth as needed.
$fromJson | ConvertTo-Json -Depth 5
Note: I'm assuming that the cattle.io/timestamp properties are inside annotations property, based on the forum link in your question.
As for what you tried:
In Windows PowerShell (but not PowerShell (Core) 7+), cat is a built in alias for Get-Content, which reads text files specified by their paths - it doesn't support receiving data via the pipeline, the way that cat Unix utility does.
However, there is no need for such a utility; the use of the ForEach-Object cmdlet, whose built-in alias % you tried to use, by itself is sufficient to process each input object via the script block ({ ... }) passed to it, inside of which the automatic $_ variable refers to the input object at hand.
The translation of the sed command into a -replace operation has several problems:
-replace doesn't support sed functions such as s - it is the s (string substitution) function in a manner of speaking.
The search regex and the replacement string must be passed as separate arguments (and each must not be enclosed in /.../ delimiters).
-replace doesn't support options (other than inline regex option in the regex operand), and it invariably replaces all occurrences; that is, sed's /g option is implied.
The external date utility is only available on Unix-like platforms by default; PowerShell's equivalent is the Get-Date cmdlet, as shown above.

Related

Using GET in PowerShell & Exporting JSON to CSV

I'm using the following CURL command, to read/fetch table data from an API:
curl -X GET \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-H 'Authorization: Bearer *myAccessToken*' \
https://www.myWebsite.com/api/orders
This command/API Call returns a table in JSON format. I need to do two things with this.
[1] I need to run this in powershell. I've tried using the above code, and returns a general syntax error:
A parameter cannot be found that matches parameter name 'X'.
[2] In PowerShell, Have the JSON output converted & saved as a CSV file
Any ideas? Thanks!
You can use Invoke-RestMethod Cmdlet to Sends an HTTP or HTTPS request to a RESTful web service.
$uri = "https://www.myWebsite.com/api/orders"
$headers = #{
'Content-Type' = 'application/json'
'Authorization' = 'Bearer <AccessToken>'
'Accept'= 'application/json'
}
$response = Invoke-RestMethod -Uri $uri -Method GET -Headers $headers
PowerShell formats the response based to the data type. For JavaScript Object Notation (JSON) or XML, PowerShell converts, or deserializes, the content into [PSCustomObject] objects. So you can select the columns you want to export and pipe it into Export-Csv Cmdlet
$response | Select ColumnName1,ColumnName2 | Export-Csv -Path "filename.csv" -NoTypeInformation

Is it possible to clone a Variable Group between PROJECTS?

I know you can clone Variable Groups, but this is limited within the Project.
Is it possible to clone it to a different Project?
There is no such a feature to clone variable groups between Projects in azure devops UI portal.
However, you can achieve this using variable group rest api.
First, you need to call get variable group rest api to get the variables content.
GET https://dev.azure.com/{organization}/{project}/_apis/distributedtask/variablegroups/{groupId}?api-version=5.1-preview.1
Then use add variable group rest api to add the a new variable group to another project with the variable content from the Get rest api.
POST https://dev.azure.com/{organization}/{project}/_apis/distributedtask/variablegroups?api-version=5.1-preview.1
Please below powershell script example:
# Get the variable group in projectA
$url = "https://dev.azure.com/{organization}/{projectA}/_apis/distributedtask/variablegroups/{groupId}?api-version=5.1-preview.1"
$PAT = "Personal access token"
$base64AuthInfo = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($PAT)"))
$result=Invoke-RestMethod -Uri $url -Headers #{Authorization = ("Basic {0}" -f $base64AuthInfo)} -Method get -ContentType "application/json"
# Call add variable group rest api to add variable group in ProjectB
$updateurl = "https://dev.azure.com/{organization}/{projectB}/_apis/distributedtask/variablegroups?api-version=5.1-preview.1"
$body = $result | ConvertTo-Json -Depth 10
Invoke-RestMethod -Uri $updateurl -Headers #{Authorization = "Basic {0}" -f $base64AuthInfo} -ContentType "application/json" -Method post -Body $body
You can also use azure devops command line az pipelines variable-group create. See here for more information.
An approach - using curl, jq and xargs - that has worked for me:
curl -H "Content-Type: application/json" -X GET -u 'username':'<PAT>' 'https://dev.azure.com/<organization>/<url encoded source project-name>/_apis/distributedtask/variablegroups?api-version=6.1-preview.1' \
| jq -c '.value[] | del(.id, .createdBy, .modifiedBy, .createdOn, .modifiedOn)' \
| xargs -d'\n' -L1 -I'{}' \
curl -H "Content-Type: application/json" -X POST -u 'username':'<PAT>' -d "{}" 'https://dev.azure.com/<organization>/<url encoded target project-name>/_apis/distributedtask/variablegroups?api-version=6.1-preview.1'
The first curl command gets all available variable groups from the source project
jq removes the specified keys from each json object and outputs them line by line (-c)
xargs splits the input by newline (-d'\n') and passes each line (-L1) to the curl command, that creates a new variable group in the target project

How to pipe wget results to subsequent curl post?

I am in Windows 10 attempting to get Confluent CE up via docker. See their instructions here.
The trouble is I believe these are specifically for MAC OS, and Windows requires slightly different syntax for the following command:
wget https://github.com/confluentinc/kafka-connect-datagen/raw/master/config/connector_pageviews_cos.config
curl -X POST -H "Content-Type: application/json" --data #connector_pageviews_cos.config http://localhost:8083/connectors
I think I am supposed to be piping the wget results to the curl. How to do that in Windows 10 ?
The powershell exception:
At line:1 char:57
+ ... ntent-Type: application/json" --data #connector_pageviews_cos.config ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~
The splatting operator '#' cannot be used to reference variables in an expression. '#connector_pageviews_cos' can be
used only as an argument to a command. To reference variables in an expression use '$connector_pageviews_cos'.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : SplattingNotPermitted
Thanks!
Both curl and wget are aliased in Powershell 5.1 to Invoke-WebRequest, although it seems that curl.exe is available under System32 (and is available in place of an alias in Powershell Core). But, it's not too difficult to translate both calls to the Powershell-native Invoke-WebRequest cmdlet.
Your wget call becomes:
Invoke-WebRequest -Uri https://github.com/confluentinc/kafka-connect-datagen/raw/master/config/connector_pageviews_cos.config -Outfile connector_pageviews_cos.config
And your curl call becomes:
Invoke-WebRequest -Uri http://localhost:8083/connectors -Method POST -ContentType 'application/json' -Body (Get-Content -Raw connector_pageviews_cos.config)
If desired, you can combine both calls into a single line, as the -Body parameter accepts pipeline input:
Invoke-WebRequest -Uri https://github.com/confluentinc/kafka-connect-datagen/raw/master/config/connector_pageviews_cos.config |
Invoke-WebRequest -Uri http://localhost:8083/connectors -Method POST -ContentType 'application/json'
Omit the -OutFile parameter on the first Invoke-WebRequest, which outputs the payload contents to the pipeline. You then pipe that output into the second Invoke-WebRequest while providing other positional and named arguments as needed.
More information on Invoke-WebRequest is available here, and you would also do well to check out the convenient-to-use Invoke-RestMethod which makes working with RESTful APIs much more comfortable.
As a note, the # operator is used for a concept called splatting in Powershell, which explains the error you were receiving. Read the link above for more information, and elsewhere I've also written an answer on how to use splatting to pass arguments to cmdlets and commands.

REST API: adapt a cURL POST to Powershell 5 (upload files)

I have this cURL command that I must adapt to Powershell in order to upload different files. In order to do that, I have mandatory fields that I need use in the request.
I've tried like a dozen of different scripts, nothing works.
Can someone help me out with this?
It should be plain simple, but I am missing something.
cURL command:
curl -H "Authorization: Bearer xxx"
-F "parentDirectoryId=1"
-F "name=AutoUpload"
-F "contents=#C:\temp\test.pdf"
https://url/v1/api/files?
This is going to depend on which version of Powershell you are using. If you have Powershell 6 you can use the simple method below which uses the form parameter. If you use another version you can use the more complicated example #4 which is outlined on the Microsoft Docs:
$Uri = 'https://url/v1/api/files?'
$Form = #{
parentDirectoryId= '1'
name = 'AutoUpload'
contents= Get-Item -Path 'C:\temp\test.pdf'
}
$token = ConvertTo-SecureString "xxx" -AsPlainText -Force
$Result = Invoke-RestMethod -Uri $Uri -Method Post -Form $Form -Authentication Bearer -Token $token
This example uses the new Authentication and Form parameters in Invoke-RestMethod. Depending on your Authentication type, you will need either a Token or Credentials parameter with additional information. The Form parameter simplifies what was previously a complicated process for adjusting the body or URI per request.

How do I upload an attachment to a JIRA issue via powershell?

I have been searching online for a while and I've not found a solid answer to this (lots of partial answers, though). Still nothing I do works.
I'm trying to write a powershell script to send attachments to JIRA using cURL (have not found another way that I can get to work).
My cURL command is:
C:\opt\curl\curl.exe -u user:pa$$word -X POST -H "X-Atlassian-Token: nocheck" -F "file=#C:\opt\attachments\75391_testingPNG.png" http://jira.ourURL.com/rest/api/2/issue/75391/attachments
This works perfectly from the command line. Anytime I try to run it via powershell it bombs out. Seems like it should be very easy to do, though. Just want to grab the files from a directory and send them to JIRA.
Anyone have any thoughts about this??? Thanks!
I suspect that the characters $ and # in the arguments could be causing you problems (In case that is what you are using). Try escaping them using the backtick symbol.
To start curl.exe using the specified parameters, try the following command:
Start-Process C:\opt\curl\curl.exe -argumentList "-u", "user:pa`$`$Word", "-X", "POST", "-H", "`"X-Atlassian-Token: nocheck`"", "-F", "`"file=`#C:\opt\attachments\75391_testingPNG.png`"", "http://jira.ourURL.com/rest/api/2/issue/75391/attachments"
Basically it means that where you would separate arguments with a space in a command prompt, you would send each argument as an element in a powershell string Array and use it as the value in the -argumentlist parameter to Start-Process.
If you're using PowerShell 3+, you can use the native method Invoke-RestMethod to talk to JIRA, and not have to worry about escaping characters to shell out to a command:
# Build a basic auth header:
$headers = #{
'Authorization' = "Basic $([System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $UserName, $Password))))"
'X-Atlassian-Token' = 'nocheck'
}
Invoke-RestMethod -Uri "http://jira.ourURL.com/rest/api/2/issue/75391/attachments" -Method Post -InFile "C:\opt\attachments\75391_testingPNG.png" -ContentType "multipart/form-data" -Headers $headers
I'm actually not sure what the nocheck header you're adding does though; I've not needed it when talking to JIRA over REST.