I'm trying to get the files added or changed in the repository push that triggered the release pipeline.
Using a PowerShell task, I've been able to identify the files that were changed if they are include in the push's most-recent commit:
$Instance = 'my.org'
$Collection = 'MyCollection'
$Project = 'MyProject'
$RepositoryId = 'my_repo'
$BaseUri = "https://{0}/{1}/{2}/_apis/git/repositories/{3}" -f $Instance, $Collection, $Project, $RepositoryId
$ReleaseDirectory = Join-Path $env:AGENT_RELEASEDIRECTORY $env:RELEASE_PRIMARYARTIFACTSOURCEALIAS
# the BUILD_BUILDID has the GUID for the latest commit
$env:BUILD_BUILDID | ForEach-Object {
$Uri = "$BaseUri/commits/$_/changes?api-version=5.0"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$Response = Invoke-WebRequest -Uri $Uri -UseDefaultCredentials -UseBasicParsing
$Content = $Response | ConvertFrom-Json
$Content.changes | Select-Object #{name='path';expression={$_.item.path}} | ForEach-Object {
$ChangedItemPath = Join-Path $ReleaseDirectory $_.path
Get-Item -Path $ChangedItemPath
} | Where-Object { $_.PSIsContainer -eq $false }
}
The Get Push Commits API method looks promising, but I can't find the pushId in the pipeline's environment variables.
Is the pushId available? Do I need to take different approach?
Related
Hello all, I've been able to get this code to get the latest download link for Nvidia Driver for a Quadro P1000.
I was wondering, if there's a better way to do this.
Here's my code:
#Get Nvidia Drivers
#This next section is JUST TO get the LINK to the LATEST driver from Nvidia
#WebContent gets the links that are "href" from the nvidia JS request. No idea if there's an easier wey to get this.
$Webcontent = (Invoke-WebRequest 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0').links.href
#The following line uses Regex regular expressions to MATCH and RETRIEVE ONE single value from the list of values in the previous line.
$NVIDIALinkToLatestDriver = [regex]::Match($Webcontent, '//(www.nvidia.com/download/driverResults.aspx/[0-9]*/en-us)').Groups[1].Value
#Link after the previous crap
$NVIDIADLPage = New-Object -COM "HTMLFile" #Creates a COM Object for easier search of the link.
[string]$htmlBody = (Invoke-WebRequest -Uri $NVIDIALinkToLatestDriver -UseBasicParsing).Content #Parses the content of the landing page to then look by id
$NVIDIADLPage.write([ref]$htmlBody)
$replace = [regex]::Replace($NVIDIADLPage.getElementById('lnkDwnldBtn').href, 'about:', 'www.nvidia.com') #Replaces the "about" with "www.nvidia.com"
$Webcontent = (Invoke-WebRequest $replace) #Replace Webcontent with the latest link.
[String]$NvidiaLinkToExe = $Webcontent.links.href -match ".*.exe$" #On this link there's the exe file for Nvidia Drivers
$NvidiaLinkToExe = $NvidiaLinkToExe -replace "^", "http:" #Replace start of line with the correct Nvidia Link.
Remove-Variable -Name NVIDIADLPage, Webcontent, replace -Force #cleanup of the previous mess.
if ($NvidiaLinkToExe -match 'http:\/\/.*[0-9]{2}\/(([0-9]{3}\.[0-9]{2}).*\.exe)') {
$NVIDIAExeToDownload = [PSCustomObject]#{
Url = $Matches[0];
Name = $Matches[1];
Version = $Matches[2]
}
}
As you can see, I have to create three Invoke-WebRequest just to get one link.
And, I think I made no use of piping, because I was unable to make it work.
Thanks!
Your code seems better than mine. I had to call Invoke-WebRequest 3 times to get the link as well and once more to download it.
$destination = 'C:\Temp'
$downloadlist = 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0'
$pattern = "(?s)<tr>.+?href='//(?<URL>.+?)'>.+?<td.+?>(?<Version>.+?)</td.+?td.+?>(?<Date>.+?)</td.+</tr>"
$content = Invoke-WebRequest $downloadlist -UseBasicParsing
$download = if($content.RawContent -match $pattern){
[PSCustomObject]#{
URL = $Matches.URL
Version = $Matches.Version
Date = $Matches.Date
}
}
$pattern = '(?s)(?<Package>/content/driver[^"]+?{0}/.+?)(?=")' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest $download.url -UseBasicParsing
if($content.RawContent -match $pattern){
$pattern = '//(?<Package>.+?{0}.+exe)' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest -Uri "https://www.nvidia.com$($Matches.Package)" -UseBasicParsing
if($content.RawContent -match $pattern){
Invoke-WebRequest "https://$($Matches.Package)" -OutFile (Join-Path $destination $($Matches.package -replace '.+/'))
}
}
If you just want the link and other info you can drop the fourth.
$downloadlist = 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0'
$content = Invoke-WebRequest $downloadlist -UseBasicParsing
$download = if($content.RawContent -match "(?s)<tr>.+?href='//(?<URL>.+?)'>.+?<td.+?>(?<Version>.+?)</td.+?td.+?>(?<Date>.+?)</td.+</tr>"){
[PSCustomObject]#{
URL = $Matches.URL
Version = $Matches.Version
Date = $Matches.Date
}
}
$pattern = '(?s)(?<Package>/content/driver[^"]+?{0}/.+?)(?=")' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest $download.url -UseBasicParsing
if($content.RawContent -match $pattern){
$pattern = '//(?<Package>.+?{0}.+exe)' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest -Uri "https://www.nvidia.com$($Matches.Package)" -UseBasicParsing
$download.URL = "https://$($Matches.Package)"
$download
}
I require to download a file programmatically from the releases section of a private GitHub repository, using powershell.
What I have tried so far:
$url = 'https://github.com/SatisfactoryModdingUE/UnrealEngine/releases/download/4.26.1-css-19/UnrealEngine-CSS-Editor-Win64-1.bin'
iwr -uri $url -outfile .\UnrealEngine-CSS-Editor-Win64-1.bin
This returns error: iwr : The request was aborted: The connection was closed unexpectedly..
This repo is a private repo. I have a username and personal access token that I can use to manually download, but I dont know how to apply these credentials to make the above download script work.
I found the solution. Thanks to:
Comment from #Justinas
API reference on list releases
API reference on the OAth authentication method
API reference on headers required for downloading binary assets
powershell script:
function get-latest-repo-release( [string]$gitPersonalAccessToken, [string]$owner, [string]$repo, [string]$dropFolder, $gitApiBase){
if ('' -eq "$gitApiBase"){
$gitApiBase = 'api.github.com'}
$parms = #{uri = "https://$gitApiBase/repos/$owner/$repo/releases"}
$headers = #{Accept = 'application/vnd.github.v3+json'}
if ('' -ne "$gitPersonalAccessToken"){
$headers['Authorization'] = "token $gitPersonalAccessToken"}
$parms['headers'] = $headers
$releases = iwr #parms | ConvertFrom-Json
$latestTag = $releases | sort -Property published_at -Descending | select -first 1 | select -expandProperty tag_name
$assets = $releases | ?{ $_.tag_name -eq $latestTag} | select -first 1 | select -expandProperty assets
$headers['Accept'] = 'application/octet-stream'
$assets | %{
$parms['uri'] = $_.url
$parms['outfile'] = "$dropFolder\$($_.name)"
iwr #parms}
$dropFolder}
$gitPersonalAccessToken = '<Redacted>'
$dropFolder = '<Redacted>'
$owner = 'SatisfactoryModdingUE'
$repo = 'UnrealEngine'
$outputDir = get-latest-repo-release -gitPersonalAccessToken $gitPersonalAccessToken -owner $owner -repo $repo -zipDropFolder $dropFolder
write-host "The latest releases can be found in this folder: $outputDir"
I have the following script that will deploy an Octopus Deploy release based on the parameters I provide it.
param(
[string] $releaseVersion,
[array] $future
)
foreach($line in Get-Content C:\admin\customers.txt) {
$ErrorActionPreference = "Stop";
# Define working variables
$octopusURL = "http://10.2.2.62:8022/api"
$octopusAPIKey = "API-Key"
$headers = #{ "X-Octopus-ApiKey" = $octopusAPIKey }
$spaceName = "Default"
$projectName = "C9-Deployment"
$environmentName = "LabFarm2"
$tenantNames = $line
$date = get-date -Format yyyy-MM-dd
$expiredate = $(date).AddDays(1).ToString("yyyy-MM-dd")
# Get space id
$spaces = Invoke-WebRequest -Uri "$octopusURL/spaces/all" -Headers $headers -ErrorVariable octoError | ConvertFrom-Json
$space = $spaces | Where-Object { $_.Name -eq $spaceName }
Write-Host "Using Space named $($space.Name) with id $($space.Id)"
# Get project by name
$projects = Invoke-WebRequest -Uri "$octopusURL/projects/all" -Headers $headers -ErrorVariable octoError | ConvertFrom-Json
$project = $projects | Where-Object { $_.Name -eq $projectName }
Write-Host "Using Project named $($project.Name) with id $($project.Id)"
# Create space specific url
$octopusSpaceUrl = "$octopusURL/$($space.Id)"
# Get release by version
$releases = Invoke-RestMethod -Uri "$octopusSpaceUrl/projects/$($project.Id)/releases" -Headers $headers -ErrorVariable octoError
$release = $releases.Items | Where-Object { $_.Version -eq $releaseVersion }
Write-Host "Using Release version $($release.Version) with id $($release.Id)"
# Get environment by name
$environments = Invoke-RestMethod -Uri "$octopusSpaceUrl/environments?partialName=$([uri]::EscapeDataString($environmentName))&skip=0&take=100" -Headers $headers -ErrorVariable octoError
$environment = $environments.Items | Where-Object { $_.Name -eq $environmentName }
Write-Host "Using Environment named $($environment.Name) with id $($environment.Id)"
$tenants = Invoke-WebRequest -Uri "$octopusSpaceUrl/tenants/all" -Headers $headers -ErrorVariable octoError | ConvertFrom-Json
$tenantNames | ForEach-Object {
$name = $_
$tenant = $tenants | Where-Object { $_.Name -eq $name }
if ($future -eq $null) {
write-host "This deployment is for tonight"
$deploymentBody = #{
ReleaseId = $release.Id
EnvironmentId = $environment.Id
TenantId = $tenant.Id
QueueTime = "${date}T23:00:00"
QueueTimeExpiry = "${expiredate}T05:00:00"
} | ConvertTo-Json
}
if ($future -ne $null) {
write-host "This deployment will take place on $future"
#Problem Line 64 below
$expirefuturedate = (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
$deploymentBody = #{
ReleaseId = $release.Id
EnvironmentId = $environment.Id
TenantId = $tenant.Id
QueueTime = "${future}T23:00:00"
#problem line 70 below
QueueTimeExpiry = "${expirefuturedate}T05:00:00"
} | ConvertTo-Json
}
Write-Host "Creating deployment with these values: $deploymentBody"
$deployment = Invoke-WebRequest -Uri $octopusSpaceUrl/deployments -Method POST -Headers $headers -Body $deploymentBody -ErrorVariable octoError
}
}
So the problem is on line 64 and 70 where I try to take add one day to the Future parameter. If I run this with only the ReleaseVersion parameter set it will run fine without issues. But if I add a parameter for future like "-Future 2021-03-11" I get the following error:
PS C:\Users\bbelden.CLOUD9\Documents\powershell\Octopus> .\Deploycustom_parm.ps1 -releaseversion 8.1.2103.193 -future 20
21-03-11
Using Space named Default with id Spaces-1
Using Project named C9-Deployment with id Projects-101
Using Release version 8.1.2103.193 with id Releases-12243
Using Environment named LabFarm2 with id Environments-161
This deployment will take place on 2021-03-11
ForEach-Object : Cannot convert 'System.Object[]' to the type 'System.DateTime' required by parameter 'Date'.
Specified method is not supported.
At C:\Users\bbelden.CLOUD9\Documents\powershell\Octopus\Deploycustom_parm.ps1:47 char:16
+ $tenantNames | ForEach-Object {
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [ForEach-Object], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgument,Microsoft.PowerShell.Commands.ForEachObjectCommand
Now if I just remove the line 64 completely and manually set a time date stamp in line 70 like the following:
if ($future -ne $null) {
write-host "This deployment will take place on $future"
#$expirefuturedate = (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
$deploymentBody = #{
ReleaseId = $release.Id
EnvironmentId = $environment.Id
TenantId = $tenant.Id
QueueTime = "${future}T23:00:00"
QueueTimeExpiry = "2021-03-11T05:00:00"
} | ConvertTo-Json
}
It will work fine.
So I am not really sure what I am missing here. Please let me know if there is something I am doing wrong. I believe it has to do with the Array, because if I comment that line out the issue will go away, but I need a way to transform the $future variable to adding one day to it.
Thanks
This should explain your error:
PS /home/> $future=[array]'2021-03-11'
PS /home/> (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
Get-Date: Cannot convert 'System.Object[]' to the type 'System.DateTime' required by parameter 'Date'. Specified method is not supported.
PS /home/> $future=[datetime]'2021-03-11'
PS /home/> (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
2021-03-12
Param(
[string]$collectionurl = "https://dev.azure.com",
[string]$project = "projectname",
[string]$token = "PAT"
)
# Base64-encodes the Personal Access Token (PAT) appropriately
$base64AuthInfo =
[Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}" -f
$token)))
$baseUrl =
"$collectionurl/$project/_apis/wit/reporting/workitemrevisions?
includeLatestOnly=true&api-version=5.0-preview.2"
$response = (Invoke-RestMethod -Uri $baseUrl -Method Get -
UseDefaultCredential -Headers #{Authorization=("Basic {0}" -f
$base64AuthInfo)}).values
$wits = $response | where({$_.fields.'System.WorkItemType' -eq
'Task'}) # Only retrieve Tasks
$witrevisions = #()
foreach($wit in $wits){
$customObject = new-object PSObject -property #{
"WitID" = $wit.fields.'System.Id'
"rev" = $wit.fields.'System.Rev'
"Title" = $wit.fields.'System.Title'
"AssignedTo" = $wit.fields.'System.AssignedTo'
"ChangedDate" = $wit.fields.'System.ChangedDate'
"ChangedBy" = $wit.fields.'System.ChangedBy'
"WorkItemType" = $wit.fields.'System.WorkItemType'
}
$witrevisions += $customObject
}
$witrevisions | Select-Object `
WitID,
rev,
Title,
AssignedTo,
ChangedDate,
ChangedBy,
WorkItemType #|export-csv -Path E:\ashwin\devdata.csv -
NoTypeInformation
Write-Output $witrevisions
I want to display the workitems in my project to be displayed using powershell with the following Rest Api using my PAT.
https://dev.azure.com/{organization}/{project}/_apis/wit/workitems/{id}?api-version=5.1
How to get the workitems from AzureDevOps with RestApi in Powershell
The result will display in the output, you will find like following:
If you do not find above output, make sure you have workitem with Task type, because you have set the condition 'System.WorkItemType' -eq 'Task' in the powershell scripts.
On the other hand, you could export the work item list to a *.csv file, this part of the code is commented in powershell:
WorkItemType #| export-csv -Path G:\temp\WIT.csv -NoTypeInformation
If you want to create a *.csv file, you need to remove the # in that line, it should be :
WorkItemType | export-csv -Path G:\temp\WIT.csv -NoTypeInformation
Now, we could get that file in our local folder:
Note: The path is G:\temp is a local path, you should use the private agent, if you are using the hosted agent, you should copy that file from the hosted agent, and publish it to pipeline artifact.
Hope this helps.
function Get-Data(){
[PSObject[]]$pid = ''
$getUri1 = 'https://playbook2.com/data/project/folder/28220'
$projectIds = wget $getUri1 -UseDefaultCredentials |
ConvertFrom-JSON | Select data | select -Expand data | select id
Write-Host $projectIds
#obtain all the project ids
ForEach-Object{
[PSObject[]]$pid += $projectIds.id
}
Write-Host $pid
$uri3 = "https://playbook2.com/data/project/export/projects-tasks?projectIds[]="
$getIds = [PSObject[]]$pid -join "&projectIds[]="
$getUri2 = $uri3 + $getIds
$of = "\\ant\dept\DCGSI\Extracts\Time_Tracking_Tasks.xlsx"
Write-Host $getUri2
#retrieve excel files of tasks from each sub-folder
wget $getUri2 -outfile $of -UseDefaultCredentials
}
This code is an adaptation of some other code that I wrote. The 5 other scripts work fine. The main difference is that the other code has to loop through multiple folders and gets the project IDs under each folder, but this code only has to go through a single folder. Now in the other code the $uri3, $getIds code works fine and I get an export. The problem I am seeing in this code is that it isn't joining the URL the way I expect.
https://playbook2.com/data/project/export/projects-tasks?projectIds[]=######&projectIds[]=####### is the expected and previously seen output to get all the project data i need.
The problem with the above script is that it is giving https://playbook2.com/data/project/export/projects-tasks?projectIds[]=&projectIds[]=######&projectIds[]=####### which is invalid.
is there a way that I can tell it to do just $pid for the first item in the object and then -join the "&projectIds[]=" on the next n until the end of the list? I tried
[PSObject[]]$pid | select -Skip 1 -join "&projectIds[]="
and
[PSObject[]]$pid | Select-Object -Skip 1 -join "&projectIds[]="
but that results in nothing being appended.
I found a couple of "mistakes" in your script.
First is that you are using the variable $pid which is an system default variable. You can check the system global variables by typing
Get-Variable
Secondly $pid is defined with an empty string. The correct way to initialize a PSObject is with $myVar = New-Object PSObject. Replace [PSObject[]]$pid = '' with $myProjectIds = New-Object PSObject
For readability I took the liberty to rewrite your script.
function Get-Data(){
$GetProjectsUri = 'https://playbook2.com/data/project/folder/28220'
$ExportProjectsUri = 'https://playbook2.com/data/project/export/projects-tasks?'
$ExportFilePath = "\\ant\dept\DCGSI\Extracts\Time_Tracking_Tasks.xlsx"
$GetProjectsJson = Invoke-WebRequest -Uri $GetProjectsUri -UseDefaultCredentials
Write-Output $GetProjectsJson
$Projects = ConvertFrom-JSON -InputObject $GetProjectsJson
Write-Output $Projects
foreach ($Project in $Projects) {
$ProjectId = $Project.data.id
# Check if ProjectId exists
if ($ProjectId) {
$ExportProjectsUri = $ExportProjectsUri + 'projectIds[]=' + $ProjectId
}
}
Write-Output $ExportProjectsUri
Invoke-WebRequest Invoke-WebRequest -Uri $ExportProjectsUri -outfile $ExportFilePath -UseDefaultCredentials
}
Cheers
Glenn