I require to download a file programmatically from the releases section of a private GitHub repository, using powershell.
What I have tried so far:
$url = 'https://github.com/SatisfactoryModdingUE/UnrealEngine/releases/download/4.26.1-css-19/UnrealEngine-CSS-Editor-Win64-1.bin'
iwr -uri $url -outfile .\UnrealEngine-CSS-Editor-Win64-1.bin
This returns error: iwr : The request was aborted: The connection was closed unexpectedly..
This repo is a private repo. I have a username and personal access token that I can use to manually download, but I dont know how to apply these credentials to make the above download script work.
I found the solution. Thanks to:
Comment from #Justinas
API reference on list releases
API reference on the OAth authentication method
API reference on headers required for downloading binary assets
powershell script:
function get-latest-repo-release( [string]$gitPersonalAccessToken, [string]$owner, [string]$repo, [string]$dropFolder, $gitApiBase){
if ('' -eq "$gitApiBase"){
$gitApiBase = 'api.github.com'}
$parms = #{uri = "https://$gitApiBase/repos/$owner/$repo/releases"}
$headers = #{Accept = 'application/vnd.github.v3+json'}
if ('' -ne "$gitPersonalAccessToken"){
$headers['Authorization'] = "token $gitPersonalAccessToken"}
$parms['headers'] = $headers
$releases = iwr #parms | ConvertFrom-Json
$latestTag = $releases | sort -Property published_at -Descending | select -first 1 | select -expandProperty tag_name
$assets = $releases | ?{ $_.tag_name -eq $latestTag} | select -first 1 | select -expandProperty assets
$headers['Accept'] = 'application/octet-stream'
$assets | %{
$parms['uri'] = $_.url
$parms['outfile'] = "$dropFolder\$($_.name)"
iwr #parms}
$dropFolder}
$gitPersonalAccessToken = '<Redacted>'
$dropFolder = '<Redacted>'
$owner = 'SatisfactoryModdingUE'
$repo = 'UnrealEngine'
$outputDir = get-latest-repo-release -gitPersonalAccessToken $gitPersonalAccessToken -owner $owner -repo $repo -zipDropFolder $dropFolder
write-host "The latest releases can be found in this folder: $outputDir"
Related
I'm trying to get the files added or changed in the repository push that triggered the release pipeline.
Using a PowerShell task, I've been able to identify the files that were changed if they are include in the push's most-recent commit:
$Instance = 'my.org'
$Collection = 'MyCollection'
$Project = 'MyProject'
$RepositoryId = 'my_repo'
$BaseUri = "https://{0}/{1}/{2}/_apis/git/repositories/{3}" -f $Instance, $Collection, $Project, $RepositoryId
$ReleaseDirectory = Join-Path $env:AGENT_RELEASEDIRECTORY $env:RELEASE_PRIMARYARTIFACTSOURCEALIAS
# the BUILD_BUILDID has the GUID for the latest commit
$env:BUILD_BUILDID | ForEach-Object {
$Uri = "$BaseUri/commits/$_/changes?api-version=5.0"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$Response = Invoke-WebRequest -Uri $Uri -UseDefaultCredentials -UseBasicParsing
$Content = $Response | ConvertFrom-Json
$Content.changes | Select-Object #{name='path';expression={$_.item.path}} | ForEach-Object {
$ChangedItemPath = Join-Path $ReleaseDirectory $_.path
Get-Item -Path $ChangedItemPath
} | Where-Object { $_.PSIsContainer -eq $false }
}
The Get Push Commits API method looks promising, but I can't find the pushId in the pipeline's environment variables.
Is the pushId available? Do I need to take different approach?
Essentially what I'm after is the results of rest API Gateways - Get Datasource Users but retaining the ID (in this example $Line.id from my imported CSV file).
The end result should be a CSV with the following fields -
ID, emailAddress, datasourceAccessRight, displayName, identifier, principalType
I'm new to PowerShell and surprised I got this far but can't figure out this final bit.
Cheers
$webclient=New-Object System.Net.WebClient
$webclient.Proxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
$Dir = "C:\pbi_pro_user_logs\"
Login-PowerBI
$GateWayFile = Import-CSV -Path "C:\pbi_pro_user_logs\Gateway_Detail.csv"
$Output = #()
foreach ($Line in $GateWayFile){
$Item = $Line.id
$url = "https://api.powerbi.com/v1.0/myorg/gateways/HIDDEN/datasources/"+$Item+"/users"
$Output += (Invoke-PowerBIRestMethod -Url $url -Method Get | ConvertFrom-Json)
}
$Result = $Output.value
$Result | Export-Csv $Dir"GateWay_users.csv" -NoTypeInformation
Try this, using a calculated property from Select-Object:
$GateWayFile = Import-CSV -Path "C:\pbi_pro_user_logs\Gateway_Detail.csv"
$Output = Foreach ($Line in $GateWayFile){
$url = "https://api.powerbi.com/v1.0/myorg/gateways/HIDDEN/datasources/"+$Line.id+"/users"
$Item = (Invoke-PowerBIRestMethod -Url $url -Method Get | ConvertFrom-Json)
# output all properties of the item, plus the ID:
$ItemWithID = $Item | Select *,#{l='Id';e={$line.id}}
Write-Output $ItemWithID
}
# This depends on how you want your csv structured, but for example:
$Result = $Output | Select Id,Value
Or, if Value is a whole object that ID should be assigned inside of, then change the selection lines:
$ItemWithID = $Item.Value | Select *,#{l='Id';e={$line.id}}
$Result = $Output
I have the following script that will deploy an Octopus Deploy release based on the parameters I provide it.
param(
[string] $releaseVersion,
[array] $future
)
foreach($line in Get-Content C:\admin\customers.txt) {
$ErrorActionPreference = "Stop";
# Define working variables
$octopusURL = "http://10.2.2.62:8022/api"
$octopusAPIKey = "API-Key"
$headers = #{ "X-Octopus-ApiKey" = $octopusAPIKey }
$spaceName = "Default"
$projectName = "C9-Deployment"
$environmentName = "LabFarm2"
$tenantNames = $line
$date = get-date -Format yyyy-MM-dd
$expiredate = $(date).AddDays(1).ToString("yyyy-MM-dd")
# Get space id
$spaces = Invoke-WebRequest -Uri "$octopusURL/spaces/all" -Headers $headers -ErrorVariable octoError | ConvertFrom-Json
$space = $spaces | Where-Object { $_.Name -eq $spaceName }
Write-Host "Using Space named $($space.Name) with id $($space.Id)"
# Get project by name
$projects = Invoke-WebRequest -Uri "$octopusURL/projects/all" -Headers $headers -ErrorVariable octoError | ConvertFrom-Json
$project = $projects | Where-Object { $_.Name -eq $projectName }
Write-Host "Using Project named $($project.Name) with id $($project.Id)"
# Create space specific url
$octopusSpaceUrl = "$octopusURL/$($space.Id)"
# Get release by version
$releases = Invoke-RestMethod -Uri "$octopusSpaceUrl/projects/$($project.Id)/releases" -Headers $headers -ErrorVariable octoError
$release = $releases.Items | Where-Object { $_.Version -eq $releaseVersion }
Write-Host "Using Release version $($release.Version) with id $($release.Id)"
# Get environment by name
$environments = Invoke-RestMethod -Uri "$octopusSpaceUrl/environments?partialName=$([uri]::EscapeDataString($environmentName))&skip=0&take=100" -Headers $headers -ErrorVariable octoError
$environment = $environments.Items | Where-Object { $_.Name -eq $environmentName }
Write-Host "Using Environment named $($environment.Name) with id $($environment.Id)"
$tenants = Invoke-WebRequest -Uri "$octopusSpaceUrl/tenants/all" -Headers $headers -ErrorVariable octoError | ConvertFrom-Json
$tenantNames | ForEach-Object {
$name = $_
$tenant = $tenants | Where-Object { $_.Name -eq $name }
if ($future -eq $null) {
write-host "This deployment is for tonight"
$deploymentBody = #{
ReleaseId = $release.Id
EnvironmentId = $environment.Id
TenantId = $tenant.Id
QueueTime = "${date}T23:00:00"
QueueTimeExpiry = "${expiredate}T05:00:00"
} | ConvertTo-Json
}
if ($future -ne $null) {
write-host "This deployment will take place on $future"
#Problem Line 64 below
$expirefuturedate = (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
$deploymentBody = #{
ReleaseId = $release.Id
EnvironmentId = $environment.Id
TenantId = $tenant.Id
QueueTime = "${future}T23:00:00"
#problem line 70 below
QueueTimeExpiry = "${expirefuturedate}T05:00:00"
} | ConvertTo-Json
}
Write-Host "Creating deployment with these values: $deploymentBody"
$deployment = Invoke-WebRequest -Uri $octopusSpaceUrl/deployments -Method POST -Headers $headers -Body $deploymentBody -ErrorVariable octoError
}
}
So the problem is on line 64 and 70 where I try to take add one day to the Future parameter. If I run this with only the ReleaseVersion parameter set it will run fine without issues. But if I add a parameter for future like "-Future 2021-03-11" I get the following error:
PS C:\Users\bbelden.CLOUD9\Documents\powershell\Octopus> .\Deploycustom_parm.ps1 -releaseversion 8.1.2103.193 -future 20
21-03-11
Using Space named Default with id Spaces-1
Using Project named C9-Deployment with id Projects-101
Using Release version 8.1.2103.193 with id Releases-12243
Using Environment named LabFarm2 with id Environments-161
This deployment will take place on 2021-03-11
ForEach-Object : Cannot convert 'System.Object[]' to the type 'System.DateTime' required by parameter 'Date'.
Specified method is not supported.
At C:\Users\bbelden.CLOUD9\Documents\powershell\Octopus\Deploycustom_parm.ps1:47 char:16
+ $tenantNames | ForEach-Object {
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [ForEach-Object], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgument,Microsoft.PowerShell.Commands.ForEachObjectCommand
Now if I just remove the line 64 completely and manually set a time date stamp in line 70 like the following:
if ($future -ne $null) {
write-host "This deployment will take place on $future"
#$expirefuturedate = (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
$deploymentBody = #{
ReleaseId = $release.Id
EnvironmentId = $environment.Id
TenantId = $tenant.Id
QueueTime = "${future}T23:00:00"
QueueTimeExpiry = "2021-03-11T05:00:00"
} | ConvertTo-Json
}
It will work fine.
So I am not really sure what I am missing here. Please let me know if there is something I am doing wrong. I believe it has to do with the Array, because if I comment that line out the issue will go away, but I need a way to transform the $future variable to adding one day to it.
Thanks
This should explain your error:
PS /home/> $future=[array]'2021-03-11'
PS /home/> (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
Get-Date: Cannot convert 'System.Object[]' to the type 'System.DateTime' required by parameter 'Date'. Specified method is not supported.
PS /home/> $future=[datetime]'2021-03-11'
PS /home/> (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
2021-03-12
Param(
[string]$collectionurl = "https://dev.azure.com",
[string]$project = "projectname",
[string]$token = "PAT"
)
# Base64-encodes the Personal Access Token (PAT) appropriately
$base64AuthInfo =
[Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}" -f
$token)))
$baseUrl =
"$collectionurl/$project/_apis/wit/reporting/workitemrevisions?
includeLatestOnly=true&api-version=5.0-preview.2"
$response = (Invoke-RestMethod -Uri $baseUrl -Method Get -
UseDefaultCredential -Headers #{Authorization=("Basic {0}" -f
$base64AuthInfo)}).values
$wits = $response | where({$_.fields.'System.WorkItemType' -eq
'Task'}) # Only retrieve Tasks
$witrevisions = #()
foreach($wit in $wits){
$customObject = new-object PSObject -property #{
"WitID" = $wit.fields.'System.Id'
"rev" = $wit.fields.'System.Rev'
"Title" = $wit.fields.'System.Title'
"AssignedTo" = $wit.fields.'System.AssignedTo'
"ChangedDate" = $wit.fields.'System.ChangedDate'
"ChangedBy" = $wit.fields.'System.ChangedBy'
"WorkItemType" = $wit.fields.'System.WorkItemType'
}
$witrevisions += $customObject
}
$witrevisions | Select-Object `
WitID,
rev,
Title,
AssignedTo,
ChangedDate,
ChangedBy,
WorkItemType #|export-csv -Path E:\ashwin\devdata.csv -
NoTypeInformation
Write-Output $witrevisions
I want to display the workitems in my project to be displayed using powershell with the following Rest Api using my PAT.
https://dev.azure.com/{organization}/{project}/_apis/wit/workitems/{id}?api-version=5.1
How to get the workitems from AzureDevOps with RestApi in Powershell
The result will display in the output, you will find like following:
If you do not find above output, make sure you have workitem with Task type, because you have set the condition 'System.WorkItemType' -eq 'Task' in the powershell scripts.
On the other hand, you could export the work item list to a *.csv file, this part of the code is commented in powershell:
WorkItemType #| export-csv -Path G:\temp\WIT.csv -NoTypeInformation
If you want to create a *.csv file, you need to remove the # in that line, it should be :
WorkItemType | export-csv -Path G:\temp\WIT.csv -NoTypeInformation
Now, we could get that file in our local folder:
Note: The path is G:\temp is a local path, you should use the private agent, if you are using the hosted agent, you should copy that file from the hosted agent, and publish it to pipeline artifact.
Hope this helps.
function Get-Data(){
[PSObject[]]$pid = ''
$getUri1 = 'https://playbook2.com/data/project/folder/28220'
$projectIds = wget $getUri1 -UseDefaultCredentials |
ConvertFrom-JSON | Select data | select -Expand data | select id
Write-Host $projectIds
#obtain all the project ids
ForEach-Object{
[PSObject[]]$pid += $projectIds.id
}
Write-Host $pid
$uri3 = "https://playbook2.com/data/project/export/projects-tasks?projectIds[]="
$getIds = [PSObject[]]$pid -join "&projectIds[]="
$getUri2 = $uri3 + $getIds
$of = "\\ant\dept\DCGSI\Extracts\Time_Tracking_Tasks.xlsx"
Write-Host $getUri2
#retrieve excel files of tasks from each sub-folder
wget $getUri2 -outfile $of -UseDefaultCredentials
}
This code is an adaptation of some other code that I wrote. The 5 other scripts work fine. The main difference is that the other code has to loop through multiple folders and gets the project IDs under each folder, but this code only has to go through a single folder. Now in the other code the $uri3, $getIds code works fine and I get an export. The problem I am seeing in this code is that it isn't joining the URL the way I expect.
https://playbook2.com/data/project/export/projects-tasks?projectIds[]=######&projectIds[]=####### is the expected and previously seen output to get all the project data i need.
The problem with the above script is that it is giving https://playbook2.com/data/project/export/projects-tasks?projectIds[]=&projectIds[]=######&projectIds[]=####### which is invalid.
is there a way that I can tell it to do just $pid for the first item in the object and then -join the "&projectIds[]=" on the next n until the end of the list? I tried
[PSObject[]]$pid | select -Skip 1 -join "&projectIds[]="
and
[PSObject[]]$pid | Select-Object -Skip 1 -join "&projectIds[]="
but that results in nothing being appended.
I found a couple of "mistakes" in your script.
First is that you are using the variable $pid which is an system default variable. You can check the system global variables by typing
Get-Variable
Secondly $pid is defined with an empty string. The correct way to initialize a PSObject is with $myVar = New-Object PSObject. Replace [PSObject[]]$pid = '' with $myProjectIds = New-Object PSObject
For readability I took the liberty to rewrite your script.
function Get-Data(){
$GetProjectsUri = 'https://playbook2.com/data/project/folder/28220'
$ExportProjectsUri = 'https://playbook2.com/data/project/export/projects-tasks?'
$ExportFilePath = "\\ant\dept\DCGSI\Extracts\Time_Tracking_Tasks.xlsx"
$GetProjectsJson = Invoke-WebRequest -Uri $GetProjectsUri -UseDefaultCredentials
Write-Output $GetProjectsJson
$Projects = ConvertFrom-JSON -InputObject $GetProjectsJson
Write-Output $Projects
foreach ($Project in $Projects) {
$ProjectId = $Project.data.id
# Check if ProjectId exists
if ($ProjectId) {
$ExportProjectsUri = $ExportProjectsUri + 'projectIds[]=' + $ProjectId
}
}
Write-Output $ExportProjectsUri
Invoke-WebRequest Invoke-WebRequest -Uri $ExportProjectsUri -outfile $ExportFilePath -UseDefaultCredentials
}
Cheers
Glenn