How to use PowerShell script to download latest private GitHub release? - powershell

I recently wrote a PowerShell script that downloads the latest release from a public repo and that works as intended. However, I want to change my script so it can access my private repo. Here is the code I have tried so far:
# Download latest release from GitHub
$credentials="myPersonalAccessToken"
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", "token $credentials")
$repo = "myUserName/MyPrivateReleaseRepo"
$file = "MyBinaries.zip"
$releases = "https://api.github.com/repos/$repo/releases"
Write-Host Determining latest release
$tag = (Invoke-WebRequest $releases -Headers $headers | ConvertFrom-Json)[0].tag_name
$download = "https://github.com/$repo/releases/download/$tag/$file"
$name = $file.Split(".")[0]
$zip = "$name-$tag.zip"
$dir = "$name-$tag"
Write-Host Dowloading latest release
Invoke-WebRequest $download -Headers $headers -Out $zip
Write-Host Extracting release files
Expand-Archive $zip -Force
# Cleaning up target dir
Remove-Item "C:\MyOutPutFolder\$name" -Recurse -Force -ErrorAction SilentlyContinue
# Moving from temp dir to target dir
Move-Item $dir\$name -Destination "C:\MyOutPutFolder\$name" -Force
# Removing temp files
Remove-Item $zip -Force
Remove-Item $dir -Recurse -Force
I get the following error only when using my private repo:
Invoke-WebRequest : The remote server returned an error: (404) Not Found
At C:\Script\DownloadLatestGitHubRelease.ps1:25 char:1
+ Invoke-WebRequest $download -Headers $headers -Out $zip
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
I've also tried providing bad credentials vs the correct credentials and got a "Bad Credentials" error when providing the incorrect ones as expected, so I believe I'm using the token correctly.
What am I doing wrong? Thanks in advance for any suggestions.

Landed here trying to do the same thing. After digging into some bash scripts that do similar, I came up with this to do the download:
$token = "<your token from https://github.com/settings/tokens>"
$downloadFolder = "C:\temp";
$ownerSlashRepo = "owner/reop";
$tag = "latest";
$json = Invoke-Webrequest -Uri "https://api.github.com/repos/$ownerSlashRepo/releases/$tag" -Headers #{'Authorization'='token '+$token; 'Accept'='application/json'}
$release = $json.Content | ConvertFrom-Json
$release.assets | %{
$asset = $_;
$x = Invoke-Webrequest -Uri $($asset.url) -OutFile "$downloadFolder\$($asset.name)" -Headers #{'Authorization'='token '+$token; 'Accept'='application/octet-stream'}
}
This downloads every file from the release, but you could filter it to just download one as needed, something like:
$release.assets | ?{$_.name -eq 'foo.zip'} | %{
To answer your question, in the original code you're attempting to use the token with https://github.com. I tried similar at first and got similar results to what you describe. I think it is only good for https://api.github.com and the REST API. I also found I had to specify Accept:application/octet-stream when downloading the asset.

Related

synchronize an onprem fileshare to a sharepoint online site collection using powershell and Microsof RestAPI

I am trying to work out a powershell script that:
retrieves an accesstoken (MSAL) to access (read/write) a sharepoint online site with subsites and documents. Preferably the Azure APP-registration ServicePrincipal can be granted access to just that site and access the sharepoint site/files without giving consent to the whole sharepoint environment. I don't know if that is possible currently as I can only grant application permission to files.readwrite.all and sites.readwrite.all. I do not see anything like files.readwrite.shared to grant access only to sites/collections that the serviceprincipal has access to. Anyone done this? I currently use the MSAL.PS powershell module to get a token using an AppRegistration with the admin-consented readwrite.all access but would like to limit that. The code for this is now:
Import-Module MSAL.PS;
$clientid = "my-appreg-client-id";
$tenantID = 'my-tenant-id';
$thumbPrint = 'certificate-thumbprint';
$ClientCertificate = Get-Item "Cert:\CurrentUser\My\$thumbPrint";
$myAccessToken = Get-MsalToken -ClientId $clientID -TenantId $tenantID -ClientCertificate
$ClientCertificate;
The script will read all files and folders from an UNC-share and build a file-collection of the onprem files. That part of the code is in place using a Get-ChildItem call to the UNC filetree.
Then, after getting the token, I need to get the current available files in the sharepoint online site document library structure and store that in a variable/hashtable which I can use to perform lookups between the onprem filecollection and the presence of those files and (sub)folders in the sharepoint site. If a folder does not yet exist I need to create that sharepoint folder and if a file is not yet present or the onprem version is newer I need to upload that file into sharepoint.
I have a script that does this using the old sharepoint.client.dll libraries but those support only basic authentication which will be unavailable any time soon for accessing the MS Online environment. So now I am searching for code to do this using the Microsoft Graph Api or other Rest API call. I am already struggling to get the contents of a site file collection so I hope that this generic problem description is enough to get some hints and tips/resources to get going.
Many thanks,
Eric
This is what I use. I'm using powershell in Linux.
## Get the Token
$clientId = "Application (Client) ID"
$clientSecret = "Client secret"
$tenantName = "TenantName.onmicrosoft.com"
$tokenBody = #{
Grant_Type = 'client_credentials'
Scope = 'https://graph.microsoft.com/.default'
Client_Id = $clientId
Client_Secret = $clientSecret
}
$tokenResponse = Invoke-RestMethod -Uri "https://login.microsoftonline.com/$TenantName/oauth2/v2.0/token" -Method POST -Body $tokenBody -ErrorAction Stop
$headers = #{
"Authorization" = "Bearer $($tokenResponse.access_token)"
"Content-Type" = "application/json"
}
## Use the SharePoint groups ObjectID. From this we'll get the drive ID.
$site_objectid = "Groups ObjectID"
## Create all the folders on the SharePoint site first. I've set microsoft.graph.conflictBehavior below to fail because I never want to rename or replace folders.
# Set the base directory.
$baseDirectory = "/test"
$directories = get-childItem -path $baseDirectory -recurse -directory
foreach ($directory in $directories) {
$URL = "https://graph.microsoft.com/v1.0/groups/$site_objectid/sites/root"
$subsite_ID = (Invoke-RestMethod -Headers $headers -Uri $URL -Method Get).ID
$URL = "https://graph.microsoft.com/v1.0/sites/$subsite_ID/drives"
$Drives = Invoke-RestMethod -Headers $headers -Uri $URL -Method Get
$Document_drive_ID = ($Drives.value | Where-Object { $_.name -eq 'Documents' }).id
$createFolderURL = "https://graph.microsoft.com/v1.0/drives/$Document_drive_ID/items/root:{0}:/children" -f $directory.parent.FullName
$file = $directory.Name
$uploadFolderRequestBody = #{
name= "$file"
folder = #{}
"#microsoft.graph.conflictBehavior"= "fail"
} | ConvertTo-Json
invoke-restMethod -headers $headers -method Post -body $uploadFolderRequestBody -contentType "application/json" -uri $createFolderURL
}
## Upload the files. I'm only adding files that are 4 days old or less because I run the script every 3 days for backup.
## These are set in the $sharefiles variable. To upload all files just remove everything after the pipe.
$sharefiles = get-childItem $baseDirectory -recurse | Where-Object {$_.LastWriteTime -gt (Get-Date).AddDays(-4)}
foreach ($sharefile in $sharefiles) {
$Filepath = $sharefile.FullName
$URL = "https://graph.microsoft.com/v1.0/groups/$site_objectid/sites/root"
$subsite_ID = (Invoke-RestMethod -Headers $headers -Uri $URL -Method Get).ID
$URL = "https://graph.microsoft.com/v1.0/sites/$subsite_ID/drives"
$Drives = Invoke-RestMethod -Headers $headers -Uri $URL -Method Get
$Document_drive_ID = ($Drives.value | Where-Object { $_.name -eq 'Documents' }).id
$Filename = $sharefile.Name
$upload_session = "https://graph.microsoft.com/v1.0/drives/$Document_drive_ID/root:{0}/$($Filename):/createUploadSession" -f $sharefile.directory.FullName
$upload_session_url = (Invoke-RestMethod -Uri $upload_session -Headers $headers -Method Post).uploadUrl
## We'll upload files in chunks.
$ChunkSize = 62259200
$file = New-Object System.IO.FileInfo($Filepath)
$reader = [System.IO.File]::OpenRead($Filepath)
$buffer = New-Object -TypeName Byte[] -ArgumentList $ChunkSize
$position = 0
$counter = 0
Write-Host "ChunkSize: $ChunkSize" -ForegroundColor Cyan
Write-Host "BufferSize: $($buffer.Length)" -ForegroundColor Cyan
$moreData = $true
While ($moreData) {
#Read a chunk
$bytesRead = $reader.Read($buffer, 0, $buffer.Length)
$output = $buffer
If ($bytesRead -ne $buffer.Length) {
#no more data to be read
$moreData = $false
#shrink the output array to the number of bytes
$output = New-Object -TypeName Byte[] -ArgumentList $bytesRead
[Array]::Copy($buffer, $output, $bytesRead)
Write-Host "no more data" -ForegroundColor Yellow
}
#Upload the chunk
$Header = #{
'Content-Range' = "bytes $position-$($position + $output.Length - 1)/$($file.Length)"
}
Write-Host "Content-Range = bytes $position-$($position + $output.Length - 1)/$($file.Length)" -ForegroundColor Cyan
#$position = $position + $output.Length - 1
$position = $position + $output.Length
Invoke-RestMethod -Method Put -Uri $upload_session_url -Body $output -Headers $Header -SkipHeaderValidation
#Increment counter
$counter++
}
$reader.Close()
}

Power shell script which silently installs softwares

# Source file location
$source1 ="https://ftp.mozilla.org/pub/firefox/releases/11.0/win32/enUS/Firefox%20Setup%2011.0.exe"
$source2 ="https://www.fosshub.com/Code-Blocks.html?dwl=codeblocks-20.03-setup.exe"
$source3 ="https://github.com/x64dbg/x64dbg/releases/download/snapshot/snapshot_2021-07-01_23-17.zip"
$source4 ="https://www.python.org/ftp/python/3.9.6/python-3.9.6-amd64.exe"
$source5 ="https://www.nasm.us/pub/nasm/releasebuilds/2.15.05/win64/nasm-2.15.05-installer-x64.exe"
# Destination to save the file
$destination1 = "C:\Users\cdac\Desktop\Shravan\Softwares\firefox.exe"
$destination2 = "C:\Users\cdac\Desktop\Shravan\Softwares\codeblocks.exe"
$destination3 = "C:\Users\cdac\Desktop\Shravan\softwares\xdbg.zip"
$destination4 = "C:\Users\cdac\Desktop\Shravan\softwares\python.exe"
$destination5 = "C:\Users\cdac\Desktop\Shravan\softwares\nasm.exe"
Invoke-WebRequest -Uri $source1 -OutFile $destination1
Invoke-WebRequest -Uri $source2 -OutFile $destination2
Invoke-WebRequest -Uri $source3 -OutFile $destination3
Invoke-WebRequest -Uri $source4 -OutFile $destination4
Invoke-WebRequest -Uri $source5 -OutFile $destination5
#Installing one software
Start-Process -Wait -FilePath 'C:\Users\cdac\Desktop\Shravan\Softwares\codeblocks.exe' -ArgumentList '/silent' -PassThru
I have written Powershell script which downloads the software tools from the internet through URL and stored in a separate directory but iam not able to install all the softwares which are stored in a separate directory silently using foreach loop. kindly help me in writing script for installing all the softwares stored in a directory using foreach loop.
Thanking you
There isn't enough information to say exactly what's going wrong here - what error are you getting specifically? What behavior are you observing when you run your script? My first guess is that your foreach loop is running through installer names, not the full path like you posted in your example.
That said, if I understand what you're trying to do, here's a quick and dirty way to do what I think you're after;
# Array of installer details
[Hashtable[]]$Installers = #();
# Firefox
$Installers += #{
SoftwareName = "Firefox"
Url = "https://ftp.mozilla.org/pub/firefox/releases/11.0/win32/enUS/Firefox%20Setup%2011.0.exe"
Destination = "C:\Users\cdac\Desktop\Shravan\Softwares\firefox.exe"
Arguments = '/s'
}
#Code Blocks
$Installers += #{
SoftwareName = "CodeBlocks"
Url = "https://www.fosshub.com/Code-Blocks.html?dwl=codeblocks-20.03-setup.exe"
Destination = "C:\Users\cdac\Desktop\Shravan\Softwares\codeblocks.exe"
Arguments = '/silent'
}
function Install-Software([Hashtable]$installer) {
Write-Host "Installing $($installer.SoftwareName)"
Write-Host "Invoke-WebRequest -Uri $($installer.Url) -OutFile $($installer.Destination)"
Write-Host "Start-Process -FilePath $($installer.Destination) -ArgumentList $($installer.Arguments) -Wait"
# Remove Write Host above - uncomment the following lines:
#Invoke-WebRequest -Uri $installer.Url -OutFile $installer.Destination
#Start-Process -FilePath $installer.Destination -ArgumentList $installer.Arguments -Wait
}
foreach($installer in $Installers) {
Install-Software -installer $installer
}
I'd recommend having a look at the PSAppDeployToolkit if you're trying to package applications silently using PowerShell - this helped me a lot back when I was doing a lot of SCCM packaging.

How to search Azure Devops Code Repos w/ Powershell using Devops REST API and output to CSV

How do you search Azure Devops Code Repos w/ Powershell using Devops REST API and output to CSV.
Couldn't find a solution for this but found enough inspiration around the internet to put this together from this article written by Tenbulls
MS Documentation on this subject can be found here
Instead of exhaustively going through the step by step process of putting this together, I just want to highlight the important hurdles that I overcame.
When the documentation says to use almsearch.dev.azure.com, that's what it means. Don't do what I did and think, oh that's probably incorrect and use dev.azure.com.
Giving your PAT (personal access token) the appropriate amount of rights is important. I, out of laziness and a propensity for anarchy chose to give my token full access after failing to give it the right amount of read access.
Here's the entire script in all of it's glory.
#set your desired org, search and PAT variables
$orgname = "YourOrgName"
$textsearch = 'YourTextString'
$personalToken = "YourPersonalAccessTokenGeneratedInDevOps"
#set your desired file and path properties
$folderpath = "C:\OutBound\"
$file = "OutputSearchResults.csv"
$FileName = $folderpath + $file
#force create directory
New-Item -ItemType Directory -Force -Path $folderpath
#initial test to remove csv file if it already exists
if (Test-Path $FileName) {
Remove-Item $FileName
}
#converting textsearch to json format for later use
$body =
#{
searchText = $textsearch
'$top' = 300
}
$json = $body | ConvertTo-Json
#Setting token and header for Get and Post requests
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($personalToken)"))
$header = #{authorization = "Basic $token"}
#outer query getting a list of all projects in the azure devops organization
$url = "https://dev.azure.com/" + $orgname + "/_apis/projects?api-version=5.1"
$output = Invoke-RestMethod -Uri $url -Method Get -ContentType "application/json" -Headers $header
$output.value | ForEach-Object {
#inner query that searches repositories for the string entered in $textsearch
#-Body $json is the where the magic happens. Many additional request properties to choose from in MS documentation
$project = $_.name
$urlsearch = "https://almsearch.dev.azure.com/" + $orgname + "/" + $project + "/_apis/search/codesearchresults?api-version=6.1-preview.1"
$output = Invoke-RestMethod -Uri $urlsearch -Method Post -ContentType "application/json" -Headers $header -Body $json
$output.results |Export-Csv -Path $FileName -Append
}

PowerShell download exe from public Github

I'm unable to download a .exe file from Github. Script works for different sites and downloads files without issues.
https://github.com/ShareX/ShareX/releases/tag/v13.1.0
This is the .exe I'm trying to download:
https://github.com/ShareX/ShareX/releases/download/v13.1.0/ShareX-13.1.0-setup.exe
>$DownloadUrl = "https://github.com/ShareX/ShareX/releases/download/v13.1.0/ShareX-13.1.0-setup.exe"
>$WebResponse = Invoke-WebRequest -Uri "$DownloadUrl" -Method Head
Invoke-WebRequest : The remote server returned an error: (403) Forbidden.
At line:2 char:16
+ $WebResponse = Invoke-WebRequest -Uri "$DownloadUrl" -Method Head
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
Full script:
[Net.ServicePointManager]::SecurityProtocol = "Tls, Tls11, Tls12, Ssl3"
$DownloadUrl = "https://github.com/ShareX/ShareX/releases/download/v13.1.0/ShareX-13.1.0-setup.exe"
$WebResponse = Invoke-WebRequest -Uri "$DownloadUrl" -Method Head
Write-Output "Downloading $DownloadUrl"
Start-BitsTransfer -Source $WebResponse.BaseResponse.ResponseUri.AbsoluteUri.Replace("%20", " ") -Destination "C:\Users\Pegavo\Desktop\PS\"
You can just simply use Invoke-WebRequest with -OutFile parameter.
Invoke-WebRequest https://github.com/ShareX/ShareX/releases/download/v13.1.0/ShareX-13.1.0-setup.exe -OutFile "ShareX-13.1.0-setup.exe"
This command will download the file from GitHub and store the result to a file in your current directory.
Or, you can use WebClient.
$webClient = New-Object System.Net.WebClient
$webClient.DownloadFile("https://github.com/ShareX/ShareX/releases/download/v13.1.0/ShareX-13.1.0-setup.exe", "E:\your\path\ShareX-13.1.0-setup.exe")
I tested both on my local machine. Both worked.
Moreover, if you want to understand why Start-BitsTransfer won't work, here.
Edit:
You can get the filename automatically like this one, using Split-Path:
$url = "https://github.com/ShareX/ShareX/releases/download/v13.1.0/ShareX-13.1.0-setup.exe"
$file= Split-Path $url -Leaf #file is ShareX-13.1.0-setup.exe now
Invoke-WebRequest $url -OutFile $file
Is there a reason for using HEAD? GET seems to work

Use variable in invoke-webrequest -Uri path

I am trying to create function to download a file from internet where the path to the files is defined in multiple text files on different client computers.
This is what I have come up with so far.
$Company = Get-Content "C:\ProgramData\test\Company.txt"
$CompanyURLFile = "https://onegeek.dk/MSI/$Company.rar"
$CompanyUpdateFile="C:\ProgramData\test\conf\conf.rar"
Invoke-WebRequest -Uri $CompanyURLFile -OutFile $CompanyUpdateFile
The code above will fail because it cant use "$Company" in line 2
If I use this insteadt everything works fine.
$CompanyURLFile = "https://onegeek.dk/MSI/CYPL.rar"
How do I fix this
The solution was to add "| Select-Object -
First 1"
$Company = Get-Content "C:\ProgramData\test\Company.txt" | Select-Object -
First 1
$CompanyURLFile = "https://onegeek.dk/MSI/$Company.rar"
$CompanyUpdateFile="C:\ProgramData\test\conf\conf.rar"
Invoke-WebRequest -Uri $CompanyURLFile -OutFile $CompanyUpdateFile