Invoke-WebRequest Not Working Inside ForEach Loop - powershell

I am trying to do a foreach loop to pull down a file from AWS. When I run Invoke-WebRequest -Uri $S3InstallerLoc -OutFile $S3OutFileoutside of a foreach loop on the server itself it pulls my test file down. When I stick it inside a foreach loop it does not pull down the file. When I attempt to do this from another server I am getting the following error: Invoke-WebRequest : The underlying connection was closed: An unexpected error occurred on a send.
Here is the entirety of the script:
$Servers = "AWS-GPOTEST"
$S3InstallerLoc = "https://s3-us-west-2.amazonaws.com/bucketname/test.txt"
$S3OutFile = "C:\Windows\Temp\Test.txt"
ForEach ($Server in $Servers)
{
Invoke-WebRequest -Uri $S3InstallerLoc -OutFile $S3OutFile
}

In the case that you are running a version of Powershell before 6.0 and that you are running this within a WinRM session you should use the UseBasicParsing parameter.
As in:
Invoke-Command -ComputerName $Servers -ScriptBlock {Invoke-WebRequest -Uri $Using:S3InstallerLoc -OutFile $Using:S3OutFile -UseBasicParsing}
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-6
Using local variables in scripts:
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_remote_variables?view=powershell-6#using-local-variables

Related

PowerShell, how to handle a page timeout from an Invoke-WebRequest?

I found that a site that I wanted to parse was down today when I went there to download a file.
The code I ran is:
$url = "https://notepad-plus-plus.org/downloads"
$page = Invoke-WebRequest -uri $url -UseBasicParsing
How can I intercept and handle this?
Is there a way to gracefully time out my request after 2 seconds if it does not get a response (which could be from the site being down or from my own internet having problems)?
What might be the best way to detect if what was returned is "connection timeout junk" as opposed to useful data that I can work with?
$url = "https://notepad-plus-plus.org/downloads" # Final url will be like: # https://github.com/notepad-plus-plus/notepad-plus-plus/releases/download/v8.4.6/npp.8.4.6.portable.x64.zip
try {
$page = Invoke-WebRequest -uri $url -UseBasicParsing -TimeoutSec 3
}
catch {
if ($_.Exception.Response.StatusCode -band 522) { "bad!"}
}
You should just call the current default error variable exception message/detail.
$url = "https://notepad-plus-plus.org/downloads"
try {$page = Invoke-WebRequest -uri $url -UseBasicParsing -TimeoutSec 3 -ErrorAction Stop}
catch {$Error[0].Exception}
# Results
<#
The operation has timed out.
#>
$url = "https://notepad-plus-plus.org/downloads"
try {$page = Invoke-WebRequest -uri $url -UseBasicParsing -ErrorAction Stop}
catch {$Error[0].Exception}
# Results
<#
The remote server returned an error: (522).
#>
Thus do what you want based on that error message.
$url = "https://notepad-plus-plus.org/downloads"
try {$page = Invoke-WebRequest -uri $url -UseBasicParsing -ErrorAction Stop}
catch
{
If (($Error[0].Exception) -match '522')
{Write-Warning -Message 'Bad stuff happened. Now, go do stuff'}
}
# Results
<#
WARNING: Bad stuff happened. Now, go do stuff
#>
Update
Timeout check.
$timer = [Diagnostics.Stopwatch]::StartNew()
$url = "https://notepad-plus-plus.org/downloads"
try {$page = Invoke-WebRequest -uri $url -UseBasicParsing -TimeoutSec 3 -ErrorAction Stop}
catch
{
$Error[0].Exception
If (($Error[0].Exception) -match 'timed out')
{Write-Warning -Message 'Timeout occurred. Do you want to set a differnet timeout'}
}
$timer.Elapsed.TotalSeconds
# Results
<#
The operation has timed out.
WARNING: Timeout occurred. Do you want to set a differnet timeout
3.0225485
#>

Power shell script which silently installs softwares

# Source file location
$source1 ="https://ftp.mozilla.org/pub/firefox/releases/11.0/win32/enUS/Firefox%20Setup%2011.0.exe"
$source2 ="https://www.fosshub.com/Code-Blocks.html?dwl=codeblocks-20.03-setup.exe"
$source3 ="https://github.com/x64dbg/x64dbg/releases/download/snapshot/snapshot_2021-07-01_23-17.zip"
$source4 ="https://www.python.org/ftp/python/3.9.6/python-3.9.6-amd64.exe"
$source5 ="https://www.nasm.us/pub/nasm/releasebuilds/2.15.05/win64/nasm-2.15.05-installer-x64.exe"
# Destination to save the file
$destination1 = "C:\Users\cdac\Desktop\Shravan\Softwares\firefox.exe"
$destination2 = "C:\Users\cdac\Desktop\Shravan\Softwares\codeblocks.exe"
$destination3 = "C:\Users\cdac\Desktop\Shravan\softwares\xdbg.zip"
$destination4 = "C:\Users\cdac\Desktop\Shravan\softwares\python.exe"
$destination5 = "C:\Users\cdac\Desktop\Shravan\softwares\nasm.exe"
Invoke-WebRequest -Uri $source1 -OutFile $destination1
Invoke-WebRequest -Uri $source2 -OutFile $destination2
Invoke-WebRequest -Uri $source3 -OutFile $destination3
Invoke-WebRequest -Uri $source4 -OutFile $destination4
Invoke-WebRequest -Uri $source5 -OutFile $destination5
#Installing one software
Start-Process -Wait -FilePath 'C:\Users\cdac\Desktop\Shravan\Softwares\codeblocks.exe' -ArgumentList '/silent' -PassThru
I have written Powershell script which downloads the software tools from the internet through URL and stored in a separate directory but iam not able to install all the softwares which are stored in a separate directory silently using foreach loop. kindly help me in writing script for installing all the softwares stored in a directory using foreach loop.
Thanking you
There isn't enough information to say exactly what's going wrong here - what error are you getting specifically? What behavior are you observing when you run your script? My first guess is that your foreach loop is running through installer names, not the full path like you posted in your example.
That said, if I understand what you're trying to do, here's a quick and dirty way to do what I think you're after;
# Array of installer details
[Hashtable[]]$Installers = #();
# Firefox
$Installers += #{
SoftwareName = "Firefox"
Url = "https://ftp.mozilla.org/pub/firefox/releases/11.0/win32/enUS/Firefox%20Setup%2011.0.exe"
Destination = "C:\Users\cdac\Desktop\Shravan\Softwares\firefox.exe"
Arguments = '/s'
}
#Code Blocks
$Installers += #{
SoftwareName = "CodeBlocks"
Url = "https://www.fosshub.com/Code-Blocks.html?dwl=codeblocks-20.03-setup.exe"
Destination = "C:\Users\cdac\Desktop\Shravan\Softwares\codeblocks.exe"
Arguments = '/silent'
}
function Install-Software([Hashtable]$installer) {
Write-Host "Installing $($installer.SoftwareName)"
Write-Host "Invoke-WebRequest -Uri $($installer.Url) -OutFile $($installer.Destination)"
Write-Host "Start-Process -FilePath $($installer.Destination) -ArgumentList $($installer.Arguments) -Wait"
# Remove Write Host above - uncomment the following lines:
#Invoke-WebRequest -Uri $installer.Url -OutFile $installer.Destination
#Start-Process -FilePath $installer.Destination -ArgumentList $installer.Arguments -Wait
}
foreach($installer in $Installers) {
Install-Software -installer $installer
}
I'd recommend having a look at the PSAppDeployToolkit if you're trying to package applications silently using PowerShell - this helped me a lot back when I was doing a lot of SCCM packaging.

How to use PowerShell script to download latest private GitHub release?

I recently wrote a PowerShell script that downloads the latest release from a public repo and that works as intended. However, I want to change my script so it can access my private repo. Here is the code I have tried so far:
# Download latest release from GitHub
$credentials="myPersonalAccessToken"
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", "token $credentials")
$repo = "myUserName/MyPrivateReleaseRepo"
$file = "MyBinaries.zip"
$releases = "https://api.github.com/repos/$repo/releases"
Write-Host Determining latest release
$tag = (Invoke-WebRequest $releases -Headers $headers | ConvertFrom-Json)[0].tag_name
$download = "https://github.com/$repo/releases/download/$tag/$file"
$name = $file.Split(".")[0]
$zip = "$name-$tag.zip"
$dir = "$name-$tag"
Write-Host Dowloading latest release
Invoke-WebRequest $download -Headers $headers -Out $zip
Write-Host Extracting release files
Expand-Archive $zip -Force
# Cleaning up target dir
Remove-Item "C:\MyOutPutFolder\$name" -Recurse -Force -ErrorAction SilentlyContinue
# Moving from temp dir to target dir
Move-Item $dir\$name -Destination "C:\MyOutPutFolder\$name" -Force
# Removing temp files
Remove-Item $zip -Force
Remove-Item $dir -Recurse -Force
I get the following error only when using my private repo:
Invoke-WebRequest : The remote server returned an error: (404) Not Found
At C:\Script\DownloadLatestGitHubRelease.ps1:25 char:1
+ Invoke-WebRequest $download -Headers $headers -Out $zip
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
I've also tried providing bad credentials vs the correct credentials and got a "Bad Credentials" error when providing the incorrect ones as expected, so I believe I'm using the token correctly.
What am I doing wrong? Thanks in advance for any suggestions.
Landed here trying to do the same thing. After digging into some bash scripts that do similar, I came up with this to do the download:
$token = "<your token from https://github.com/settings/tokens>"
$downloadFolder = "C:\temp";
$ownerSlashRepo = "owner/reop";
$tag = "latest";
$json = Invoke-Webrequest -Uri "https://api.github.com/repos/$ownerSlashRepo/releases/$tag" -Headers #{'Authorization'='token '+$token; 'Accept'='application/json'}
$release = $json.Content | ConvertFrom-Json
$release.assets | %{
$asset = $_;
$x = Invoke-Webrequest -Uri $($asset.url) -OutFile "$downloadFolder\$($asset.name)" -Headers #{'Authorization'='token '+$token; 'Accept'='application/octet-stream'}
}
This downloads every file from the release, but you could filter it to just download one as needed, something like:
$release.assets | ?{$_.name -eq 'foo.zip'} | %{
To answer your question, in the original code you're attempting to use the token with https://github.com. I tried similar at first and got similar results to what you describe. I think it is only good for https://api.github.com and the REST API. I also found I had to specify Accept:application/octet-stream when downloading the asset.

invoke-webrequest returns 401 with Windows Authentication

I'm working on a script to login to a sharepoint 2013 site and navigate to a few pages to make sure the site is working after updates and DR drills. I'm calling Invoke-WebRequest like this:
$site = Invoke-WebRequest -uri 'https://spsite' -Credential $(Get-Credential) -SessionVariable s
when I make the call I get a 401 Unauthorized error back. I have tried using basic authentication and building out the headers like this:
$u = 'domain\user'
$p = 'password'
$header = #{ Authorization = "Basic {0}" -f [convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $u,$p))) }
$site = Invoke-WebRequest -uri 'https://spsite' -Headers $header
with the same result. I'm hoping someone could offer another way to make this connection?
so I found a way to make this work in my situation and wanted to post the basics in case someone else runs into this.
I found that when the exception is thrown you can get the actual response from the web server from the exception object like this:
try{
$site = Invoke-WebRequest -uri 'https://spsite' -Credential $(Get-Credential) -SessionVariable s
}
catch{
$site = $_.Exception.Response
}
after that I was able to manipulate the $site variable to follow the redirection and submit the credentials as needed.
Use Export-PSCredential and Import-PSCredential from WFTools - you'll only have to enter your credentials once per box, and it will last as long as your password doesn't change: https://github.com/RamblingCookieMonster/PowerShell
Install-Module -Name WFTools -RequiredVersion 0.1.44
Import-Module WFTools;
$getCredentialMessage = "Please provide your Windows credentials";
$importedCredential = Import-PSCredential;
if ($importedCredential) {
Write-Host -ForegroundColor Yellow "Imported your cached credential."
while (-not $(Test-Credential -Credential $credential)) {
Write-Host -ForegroundColor Yellow "Your cached credentials are not valid. Please re-enter."
$credential = Get-Credential -Message $getCredentialMessage;
}
$credential = $importedCredential;
}
else {
$credential = Get-Credential -Message $getCredentialMessage;
while (-not $(Test-Credential -Credential $credential)) {
$credential = Get-Credential -Message $getCredentialMessage;
}
Export-PSCredential $credential;
}
# Here is where the magic happens
$site = Invoke-WebRequest -uri 'https://spsite' -Credential $credential

Use variable in invoke-webrequest -Uri path

I am trying to create function to download a file from internet where the path to the files is defined in multiple text files on different client computers.
This is what I have come up with so far.
$Company = Get-Content "C:\ProgramData\test\Company.txt"
$CompanyURLFile = "https://onegeek.dk/MSI/$Company.rar"
$CompanyUpdateFile="C:\ProgramData\test\conf\conf.rar"
Invoke-WebRequest -Uri $CompanyURLFile -OutFile $CompanyUpdateFile
The code above will fail because it cant use "$Company" in line 2
If I use this insteadt everything works fine.
$CompanyURLFile = "https://onegeek.dk/MSI/CYPL.rar"
How do I fix this
The solution was to add "| Select-Object -
First 1"
$Company = Get-Content "C:\ProgramData\test\Company.txt" | Select-Object -
First 1
$CompanyURLFile = "https://onegeek.dk/MSI/$Company.rar"
$CompanyUpdateFile="C:\ProgramData\test\conf\conf.rar"
Invoke-WebRequest -Uri $CompanyURLFile -OutFile $CompanyUpdateFile