I am trying to create function to download a file from internet where the path to the files is defined in multiple text files on different client computers.
This is what I have come up with so far.
$Company = Get-Content "C:\ProgramData\test\Company.txt"
$CompanyURLFile = "https://onegeek.dk/MSI/$Company.rar"
$CompanyUpdateFile="C:\ProgramData\test\conf\conf.rar"
Invoke-WebRequest -Uri $CompanyURLFile -OutFile $CompanyUpdateFile
The code above will fail because it cant use "$Company" in line 2
If I use this insteadt everything works fine.
$CompanyURLFile = "https://onegeek.dk/MSI/CYPL.rar"
How do I fix this
The solution was to add "| Select-Object -
First 1"
$Company = Get-Content "C:\ProgramData\test\Company.txt" | Select-Object -
First 1
$CompanyURLFile = "https://onegeek.dk/MSI/$Company.rar"
$CompanyUpdateFile="C:\ProgramData\test\conf\conf.rar"
Invoke-WebRequest -Uri $CompanyURLFile -OutFile $CompanyUpdateFile
Related
I am trying to loop through a folder, grab all files, read their contents then post each file content individually to SOAP.
This is how I would do it, but PowerShell returns an error.
Invoke-Webrequest : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties do not match any of the parameters that take pipeline input.
Below is my code:
$dataAPI = Get-ChildItem 'C:\Users\..\Output'
$uri = 'http://server-name.com:8080/name/name2'
ForEach ($Item in $dataAPI) {
Get-Content $Item.FullName | Invoke-Webrequest -Headers #{"Content-Type" = "text/xml;charset=UTF-8"; "SOAPAction" = "http://server-name.com:8080/name/name2"} -Method 'POST' -Body $dataAPI -Uri $uri -UseDefaultCredential
}
I am not really sure where I should place the Invoke-WebRequest...
Any help would be appreciated. Thanks.
Continuing from my comments,
Add switch -Raw to the Get-Content call to receive a single multiline string instead of an array of lines
Add switch -File to the Get-ChildItem call to ensure you will only deal with files in the loop, not directories too
Try
# if all files you need have a common extension, add `-Filter '*.xml'` to below line
# '*.xml' is just an example here..
$files = Get-ChildItem -Path 'C:\Users\Gabriel\Output' -File
$uri = 'http://server-name.com:8080/name/name2'
$header = #{"Content-Type" = "text/xml;charset=UTF-8"; "SOAPAction" = "http://server-name.com:8080/name/name2"}
foreach ($Item in $files) {
$content = Get-Content $Item.FullName -Raw
Invoke-Webrequest -Headers $header -Method 'POST' -Body $content -Uri $uri -UseDefaultCredential
}
I recently wrote a PowerShell script that downloads the latest release from a public repo and that works as intended. However, I want to change my script so it can access my private repo. Here is the code I have tried so far:
# Download latest release from GitHub
$credentials="myPersonalAccessToken"
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", "token $credentials")
$repo = "myUserName/MyPrivateReleaseRepo"
$file = "MyBinaries.zip"
$releases = "https://api.github.com/repos/$repo/releases"
Write-Host Determining latest release
$tag = (Invoke-WebRequest $releases -Headers $headers | ConvertFrom-Json)[0].tag_name
$download = "https://github.com/$repo/releases/download/$tag/$file"
$name = $file.Split(".")[0]
$zip = "$name-$tag.zip"
$dir = "$name-$tag"
Write-Host Dowloading latest release
Invoke-WebRequest $download -Headers $headers -Out $zip
Write-Host Extracting release files
Expand-Archive $zip -Force
# Cleaning up target dir
Remove-Item "C:\MyOutPutFolder\$name" -Recurse -Force -ErrorAction SilentlyContinue
# Moving from temp dir to target dir
Move-Item $dir\$name -Destination "C:\MyOutPutFolder\$name" -Force
# Removing temp files
Remove-Item $zip -Force
Remove-Item $dir -Recurse -Force
I get the following error only when using my private repo:
Invoke-WebRequest : The remote server returned an error: (404) Not Found
At C:\Script\DownloadLatestGitHubRelease.ps1:25 char:1
+ Invoke-WebRequest $download -Headers $headers -Out $zip
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
I've also tried providing bad credentials vs the correct credentials and got a "Bad Credentials" error when providing the incorrect ones as expected, so I believe I'm using the token correctly.
What am I doing wrong? Thanks in advance for any suggestions.
Landed here trying to do the same thing. After digging into some bash scripts that do similar, I came up with this to do the download:
$token = "<your token from https://github.com/settings/tokens>"
$downloadFolder = "C:\temp";
$ownerSlashRepo = "owner/reop";
$tag = "latest";
$json = Invoke-Webrequest -Uri "https://api.github.com/repos/$ownerSlashRepo/releases/$tag" -Headers #{'Authorization'='token '+$token; 'Accept'='application/json'}
$release = $json.Content | ConvertFrom-Json
$release.assets | %{
$asset = $_;
$x = Invoke-Webrequest -Uri $($asset.url) -OutFile "$downloadFolder\$($asset.name)" -Headers #{'Authorization'='token '+$token; 'Accept'='application/octet-stream'}
}
This downloads every file from the release, but you could filter it to just download one as needed, something like:
$release.assets | ?{$_.name -eq 'foo.zip'} | %{
To answer your question, in the original code you're attempting to use the token with https://github.com. I tried similar at first and got similar results to what you describe. I think it is only good for https://api.github.com and the REST API. I also found I had to specify Accept:application/octet-stream when downloading the asset.
How do you search Azure Devops Code Repos w/ Powershell using Devops REST API and output to CSV.
Couldn't find a solution for this but found enough inspiration around the internet to put this together from this article written by Tenbulls
MS Documentation on this subject can be found here
Instead of exhaustively going through the step by step process of putting this together, I just want to highlight the important hurdles that I overcame.
When the documentation says to use almsearch.dev.azure.com, that's what it means. Don't do what I did and think, oh that's probably incorrect and use dev.azure.com.
Giving your PAT (personal access token) the appropriate amount of rights is important. I, out of laziness and a propensity for anarchy chose to give my token full access after failing to give it the right amount of read access.
Here's the entire script in all of it's glory.
#set your desired org, search and PAT variables
$orgname = "YourOrgName"
$textsearch = 'YourTextString'
$personalToken = "YourPersonalAccessTokenGeneratedInDevOps"
#set your desired file and path properties
$folderpath = "C:\OutBound\"
$file = "OutputSearchResults.csv"
$FileName = $folderpath + $file
#force create directory
New-Item -ItemType Directory -Force -Path $folderpath
#initial test to remove csv file if it already exists
if (Test-Path $FileName) {
Remove-Item $FileName
}
#converting textsearch to json format for later use
$body =
#{
searchText = $textsearch
'$top' = 300
}
$json = $body | ConvertTo-Json
#Setting token and header for Get and Post requests
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($personalToken)"))
$header = #{authorization = "Basic $token"}
#outer query getting a list of all projects in the azure devops organization
$url = "https://dev.azure.com/" + $orgname + "/_apis/projects?api-version=5.1"
$output = Invoke-RestMethod -Uri $url -Method Get -ContentType "application/json" -Headers $header
$output.value | ForEach-Object {
#inner query that searches repositories for the string entered in $textsearch
#-Body $json is the where the magic happens. Many additional request properties to choose from in MS documentation
$project = $_.name
$urlsearch = "https://almsearch.dev.azure.com/" + $orgname + "/" + $project + "/_apis/search/codesearchresults?api-version=6.1-preview.1"
$output = Invoke-RestMethod -Uri $urlsearch -Method Post -ContentType "application/json" -Headers $header -Body $json
$output.results |Export-Csv -Path $FileName -Append
}
I have written 2 scripts. One in python and one in powershell. Python is able to call a rest api and is returned the string "JSESSIONID=8kfv0fi1bc84gtw2xvnqsrt4;Path=/;Secure;HttpOnly ". When I use the following code in powershell, it returns "success". What am I doing wrong?
$getEncCode = "Er6TmdhXn09Y9C1I"
$dataPart1 = #{EncCode=$getEncCode}
$dataPart = $dataPart1 | ConvertTo-Json
$uri = "https://10.164.42.77:8092/getEnc/2252953/login"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$result = Invoke-RestMethod -Method Post -Body $dataPart -uri $uri -ContentType "application/json"
Write-Host $result
Write-Host $result uses the $result.ToString() method to display that object as a string.
Likely there is more data there to show. Simply remove Write-Host to see it. Or do something like Get-Member -InputObject $result to see all immediate properties, methods etc.
Have a look at Printing object properties in Powershell to see other ways to deal with this.
I have an url in which there are 6 digits which are changing daily.
Sample website:
https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/ea_csv_160126.csv
This is the part which changes: 160126
I don't know the correct syntax but as a form of pseudo code:
$url = "https://www.ecb.europa.eu/paym/coll/assets
/html/dla/ea_MID/ea_csv_" + [0-9][0-9][0-9][0-9][0-9][0-9]+ ".csv"
How can I write this string?
To answer the comments, I use it to download that file to a folder, like this:
"https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/ea_csv_" + [0-9]+[0-9]+[0-9]+[0-9]+[0-9]+[0-9] +".csv"
$output = "C:\MyFolder\SomeSubFolder\ScriptDownload"
$start_time = Get-Date
Invoke-WebRequest -Uri $url -OutFile $output
Write-Output "Time taken: $((Get-Date).Subtract($start_time).Seconds) second(s)"
You can parse the download page for your filenames by downloading the page or use get-ElementById etc.
I assume, this is the original download page
This is your Download URL:
$Url = "https://www.ecb.europa.eu/paym/coll/assets/html/list-MID.en.html"
$page = Invoke-WebRequest -Uri $Url
$a = ($page.ParsedHtml.getElementsByTagName('table') | ? {$_.classname -eq 'ecb-contentTable'}).textContent
$filename = $a.Substring($a.IndexOf('ea_csv_'), 17)
$DLURL = "https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/" + $filename
Gives:
$DLURL
https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/ea_csv_160126.csv
Complete that with your
$output = "C:\MyFolder\SomeSubFolder\ScriptDownload\" + $filename
Invoke-WebRequest -Uri $DLURL -OutFile $Output
and its done.
What you are asking for cannot be done. However there are better, more reliable ways to get the same result you are looking for.
I'm with Martin. I also found the download page he did. The better way to do this is get the link. Now this is probably not the best way to get the information but it is a start in the right direction.
Note this is slow as hell. Mostly because of Invoke-WebRequest
$start_time = Get-Date
$output = "C:\MyFolder\SomeSubFolder\ScriptDownload"
# Browse to the page hosting the csv file.
$request = Invoke-WebRequest "https://www.ecb.europa.eu/paym/coll/assets/html/list-MID.en.html"
# Locate the uncompressed CSV file name from the page
$filename = $request.ParsedHtml.getElementsByTagName("a") | Where-Object{$_.nameProp -match "^ea_csv_\d{6}\.csv$"} | Select -ExpandProperty nameProp
$fileurl = "https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/$filename"
# Get the file the is hosted today.
Invoke-WebRequest -Uri $fileurl -OutFile "$output\$filename"
Write-Output "Time taken: $((Get-Date).Subtract($start_time).Seconds) second(s)"
The way we find the right file name is with ^ea_csv_\d{6}\.csv$ which matches a name where it is exactly "ea_csv_[6 digits].csv".
The 6 digits are the date encoded as YYMMDD, correct? If so, you can generate a URL for the current day with:
$currentDay = $(get-date).ToString("yyMMdd")
$url = "https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/ea_csv_$currentDay.csv"