just as the title states, I'd like to download a file from the internet, specifically the download on this webpage. I have looked into using Invoke-WebRequest, curl, and certutil. All of these options download the HTML of the site. The specific URL of the download looks like this: https://bluetoothinstaller.com/bluetooth-command-line-tools/BluetoothCLTools-1.2.0.56.exe.
Calling things like the following just downloads the HTML:
Invoke-WebRequest -Uri 'https://bluetoothinstaller.com/bluetooth-command-line-tools/BluetoothCLTools-1.2.0.56.exe' -OutFile 'test.exe'
Alternatively, if anyone knows how to download the link via the HTML, please do share.
I'd prefer it if the solution did not require any additional software, but am flexible.
Thanks!
Looking at some code I wrote near the end of last year and I found this line:
(New-Object System.Net.WebClient).DownloadFile($URL, $ZipFile)
In my case I was trying to download the latest SQLite and it worked. In your case you will probably want to rename the $ZipFile variable to something like $ExeFile.
The command to build the file path/name, and define where I wanted the file saved, was this:
$ZipFile = "$PSScriptRoot\$(Split-Path -Path $URL -Leaf)"
As for extracting the file's download path form a webpage, I haven't done that yet. It is something aim to do but it will be awhile before I get around to trying to figure that out.
The following worked for me, note the OutFile comment. You might find something useful on the Network tab of your browser's dev-tools.
$params = #{
UseBasicParsing = $true
Uri = "https://bluetoothinstaller.com/bluetooth-command-line-tools/BluetoothCLTools-1.2.0.56.exe"
Headers = #{
Referer = "https://bluetoothinstaller.com/bluetooth-command-line-tools/download.html"
}
OutFile = 'path/to/download.exe' # Change this
}
Invoke-RestMethod #params
Related
I am trying to copy a xlsx file from my Teams channel to a location on a file server.
I've seen various articles on line that suggest Invoke-WebRequest "https://teams.microsoft.com/l/file/rest of URL here" -OutFile C:\Test\CricketQuiz.xlsx. While this works in terms of being able to see the file at the desired file location, I can't actually open it as I get this error:
I get the same error when I tried the approach suggested in this article https://blog.jourdant.me/post/3-ways-to-download-files-with-powershell .
$url = "https://teams.microsoft.com/l/file/rest of my URL here"
$output = "C:\Test\SportsQuiz.xlsx"
$start_time = Get-Date
$wc = New-Object System.Net.WebClient
$wc.DownloadFile($url, $output)
I'm guessing this is something relatively straightforward to resolve for those with more experience.
The problem here is that the link you've got (the Teams link) is not a direct link to the file at all - it's a link to an embedded version of the file, inside the Teams client (basically like a deep link). To -actually- download the file try the following:
from the url you've got, parse out the "objectUrl" part of the query string. As an example, I have:
https://teams.microsoft.com/l/file/[guid]?tenantId=[guid2]&fileType=xlsx&objectUrl=https%3A%2F%2F[tenantname].sharepoint.com%2Fsites%2FHR%2FShared%2520Documents%2FEmployee%2520Sentiment%2520Analysis.xlsx&serviceName=recent
you want (in my example): https%3A%2F%2F[tenantname].sharepoint.com%2Fsites%2FHR%2FShared%2520Documents%2FEmployee%2520Sentiment%2520Analysis.xlsx
then you need to querystring decode this, to get (e.g.) https://[tenantname].sharepoint.com/sites/HR/Shared%20Documents/Employee%20Sentiment%20Analysis.xlsx
finally, you should use the PnP-PowerShell module's Get-PnPFile to download the file. This itself is a few steps though:
3.1 you need to connect the session, using Connect-PnPOnline, but you also need to connect to the right "SPWeb". In this case, it would be Connect-PnPOnline https://[tenantname].sharepoint.com/sites/HR
3.1 after that you can download the file, but you need to url decode it again, to get rid of %20 and similar, something like:
Get-PnPFile -Url "/Shared Documents/Employee Sentiment Analysis.xlsx" -AsFile -Path "c:\temp\"
This will give you a copy of Employee Sentiment Analysis.xlsx (in my example) inside c:\temp
Obviously this can all be automated, like the querystring decoding, the connect-pnp credentials, etc., but hopefully this gets you on the right path.
I used Invoke-WebRequest in Powershell to download a file without using the -OutFile parameter, and, from the documentation here, the file should've ended up in the directory I was in. However, there is nothing. Response was OK, no error was shown.
What could've happened to that file? Am I mistaken about how Invoke-WebRequest should work without an Out parameter?
Thanks!
Note: I know I can easily download the file using the parameter, but it's pretty big and I'd like to make sure it doesn't end up clogging disk space somewhere I don't need
From the linked docs:
By default, Invoke-WebRequest returns the results to the pipeline.
That is, in the absence of -OutFile no file is created.
(If you don't capture or redirect the output, it will print to the host (console).)
As techguy1029 notes in a comment, the current directory only comes into play if you do use -OutFile but specify a mere file name rather than a path.
As an aside: To-pipeline output is a response object of (a) type (derived from) WebResponseObject, whereas only the value of the response's body (the equivalent of property value .Content) is saved with -OutFile.
Lets talk about what the Microsoft documentation says for Invoke-WebRequest
"
-OutFile : Specifies the output file for which this cmdlet saves the response body. Enter a path and file name. If you omit the path, the
default is the current location. "
The Key word here is if a Path is omitted it will use the current path.
the -OutFile is a parameter of type String
The usage to save to current path would be
Invoke-webrequest "http://Test.com/test.pdf" -OutFile "Test.pdf"
else to have a custom path
Invoke-webrequest "http://Test.com/test.pdf" -OutFile "C:\Test\Test.Pdf"
I'm trying to create a list of all the links from an FTP site. The links are to download zip files.
My end goal is to analyze each link as a string and match the beginning to a set phrase. The end of each link contains a date and I have to find the newest one to download.
In this example I want to find ABC_20170323.zip out of this list:
ABC_20170323.zip
ABC_20160102.zip
EFG_20170324.zip
I need to figure out how to acquire the links before analysis. I've tried a variety of methods and the only one that has returned any information from the site is to gather the source code:
Invoke-WebRequest $sourceuri -UseBasicParsing -Credential $user
But then I find it difficult to gather all the links from there. Anyone have a method for easily getting these file download links?
Okay, so I know it's been ages, but I figured out how to do it. Admittedly, it's the hard way. What ended up happening is I gathered the source code and saved it like so:
$r = Invoke-WebRequest $sourceuri -UseBasicParsing -credential $user
Then I converted it to a string and used -split to separate out the links by their html tag and what I expected the beginning to look like (in this case 'ABC'):
$c = $r.ToString() #convert to string
$datelist = #()
$f = ($c -split 'A HREF="' -split '.zip</A>') #split by html tag (and .zip)
foreach($link in $f){
if($link -match 'ABC') { #if the beginning of the link is 'ABC'
$datelist += ($link.substring($link.Length-8)) #isolate the date on the end
}
} #more logic for comparing $datelist items...
Then I wrote some logic for comparing the items in $datelist (Omitted from answer) and created a variable that had all the components I needed:
$ExactLink = "ABC_$GreatestDate" + ".zip"
Then went on to download the $ExactLink I needed.
I'm trying to write a script that reads the mail headers from a directory full of .msg files so I can later parse them via regex. I tried $MSG = Get-Content .\message.msg, which could work, but it's a pretty dirty output. Has anyone tried this? I can't seem to find a working example online.
You have a few options depending on your environment. If you are on a computer with Outlook installed you can easily do this with an Outlook com object. The problem is that the headers are not exposed by default so you have to dig for them.
$ol = New-Object -ComObject Outlook.Application
$msg = $ol.CreateItemFromTemplate("SOME\PATH\TO\A\MSG\FILE.msg")
$headers = $msg.PropertyAccessor.GetProperty("http://schemas.microsoft.com/mapi/proptag/0x007D001E")
$headers
At this point you have a text block with all of the header information in it. If you want a specific header you will need to write a regex to extract it.
You could also write a class that reads the raw content based on the specification. Or read in the raw content with powershell and write a regex to attempt to extract it.
How to call System.ServiceModel.Syndication.SyndicationFeed from PowerShell?
I was trying to read RSS feed from Powershell, and need to traverse through feed, I am wondering what would be the way to do it?
I could not find any example on it.
Thanks in advance!!
Ramani
What you're really trying to do is parse XML with PowerShell. This one liner will get you the entries from this question:
((New-Object Net.Webclient).DownloadString("http://stackoverflow.com/feeds/question/10589059") -as [xml]).feed.entry
It creates a new webclient, and downloads the URL, turns it into XML, and then just dots into the XML elements (as you can do in PowerShell).
It's a very simple one liner that packs in a lot of the power of PowerShell. A similar example is at the beginning of Bruce Payette's "PowerShell In Action"
Hope this Helps
Here's what I came up with in a few seconds:
$url = "http://stackoverflow.com/feeds/question/10589059"
[System.Reflection.Assembly]::LoadWithPartialName("System.ServiceModel") | Out-Null
[System.ServiceModel.Syndication.SyndicationFeed] $feed = [System.ServiceModel.Syndication.SyndicationFeed]::Load([System.Xml.XmlReader]::Create($url))
$feed | Get-Member