Programmatically downloading from Dropbox with PowerShell is trivial. With a link to a shared file https://www.dropbox.com/s/sdfjjthwrf32yFileName.pdf?dl=0, you just replace www.dropbox.com by dl.dropboxusercontent.com to give https://dl.dropboxusercontent.com/s/sdfjjthwrf32yFileName.pdf?dl=0 and can then use Invoke-WebRequest to fetch that file via a script (as long as the permissions on it are open for anyone to read).
With OneDrive, I have spent a few hours looking, and found no way to download a file with PowerShell. Some websites talk about OneDrive for Business, this question is not about OneDrive for Business as I do not know about or use that, this is specifically about OneDrive.
What I want to do is really simple, i.e. share a file in OneDrive, then get the share link, then be able to programmatically download that file using PowerShell (as the file is set to "anyone can view" so I should not really require authentication). Here is how I would share that file on the OneDrive.live.com site:
I then change the file from "Anyone can Edit" to "Anyone can View", and this gives me a link that looks like this:
$url = "https://1drv.ms/u/s!Bjfhd-pwUc_asdcadfasdfTYFASDFASDf?e=aksdHDG"
I then try to download it using Invoke-WebRequest:
Invoke-WebRequest -UseBasicParsing -Uri $url -OutFile "C:\MyFile.txt"
But all I get is a small stub of javascript output (note that the above is not a real link, but an example)
With Dropbox, it is trivially simple, so how can we do this with OneDrive?
The closest I have come might be the OneDrive module, but I don't know how to generate an OAuth2 token that I can pass to the script (and it completely breaks the purpose of this exercise, as if I have to provide credentials to download an "Anyone can View" file, then I cannot create a script that others can use to download a file without them also providing credentials; we should not need to provide credentials if the file in question is set as "anyone can view". That is the case for the Dropbox example; if the file is to read-only for everyone, then no credentials are required, as that would obviously be sort of ludicrous - it would be like demanding a key to get through an unlocked and open doorway that has a sign above it saying "Anyone may freely enter here without a key").
Install-Module -Name OneDrive -Force
Please note: To stress, this question is not about OneDrive for Business, that is something that I do not know or use.
You can download the shared file directly by using the OneDrive API. To do so, you have to convert your sharing URL into a sharing token and pass it to the OneDrive API. This example shows the needed C# code. You can derive the following PowerShell function from it:
function Get-OneDriveDirectDownloadUrl {
param(
[string]
$SharingUrl
)
$base64Value = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($SharingUrl))
$encodedUrl = 'u!' + $base64Value.TrimEnd('=').Replace('/','_').Replace('+','-')
return 'https://api.onedrive.com/v1.0/shares/' + $encodedUrl + '/root/content'
}
With that function, you can convert your sharing URL like this:
Get-OneDriveDirectDownloadUrl -SharingUrl 'https://1drv.ms/u/s!Bjfhd-pwUc_asdcadfasdfTYFASDFASDf?e=aksdHDG'
And you will receive the following direct download URL:
https://api.onedrive.com/v1.0/shares/u!aHR0cHM6Ly8xZHJ2Lm1zL3UvcyFCamZoZC1wd1VjX2FzZGNhZGZhc2RmVFlGQVNERkFTRGY_ZT1ha3NkSERH/root/content
Related
Looking for solution how to test website login with powershell without IE COM method. There is lot of examples how to do it with IE COM method: -ComObject 'internetExplorer.Application, but unfortunately this is not working for me as site is not working with IE at all.
Google chrome has a nice feature in its developer tools, on the network tab.
Clear the network log, filter so that it shows "All" requests/responses.
Log into the site and then click the red button or hit "ctrl + e" to stop the log from increasing.
Go through each request until you see the login request (These can sometimes be hard to spot.)
Once you have found the request right click on the name of the request, you can then select "Copy -> Copy As PowerShell".
Paste this into ISE etc.
I would personally then remove all of the $Session lines from above the request and change -WebSession $session to -SessionVariable $session and subsequently call -WebSession $session in each web request there after. (Only use -Sessionvariable once as this is the what instantiates the session). The reason I do this is so that the login creates its own session data rather than using any possible data from the browser.
Last of all you will want to clear any unwanted headers out of the webrequest.
Depending on the login mechanism the request may need tweaking to allow it to be done through automation, I have only had a couple of circumstances where this has been the case and multiple requests have been required due to AJAX or other elements on the page.
I'm building an app that uses the ms-graph v1.0 API to write data to excelsheets in my OneDrive. It works with excel files that I uploaded to my drive but doesn't work with excel files that've been shared with me.
I know that I can get a list of all shared files with me/drive/sharedWithMe and the file that i want to edit is amongst the files that are being returned.
However, when i try to get one shared drive item using its driveItem property parentReference: driveID like this: /drives/{driveID}/items/{itemID} it returns : 403 - acces denied.
Here are my permissions:
"user.read",
"calendars.read",
"directory.accessasuser.all",
"files.readwrite.all"
I couldn't try the shares path /shares/{shareID} because I don't know how to figure out the shareId. It doesn't seem to be among the properties of the item that is returned by /sharedWithMe. Where can I get it?
Figured it out by myself.
I got the error
"message": "Cannot reference a user's drive from another user's personal site"
so I removed the me/from the route me/drives/{driveID}/items/{itemId} and it worked.
I am looking for assistance to find out how I can download and move a OneDrive file that is accessed through a shared directory, via the shared link method of sharing.
I have two users:
user 'A' who is a Microsoft Consumer and has a regular OneDrive account and will host a csv file 'test.csv' in a folder 'toshare'
and user 'B' who is also a regular Microsoft Consumer who should use the graph API to download test.csv and then move the file to a subdirectory /toshare/archive
Aside: I am currently using the chrome app "advanced REST client" to manually make the REST calls, and am getting Authenticated OAuth BEARER tokens by inspecting network traffic from Microsoft's online "Graph Explorer" tool. After we understand the calls, we'll integrate it into our Java app.
I have succesfully followed the instructions here:
https://developer.microsoft.com/en-us/graph/docs/api-reference/v1.0/api/shares_get
to view the folder contents.
To be more explicit, user 'A' has went into OneDrive and has right clicked the folder 'toshare' and selected shareLink. I have converted the shareLink to a share token and then used the following API call with the Graph API as user 'B':
GET https://graph.microsoft.com/v1.0/shares/<share-token>/root?$expand=children
this shows me all the files in the directory, which includes 'test.csv'
Now, using this information, how can I download test.csv? Assuming user 'B' doesn't know the name of the file, but can identify it by being a .csv file (we can do this in code). There does not appear to be much documentation on how to download the files through a share.
The closest I've gotten was to take the "webUrl" attribute of the children object for my file, and then turn that into a share token and call
GET https://graph.microsoft.com/v1.0/shares/<child-share-token>/root
This will show me the file meta-data. and then I try to download it by roughly following the api documentation to download https://developer.microsoft.com/en-us/graph/docs/api-reference/v1.0/api/item_downloadcontent
GET https://graph.microsoft.com/v1.0/shares/<child-share-token>/root/content
This is interesting because this works if I make the call with user 'A' but does not work for user 'B' who instead gets a 403 in advanced REST client. (If I run it in Graph Explorer, I get "The site in the encoded share URI is invalid." instead, which I've discovered with other experimentation, really means there's an authorization issue.)
GET https://graph.microsoft.com/v1.0/shares/<share-token>/root:/test.csv:/content
Also does not work, it returns: "400 Bad Request" with message: "Resource not found for the segment 'root:'." It seems like the path style file navigation does not work for shared directories?
At this point I'm rather stuck. After downloading the file, I also would like to move it into a subdirectory, denoting that it has already been read in. I'd also like to get this working for OneDrive for Business, but that seems to be another set of challenges that I'll leave for another day.
Any insight would be great thanks,
Jeremy
It's best to consider the shares/{id} segments to be similar to drives/{id}, at which point all of the previous documentation around children access is applicable. Given your scenario I'd use the path syntax:
https://graph.microsoft.com/v1.0/shares/<share-token>/root/children/test.csv
This obviously necessitates knowing the file name, but it sounds like you already have an algorithm to do that.
Theoretically your approach for creating a child-share-token would work, but it would now require that User B both provide authentication as well as to have explicit permissions. Since your share-token was a sharing link User B is most likely getting permission by virtue of the fact that they have the URL, in which case generating a new one is probably removing the special token that allows this to work. That's why it's best to always use the original share-token where possible.
Similar rules will apply to move the file. First off, we'll assume that the sharing link provides the ability to "Edit" otherwise none of this will work :). Second, we'll assume that the archive folder already exists (if it doesn't you'd need to create it using a POST to https://graph.microsoft.com/v1.0/shares/<share-token>/root/children that looks like what we've documented here).
To move the file you'd want to PATCH to https://graph.microsoft.com/v1.0/shares/<share-token>/root/children/test.csv and provide a new parentReference as documented here. It's always best to use id values if you have them, but you should also be able to provide the path to the parent in the form of /shares/<share-token>/root/children/archive.
Let's say you only have the artist and title from a music file but you don't know the album name.
When you do a Google search in Chrome for i.e Golden Earring Radar Love Album you get:
You see the album name (Moontan), release date (July 1973) and even the correct album cover. How is this page section called? Google Preview? Google Instant Page? I don't know
My question is
How do I programmatically get these information via PowerShell?
What I have tried
Invoke-Webrequest: Not working, specific content not in response
$Response = Invoke-WebRequest -URI "https://www.google.com/search?hl=en&q=Golden+Earring+Radar+Love+Album"
$Response.content | Set-Content D:\test.txt
XmlHttpRequest: Not working, specific content not in response
$objXmlHttp = New-Object -ComObject MSXML2.ServerXMLHTTP
$objXmlHttp.Open("GET", "https://www.google.com/search?hl=en&q=Golden+Earring+Radar+Love+Album")
$objXmlHttp.Send()
$objXmlHttp.responseText | Set-Content D:\test.txt
Invoke-RestMethod: Not working, retrieves only URLs and their snippets
$Response = Invoke-RestMethod -Uri 'https://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Golden%20Earring%20Radar%20Love%20Album'
$Response.responseData.results
I looked for a Google Play or Google Music API which can be used within PowerShell
I believe the problem is, that these information are loaded via Javascript which is not executed when using methods like Invoke-WebRequest. I could be wrong here.
I see two solutions: 1) Imitate a web browser within PowerShell and load the whole website into a form. Or 2) Use fiddler to see when and how these extra information are loaded. I would prefer the second solution but both are beyond my knowledge.
Background to avoid comments saying There are other services like XYZ which better fit your needs
I already have working PowerShell scripts to get album name and additional info only by a given artist and track title for numerous services including Amazon, Deezer, Discogs, EchoNest, iTunes, Last.fm, MusicBrainz, Napster, rdio and Spotify. Because they all offer an easy to use API (except Amazon. Their implementation is pretty hard).
I ran some tests against ~3000 music files only given the artist and track title to retrieve the according album name. And when I compared the results with Google I noticed that none of the above services were so accurate as Google was.
open the powershell profile ii $profile
paste the following snippet to the profile
Function search-google {
$query = 'https://www.google.com/search?q='
$args | % { $query = $query + "$_+" }
$url = $query.Substring(0, $query.Length - 1)
start "$url"
}
Set-Alias glg search-google
restart the powershell session
from the console just run the new command glg hello world
no quotes for strings needed
It's quite possible that Google returns different results depending on the user-agent making the request. So in your case you're not passing a user-agent so Google assumes that it's not a browser and is limiting the amount of information that they are returning (maybe to make your parsing a little easier).
So you have a few options, two of them are:
As suggested by #AlexanderObersht, use Fiddler to sniff some of the network traffic and see what additional headers are being provided by default and fiddle around (pun-intended) with them to see if you can make it work.
With Invoke-RestMethod or Invoke-WebRequest you will need to add a -Headers parameter
With XMLHttpRequest you will have to add the headers in the appropriate properties.
If you don't want to deal with the browser details you can just automate IE directly from Powershell. I've got a sample shown below.
-
$ie = New-Object -com InternetExplorer.Application -ErrorAction Stop
$ie.Visible = $true
$ie.Navigate("https://www.bing.com")
while($ie.Busy) { Start-Sleep -Milliseconds 1 }
$ie.Document.DoStuff()
I am trying to upload an archive file (a Zip archive if that matters) to an existing API at an external site and while I can get the webclient to connect and get a token to the site the upload never seems to work, I continually get a 404 error. The problem may be on their side, but I seem to be the only one who has posted about it on their site, yet I wonder if I am using the UploadFile correctly, or if there is a way to capture the response from the command, which I'd like to know if only to improve my PowerShell coding.
What I am running is the following:
# Variables I want to use
$appArchive = "\\networkpath\Releases\CodeReleasePackage.zip"
$localArchive = "c:\temp\CodeReleasePackage.zip"
[string]$appUrl = "https://site.com/api/v1/apps/(id)/"
[string]$tokenUrl = "https://site.com/token"
[string]$uploadUrl = "https://site.com/api/v1/apps/(id)/?auth_token=$mytoken"
# $appCred = Get-Credential
$username = "email#address"
$password = "password"
# Create the web client
"Creating the client.`n"
$pgup = new-object System.Net.WebClient
$pgup.Credentials = New-Object net.NetworkCredential($username,$password)
"Going to the token page"
$pgToken = $pgup.UploadString($tokenUrl,"")
$mytoken = $pgToken -replace '{|}|"|:|token',''
"Now trying to upload the latest file with my token: $mytoken"
$pgup.UploadFile($uploadUrl,$appArchive)
I get the token which is something like {"token","hkjuihgkjhiuhhlkhuhk"} and not really knowing if there is a better way to handle the resulting token I did the replace to get rid of everything but the token string. Adding it to the URL is supposed to work but I just get 404 errors accorinding to Fiddler.
So I'd like to be able to capture the response from the site when I run $pgup.UploadFile() but when I do it seems like nothing is coming back, is there a particular way to capture that? I couldn't find a clear example and when I do something like $response = $pgup.UploadFile() I have nothing in $response. Anything that is useful for diagnosing webclient errors would be great, I'd like to learn more about this since it seems pretty useful
Im not sure if this is what your looking for, however here is a bit of code that I use to upload a zip file to a server.
$File = "yourfile.zip" #File to be uploaded
$ftp = "ftp://yourserver/directory #Location for the zip file to be uploaded
$webclient = New-Object System.Net.WebClient
$uri = New-Object System.Uri($ftp)
$webclient.Credentials = Get-Credential #Will prompt for user info. You can use your above way to gather credentials as well
$webclient.UploadFile($uri,$File)
As you can see its rather short and simple, and almost identical to your code but does not handle tokens.
It looks like I cannot access the site with PowerShell as the driver, for some reason I can never get the upload to work but I can read the pages on the site and actually get responses for my account. It may be their API doesn't like the way PowerShell is running the connection, although I have never been able to confirm anything from PhoneGap since they have not yet responded to my issues on this yet.
Edited: 6/14/12
The whole reason this was failing was due to the Apache implementation on their side not accepting the HTTP 1.1 request the way WebClient sends it. If I can get back to HTTP 1.0 then they say I should be able to complete this. Or I can wait for the OAUTH 2.0 implementation and just go from there.