I am trying to upload an archive file (a Zip archive if that matters) to an existing API at an external site and while I can get the webclient to connect and get a token to the site the upload never seems to work, I continually get a 404 error. The problem may be on their side, but I seem to be the only one who has posted about it on their site, yet I wonder if I am using the UploadFile correctly, or if there is a way to capture the response from the command, which I'd like to know if only to improve my PowerShell coding.
What I am running is the following:
# Variables I want to use
$appArchive = "\\networkpath\Releases\CodeReleasePackage.zip"
$localArchive = "c:\temp\CodeReleasePackage.zip"
[string]$appUrl = "https://site.com/api/v1/apps/(id)/"
[string]$tokenUrl = "https://site.com/token"
[string]$uploadUrl = "https://site.com/api/v1/apps/(id)/?auth_token=$mytoken"
# $appCred = Get-Credential
$username = "email#address"
$password = "password"
# Create the web client
"Creating the client.`n"
$pgup = new-object System.Net.WebClient
$pgup.Credentials = New-Object net.NetworkCredential($username,$password)
"Going to the token page"
$pgToken = $pgup.UploadString($tokenUrl,"")
$mytoken = $pgToken -replace '{|}|"|:|token',''
"Now trying to upload the latest file with my token: $mytoken"
$pgup.UploadFile($uploadUrl,$appArchive)
I get the token which is something like {"token","hkjuihgkjhiuhhlkhuhk"} and not really knowing if there is a better way to handle the resulting token I did the replace to get rid of everything but the token string. Adding it to the URL is supposed to work but I just get 404 errors accorinding to Fiddler.
So I'd like to be able to capture the response from the site when I run $pgup.UploadFile() but when I do it seems like nothing is coming back, is there a particular way to capture that? I couldn't find a clear example and when I do something like $response = $pgup.UploadFile() I have nothing in $response. Anything that is useful for diagnosing webclient errors would be great, I'd like to learn more about this since it seems pretty useful
Im not sure if this is what your looking for, however here is a bit of code that I use to upload a zip file to a server.
$File = "yourfile.zip" #File to be uploaded
$ftp = "ftp://yourserver/directory #Location for the zip file to be uploaded
$webclient = New-Object System.Net.WebClient
$uri = New-Object System.Uri($ftp)
$webclient.Credentials = Get-Credential #Will prompt for user info. You can use your above way to gather credentials as well
$webclient.UploadFile($uri,$File)
As you can see its rather short and simple, and almost identical to your code but does not handle tokens.
It looks like I cannot access the site with PowerShell as the driver, for some reason I can never get the upload to work but I can read the pages on the site and actually get responses for my account. It may be their API doesn't like the way PowerShell is running the connection, although I have never been able to confirm anything from PhoneGap since they have not yet responded to my issues on this yet.
Edited: 6/14/12
The whole reason this was failing was due to the Apache implementation on their side not accepting the HTTP 1.1 request the way WebClient sends it. If I can get back to HTTP 1.0 then they say I should be able to complete this. Or I can wait for the OAUTH 2.0 implementation and just go from there.
Related
Programmatically downloading from Dropbox with PowerShell is trivial. With a link to a shared file https://www.dropbox.com/s/sdfjjthwrf32yFileName.pdf?dl=0, you just replace www.dropbox.com by dl.dropboxusercontent.com to give https://dl.dropboxusercontent.com/s/sdfjjthwrf32yFileName.pdf?dl=0 and can then use Invoke-WebRequest to fetch that file via a script (as long as the permissions on it are open for anyone to read).
With OneDrive, I have spent a few hours looking, and found no way to download a file with PowerShell. Some websites talk about OneDrive for Business, this question is not about OneDrive for Business as I do not know about or use that, this is specifically about OneDrive.
What I want to do is really simple, i.e. share a file in OneDrive, then get the share link, then be able to programmatically download that file using PowerShell (as the file is set to "anyone can view" so I should not really require authentication). Here is how I would share that file on the OneDrive.live.com site:
I then change the file from "Anyone can Edit" to "Anyone can View", and this gives me a link that looks like this:
$url = "https://1drv.ms/u/s!Bjfhd-pwUc_asdcadfasdfTYFASDFASDf?e=aksdHDG"
I then try to download it using Invoke-WebRequest:
Invoke-WebRequest -UseBasicParsing -Uri $url -OutFile "C:\MyFile.txt"
But all I get is a small stub of javascript output (note that the above is not a real link, but an example)
With Dropbox, it is trivially simple, so how can we do this with OneDrive?
The closest I have come might be the OneDrive module, but I don't know how to generate an OAuth2 token that I can pass to the script (and it completely breaks the purpose of this exercise, as if I have to provide credentials to download an "Anyone can View" file, then I cannot create a script that others can use to download a file without them also providing credentials; we should not need to provide credentials if the file in question is set as "anyone can view". That is the case for the Dropbox example; if the file is to read-only for everyone, then no credentials are required, as that would obviously be sort of ludicrous - it would be like demanding a key to get through an unlocked and open doorway that has a sign above it saying "Anyone may freely enter here without a key").
Install-Module -Name OneDrive -Force
Please note: To stress, this question is not about OneDrive for Business, that is something that I do not know or use.
You can download the shared file directly by using the OneDrive API. To do so, you have to convert your sharing URL into a sharing token and pass it to the OneDrive API. This example shows the needed C# code. You can derive the following PowerShell function from it:
function Get-OneDriveDirectDownloadUrl {
param(
[string]
$SharingUrl
)
$base64Value = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($SharingUrl))
$encodedUrl = 'u!' + $base64Value.TrimEnd('=').Replace('/','_').Replace('+','-')
return 'https://api.onedrive.com/v1.0/shares/' + $encodedUrl + '/root/content'
}
With that function, you can convert your sharing URL like this:
Get-OneDriveDirectDownloadUrl -SharingUrl 'https://1drv.ms/u/s!Bjfhd-pwUc_asdcadfasdfTYFASDFASDf?e=aksdHDG'
And you will receive the following direct download URL:
https://api.onedrive.com/v1.0/shares/u!aHR0cHM6Ly8xZHJ2Lm1zL3UvcyFCamZoZC1wd1VjX2FzZGNhZGZhc2RmVFlGQVNERkFTRGY_ZT1ha3NkSERH/root/content
Looking for solution how to test website login with powershell without IE COM method. There is lot of examples how to do it with IE COM method: -ComObject 'internetExplorer.Application, but unfortunately this is not working for me as site is not working with IE at all.
Google chrome has a nice feature in its developer tools, on the network tab.
Clear the network log, filter so that it shows "All" requests/responses.
Log into the site and then click the red button or hit "ctrl + e" to stop the log from increasing.
Go through each request until you see the login request (These can sometimes be hard to spot.)
Once you have found the request right click on the name of the request, you can then select "Copy -> Copy As PowerShell".
Paste this into ISE etc.
I would personally then remove all of the $Session lines from above the request and change -WebSession $session to -SessionVariable $session and subsequently call -WebSession $session in each web request there after. (Only use -Sessionvariable once as this is the what instantiates the session). The reason I do this is so that the login creates its own session data rather than using any possible data from the browser.
Last of all you will want to clear any unwanted headers out of the webrequest.
Depending on the login mechanism the request may need tweaking to allow it to be done through automation, I have only had a couple of circumstances where this has been the case and multiple requests have been required due to AJAX or other elements on the page.
I have the following script for logging in to salesforce on IE.
I want to login using chrome or firefox.
If I need a completely new script I will use it.
$username = "username#domain.com"
$password = "Mypassword"
$ie = New-Object -com InternetExplorer.Application
$ie.visible=$true
$ie.navigate("https://login.salesforce.com")
while ($ie.Busy -eq $true)
{
Start-Sleep -Milliseconds 1000;
}
$ie.document.getElementById("username").value= "$username
$ie.document.getElementById("password").value = "$password"
$login = $ie.document.getElementsByClassName("button r4 wide primary") | Select-Object -first 1
$login.click()
TL;DR: There's no built-in way to do what you're asking.
Let me preface this by saying that, in general, trying to automate logins in this manner is a really, really bad idea. I cannot stress enough how dangerous this is from a security perspective. I would strongly recommend against attempting to write a Powershell script to automate a login in this manner unless it is done with a fake username and password for learning/test purposes.
That being said, IE has a rich COM interface - Chrome and Firefox, not so much. You will need to install an extension to enable the same functionality.
This is evil, if you want systematic access and automation you should look into proper REST or SOAP API access, with OAuth2 flows. There's even a flow where you don't share password, the trust is established by signing a request with certificate that was earlier uploaded to SF and the user preauthorised...
This should never be part of any serious deliverable. If this gets hacked - your company / client can sue for damages.
If you really need it you can use simple GET like https://login.salesforce.com/?un=username#example.com&pw=hunter2, no need for fancy scripting. In Setup -> Login History you'll see that the method was GET instead of POST (submitting the real form). I don't think it's officially supported, they could change it anytime. It'd pass your password plaintext over the web...
Edit:
Check what your company signed. For example the Master Service Agreement
https://a.sfdcstatic.com/content/dam/www/ocms-backup/assets/pdf/misc/salesforce_MSA.pdf
(…)the Service or Content may not be accessed by more than that number
of Users, (b) a User’s password may not be shared with any other
individual(…)
Let's say you only have the artist and title from a music file but you don't know the album name.
When you do a Google search in Chrome for i.e Golden Earring Radar Love Album you get:
You see the album name (Moontan), release date (July 1973) and even the correct album cover. How is this page section called? Google Preview? Google Instant Page? I don't know
My question is
How do I programmatically get these information via PowerShell?
What I have tried
Invoke-Webrequest: Not working, specific content not in response
$Response = Invoke-WebRequest -URI "https://www.google.com/search?hl=en&q=Golden+Earring+Radar+Love+Album"
$Response.content | Set-Content D:\test.txt
XmlHttpRequest: Not working, specific content not in response
$objXmlHttp = New-Object -ComObject MSXML2.ServerXMLHTTP
$objXmlHttp.Open("GET", "https://www.google.com/search?hl=en&q=Golden+Earring+Radar+Love+Album")
$objXmlHttp.Send()
$objXmlHttp.responseText | Set-Content D:\test.txt
Invoke-RestMethod: Not working, retrieves only URLs and their snippets
$Response = Invoke-RestMethod -Uri 'https://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Golden%20Earring%20Radar%20Love%20Album'
$Response.responseData.results
I looked for a Google Play or Google Music API which can be used within PowerShell
I believe the problem is, that these information are loaded via Javascript which is not executed when using methods like Invoke-WebRequest. I could be wrong here.
I see two solutions: 1) Imitate a web browser within PowerShell and load the whole website into a form. Or 2) Use fiddler to see when and how these extra information are loaded. I would prefer the second solution but both are beyond my knowledge.
Background to avoid comments saying There are other services like XYZ which better fit your needs
I already have working PowerShell scripts to get album name and additional info only by a given artist and track title for numerous services including Amazon, Deezer, Discogs, EchoNest, iTunes, Last.fm, MusicBrainz, Napster, rdio and Spotify. Because they all offer an easy to use API (except Amazon. Their implementation is pretty hard).
I ran some tests against ~3000 music files only given the artist and track title to retrieve the according album name. And when I compared the results with Google I noticed that none of the above services were so accurate as Google was.
open the powershell profile ii $profile
paste the following snippet to the profile
Function search-google {
$query = 'https://www.google.com/search?q='
$args | % { $query = $query + "$_+" }
$url = $query.Substring(0, $query.Length - 1)
start "$url"
}
Set-Alias glg search-google
restart the powershell session
from the console just run the new command glg hello world
no quotes for strings needed
It's quite possible that Google returns different results depending on the user-agent making the request. So in your case you're not passing a user-agent so Google assumes that it's not a browser and is limiting the amount of information that they are returning (maybe to make your parsing a little easier).
So you have a few options, two of them are:
As suggested by #AlexanderObersht, use Fiddler to sniff some of the network traffic and see what additional headers are being provided by default and fiddle around (pun-intended) with them to see if you can make it work.
With Invoke-RestMethod or Invoke-WebRequest you will need to add a -Headers parameter
With XMLHttpRequest you will have to add the headers in the appropriate properties.
If you don't want to deal with the browser details you can just automate IE directly from Powershell. I've got a sample shown below.
-
$ie = New-Object -com InternetExplorer.Application -ErrorAction Stop
$ie.Visible = $true
$ie.Navigate("https://www.bing.com")
while($ie.Busy) { Start-Sleep -Milliseconds 1 }
$ie.Document.DoStuff()
I have a PowerShellScript that will call a webpage and the webpage is doing some code-behind work. The issue I am trying to resolve is that I'd like to make the webpage as a secured web page, but once I do that, I couldn't figure out a way to pass the username and password to the secured webpage from PowerShell Script. I am using the form authentication. I'd appreciate if anyone could provide a completed code sample so that I can just plug-in and do the magic. Thanks so much!
p.s. I've already tried the System.Net.NetworkCredential, but it seems it is for the basic authentication.
$url = "someurl"
$webclient = New-Object System.Net.WebClient
$webclient.DownloadString($url)
Need a little more information. You may be able to solve your issue with the following answer
How to post data to specific URL using WebClient in C#
But I'm curious, what page you are submitting your authentication form to? That page would have this information. If the form is returning data into PS variables. Then the above link shows how to post directly to a page