I've been looking to ways to read HTML on a opened custom chrome profile, but with no luck.
& "C:\Program Files\Google\Chrome\Application\chrome.exe" --profile-directory="Profile 1" $url;
I tried using invoke-webrequest but couldn't figure out how to get it to work through a custom profile.
The point is to read if an item was bought on $url, where "Profile 1" is logged into.
Alternatively I thought about logging into $url with POST method and then returning the HTML as a variable, but I couldn't figure out a way to do that either.
This won't work. This can't work.
You can :
Launch Chrome on a specific profile
Use Invoke-RestMethod and / or Invoke-WebRequest
The two are not correlated though.
The Powershell cmdlets do not depend on Chrome for their operations.
If you want to do stuff through Chrome, you need to do browser automation.
That's a different thing that require specialized tools, such as Selenium.
You can use the .Net interface of selenium directly or use the Selenium powershell module that will help you interface with selenium (and your browser of choice / browser profile) more easily.
To install :
Install-Module -Name Selenium -AllowPrerelease
Note that this tool is a lot more complex to learn and use than just Invoke-WebRequest
References
Github - Selenium-Powershell
Powershell Gallery - Selenium
Related
For a project I'm working on I want to export IIS bindings and then import them back using PowerShell. The export part is working, but now I want to import the IIS site bindings back into IIS. I'm using the New-IISSiteBinding command in order to import them like this:
New-IISSiteBinding -Name "Portal" -BindingInformation "*:443:domainname.com" -CertificateThumbPrint "theactualthumbprint" -CertStoreLocation "Cert:\LocalMachine\My" -Protocol https
When running the command it says WARNING: Web site 'Portal' does not exist.. Get-IISSite is returning the same result. When I look in the IIS management console, I do see the site with that name.
Why is it saying that the Web site does not exist when it actually does?
It was actually PowerShell ISE that was the issue. When I executed the scripts in a separate PowerShell window it was working fine and importing the binding. I wasn't aware that ISE is no longer actively maintained, so that's probably why my command wasn't working.
I'm trying to do something like this to parse a homepage with a login page but Invoke-WebRequest doesn't return anything.
The page I'm trying to access is https://www.suidoapp.waterworks.metro.tokyo.lg.jp/#/login and the code I'm running is this:
$TopURI = "https://www.suidoapp.waterworks.metro.tokyo.lg.jp/#/login"
$TopPage = Invoke-WebRequest -Method Get -Uri $TopURI -SessionVariable MySession -UseBasicParsing
When I look at the Content or RawContent of the $TopPage I can see that it just says "please enable JavaScript" (I've tried both with and without -UseBasicParsing). If I open the page in developer tool in my browser I can see that response for the initial document is the same:
But the interesting thing is that even though the initial page says "please enable JavaScript" the page actually loads:
Has anyone seen this before, where Invoke-WebRequest fails because the response is "please enable JavaScript" yet the page should actually be able to load? Is there another way for me to parse a homepage and send in login forms when Invoke-WebRequest fails like this?
I am having the same issue. The simple answer is: Invoke-WebRequest is not allowed to run javascript for fear of XSS attacks which makes total sense. In my case, I needed to run a Vue.js app via Task Scheduler (on Windows machine in conjunction with IIS). I eventually let PowserShell open a browser and finish the work then close it.
Start-Process -file iexplore -arg 'http://localhost:8080/ (or any URL)' -PassThru
sleep 10
(Get-Process -Name iexplore).Kill()
If you would like to run Firefox instead,
Start-Process -file 'C:\Program Files\Mozilla Firefox\firefox.exe (or your ff location)' -arg 'http://localhost:8080/ (or any URL)' -PassThru
sleep 10
(Get-Process -Name firefox).Kill()
Going back to your question, the loaded page is probably not functional if it invokes any javascript functions. If you talk about form login, you can find resources easily such as this: https://community.auth0.com/t/forms-login-via-curl-or-powershell/17456/3
I've inherited a lot of scripts that rely on Invoke-WebRequest and am aware that this commandlet requires IE to run.
Is there a way to configure Invoke-WebRequest so that it uses Edge instead?
Considering that Internet Explorer 11 desktop application will be retired and go out of support on June 15, 2022; I'd imagine that MS would have some drop-in replacement for the Invoke-WebRequest command that would allow PowerShell scripts using it to continue to function after IE11's retirement.
PowerShell 5.1 -UseBasicParsing
Indicates that the cmdlet uses the response object for HTML content without Document Object Model (DOM) parsing. This parameter is required when Internet Explorer is not installed on the computers, such as on a Server Core installation of a Windows Server operating system.
PowerShell 7.2 -UseBasicParsing
This parameter has been deprecated. Beginning with PowerShell 6.0.0, all Web requests use basic parsing only. This parameter is included for backwards compatibility only and any use of it has no effect on the operation of the cmdlet.
I am attempting to get a script working that does not use invoke-webrequest. The problem I am having is that when I run the script a popup prompt occurs, the popup consists of the following message;
"Windows Security Warning
To allow this website to provide information personalized for you, will you allow it to put a small file (called a cookie) on your computer?"
with yes no response from user
The code I am executing is the following:
$ParsedHTML = New-Object -com "HTMLFILE"
$webresponse = New-Object System.Net.WebClient
$webresponse.Headers.Add("Cookie", $CookieContainer.GetCookieHeader($url))
$result = $webresponse.DownloadString($url)
$ParsedHTML.IHTMLDocument2_write($webresponse)
$ParsedHTML.body.innerText
The main problem with this code is that the $url I am using part of the weblink checks to see if cookies are enabled and this code causes a returned value of disabled.
My question, is there a way to handle the cookie request without changing the output response from the test url site.
Note: This script will be automating a process over hundreds of remote computers and thus having an unhandled popup will just prevent the script from running.
I found the answer in another SO question Using Invoke-Webrequest in PowerShell 3.0 spawns a Windows Security Warning
add the parameter -UseBasicParsing
Technet https://technet.microsoft.com/en-us/library/hh849901.aspx notes that the parameter stops the DOM processing. and has the caveat "This parameter is required when Internet Explorer is not installed on the computers, such as on a Server Core installation of a Windows Server operating system."
So, you mileage may vary.
I want to write a script that loads a url (eg. http://google.com) automatically. But I don't want to install any 3rd party libraries or programs to the server. what's the easiest way to do this?
I just my options are batch script, vb script or powershell right?
FYI from PowerShell, if you want to retrieve the contents of the URL you can do this;
$page = (new-object net.webclient).DownloadString("http://www.bing.com")
$page # writes the contents to the console
If you just want to open it in a browser:
Start-Process http://www.bing.com
Or using the start alias
start http://www.bing.com
Start-Process is new in PowerShell 2.0.
The beauty of Powershell is it has so many ways to do something.
This is my Powershell 2.0 example code - consisting of a Pause function to allow the site to open. It uses Internet Explorer as the browser. In this case - IE is a better browser than the others because it integrates with Powershell through a verbose API.
$url = "http://www.google.com/"
$ie = new-object -com "InternetExplorer.Application"
$ie.Navigate($url)
There are many different functions attached to this object. I recommend loading up the Powershell command line, typing in the above code, and checking what other functions this object has. Type $ie. and pressing TAB iterates through all the methods of this library.
The more I learn of Powershell, the more exciting it becomes. There is nothing it cannot do on Windows.
you can use vbscript
url="http://somewhere.com"
Set objHTTP = CreateObject( "WinHttp.WinHttpRequest.5.1" )
objHTTP.Open "GET", url, False
objHTTP.Send
wscript.Echo objHTTP.ResponseText
objFile.Close