I am pretty new to Powershell and just using it for personal stuff. I have been experimenting with pulling specific info from websites to include in emails to family. By reading the forums I got pretty good using the Invoke-WebRequest cmdlet, but soon hit upon its limitation of not having access to content constructed dynamically at the time the page is loaded.
Thanks to these forums, I then discovered the IE object and how to pull the data. I had luck with one website, but another I tried does not work the same. Hoping for a little help figuring it out.
Here is a snippet of the inspected code for the page, with my target of interest highlighted.
Below is the code where I am trying to extract that text string. I have tried many iterations and approaches with no success. What is odd, though, the $ie.Document object supposedly has a "body" object, but when I tried to access it, I get a null object error. I notice the Document object itself has a getElementsByTagName method, so I tried that. It does not have a getElementsByClassName method.
Note that the URL I am loading is "https" so I am wondering if this is causing issues. Suggestions appreciated! If I can just get a surrounding chunk of the HTML, I am fine doing some string manipulation to get what I want.
# Create IE object and load URL
$WeatherURL = "https://weather.com/weather/today/l/77630"
$ie = New-Object -comobject "InternetExplorer.Application"
$ie.visible = $true
$ie.navigate($WeatherURL)
# Wait for the page to load
while ($ie.Busy -eq $true -Or $ie.ReadyState -ne 4) {Start-Sleep 2}
$Doc = $ie.Document
$Weather0 = $Doc.getElementsByTagName('span') `
| ?{$_.getAttribute('class') -eq "today-wx-descrip"} | Select-Object -First 1
You should replace
$Weather0 = $Doc.getElementsByTagName('span') `
| ?{$_.getAttribute('class') -eq "today-wx-description"} | Select-Object -First 1
With
$Weather0 = $Doc.getElementsByTagName('span') `
| ?{$_.getAttribute('class') -eq "today-wx-descrip"} | Select-Object -First 1
Note today-wx-description vs today-wx-descrip.
Related
I was asked to retrieve a .csv list with all users that registered to the FIM portal. I did some searching until I stumbled accross this script:
set-variable -name URI -value "http://localhost:5725/resourcemanagementservice' " -option constant
set-variable -name CSV -value "RegistredResetPassUsers.csv" -option constant
clear
If(#(Get-PSSnapin | Where-Object {$_.Name -eq "FIMAutomation"} ).count -eq 0) {Add-PSSnapin FIMAutomation}
$WFDFilter = "/WorkflowDefinition[DisplayName='Password Reset AuthN Workflow']"
$curObjectWFD = export-fimconfig -uri $URI –onlyBaseResources -customconfig ($WFDFilter) -ErrorVariable Err -ErrorAction SilentlyContinue
$WFDObjectID = (($curObjectWFD.ResourceManagementObject.ResourceManagementAttributes | Where-Object {$_.AttributeName -eq "ObjectID"}).value).split(":")[2]
$Filter = "/Person[AuthNWFRegistered = '$WFDObjectID']"
$curObject = export-fimconfig -uri $URI –onlyBaseResources -customconfig ($Filter) -ErrorVariable Err -ErrorAction SilentlyContinue
[array]$users = $null
foreach($Object in $curObject)
{
$ResetPass = New-Object PSObject
$UserDisplayName = (($Object.ResourceManagementObject.ResourceManagementAttributes | Where-Object {$_.AttributeName -eq "DisplayName"}).Value)
$ResetPass | Add-Member NoteProperty "DisplayName" $UserDisplayName
$Users += $ResetPass
}
$users | export-csv -path $CSV
The script works without errors except that the actual list that it exports only contains my display name. I've been trying to figure out why its not exporting the complete list of all users and only shows my name, but I haven't been able to figure it out so I was wondering if any one could help me shed some light into this issue.
Thanks again for any help you guys can provide!
No experience with this snapin/product, but seeing as the code fundamentally works (it returns your object) this could be a permissions issue. You may not be able to read the other user objects, so they're not being exposed to you.
If you can view them in a UI console of some kind, then check for separate permissions related to API actions, and ensure you have access that way.
Another course of action may be to run the code line by line and see what results you receive from each line, to make sure you get what you're expecting.
Try replacing:
[array]$users = $null
With:
$users = #()
This is likely due to permission setup.
By default you have a permissions to see your own attributes.
There is likely some Management Policy Rule setup so user accounts in a specific Set can read AuthNWFRegistered attribute of other users to support for troubleshooting and customer support.
You will need to use one of the options:
Add the account used for this script into the Set that delegates out this Read permission already
or
Create a separate MPR for this particular reporting (this is what I would recommend) that Grants permissions for a specific user account to read AuthNWFRegistered attribute.
Also make sure there is really only one Workflow that is associated with user registration. If there are multiple, you'd want to target Set with all register Workflows in your XPath filter instead of particular Workflow name.
On a separate note - while FIMAutomation is sometimes necessary snapin to use with standard tooling, for your custom work I strongly suggest to use Lithnet's LithnetRMA PowerShell module (https://github.com/lithnet/resourcemanagement-powershell).
You will be much more productive with it and most of the operations will be without boilerplate code FIMAutomation needs. This will be your code using LithnetRMA.
Set-ResourceManagementClient -BaseAddress 'http://localhost:5725/resourcemanagementservice'
$scope = Search-Resources -XPath "/Person[AuthNWFRegistered = /WorkflowDefinition[DisplayName='Password Reset AuthN Workflow']]" -ExpectedObjectType Person
$scope | select DisplayName | Export-Csv 'RegistredResetPassUsers.csv' -Encoding Unicode
I am very new to powershell, and I am trying to automate a login to website, I have browsed many examples but NONE is working without errors.
For example with this very simple script:
$ie = New-Object -comobject InternetExplorer.Application
$ie.Visible = $true
$ie.Navigate($URL)
$ie.Navigate("www.facebook.com")
$ie.document.getElementById("pass")
$ie.document.getElementById("pass").value = "Hi"
I am supposed to get "Hi" in the password field but I get instead this
I am getting this error in ALL the examples I found, what's going on? Is this something no longer supported in Powershell?
I have windows 10.
Don't use getElementById. Use IHTMLDocument3_getElementById, e.g.:
$ie.Document.IHTMLDocument3_getElementById("email").value = "asd123"
You can do this for other methods as well, such as IHTMLDocument3_getElementsByName.
I should add that I don't know why this works - I just used the following query to find any useful looking members on the Document object and played around with them
$ie.Document | Get-Member | where Name -like '*get*'
Been wrestling with parsing the HP Warranty website since the web service method seems to be constantly changing.
Problem I am having is I have tried to use the internet explorer com and its acting different depending on a version difference somewhere either PS or IE of which I am not certain how to tell.
Here is my core portion of code that I am testing with. You would need to find an HP Serial Number to test with.
$SerialNumber = ""
$link = "http://h20564.www2.hp.com/hpsc/wc/public/find?rows%5B0%5D.item.serialNumber=" + $SerialNumber + "&rows%5B0%5D.item.countryCode=US&submitButton=Submit"
Write-Verbose "Calling: $link"
$request = New-Object System.Net.WebClient
$request.UseDefaultCredentials = "True"
$request.Proxy = [System.Net.GlobalProxySelection]::GetEmptyWebProxy()
$results = $request.downloadstring($link)
if($results -match "Base Warranty") {
Write-Verbose "Results contain Warranty Data"
$results | Out-File "C:\Temp\Warranty.html" -Encoding ascii
$oIE = New-Object -ComObject InternetExplorer.Application
$oIE.Navigate("C:\Temp\Warranty.html")
$ohtmldoc = $oIE.Document
$rows = $ohtmldoc.documentElement.getElementsByClassName("hpui-normal-row")
$data = $rows[0].ChildNodes | ? {$_.TagName -eq "td"}
$ExpireDate = Date($data[4].InnerText)
} else {
Write-Verbose "Results do not contain Warranty Data"
$ExpireDate = ""
}
Works fine on my win 10 machine with IE11 v11.212.10586.0; PSVersion 5.0
Failing win 2008 R2 Term servers with IE11 v11.0.9600.17873CO in two different fashions; PSVersion 4.0
It appears that the DOM is not populating the ChildNodes property from the $rows[0] variable after it parses the ClassName.
Aside from the above being my main issue,
Another 2008 R2 Term server as soon as I invoke the $oIE.Navigate Method it opens an instance of an IE window for the link being called then the rest of the commands fail which I haven't seen happen yet on my other two environments.
Any thoughts would be greatly appreciated as I don't often use the InternetExplorer.Application COM. I really just need to be able to parse this HTML content to get the table data effectively. I had tried ditching the COM and ingesting the HTML contents as XML but HP has several malformed lines on the page this warranty result kicks back.
I don't appear to be able to write changes to Outlook via MAPI, the .UnRead variable is being set correctly to false within the script if you Write-Output it, but the variables don't appear to manipulate the actual .PST file. The select produces the correct emails, so read access to the .PST is fine.
Here is the code I am using to retrieve a list of unread emails from a PST folder, and set one of them to read:
$Outlook = new-object -comobject "Outlook.Application";
$Mapi = $Outlook.getnamespace("mapi");
$Pst = $Mapi.Folders.Item("Personal Folders")
$Folder = $Pst.Folders.Item("Test")
$Emails = $Folder.Items | Select UnRead, SenderEmailAddress, Subject, ReceivedTime, Body | Where {$_.Unread -eq "True"}
$Emails[1].UnRead = $false
Most examples I have seen say to place the variable in brackets, e.g.
$($Emails)[1].UnRead = $false
But this has made no difference for me.
Interestingly I get a 'method not found' error when I try to use the .delete() as well, hence I think I must be missing something.
Many thanks in advance for any advice.
Call MailItem.Save.
Do not loop through all items in a folder, use Items.Find/FindNext or Items.Restrict.
You've change property of your own object, but not on mail server
Last year I had powershell (v3) script that parsed HTML of one festival page (and generate XML for my Windows Phone app).
I also was asking a question about it here and it worked like a charm.
But when I run the script this year, it is not working. To be specific - the method getElemntsByClassName is not returning anything. I tried that method also on other web pages with no luck.
Here is my code from last year, that is not working now:
$tmpFile_bandInfo = "C:\band.txt"
Write-Host "Stahuji kapelu $($kap.Nazev) ..." -NoNewline
Invoke-WebRequest http://www.colours.cz/ucinkujici/the-asteroids-galaxy-tour/ -OutFile $tmpFile_bandInfo
$content = gc $tmpFile_bandInfo -Encoding utf8 -raw
$ParsedHtml = New-Object -com "HTMLFILE"
$ParsedHtml.IHTMLDocument2_write($content)
$ParsedHtml.Close()
$bodyK = $ParsedHtml.body
$bodyK.getElementsByClassName("body four column page") # this returns NULL
$page = $page.item(0)
$aside = $page.getElementsByTagName("aside").item(0)
$img = $aside.getElementsByTagName("img").item(0)
$imgPath = $img.src
this is code I used to workaround this:
$sec = $bodyK.getElementsByTagName("section") | ? ClassName -eq "body four column page"
# but now I have no innerHTML, only the lonely tag SECTION
# so I am walking through siblings
$img = $sec.nextSibling.nextSibling.nextSibling.getElementsByTagName("img").item(0)
$imgPath = $img.src
This works, but this seems silly solution to me.
Anyone knows what I am doing wrong?
I actually solved this problem by abandoning Invoke-WebRequest cmdlet and by adopting HtmlAgilityPack.
I transformed my former sequential HTML parsing into few XPath queries (everything stayed in powershell script). This solution is much more elegant and HtmlAgilityPack is real badass ;) It is really honour to work with project like this!
The issue is not a bug but rather that the return where you're seeing NULL is because it's actually a reference to a proxy HTMLFile COM call to the DOM model.
You can force this to operate and return the underlying strings by boxing it into an array #() as such:
#($mybody.getElementsByClassName("body four column page")).textContent
If you do a Select-Object on it, that also automatically happens and it will unravel it via COM and return it as a string
$mybody.getElementsByClassName("body four column page") | Select-Object -Property TextContent