PowerShell To Retreive HTML from an open tab - powershell

Question

I am trying to retrieve links from a dynamic JavaScript-rendered web page using PowerShell.
This is the code I have so far:
# Create IE object and load URL
$ie = New-Object -comobject "InternetExplorer.Application"
$ie.visible = $true
$ie.navigate($url)
# Wait for the page to load
while ($ie.Busy -eq $true -Or $ie.ReadyState -ne 4) {Start-Sleep 2}
$Doc = $ie.Document
$divs = $Doc.getElementsByTagName('a')
foreach ($div in $divs){
write-host $div['id'].value
write-host $div['tagName'].value
write-host $div['parentElement'].value
write-host $div['style'].value
write-host $div['document'].value
write-host $div['sourceIndex'].value
write-host $div['offsetLeft'].value
write-host $div['offsetTop'].value
write-host $div['offsetWidth'].value
write-host $div['offsetHeight'].value
write-host $div['offsetParent'].value
write-host $div['innerHTML'].value
write-host $div['innerText'].value
write-host $div['outerHTML'].value
write-host $div['outerText'].value
write-host $div['parentTextEdit'].value
}
However, all of the output is blank lines.
(FYI - if I output just the $div, then I get System.__ComObject
Can anyone explain what I need to do to get the information?
Thank you.

Answer 1

First of all, I would change the names of $divs to $tags and use $tag rather then $div in the foreach loop, because you are not really looking for divs, but <a> tags.
Each $tag has a getAttribute method you should use instead of $tag['attribName'].value.
Your code could then look like this
$ie = New-Object -comobject "InternetExplorer.Application"
$ie.visible = $true
$ie.navigate($url)
# Wait for the page to load
while ($ie.Busy -eq $true -Or $ie.ReadyState -ne 4) {Start-Sleep 2}
$Doc = $ie.Document
$tags = $Doc.getElementsByTagName("a")
foreach ($tag in $tags){
Write-Host $tag.getAttribute("id")
Write-Host $tag.getAttribute("tagName")
Write-Host $tag.getAttribute("parentElement")
Write-Host $tag.getAttribute("style")
Write-Host $tag.getAttribute("document")
Write-Host $tag.getAttribute("sourceIndex")
Write-Host $tag.getAttribute("offsetLeft")
Write-Host $tag.getAttribute("offsetTop")
Write-Host $tag.getAttribute("offsetWidth")
Write-Host $tag.getAttribute("offsetHeight")
Write-Host $tag.getAttribute("offsetParent")
Write-Host $tag.getAttribute("innerHTML")
Write-Host $tag.getAttribute("innerText")
Write-Host $tag.getAttribute("outerHTML")
Write-Host $tag.getAttribute("outerText")
Write-Host $tag.getAttribute("parentTextEdit")
}
Also, when finished you should cleanup the COM object you have created:
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($ie) | Out-Null
[System.GC]::Collect()
[System.GC]::WaitForPendingFinalizers()
Edit
to make better output i suggest this:
$Doc = $ie.Document
$tags = $Doc.getElementsByTagName("a")
#$tags = $doc.all.tags("a")
$attribs = #("id", "tagName", "parentElement", "style", "document", "sourceIndex",
"offsetLeft", "offsetTop", "offsetWidth", "offsetHeight", "offsetParent",
"innerHTML", "innerText", "outerHTML", "outerText", "parentTextEdit")
foreach ($tag in $tags){
$attribs | ForEach-Object {
#{ $_ = $tag.getAttribute($_) }
}
}