Using PowerShell & Invoke-Webrequest to get specific information - powershell

My code is below, it's pretty much complete, however a tiny part I am stuck on.
Basically what the code does is, based on a Song, the script goes out to a couple of websites on the internet to bring back the song label, composer & year that the song was released.
Only thing is, with the particular song Alicia Keys - Girl On Fire, if you manually go to the website http://staff.australian-charts.com/showitem.asp?interpret=Alicia+Keys+feat%2E+Nicki+Minaj&titel=Girl+On+Fire&cat=s you will notice that there is more than one Composer listed under the Music/Lyrics section.
In the event the song has more than one composer, my script below in it's current state will grab the first composer listed only.
What I need is for the script to grab all the composers. If there's one composer, or if there are multiple composers, I need them captured in the format of "Composer1, Composer2, Composer3, Composer4" etc (Commas included)
I am thinking of changing the Invoke-Webrequest to get TABLES back and particular tables with rows etc, not sure....
$song = "Alicia Keys - Girl On Fire"
Write-Host $song
$SearchSong = $song -replace '\(' -replace '\)' -replace '&' -replace ' - ', ' ' -replace '\s','+'
$MatchSong = $song -replace ' - ', '&titel=' -replace '\s','\+'
#Check iTunes for music Label information
$uri = "https://itunes.apple.com/search?term=$SearchSong&country=au&entity=song"
$x = Invoke-WebRequest -Uri $uri
$iTunesResults = ($x.Content | ConvertFrom-Json).results
$y = Invoke-WebRequest -Uri $iTunesResults[0].trackViewUrl
$iTunesSongCopyright = ($y.ParsedHtml.getElementsByTagName('li') | ? {$_.ClassName -eq 'copyright'}).innerText -replace '℗ '
$iTunesSongLabel = $iTunesSongCopyright -replace '.*\d\s'
#The check australian-charts for Composer & Year infomation
$domain = 'http://staff.australian-charts.com/'
$uri = $domain + "search.asp?search=$SearchSong&cat=s"
$x = Invoke-WebRequest -Uri $uri
$x.AllElements[462].outerHTML -match 'A.href="(.*)"';$resultURL = $domain + $Matches[1]
$resultURL = $resultURL -replace("&","&") -replace('"','"')
$y = Invoke-WebRequest -Uri $resultURL
$Element = ($y.AllElements | ? {$_.tagName -eq 'HTML'})
if($Element.innerText -match 'Music\/Lyrics:(.*)')
{
$Element.innerText -match 'Music\/Lyrics:(.*)'
$Composer = $Matches[1]
Write-Host $Composer
} else {
$Composer = $null
}
if($Element.innerText -match 'Year:(.*)')
{
$Element.innerText -match 'Year:(.*)'
$Year = $Matches[1]
Write-Host $Year
} else {
$Year = $null
}
Write-Host $iTunesSongLabel

you can use this for getting composer list :
if($Element.innerText -match 'Music\/Lyrics:(.*)')
{
$startpos = $Element.innertext.IndexOf("Lyrics:") + 7
$endpos = $Element.innertext.IndexOf("Producer:") -1
$composer=$Element.innertext.substring($startpos,($endpos - $startpos))
#even the below line will give the same result as the above line if uncommented
#$composer = $Element.innertext[$startpos..$endpos] -join ""
Write-Host $Composer
}

Related

using PowerShell, Is there an easier or better optimized way to get the download link to the latest Nvidia Driver?

Hello all, I've been able to get this code to get the latest download link for Nvidia Driver for a Quadro P1000.
I was wondering, if there's a better way to do this.
Here's my code:
#Get Nvidia Drivers
#This next section is JUST TO get the LINK to the LATEST driver from Nvidia
#WebContent gets the links that are "href" from the nvidia JS request. No idea if there's an easier wey to get this.
$Webcontent = (Invoke-WebRequest 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0').links.href
#The following line uses Regex regular expressions to MATCH and RETRIEVE ONE single value from the list of values in the previous line.
$NVIDIALinkToLatestDriver = [regex]::Match($Webcontent, '//(www.nvidia.com/download/driverResults.aspx/[0-9]*/en-us)').Groups[1].Value
#Link after the previous crap
$NVIDIADLPage = New-Object -COM "HTMLFile" #Creates a COM Object for easier search of the link.
[string]$htmlBody = (Invoke-WebRequest -Uri $NVIDIALinkToLatestDriver -UseBasicParsing).Content #Parses the content of the landing page to then look by id
$NVIDIADLPage.write([ref]$htmlBody)
$replace = [regex]::Replace($NVIDIADLPage.getElementById('lnkDwnldBtn').href, 'about:', 'www.nvidia.com') #Replaces the "about" with "www.nvidia.com"
$Webcontent = (Invoke-WebRequest $replace) #Replace Webcontent with the latest link.
[String]$NvidiaLinkToExe = $Webcontent.links.href -match ".*.exe$" #On this link there's the exe file for Nvidia Drivers
$NvidiaLinkToExe = $NvidiaLinkToExe -replace "^", "http:" #Replace start of line with the correct Nvidia Link.
Remove-Variable -Name NVIDIADLPage, Webcontent, replace -Force #cleanup of the previous mess.
if ($NvidiaLinkToExe -match 'http:\/\/.*[0-9]{2}\/(([0-9]{3}\.[0-9]{2}).*\.exe)') {
$NVIDIAExeToDownload = [PSCustomObject]#{
Url = $Matches[0];
Name = $Matches[1];
Version = $Matches[2]
}
}
As you can see, I have to create three Invoke-WebRequest just to get one link.
And, I think I made no use of piping, because I was unable to make it work.
Thanks!
Your code seems better than mine. I had to call Invoke-WebRequest 3 times to get the link as well and once more to download it.
$destination = 'C:\Temp'
$downloadlist = 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0'
$pattern = "(?s)<tr>.+?href='//(?<URL>.+?)'>.+?<td.+?>(?<Version>.+?)</td.+?td.+?>(?<Date>.+?)</td.+</tr>"
$content = Invoke-WebRequest $downloadlist -UseBasicParsing
$download = if($content.RawContent -match $pattern){
[PSCustomObject]#{
URL = $Matches.URL
Version = $Matches.Version
Date = $Matches.Date
}
}
$pattern = '(?s)(?<Package>/content/driver[^"]+?{0}/.+?)(?=")' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest $download.url -UseBasicParsing
if($content.RawContent -match $pattern){
$pattern = '//(?<Package>.+?{0}.+exe)' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest -Uri "https://www.nvidia.com$($Matches.Package)" -UseBasicParsing
if($content.RawContent -match $pattern){
Invoke-WebRequest "https://$($Matches.Package)" -OutFile (Join-Path $destination $($Matches.package -replace '.+/'))
}
}
If you just want the link and other info you can drop the fourth.
$downloadlist = 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0'
$content = Invoke-WebRequest $downloadlist -UseBasicParsing
$download = if($content.RawContent -match "(?s)<tr>.+?href='//(?<URL>.+?)'>.+?<td.+?>(?<Version>.+?)</td.+?td.+?>(?<Date>.+?)</td.+</tr>"){
[PSCustomObject]#{
URL = $Matches.URL
Version = $Matches.Version
Date = $Matches.Date
}
}
$pattern = '(?s)(?<Package>/content/driver[^"]+?{0}/.+?)(?=")' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest $download.url -UseBasicParsing
if($content.RawContent -match $pattern){
$pattern = '//(?<Package>.+?{0}.+exe)' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest -Uri "https://www.nvidia.com$($Matches.Package)" -UseBasicParsing
$download.URL = "https://$($Matches.Package)"
$download
}

PowerShell & Power BI Rest API

Essentially what I'm after is the results of rest API Gateways - Get Datasource Users but retaining the ID (in this example $Line.id from my imported CSV file).
The end result should be a CSV with the following fields -
ID, emailAddress, datasourceAccessRight, displayName, identifier, principalType
I'm new to PowerShell and surprised I got this far but can't figure out this final bit.
Cheers
$webclient=New-Object System.Net.WebClient
$webclient.Proxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
$Dir = "C:\pbi_pro_user_logs\"
Login-PowerBI
$GateWayFile = Import-CSV -Path "C:\pbi_pro_user_logs\Gateway_Detail.csv"
$Output = #()
foreach ($Line in $GateWayFile){
$Item = $Line.id
$url = "https://api.powerbi.com/v1.0/myorg/gateways/HIDDEN/datasources/"+$Item+"/users"
$Output += (Invoke-PowerBIRestMethod -Url $url -Method Get | ConvertFrom-Json)
}
$Result = $Output.value
$Result | Export-Csv $Dir"GateWay_users.csv" -NoTypeInformation
Try this, using a calculated property from Select-Object:
$GateWayFile = Import-CSV -Path "C:\pbi_pro_user_logs\Gateway_Detail.csv"
$Output = Foreach ($Line in $GateWayFile){
$url = "https://api.powerbi.com/v1.0/myorg/gateways/HIDDEN/datasources/"+$Line.id+"/users"
$Item = (Invoke-PowerBIRestMethod -Url $url -Method Get | ConvertFrom-Json)
# output all properties of the item, plus the ID:
$ItemWithID = $Item | Select *,#{l='Id';e={$line.id}}
Write-Output $ItemWithID
}
# This depends on how you want your csv structured, but for example:
$Result = $Output | Select Id,Value
Or, if Value is a whole object that ID should be assigned inside of, then change the selection lines:
$ItemWithID = $Item.Value | Select *,#{l='Id';e={$line.id}}
$Result = $Output

how to export powershel result to csv

I want to export what I already filtered in ForEach-Object. The problem is that I can't export the filtered data.
I tried the following:
$getTapes.rows | Export-Csv C:\\123\\123456.txt but this has exported all the information without filter.
$getTapes = Invoke-RestMethod -Method GET -ContentType $content -Uri $Uri -Headers #{'Authorization' = $Authorization}
$today = Get-Date
$getTapes.rows | ForEach-Object {
$tape = $_;
if ( $tape.custom_fields.Ueberschreibschutz.value -ge $today ) {
Write-Host "Treffer ID=" $tape.asset_tag " Name=" $tape.name " SNR=" $tape.serial " Mediensatz=" $tape.custom_fields.Mediensatz.value
}
}
$getTapes.rows |export-Csv C:\\123\\123456.txt
I expect:
Treffer ID= 1 Name= 12 SNR= 12345 Mediensatz= M
Treffer ID= 2 Name= 32 SNR= 54321 Mediensatz= W
You should not use Write-Host to collect data. That's only to output pixels on the screen. Instead you should create a custom object you can use as you want later on ... like this:
$Result = $getTapes.rows | ForEach-Object {
if ( $_.custom_fields.Ueberschreibschutz.value -ge $today ) {
[PSCustomObject]#{
TrefferID = $_.asset_tag
Name = $_.name
SNR = $_.serial
Mediensatz = $_.custom_fields.Mediensatz.value
}
}
}
$Result | Export-Csv -Path C:\123\123456.csv -NoTypeInformation
Write-host do nothing except it shows you the result in the console, so it will not modify or delete the things you don't want in $getTapes.rows.
Instead you can define a variable $result and iterate over the $getTapes.rows using Foreach-Object, and add the result if it meets your if condition.
Try this:
$getTapes = Invoke-RestMethod -Method GET -ContentType $content -Uri $Uri -Headers #{'Authorization' = $Authorization}
$today = Get-Date
$getTapes.rows | ForEach-Object -begin {$result = "" } {
$tape = $_;
if ( $tape.custom_fields.Ueberschreibschutz.value -ge $today ) {
$result += "Treffer ID= $($tape.asset_tag) Name= $($tape.name) SNR= $($tape.serial) Mediensatz= $($tape.custom_fields.Mediensatz.value)`n"
}
} -end {$result | export-Csv C:\123\123456.txt}

Appending to a URL with powershell

function Get-Data(){
[PSObject[]]$pid = ''
$getUri1 = 'https://playbook2.com/data/project/folder/28220'
$projectIds = wget $getUri1 -UseDefaultCredentials |
ConvertFrom-JSON | Select data | select -Expand data | select id
Write-Host $projectIds
#obtain all the project ids
ForEach-Object{
[PSObject[]]$pid += $projectIds.id
}
Write-Host $pid
$uri3 = "https://playbook2.com/data/project/export/projects-tasks?projectIds[]="
$getIds = [PSObject[]]$pid -join "&projectIds[]="
$getUri2 = $uri3 + $getIds
$of = "\\ant\dept\DCGSI\Extracts\Time_Tracking_Tasks.xlsx"
Write-Host $getUri2
#retrieve excel files of tasks from each sub-folder
wget $getUri2 -outfile $of -UseDefaultCredentials
}
This code is an adaptation of some other code that I wrote. The 5 other scripts work fine. The main difference is that the other code has to loop through multiple folders and gets the project IDs under each folder, but this code only has to go through a single folder. Now in the other code the $uri3, $getIds code works fine and I get an export. The problem I am seeing in this code is that it isn't joining the URL the way I expect.
https://playbook2.com/data/project/export/projects-tasks?projectIds[]=######&projectIds[]=####### is the expected and previously seen output to get all the project data i need.
The problem with the above script is that it is giving https://playbook2.com/data/project/export/projects-tasks?projectIds[]=&projectIds[]=######&projectIds[]=####### which is invalid.
is there a way that I can tell it to do just $pid for the first item in the object and then -join the "&projectIds[]=" on the next n until the end of the list? I tried
[PSObject[]]$pid | select -Skip 1 -join "&projectIds[]="
and
[PSObject[]]$pid | Select-Object -Skip 1 -join "&projectIds[]="
but that results in nothing being appended.
I found a couple of "mistakes" in your script.
First is that you are using the variable $pid which is an system default variable. You can check the system global variables by typing
Get-Variable
Secondly $pid is defined with an empty string. The correct way to initialize a PSObject is with $myVar = New-Object PSObject. Replace [PSObject[]]$pid = '' with $myProjectIds = New-Object PSObject
For readability I took the liberty to rewrite your script.
function Get-Data(){
$GetProjectsUri = 'https://playbook2.com/data/project/folder/28220'
$ExportProjectsUri = 'https://playbook2.com/data/project/export/projects-tasks?'
$ExportFilePath = "\\ant\dept\DCGSI\Extracts\Time_Tracking_Tasks.xlsx"
$GetProjectsJson = Invoke-WebRequest -Uri $GetProjectsUri -UseDefaultCredentials
Write-Output $GetProjectsJson
$Projects = ConvertFrom-JSON -InputObject $GetProjectsJson
Write-Output $Projects
foreach ($Project in $Projects) {
$ProjectId = $Project.data.id
# Check if ProjectId exists
if ($ProjectId) {
$ExportProjectsUri = $ExportProjectsUri + 'projectIds[]=' + $ProjectId
}
}
Write-Output $ExportProjectsUri
Invoke-WebRequest Invoke-WebRequest -Uri $ExportProjectsUri -outfile $ExportFilePath -UseDefaultCredentials
}
Cheers
Glenn

set cache refresh schedule for report ssrs

I have s h*tload of reports on my reportserver. Most of them have a Cache Refresh Plan using a shared schedule. Is it programmatically possible to set a Cache Refresh Plan on a report?
Enabling caching, set expiration for a cache using a shared schedule, running snapshots according to a shared schedule all works runs fine using SetExecutionOptions-method and SetCacheOptions-method.
Setting a Cached Refreshplan for a report however does NOT run fine. Suggestions?
edit: I would like to do the same for all, datasets set them to refresh on a shared schedule.
Below is the code I am using (Powershell V3)
$reportServerURI = "http://localhost/Reportserver"
$ReportPathWildCard = "/SOME/FOLDER/ON/SERVER";
$NameSharedSchedule="NAMEOFSCHEDULE";
# init WS proxy
$reportServerURI2010 = "$reportServerURI/ReportService2010.asmx?WSDL"
$RS = New-WebServiceProxy -Uri $reportServerURI2010 -UseDefaultCredential
$proxyNamespace = $RS.GetType().Namespace
#Get Schedule Reference
$NeverExpireSchedule= $RS.ListSchedules([System.Management.Automation.Language.NullString]::Value) | where {$_.Name -eq "$NameSharedSchedule"}
$NeverExpireScheduleID = $NeverExpireSchedule.scheduleid;
$NeverExpireDescription = $NeverExpireSchedule.Description;
$NeverExpireDefinition = $NeverExpireSchedule.Definition;
Write-Host "Found Shared Schedule: '$NameSharedSchedule' with id $NeverExpireScheduleID and definition $NeverExpireDescription";
$NeverExpireScheduleRef =New-Object("$proxyNamespace.ScheduleReference");
$NeverExpireScheduleRef.ScheduleID=$NeverExpireScheduleID;
#get all needed items
$items = $RS.ListChildren($ReportPathWildCard, $true) | Where-Object {"Report" -contains $_.TypeName}
#process all items
foreach ($item in $items) {
$xpath = $item.path
$xtype = $item.TypeName
Write-Host "Processing $xtype $xpath"
##SET Refresh
$r= $RS.SetExecutionOptions( $xpath,"Snapshot",$NeverExpireDefinition)
}
Actually found it. Somehow me ànd several colleagues of mine overlooked it:
CreateCacheRefreshPlan Method is the solution...
It does not look nice (but hey, I am not a developer) and half of it are shameless rip-offs but it DOES do the trick.... :)
Thnx and kudo's to all the people who posted the tidbits I needed...
$reportServerURI = "http://localhost/Reportserver"
$ReportPathWildCard = "/";
$NameSharedSchedule="NAME OF SCHEDULE";
# init WS proxy
$reportServerURI2010 = "$reportServerURI/ReportService2010.asmx?WSDL"
$RS = New-WebServiceProxy -Uri $reportServerURI2010 -UseDefaultCredential
$proxyNamespace = $RS.GetType().Namespace
# Get Schedule Reference
$NeverExpireSchedule= $RS.ListSchedules([System.Management.Automation.Language.NullString]::Value) | where {$_.Name -eq "$NameSharedSchedule"}
$NeverExpireScheduleID = $NeverExpireSchedule.scheduleid;
$NeverExpireDescription = $NeverExpireSchedule.Description;
$NeverExpireDefinition = $NeverExpireSchedule.Definition;
#Write-Host "Found Shared Schedule: '$NameSharedSchedule' with id $NeverExpireScheduleID and definition $NeverExpireDescription";
$NeverExpireScheduleRef =New-Object("$proxyNamespace.ScheduleReference");
$NeverExpireScheduleRef.ScheduleID=$NeverExpireScheduleID;
# Wat dingen voorbereiden
#delivery Extension
#$setting = "Report Server Email"
$matchdata = $NeverExpireScheduleID
$description = "Automatisch ingesteld op " + $NameSharedSchedule
$eventtype = "RefreshCache"
$parameters
#get all needed items
$items = $RS.ListChildren($ReportPathWildCard, $true) | Where-Object {"Dataset" -contains $_.TypeName}
#process all items
foreach ($item in $items) {
$xpath = $item.path
$xtype = $item.TypeName
Write-Host "Processing $xtype $xpath"
$report = $xpath
##SET Cache
$r= $RS.SetCacheOptions( $xpath,[System.Management.Automation.Language.NullString]::Value, $o)
$r= $RS.CreateCacheRefreshPlan( $report,
$description,
$eventtype,
$matchdata,
$parameters
)
}