Appending to a URL with powershell - powershell

function Get-Data(){
[PSObject[]]$pid = ''
$getUri1 = 'https://playbook2.com/data/project/folder/28220'
$projectIds = wget $getUri1 -UseDefaultCredentials |
ConvertFrom-JSON | Select data | select -Expand data | select id
Write-Host $projectIds
#obtain all the project ids
ForEach-Object{
[PSObject[]]$pid += $projectIds.id
}
Write-Host $pid
$uri3 = "https://playbook2.com/data/project/export/projects-tasks?projectIds[]="
$getIds = [PSObject[]]$pid -join "&projectIds[]="
$getUri2 = $uri3 + $getIds
$of = "\\ant\dept\DCGSI\Extracts\Time_Tracking_Tasks.xlsx"
Write-Host $getUri2
#retrieve excel files of tasks from each sub-folder
wget $getUri2 -outfile $of -UseDefaultCredentials
}
This code is an adaptation of some other code that I wrote. The 5 other scripts work fine. The main difference is that the other code has to loop through multiple folders and gets the project IDs under each folder, but this code only has to go through a single folder. Now in the other code the $uri3, $getIds code works fine and I get an export. The problem I am seeing in this code is that it isn't joining the URL the way I expect.
https://playbook2.com/data/project/export/projects-tasks?projectIds[]=######&projectIds[]=####### is the expected and previously seen output to get all the project data i need.
The problem with the above script is that it is giving https://playbook2.com/data/project/export/projects-tasks?projectIds[]=&projectIds[]=######&projectIds[]=####### which is invalid.
is there a way that I can tell it to do just $pid for the first item in the object and then -join the "&projectIds[]=" on the next n until the end of the list? I tried
[PSObject[]]$pid | select -Skip 1 -join "&projectIds[]="
and
[PSObject[]]$pid | Select-Object -Skip 1 -join "&projectIds[]="
but that results in nothing being appended.

I found a couple of "mistakes" in your script.
First is that you are using the variable $pid which is an system default variable. You can check the system global variables by typing
Get-Variable
Secondly $pid is defined with an empty string. The correct way to initialize a PSObject is with $myVar = New-Object PSObject. Replace [PSObject[]]$pid = '' with $myProjectIds = New-Object PSObject
For readability I took the liberty to rewrite your script.
function Get-Data(){
$GetProjectsUri = 'https://playbook2.com/data/project/folder/28220'
$ExportProjectsUri = 'https://playbook2.com/data/project/export/projects-tasks?'
$ExportFilePath = "\\ant\dept\DCGSI\Extracts\Time_Tracking_Tasks.xlsx"
$GetProjectsJson = Invoke-WebRequest -Uri $GetProjectsUri -UseDefaultCredentials
Write-Output $GetProjectsJson
$Projects = ConvertFrom-JSON -InputObject $GetProjectsJson
Write-Output $Projects
foreach ($Project in $Projects) {
$ProjectId = $Project.data.id
# Check if ProjectId exists
if ($ProjectId) {
$ExportProjectsUri = $ExportProjectsUri + 'projectIds[]=' + $ProjectId
}
}
Write-Output $ExportProjectsUri
Invoke-WebRequest Invoke-WebRequest -Uri $ExportProjectsUri -outfile $ExportFilePath -UseDefaultCredentials
}
Cheers
Glenn

Related

Getting URL after InvokeWebRequest, when Status Code is 200 and store in CSV

I am attempting to pull the URL once an InvokeWebRequest has been called in my script. I wish for it do this when the HTTP Status Code is 200. So far, I have got it to tell the Status Code, and give the re-directed URL is 301/302 is the Status Code. Also an Error Message when the Status Code is 4xx//5xx.
I deal with a lot of URLs on clients' ecommerce sites, so need know what the final URL is after a end-user clicks on one of the URLs - essentially something like the query strings being dropped or something like that, on Status Code 200. As mentioned before, I already have 3xx/4xx/5xx reporting.
Attempted -
Reading the page and pulling the URL from the Header
Searching for the 'href' and pulling it
I wish for the CSV out to have the following output -
Expected CSV Output
Actual CSV Output (Row 3)
The "Landing Page" field is what I am trying to fill - the rest are working.
Script below -
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$urlInput = "C:\Users\User\Desktop\testurls.txt"
$csvOutput = "C:\Users\User\Desktop\testurl_result.csv"
#$totalUrls = Get-Content $urlInput | Measure-Object –Line
#Write-Host "Total URLs = $totalUrls"
$results = Test-Path "C:\Users\User\Desktop\testurl_result.csv"
If ($results -eq $false) {
Remove-Item "testurl_result.csv" -Force
}
Get-Content $urlInput | ForEach-Object { $uri = $_;
try
{
Start-Sleep 5
Invoke-WebRequest -Uri $uri -Method HEAD -MaximumRedirection 0 -ErrorAction SilentlyContinue
$link = Invoke-WebRequest -Uri $uri -Method HEAD -MaximumRedirection 0 -ErrorAction SilentlyContinue
}
catch
{
New-Object -TypeName psobject -Property #{Error = $_}
}
} | Select-Object #{Name='Click URL'; Expression={$uri}}, StatusCode, #{Name='Path'; Expression={([uri]$uri).AbsolutePath}}, #{Name='UTMs'; Expression={([uri]$uri).Query}}, #{Name='RedirectTo';Expression={$_.Headers["Location"]}}, #{Name='Landing Page';Expression={$link.Links | select -ExpandProperty href}}, Error | Export-Csv $csvOutput -NoTypeInformation
Thanks.

using PowerShell, Is there an easier or better optimized way to get the download link to the latest Nvidia Driver?

Hello all, I've been able to get this code to get the latest download link for Nvidia Driver for a Quadro P1000.
I was wondering, if there's a better way to do this.
Here's my code:
#Get Nvidia Drivers
#This next section is JUST TO get the LINK to the LATEST driver from Nvidia
#WebContent gets the links that are "href" from the nvidia JS request. No idea if there's an easier wey to get this.
$Webcontent = (Invoke-WebRequest 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0').links.href
#The following line uses Regex regular expressions to MATCH and RETRIEVE ONE single value from the list of values in the previous line.
$NVIDIALinkToLatestDriver = [regex]::Match($Webcontent, '//(www.nvidia.com/download/driverResults.aspx/[0-9]*/en-us)').Groups[1].Value
#Link after the previous crap
$NVIDIADLPage = New-Object -COM "HTMLFile" #Creates a COM Object for easier search of the link.
[string]$htmlBody = (Invoke-WebRequest -Uri $NVIDIALinkToLatestDriver -UseBasicParsing).Content #Parses the content of the landing page to then look by id
$NVIDIADLPage.write([ref]$htmlBody)
$replace = [regex]::Replace($NVIDIADLPage.getElementById('lnkDwnldBtn').href, 'about:', 'www.nvidia.com') #Replaces the "about" with "www.nvidia.com"
$Webcontent = (Invoke-WebRequest $replace) #Replace Webcontent with the latest link.
[String]$NvidiaLinkToExe = $Webcontent.links.href -match ".*.exe$" #On this link there's the exe file for Nvidia Drivers
$NvidiaLinkToExe = $NvidiaLinkToExe -replace "^", "http:" #Replace start of line with the correct Nvidia Link.
Remove-Variable -Name NVIDIADLPage, Webcontent, replace -Force #cleanup of the previous mess.
if ($NvidiaLinkToExe -match 'http:\/\/.*[0-9]{2}\/(([0-9]{3}\.[0-9]{2}).*\.exe)') {
$NVIDIAExeToDownload = [PSCustomObject]#{
Url = $Matches[0];
Name = $Matches[1];
Version = $Matches[2]
}
}
As you can see, I have to create three Invoke-WebRequest just to get one link.
And, I think I made no use of piping, because I was unable to make it work.
Thanks!
Your code seems better than mine. I had to call Invoke-WebRequest 3 times to get the link as well and once more to download it.
$destination = 'C:\Temp'
$downloadlist = 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0'
$pattern = "(?s)<tr>.+?href='//(?<URL>.+?)'>.+?<td.+?>(?<Version>.+?)</td.+?td.+?>(?<Date>.+?)</td.+</tr>"
$content = Invoke-WebRequest $downloadlist -UseBasicParsing
$download = if($content.RawContent -match $pattern){
[PSCustomObject]#{
URL = $Matches.URL
Version = $Matches.Version
Date = $Matches.Date
}
}
$pattern = '(?s)(?<Package>/content/driver[^"]+?{0}/.+?)(?=")' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest $download.url -UseBasicParsing
if($content.RawContent -match $pattern){
$pattern = '//(?<Package>.+?{0}.+exe)' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest -Uri "https://www.nvidia.com$($Matches.Package)" -UseBasicParsing
if($content.RawContent -match $pattern){
Invoke-WebRequest "https://$($Matches.Package)" -OutFile (Join-Path $destination $($Matches.package -replace '.+/'))
}
}
If you just want the link and other info you can drop the fourth.
$downloadlist = 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0'
$content = Invoke-WebRequest $downloadlist -UseBasicParsing
$download = if($content.RawContent -match "(?s)<tr>.+?href='//(?<URL>.+?)'>.+?<td.+?>(?<Version>.+?)</td.+?td.+?>(?<Date>.+?)</td.+</tr>"){
[PSCustomObject]#{
URL = $Matches.URL
Version = $Matches.Version
Date = $Matches.Date
}
}
$pattern = '(?s)(?<Package>/content/driver[^"]+?{0}/.+?)(?=")' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest $download.url -UseBasicParsing
if($content.RawContent -match $pattern){
$pattern = '//(?<Package>.+?{0}.+exe)' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest -Uri "https://www.nvidia.com$($Matches.Package)" -UseBasicParsing
$download.URL = "https://$($Matches.Package)"
$download
}

PowerShell & Power BI Rest API

Essentially what I'm after is the results of rest API Gateways - Get Datasource Users but retaining the ID (in this example $Line.id from my imported CSV file).
The end result should be a CSV with the following fields -
ID, emailAddress, datasourceAccessRight, displayName, identifier, principalType
I'm new to PowerShell and surprised I got this far but can't figure out this final bit.
Cheers
$webclient=New-Object System.Net.WebClient
$webclient.Proxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
$Dir = "C:\pbi_pro_user_logs\"
Login-PowerBI
$GateWayFile = Import-CSV -Path "C:\pbi_pro_user_logs\Gateway_Detail.csv"
$Output = #()
foreach ($Line in $GateWayFile){
$Item = $Line.id
$url = "https://api.powerbi.com/v1.0/myorg/gateways/HIDDEN/datasources/"+$Item+"/users"
$Output += (Invoke-PowerBIRestMethod -Url $url -Method Get | ConvertFrom-Json)
}
$Result = $Output.value
$Result | Export-Csv $Dir"GateWay_users.csv" -NoTypeInformation
Try this, using a calculated property from Select-Object:
$GateWayFile = Import-CSV -Path "C:\pbi_pro_user_logs\Gateway_Detail.csv"
$Output = Foreach ($Line in $GateWayFile){
$url = "https://api.powerbi.com/v1.0/myorg/gateways/HIDDEN/datasources/"+$Line.id+"/users"
$Item = (Invoke-PowerBIRestMethod -Url $url -Method Get | ConvertFrom-Json)
# output all properties of the item, plus the ID:
$ItemWithID = $Item | Select *,#{l='Id';e={$line.id}}
Write-Output $ItemWithID
}
# This depends on how you want your csv structured, but for example:
$Result = $Output | Select Id,Value
Or, if Value is a whole object that ID should be assigned inside of, then change the selection lines:
$ItemWithID = $Item.Value | Select *,#{l='Id';e={$line.id}}
$Result = $Output

How to get the workitems from AzureDevOps with RestApi in Powershell

Param(
[string]$collectionurl = "https://dev.azure.com",
[string]$project = "projectname",
[string]$token = "PAT"
)
# Base64-encodes the Personal Access Token (PAT) appropriately
$base64AuthInfo =
[Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}" -f
$token)))
$baseUrl =
"$collectionurl/$project/_apis/wit/reporting/workitemrevisions?
includeLatestOnly=true&api-version=5.0-preview.2"
$response = (Invoke-RestMethod -Uri $baseUrl -Method Get -
UseDefaultCredential -Headers #{Authorization=("Basic {0}" -f
$base64AuthInfo)}).values
$wits = $response | where({$_.fields.'System.WorkItemType' -eq
'Task'}) # Only retrieve Tasks
$witrevisions = #()
foreach($wit in $wits){
$customObject = new-object PSObject -property #{
"WitID" = $wit.fields.'System.Id'
"rev" = $wit.fields.'System.Rev'
"Title" = $wit.fields.'System.Title'
"AssignedTo" = $wit.fields.'System.AssignedTo'
"ChangedDate" = $wit.fields.'System.ChangedDate'
"ChangedBy" = $wit.fields.'System.ChangedBy'
"WorkItemType" = $wit.fields.'System.WorkItemType'
}
$witrevisions += $customObject
}
$witrevisions | Select-Object `
WitID,
rev,
Title,
AssignedTo,
ChangedDate,
ChangedBy,
WorkItemType #|export-csv -Path E:\ashwin\devdata.csv -
NoTypeInformation
Write-Output $witrevisions
I want to display the workitems in my project to be displayed using powershell with the following Rest Api using my PAT.
https://dev.azure.com/{organization}/{project}/_apis/wit/workitems/{id}?api-version=5.1
How to get the workitems from AzureDevOps with RestApi in Powershell
The result will display in the output, you will find like following:
If you do not find above output, make sure you have workitem with Task type, because you have set the condition 'System.WorkItemType' -eq 'Task' in the powershell scripts.
On the other hand, you could export the work item list to a *.csv file, this part of the code is commented in powershell:
WorkItemType #| export-csv -Path G:\temp\WIT.csv -NoTypeInformation
If you want to create a *.csv file, you need to remove the # in that line, it should be :
WorkItemType | export-csv -Path G:\temp\WIT.csv -NoTypeInformation
Now, we could get that file in our local folder:
Note: The path is G:\temp is a local path, you should use the private agent, if you are using the hosted agent, you should copy that file from the hosted agent, and publish it to pipeline artifact.
Hope this helps.

Using PowerShell & Invoke-Webrequest to get specific information

My code is below, it's pretty much complete, however a tiny part I am stuck on.
Basically what the code does is, based on a Song, the script goes out to a couple of websites on the internet to bring back the song label, composer & year that the song was released.
Only thing is, with the particular song Alicia Keys - Girl On Fire, if you manually go to the website http://staff.australian-charts.com/showitem.asp?interpret=Alicia+Keys+feat%2E+Nicki+Minaj&titel=Girl+On+Fire&cat=s you will notice that there is more than one Composer listed under the Music/Lyrics section.
In the event the song has more than one composer, my script below in it's current state will grab the first composer listed only.
What I need is for the script to grab all the composers. If there's one composer, or if there are multiple composers, I need them captured in the format of "Composer1, Composer2, Composer3, Composer4" etc (Commas included)
I am thinking of changing the Invoke-Webrequest to get TABLES back and particular tables with rows etc, not sure....
$song = "Alicia Keys - Girl On Fire"
Write-Host $song
$SearchSong = $song -replace '\(' -replace '\)' -replace '&' -replace ' - ', ' ' -replace '\s','+'
$MatchSong = $song -replace ' - ', '&titel=' -replace '\s','\+'
#Check iTunes for music Label information
$uri = "https://itunes.apple.com/search?term=$SearchSong&country=au&entity=song"
$x = Invoke-WebRequest -Uri $uri
$iTunesResults = ($x.Content | ConvertFrom-Json).results
$y = Invoke-WebRequest -Uri $iTunesResults[0].trackViewUrl
$iTunesSongCopyright = ($y.ParsedHtml.getElementsByTagName('li') | ? {$_.ClassName -eq 'copyright'}).innerText -replace '℗ '
$iTunesSongLabel = $iTunesSongCopyright -replace '.*\d\s'
#The check australian-charts for Composer & Year infomation
$domain = 'http://staff.australian-charts.com/'
$uri = $domain + "search.asp?search=$SearchSong&cat=s"
$x = Invoke-WebRequest -Uri $uri
$x.AllElements[462].outerHTML -match 'A.href="(.*)"';$resultURL = $domain + $Matches[1]
$resultURL = $resultURL -replace("&","&") -replace('"','"')
$y = Invoke-WebRequest -Uri $resultURL
$Element = ($y.AllElements | ? {$_.tagName -eq 'HTML'})
if($Element.innerText -match 'Music\/Lyrics:(.*)')
{
$Element.innerText -match 'Music\/Lyrics:(.*)'
$Composer = $Matches[1]
Write-Host $Composer
} else {
$Composer = $null
}
if($Element.innerText -match 'Year:(.*)')
{
$Element.innerText -match 'Year:(.*)'
$Year = $Matches[1]
Write-Host $Year
} else {
$Year = $null
}
Write-Host $iTunesSongLabel
you can use this for getting composer list :
if($Element.innerText -match 'Music\/Lyrics:(.*)')
{
$startpos = $Element.innertext.IndexOf("Lyrics:") + 7
$endpos = $Element.innertext.IndexOf("Producer:") -1
$composer=$Element.innertext.substring($startpos,($endpos - $startpos))
#even the below line will give the same result as the above line if uncommented
#$composer = $Element.innertext[$startpos..$endpos] -join ""
Write-Host $Composer
}