cutting a portion of url in powershell - powershell

I have a few URLs which would need to cut and separate the first part of the each URL, i.e example1.com, example2.com, example3.com from each line and store in a variable
Contents in url.csv
https://example1.com/v1/test/f3de-a8c6-464f-8166-9fd4
https://example2.com/v1/test/14nf-d7jc-54lf-fd90-fds8
https://example3.com/v1/test/bd38-17gd-2h65-0j3b-4jf6
Script:
$oldurl = Import-CSV "url.csv"
$newurl = $oldurl.list -replace "https://"
This would replace https://, however the rest of each cannot be hard coded as those values can change.
What could be change code change required to cut anything from and after /v1/ along with https://?


$list = #(
"https://example1.com/v1/test/f3de-a8c6-464f-8166-9fd4",
"https://example2.com/v1/test/14nf-d7jc-54lf-fd90-fds8",
"https://example3.com/v1/test/bd38-17gd-2h65-0j3b-4jf6"
)
$result = $list | %{
$uri = [System.Uri] $_
$uri.Authority
}
$result
See System.Uri properties to potentially assemble the information you need in your result list.


This will cut off anything after "/v1/" and it self. Is that what you want?
$string = "https://example1.com/v1/test/f3de-a8c6-464f-8166-9fd4"
$string = $string -replace "https://"
$pos = $string.IndexOf("/v1/")
$result = $string.Substring(0, $pos)
$result
Output: example1.com

Related

Powershell Most efficient way to combine customobject

I'm parsing a webpage, but having difficulty combining into one variable.
I'm looking for the most efficient way to do so as well.
This is code I have so far. Any help is appreciated.
$WebResponse = Invoke-WebRequest -Uri "https://finviz.com/news.ashx"
$title = ($WebResponse.AllElements | Where {$_.class -match 'nn-tab-link'}).innertext
$time = ($WebResponse.AllElements | Where {$_.class -match 'nn-date'}).innertext
$link = ($WebResponse.AllElements | Where {$_.class -match 'nn-tab-link'}).href
$r = {
[PSCustomObject]#{
{Time(E/T)} = $time
Headline = $title
Link = $link
}
}
$R
Thanks

Use an index-based loop (which assumes that all three arrays have corresponding elements and the same element count):
$objects = foreach ($i in 0..($title.Count-1)) {
[pscustomobject] #{
'Time(E/T)' = $time[$i]
Headline = $title[$i]
Link = $link[$i]
}
}
Note how property name Time(E/T) is enclosed in '...' - a verbatim string - rather than in {...} - a script block; the latter only works accidentally, because script blocks stringify[1] to their verbatim content (without the { and }).
[1] When using the [pscustomobject] #{ ... } syntactic sugar, the keys of the hashtable (#{ ... }) are implicitly stringified, given that objects' property names are invariably strings.

For-each loop unable to process all strings

I am trying to genrate URLs using a bunch of strings and an anchor URL
strings in s.txt are
123
234
345
anchor URL is https://testurl.com/prod/hostdetails.php?qs=
The code I am using
$ur = gc C:\temp\s.txt
foreach($u in $ur) {
$test = invoke-webrequest -uri "https://testurl.com/prod/hostdetails.php?qs=$u" -UseDefaultCredentials
}
$test
but it returns data only for
https://testurl.com/prod/hostdetails.php?qs=345

Its because you reassign $test each time within your loop. I suggest this solution:
$test = 'https://testurl.com/prod/hostdetails.php?{0}' -f ((C:\temp\s.txt) -join '&')
You edited your question. This is probably what you are looking for. Please notice the += operator to concat the results:
$result = #()
Get-Content C:\temp\s.txt | ForEach-Object {
$result += invoke-webrequest -uri "https://testurl.com/prod/hostdetails.php?qs=$($_)"
}

Grab parameters from url and drop them to powershell variables

i have a powershell listener running on my windows-box. Code from Powershell-Gallery: https://www.powershellgallery.com/packages/HttpListener/1.0.2
The "Client" calls the listener with the following example url:
With Powershell i do:
invoke-webrequest -Uri "http://127.0.0.1:8888/Test?id='1234'&param2='##$$'&param3='This is a test!'"
I have no idea, how to drop the parameters from the url to variables with the same name in powershell. I need to bring the parameters to powershellvariables to simply echo them. this is my last missing part. The parameters are separated with & and parameternames are case-sensitive.
To be more detailed, the id from the url should be in a powershell-variable named $id with the value 1234. The Variables can contain spaces, special characters, numbers, alphas. They are case sensitive. The parametervalue could be "My Name is "Anna"! My Pets Name is 'Bello'. " with all the "dirty" Characters like '%$"!{[().
Can someone point me to the right way how to get this solved?

In a valid url, the $ characters should have been escaped to %24
The other 'dirty' character % in Url escaped form is %25
This means the example url is invalid and should be:
$url = "http://127.0.0.1:8888/Test?id='1234'&param2='##%24%24'&param3='This is a test!'"
Then the following does work
$url = "http://127.0.0.1:8888/Test?id='1234'&param2='##%24%24'&param3='This is a test!'"
if ($url -is [uri]) {
$url = $url.ToString()
}
# test if the url has a query string
if ($url.IndexOf('?') -ge 0) {
# get the part of the url after the question mark to get the query string
$query = ($url -split '\?')[1]
# or use: $query = $url.Substring($url.IndexOf('?') + 1)
# remove possible fragment part of the query string
$query = $query.Split('#')[0]
# detect variable names and their values in the query string
foreach ($q in ($query -split '&')) {
$kv = $($q + '=') -split '='
$varName = [uri]::UnescapeDataString($kv[0]).Trim()
$varValue = [uri]::UnescapeDataString($kv[1])
New-Variable -Name $varname -Value $varValue -Force
}
}
else {
Write-Warning "No query string found as part of the given URL"
}
Prove it by writing the newly created variables to the console
Write-Host "`$id = $id"
Write-Host "`$param2 = $param2"
Write-Host "`$param3 = $param3"
which in this example would print
$id = '1234'
$param2 = '##$$'
$param3 = 'This is a test!'
However, I personally would not like to create variables like this, because of the risk of overwriting already existing ones.
I think it would be better to store them in a hash like this:
# detect variable names and their values in the query string
# and store them in a Hashtable
$queryHash = #{}
foreach ($q in ($query -split '&')) {
$kv = $($q + '=') -split '='
$name = [uri]::UnescapeDataString($kv[0]).Trim()
$queryHash[$name] = [uri]::UnescapeDataString($kv[1])
}
$queryHash
which outputs
Name Value
---- -----
id '1234'
param2 '##$$'
param3 'This is a test!'

PowerShell split input and replace/combine?

I want to be able to automatically generate an output if I run a PowerShell script that splits the input by a period "." and adds "DC=" in each item/object that has been split.
$DomainFQDN = "prod.mydomain.com" # This varies depending on the input. It could be "prod.boston.us.mydomain.com" as the input.
$DistinguishedName = $DomainFQDN -split "\."
...
...
...I just don't know how to proceed
How do I get an output of "DC=prod,DC=mydomain,DC=com" for prod.mydomain.com as the input or DC=prod,DC=boston,DC=us,DC=mydomain,DC=com for prod.boston.us.mydomain.com?

Well, you can use foreach construct with $DistinguishedName and use -join like this (if you want to output directly the joined string):
$AddDC = foreach ($e in $DistinguishedName) { "DC=$e" }
Write-Host $($AddDC -join ",")
-join works like -split, you just specify the character that you need to join by.
Other way to do it is to store $AddDC
$AddDC = foreach ($e in $DistinguishedName) { "DC=$e" }
$new_string = $AddDC -join ","
Write-Host $new_string
You can consult this page for more info.

If I got it right, this is what needed:
$fqdn='prod.boston.us.mydomain.com'
$dn="DC=$($fqdn.replace('.',',DC='))"
$dn

$DomainFQDN = "prod.mydomain.com"
$DomainFQDN = $DomainFQDN.Split(".")
For ($i = 0; $i -lt $DomainFQDN.Count; $i++) {
$DomainFQDN[$i] = "DC=" + $DomainFQDN[$i]
}
$DomainFQDN = $DomainFQDN -join ","
Write-Host $DomainFQDN
Output:
DC=prod,DC=mydomain,DC=com
I'm not sure why I couldn't get the -Split "." operator to work. It should function the same as .Split(".") but for some reason it gives a different result. But anyway this should work for you.
I feel like I should mention that a real FQDN would not be DC= on every line. It would look more like:
DC=Com,DC=MyDomain,OU=Prod

I usually do a single replace operation in an expandable to convert from FQDN to distinguished name of the domain root:
$DistinguishedName = "DC=$($DomainFQDN.TrimEnd('.') -replace '\.',',DC=')"
The TrimEnd('.') call strips any dot from rooted FQDNs, and the replace operation replaces each remaining dot with ,DC=

How to properly string replace in Powershell without appending the replaced variable to a newline?

I'm pretty new to powershell/programming so bear with me. I have this bug that appends the new renamed path to a new-line without the rest of path.
The console output:
/content/pizza/en/ingredients/
helloworld/menu-eng.html
What I want:
/content/pizza/en/ingredients/helloworld/menu-eng.html
What the code below is supposed to do is rename a bunch paths. Right now testName is hard-coded but after I get this to work properly it will be dynamic.
My code:
$testName = "helloworld"
$text = (Get-Content W:\test\Rename\rename.csv) | Out-String
$listOfUri = Import-Csv W:\test\Rename\rename.csv
foreach ($element in $listOfUri) {
if ($element -match "menu-eng.html") {
$elementString = $element.'ColumnTitle' | Out-String
$elementString = $elementString.Replace('menu-eng.html', '')
$varPath1 = $elementString
$elementString = $elementString.Insert('', 'http://www.pizza.com')
$elementName = ([System.Uri]$elementString).Segments[-1]
$elementString = $elementString.Replace($elementName, '')
$elementString = $elementString.Replace('http://www.pizza.com', '')
$varPath2 = $elementString.Insert($elementString.Length, $testName + '/')
$text = $text.Replace($varPath1.Trim(), $varPath2)
}
}
$text

Assuming your .csv file looks like this:
ColumnTitle,Idk
/content/pizza/en/ingredients/SPAM/menu-eng.html,Stuff
Then:
$testName = 'helloworld'
foreach ($row in Import-CSV d:\rename.csv) {
$bit = $row.'ColumnTitle'.Split('/')[-2]
$row.'ColumnTitle'.replace($bit, $testName)
}
I have no real idea what all the rest of your code is for, particularly my earlier comment, your line:
$text = (Get-Content W:\test\Rename\rename.csv) | Out-String
is making $text into an /array/ of all the lines in the file, including the headers. You can still use .Replace() on it in PowerShell, but it's going to do the replace on every line. I can't quite see how that gives you the output you get, but it will give you multiple lines for every line in the input file.