I'm looking to strip out the domain in this scenario using PowerShell. What is the most effective method to get example.com out of the following variable?
$URL = "http://www.example.com/folder/"
(some sort of regex command here to convert/strip $URL into $DOMAIN using PowerShell)
$DOMAIN = "example.com" #<-- taken from $URL
I've searched and I've found results for finding the IP address from a domain but I need to establish what the domain is first using regex (or another method).
Try the URI class:
PS> [System.Uri]"http://www.example.com/folder/"
AbsolutePath : /folder/
AbsoluteUri : http://www.example.com/folder/
LocalPath : /folder/
Authority : www.example.com
HostNameType : Dns
IsDefaultPort : True
IsFile : False
IsLoopback : False
PathAndQuery : /folder/
Segments : {/, folder/}
IsUnc : False
Host : www.example.com
Port : 80
Query :
Fragment :
Scheme : http
OriginalString : http://www.example.com/folder/
DnsSafeHost : www.example.com
IsAbsoluteUri : True
UserEscaped : False
UserInfo :
And remove the www prefix:
PS> ([System.Uri]"http://www.example.com/folder/").Host -replace '^www\.'
example.com
Like this:
PS C:\ps> [uri]$URL = "http://www.example.com/folder/"
PS C:\ps> $domain = $url.Authority -replace '^www\.'
PS C:\ps> $domain
example.com
For properly calculating the sub-domain, the trick is you need to know the second to last period. Then you take a substring of that second to last period (or none if it is only one .) to the final position by subtracting the location of second period (or 0) from total length of domain. This will return the proper domain-only and will work regardless of how many sub-domains are nested under the TLD:
$domain.substring((($domain.substring(0,$domain.lastindexof("."))).lastindexof(".")+1),$domain.length-(($domain.substring(0,$domain.lastindexof("."))).lastindexof(".")+1))
Also note that System URI itself is valid 99% of the time but I'm parsing my IIS logs and finding that with VERY long (often invalid/malicious requests) URIs it does not properly parse and fails those.
I have this in function form as:
Function Get-DomainFromURL {
<#
.SYNOPSIS
Takes string URL and returns domain only
.DESCRIPTION
Takes string URL and returns domain only
.PARAMETER URL
URL to parse for domain
.NOTES
Author: Dane Kantner 9/16/2016
#>
[CmdletBinding()]
param(
[Alias("URI")][parameter(Mandatory=$True,ValueFromPipeline=$True)][string] $URL
)
try { $URL=([System.URI]$URL).host }
catch { write-error "Error parsing URL"}
return $URL.substring((($URL.substring(0,$URL.lastindexof("."))).lastindexof(".")+1),$URL.length-(($URL.substring(0,$URL.lastindexof("."))).lastindexof(".")+1))
}
Related
I have an URL like so:
https://www.example.com/oauth/connect/token
I want to get only https://www.example.com and I have tried a bunch of ways to get this but all the ways I have tried require me to hardcode multiple /s.
Example:
$Url.Split('/')[0] + "//" + $url.split('/')[2]
Is there a way to do this without using the harcoding?
You may use [System.Uri], to 'cast' the string into an System.Uri class.
$url = [System.Uri]"https://www.example.com/oauth/connect/token"
You can then use the host and scheme to get the part of the URL you want.
$result = $url.Scheme, $url.Host -join "://"
You could also remove the AbsolutePath from the entire URL.
$url = [System.Uri]"https://www.example.com/oauth/connect/token"
$result = $url.AbsoluteUri.Replace($url.AbsolutePath, "")
This is the complete list of attributes that the System.Uri instance will have:
AbsolutePath : /oauth/connect/token
AbsoluteUri : https://www.example.com/oauth/connect/token
LocalPath : /oauth/connect/token
Authority : www.example.com
HostNameType : Dns
IsDefaultPort : True
IsFile : False
IsLoopback : False
PathAndQuery : /oauth/connect/token
Segments : {/, oauth/, connect/, token}
IsUnc : False
Host : www.example.com
Port : 443
Query :
Fragment :
Scheme : https
OriginalString : https://www.example.com/oauth/connect/token
DnsSafeHost : www.example.com
IdnHost : www.example.com
IsAbsoluteUri : True
UserEscaped : False
UserInfo :
I am having some trouble converting a Powershell script from PS5 to PS7.
My goal is to get information on a remote server cert.
This IP can be anything with a cert:
$url = "https://192.168.1.1"
[Net.ServicePointManager]::ServerCertificateValidationCallback = { $true }
$req = [Net.HttpWebRequest]::Create($url)
$req.GetResponse()
[datetime]$expiration =
[System.DateTime]::Parse($req.ServicePoint.Certificate.GetExpirationDateString())
Write-Output $expiration
When run in Powershell 5, this script completes successfully with the output below:
IsMutuallyAuthenticated : False
Cookies : {}
Headers : {Accept-Ranges, Content-Length, Content-Type, Date...}
SupportsHeaders : True
ContentLength : 98540
ContentEncoding :
ContentType : text/html
CharacterSet : ISO-8859-1
Server : Microsoft-IIS/10.0
...
Date : 7/7/2020 12:00:00 AM
...
This script errors on $req.GetResponse when run in Powershell 7.
MethodInvocationException: Exception calling "GetResponse" with "0" argument(s): "The SSL connection could not be established, see inner exception. There is no Runspace available to run scripts in this thread. You can provide one in the DefaultRunspace property of the System.Management.Automation.Runspaces.Runspace type. The script block you attempted to invoke was: $true "
Any ideas?
I want to do this things :
select a session
make a dump of this session (the issue is here)
Plus, I want to do that without modification of Fiddler Rules. I have done this with a modification of Fiddler Rules but the program will be used on several machines and it can be difficult to change Fiddler Rules in every machines.
I don't know if it is possible.
The code to do that with the modification of the fiddler Rules is :
PowerShell :
$filePath = "...\nameFile.txt" # file which contain the names of fiddler ZIP files
$file = Get-Content $filePath # content of nameFile
foreach ($line in $file) {
start $line # open the file
Write-Host "File : $line open"
Start-Sleep -s 1
}
& "...\Fiddler\ExecAction.exe" "failuresselection" # select all failures and make another file (see Fiddler Rules)
Fiddler Rules :
static function OnExecAction(sParams: String[]): Boolean {
[...]
// Select all failures and put them in a new ZIP file
case "failuresselection":
var path = "...\\Newlogs";
UI.actSelectSessionsWithResponseCode(449);
if (UI.GetFirstSelectedSession() != null){
UI.actSaveSessionsToZip(path+"\\Logs" + 449 + ".saz");
}
// Confirmation
FiddlerObject.StatusText = "Dumped all failures sessions to " + path;
UI.actExit();
return true;
[...]
}
I try this but it doesn't worked :
& "...\Fiddler\ExecAction.exe" "FiddlerApplication.UI.actSelectAll();"
It's to select all line but I think that ExecAction.exe replace QuickExec.
In summary, I am searching a way to do the same thing without modification of Fiddler Rules.
I want to make a backup script for ZyWall USG 20.
I used this Powershell code to get a list of the content in the directory:
[System.Net.FtpWebRequest]$ftp = [System.Net.WebRequest]::Create("ftp://*server*/*path*")
$ftp.Credentials = New-Object System.Net.NetworkCredential("*user*","*password*")
$ftp.Method = [System.Net.WebRequestMethods+FTP]::ListDirectory
$response = $ftp.getresponse()
echo $response
I expected to get a list of all the files..instead I get this:
PS Z:\> ...\list-files-test.ps1
ContentLength : -1
Headers : {}
SupportsHeaders : True
ResponseUri : ftp://*server*/*path*/
StatusCode : OpeningData
StatusDescription : 150 Opening BINARY mode data connection for file list
LastModified : 01.01.0001 00:00:00
BannerMessage : 220 FTP Server (ZyWALL USG 20) [::ffff:*server*]
WelcomeMessage : 230 User *user* logged in
ExitMessage :
IsFromCache : False
IsMutuallyAuthenticated : False
ContentType :
PS Z:\>
Why don' I get a list of files?
I checked it with filezilla - the folder is not empty.
And I can download specific files using [System.Net.WebRequestMethods+Ftp]::DownloadFile.
WebRequest.GetResponse() makes the remote call and gets the response details, but there's no content within the response itself.
Instead, you need to continue on and read the response stream out of the response with WebResponse.GetResponseStream().
The code:
New-Object System.Net.WebProxy($Env:http_proxy, $true, #('localhost', '*.domain.com')
fails with the error:
New-Object : Exception calling ".ctor" with "3" argument(s): "parsing "*.domain.com" - Quantifier {x,y} following nothing."
At line:1 char:6
+ $p = New-Object System.Net.WebProxy($Env:http_proxy, $true, #('*.domain.com', 'l ...
The Quantifier {x,y} following nothing is regex error which is strange. I tried to use regex escape chars but nothing.
Any solutions ?
I messed that up at least twice - the following however seems to add it correctly though one at a time:
$wp = New-Object System.Net.WebProxy($Env:http_proxy, $true)
$wp.BypassArrayList.Add('localhost')
$wp.BypassArrayList.Add('*.domain.com')
Output
Address :
BypassProxyOnLocal : True
BypassList : {localhost, *.domain.com}
Credentials :
UseDefaultCredentials : False
BypassArrayList : {localhost, *.domain.com}
Looking at this it's stated the third parameter is an array of regex strings - *.domain.com isn't a valid regex as a character class must precede the *.
It works if you change it to .*.domain.com however:
[PS] > New-Object System.Net.WebProxy($Env:http_proxy, $true, #("localhost.domain.com",".*.domain.com"))
Address :
BypassProxyOnLocal : True
BypassList : {localhost.domain.com, .*.domain.com}
Credentials :
UseDefaultCredentials : False
BypassArrayList : {localhost.domain.com, .*.domain.com}