I googled but did not found what I'm looking for.
I have a big file containing list of countries and people. I am aware how to do multiple piping in Linux, but the same way did not work for me in Powershell.
This is what I looked for and got nothing:
Select-String .\file -pattern 'country:[ ]{8}IR' -context 5 | Select-String -pattern 'names'
But if I separate this command into to, like below, works (in which I want to avoid creating a file to search):
Select-String .\file -pattern 'country:[ ]{8}IR' -context 5 > country
Select-String .\file -patern 'names'
*Update 1
Sample data after first grep is:
file:1407215:names: Hadi
file:1407216:company: sample
file:1407217:city: Tehran
file:1407218:district: 8
file:1407219:country: IR
file:1407220:admin: Mahmoud
file:1407221:tech: Hamed
file:1407222:seller: sell#company
file:1407223:status: Active
file:1407224:id: 12456
Select-String doesn't return a [string] (or array of strings) but an object of type [MatchInfo]. The output of a MatchInfo may look like a multi line text but is split in the properties .Context.PreContext, .Line and .Context.PostContext. So you can't use this object directly to pipe it into Select-String again.
However you can cast the output to [String], -split it at the new lines and use Select-String over this array:
$MatchInfo = Select-String $file -pattern 'country:[ ]{8}IR' -context 5
[string]$MatchInfo -split [environment]::NewLine | Select-String -pattern 'names'
From a PowerShell perspective you will be dealing with objects most of the time, it might be a good idea to get the hang of dealing with them, hence this answer can show you an alternative to parsing your file into an array of objects which can be easily manipulated, filtered, sorted and exported into structured data (such as Csv).
Supposing the test.txt looks similar to this:
names: Hadi
company: sample
city: Tehran
district: 8
country: IR
admin: Mahmoud
tech: Hamed
seller: sell#company
status: Active
id: 12456
names: John
company: sample
city: Tehran
district: 8
country: IR
admin: Doe
tech: Hamed
seller: sell#company
status: Disabled
id: 12456
For this particular case we can use a switch with the -File parameter to read the file and the -Regex switch for the conditional clauses to start capturing and outputting the capture data as objects:
$parsed = switch -Regex -File .\test.txt {
# line starts with "names", signal to start capturing
'^names' {
$startcapture = $true
$out = [ordered]#{}
}
# boolean is set to `$true`, capture this line and add it to the ordered dictionary
{ $startcapture } {
$key, $value = $_.Split(':').Trim()
$out[$key] = $value
}
# line starts with "id", signal to output the object, and restart the capture boolean
'^id' {
$startcapture = $false
[pscustomobject] $out
}
}
After parsing the test.txt file with above switch, $parsed would look like this:
names company city district country admin tech seller status id
----- ------- ---- -------- ------- ----- ---- ------ ------ --
Hadi sample Tehran 8 IR Mahmoud Hamed sell#company Active 12456
John sample Tehran 8 IR Doe Hamed sell#company Disabled 12456
Now $parsed can be exported to structured data at ease with Export-Csv and imported back as objects with Import-Csv:
$parsed | Export-Csv parseddata.csv -NoTypeInformation
$csv = Import-Csv parseddata.csv
It can also be filtered very easily a filtering cmdlet such as Where-Object:
# this array of objects where the `country` property value is equal to "IR"
# AND this array of objects where the `names` property value is equal to "Hadi"
$parsed | Where-Object { $_.country -eq 'IR' -and $_.names -eq 'Hadi' }
Which results in:
names : Hadi
company : sample
city : Tehran
district : 8
country : IR
admin : Mahmoud
tech : Hamed
seller : sell#company
status : Active
id : 12456
Related
Currently I want to get a list of active user on Windows 10 using Powershell.
After some searching I found this helps:
Get-LocalUser | Where-Object -Property Enabled -eq True
And here is the output:
Name Enabled Description
---- ------- -----------
User_1 True
User_2 True
I just want to concatenate strings to a list of values of Name column from above, which will be like below:
Active user(s): User_1, User_2
Do you have any idea how I can do that? I'm just a non-tech guy trying to learn some useful command so forgive me if this is a basic to you.
You can do the following:
# Retrieve the name values
$Users = Get-LocalUser |
Where-Object Enabled -eq True | Select-Object -Expand Name
# Create the output string
"Active user(s): {0}" -f ($Users -join ', ')
Using -Expand (or -ExpandProperty) from Select-Object, the target property's value is returned rather than the object that contains the property.
-f is the string format operator. It uses substitution for the {position} syntax. The -join operator creates a string from a list with , as the delimiter.
Say [hypothetically], I have two .CSVs I'm comparing to try and see which of my current members are original members... I wrote a nested ForEach-Object comparing every $name and $memberNumber from each object against every other object. It works fine, but is taking way to long, especially since each CSV has 10s of thousands of objects. Is there another way I should approach this?
Original_Members.csv
Name, Member_Number
Alice, 1234
Jim , 4567
Current_Members.csv
Alice, 4599
Jim, 4567
$currentMembers = import-csv $home\Desktop\current_members.csv |
ForEach-Object {
$name = $_.Name
$memNum = $_."Member Number"
$ogMembers = import-csv $home\Desktop\original_members.csv" |
ForEach-Object {
If ($ogMembers.Name -eq $name -and $ogMembers."Member Number" -eq $memNum) {
$ogMember = "Yes"
}
Else {
$ogMember = "No"
}
}
[pscustomobject]#{
"Name"=$name
"Member Number"=$memNum
"Original Member?"=$ogMember
}
} |
select "Name","Member Number","Original Member?" |
Export-CSV "$home\Desktop\OG_Compare_$(get-date -uformat "%d%b%Y").csv" -Append -NoTypeInformation
Assuming both of your files are like the below:
Original_Members.csv
Name, Member_Number
Alice, 1234
Jim, 4567
Current_Members.csv
Name, Member_Number
Alice, 4599
Jim, 4567
You could store the original member names in a System.Collections.Generic.HashSet<T> for constant time lookups, instead of doing a linear search for each name. We can use System.Linq.Enumerable.ToHashSet to create a hashset of string[] names.
We can then use Where-Object to filter current names by checking if the hashset contains the original name with System.Collections.Generic.HashSet<T>.Contains(T), which is an O(1) method.
$originalMembers = Import-Csv -Path .\Original_Members.csv
$currentMembers = Import-Csv -Path .\Current_Members.csv
$originalMembersLookup = [Linq.Enumerable]::ToHashSet(
[string[]]$originalMembers.Name,
[StringComparer]::CurrentCultureIgnoreCase
)
$currentMembers |
Where-Object {$originalMembersLookup.Contains($_.Name)}
Which will output the current members that were original members:
Name Member_Number
---- -------------
Alice 4599
Jim 4567
Update
As requested in the comments, If we want to check both Name and Member_Number, we can concatenate both strings to use for lookups:
$originalMembers = Import-Csv -Path .\Original_Members.csv
$currentMembers = Import-Csv -Path .\Current_Members.csv
$originalMembersLookup = [Linq.Enumerable]::ToHashSet(
[string[]]($originalMembers |
ForEach-Object {
$_.Name + $_.Member_Number
}),
[StringComparer]::CurrentCultureIgnoreCase
)
$currentMembers |
Where-Object {$originalMembersLookup.Contains($_.Name + $_.Member_Number)}
Which will now only return:
Name Member_Number
---- -------------
Jim 4567
I need to check if the column value from the first csv file, exists in any of the three other csv files and return a value into a new csv file, in order of precedence.So if the username field from allStaff.csv exists in the list of usernames in the sessionVPNct.csv file, put the static text into the final csv file as 'VPN'. If it does not exist, check the next csv file: sessionCRXct.csv then put the static text 'CRX', if not check the last csv file: sessionTMSct.csv then put the static text: TM if not the put the static text 'none' into the final csv file.
I have four csv files as below:
1. allStaff.csv
2. VPN.csv
3. CRX.csv
4. TMS.csv
I have imported the csv files into variables as below:
$allUsers = Import-Csv -Path "C:\allStaff.csv"
$vpn = Import-Csv -Path "C:\VPN.csv" | Select-Object -ExpandProperty UserName
$crx = Import-Csv -Path "C:\CRX.csv" | Select-Object -ExpandProperty UserName
$tms = Import-Csv -Path "C:\TMS.csv" | Select-Object -ExpandProperty UserName
The $allUsers variable displays the following:
Firstname LastName Username Position Area
--------- -------- -------- -------- ----
Joe Bloggs jbloggs Gardener Maintenance
Jim Smith jsmith Accountant Finance
Bob Seger bseger HR Advisor Human Resources
Adam Boson aboson Customer Support IT
Adele bree abree Payroll Finance
The $vpn variable displays the following:
Username
--------
jbloggs
jsmith
The $crx variable displays the following:
Username
--------
jbloggs
jsmith
bseger
The $tms variable displays the following:
Username
--------
jbloggs
jsmith
bseger
aboson
Then I have the following line to start the result csv file
$result = $allUsers | Select-Object *,ConnectionMethod
Not quite sure how to do the final query, which I believe should be an if else loop to go through all rows in the $result variable, and check the other csv if the username field exists, then return the static text.
$result | Export-Csv -Path "C:\allStaffConnections.csv"
This is how I need the final allStaffConnections.csv file to be displayed.
Firstname LastName Username Position Area ConnectionMethod
--------- -------- -------- -------- ---- --------------
Joe Bloggs jbloggs Gardener Maintenance VPN
Jim Smith jsmith Accountant Finance VPN
Bob Seger bseger HR Advisor Human Resources CRX
Adam Boson aboson Customer Support IT TMS
Adele bree abree Payroll Finance none
Am I on the right track with the below code?
$allUsers = Import-Csv -Path "C:\allStaff.csv"
$vpn = Import-Csv -Path "C:\VPN.csv" | Select-Object -ExpandProperty UserName
$crx = Import-Csv -Path "C:\CRX.csv" | Select-Object -ExpandProperty UserName
$tms = Import-Csv -Path "C:\TMS.csv" | Select-Object -ExpandProperty UserName
$vpnText = 'VPN'
$crxText = 'CRX'
$txsText = 'TMS'
$noneText = 'none'
$allUsersExtended = $allUsers | Select-Object *,ConnectionMethod
$results = $allUsersExtended.ForEach(
{
if($vpn -Contains $PSItem.UserName) {
# add $vpnText to ConnectionMethod column for that row in the $result
$PSItem.ConnectionMethod = $vpnText
}elseif($crx -Contains $PSItem.UserName) {
# add $crxText to ConnectionMethod column for that row in the $result
$PSItem.ConnectionMethod = $crxText
}elseif($tms -Contains $PSItem.UserName) {
# add $txsText to ConnectionMethod column for that row in the $result
$PSItem.ConnectionMethod = $tmsText
}else {
# add $noneText to ConnectionMethod column for that row in the $result
$PSItem.ConnectionMethod = $noteText
}
})
$results | Export-Csv -Path "C:\allStaffConnections.csv" -NoTypeInformation
This gives me an empty allStaffConnections.csv file.
I have run the code line by line and can get as far as:
$allUsersExtended = $allUsers | Select-Object *,ConnectionMethod
Which gives me the extra column "ConnectionMethod", but after running the loop, it gives me an empty allStaffConnections.csv file.
here is one way to do the job. [grin] it presumes that you only want to 1st connection type found. if you want all of them [for instance, JBloggs has all 3 types listed], you will need to concatenate them.
what it does ...
fakes reading in the CSV files
when ready to use real data, comment out or remove the entire #region/#endregion section and use Get-Content.
iterates thru the main collection
uses a switch to test for membership in each connection type list
this breaks out of the switch when it finds a match since it presumes you only want the 1st match. if you want all of them, then you will need to accumulate them instead of breaking out of the switch block.
sets the $ConnectionType as appropriate
builds a PSCO with all the wanted props
this likely could be shortened by using Select-Object, a wildcard property selector, and a calculated property.
sends it out to the $Results collection
shows it on screen
saves it to a CSV file
the code ...
#region >>> fake reading in CSV files
# in real life, use Import-CSV
$AllUsers = #'
FirstName, LastName, UserName, Position, Area
Joe, Bloggs, jbloggs, Gardener, Maintenance
Jim, Smith, jsmith, Accountant, Finance
Bob, Seger, bseger, HR Advisor, Human Resources
Adam, Boson, aboson, Customer Support, IT
Adele, bree, abree, Payroll, Finance
'# | ConvertFrom-Csv
$Vpn = #'
UserName
jbloggs
jsmith
'# | ConvertFrom-Csv
$Crx = #'
UserName
jbloggs
jsmith
bseger
'# | ConvertFrom-Csv
$Tms = #'
UserName
jbloggs
jsmith
bseger
aboson
'# | ConvertFrom-Csv
#endregion >>> fake reading in CSV files
$Results = foreach ($AU_Item in $AllUsers)
{
# this presumes you want only the 1st connection type found
# if you want all of them, then you will need to concatenate them
switch ($AU_Item.UserName)
{
{$_ -in $Vpn.UserName}
{
$ConnectionType = 'VPN'
break
}
{$_ -in $Crx.UserName}
{
$ConnectionType = 'CRX'
break
}
{$_ -in $Tms.UserName}
{
$ConnectionType = 'TMS'
break
}
default
{
$ConnectionType = 'None'
}
}
[PSCustomObject]#{
FirstName = $AU_Item.FirstName
LastName = $AU_Item.LastName
UserName = $AU_Item.UserName
Position = $AU_Item.Position
Area = $AU_Item.Area
ConnectionTYpe = $ConnectionType
}
}
# on screen
$Results
# send to CSV
$Results |
Export-Csv -LiteralPath "$env:TEMP\brokencrow_-_UserConnectionType.csv" -NoTypeInformation
truncated on screen output ...
FirstName : Joe
LastName : Bloggs
UserName : jbloggs
Position : Gardener
Area : Maintenance
ConnectionTYpe : VPN
[*...snip...*]
FirstName : Adele
LastName : bree
UserName : abree
Position : Payroll
Area : Finance
ConnectionTYpe : None
the CSV file content from brokencrow_-_UserConnectionType.csv ...
"FirstName","LastName","UserName","Position","Area","ConnectionTYpe"
"Joe","Bloggs","jbloggs","Gardener","Maintenance","VPN"
"Jim","Smith","jsmith","Accountant","Finance","VPN"
"Bob","Seger","bseger","HR Advisor","Human Resources","CRX"
"Adam","Boson","aboson","Customer Support","IT","TMS"
"Adele","bree","abree","Payroll","Finance","None"
$dat = query user /server:$SERVER
this query gives below data
USERNAME SESSIONNAME ID STATE IDLE TIME LOGON TIME
>vm82958 console 1 Active 1:28 2/9/2018 9:18 AM
adminhmc 2 Disc 1:28 2/13/2018 10:25 AM
nn82543 3 Disc 2:50 2/13/2018 3:07 PM
I would like to get each independent user details like STATE, USERNAME, ID details. I tried below code but it is not giving any data
foreach($proc in $dat) {
$proc.STATE # This is not working this command not giving any data.
$proc.ID # This is not working this command not giving any data.
}
Please help me on this.
The result of $dat.GetType() is:
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True String System.Object
This is very similar to this StackOverflow post, but you have blank fields in your data.
One solution is to deal with this first. Example below but this may break given data that is very different to your example. For a more robust and complete solution see Matt's comment
# replace 20 spaces or more with TWO commas, because it signifies a missing field
$dat2 = $dat.Trim() -replace '\s{20,}', ',,'
# replace 2 spaces or more with a single comma
$datTable = $dat2.Trim() -replace '\s{2,}', ',,'
foreach($proc in $datTable) {
$proc.STATE
$proc.ID
}
Another option is to use fixed Columns with string.Insert , like this:
$content = quser /server:$SERVER
$columns = 14,42,46,54,65 | Sort -Descending
$Delimiter = ','
$dat = $content | % {
$line = $_
$columns | % {
$line = $line.Insert($_, $Delimiter)
}
$line -replace '\s'
} |
ConvertFrom-Csv -Delimiter $Delimiter
And Then:
foreach($proc in $dat) {
$proc.STATE
$proc.ID # Will show the relevant Data
}
Using PowerShell, I can import the CSV file and count how many objects are equal to "a". For example,
#(Import-csv location | where-Object{$_.id -eq "a"}).Count
Is there a way to go through every column and row looking for the same String "a" and adding onto count? Or do I have to do the same command over and over for every column, just with a different keyword?
So I made a dummy file that contains 5 columns of people names. Now to show you how the process will work I will show you how often the text "Ann" appears in any field.
$file = "C:\temp\MOCK_DATA (3).csv"
gc $file | %{$_ -split ","} | Group-Object | Where-Object{$_.Name -like "Ann*"}
Don't focus on the code but the output below.
Count Name Group
----- ---- -----
5 Ann {Ann, Ann, Ann, Ann...}
9 Anne {Anne, Anne, Anne, Anne...}
12 Annie {Annie, Annie, Annie, Annie...}
19 Anna {Anna, Anna, Anna, Anna...}
"Ann" appears 5 times on it's own. However it is a part of other names as well. Lets use a simple regex to find all the values that are only "Ann".
(select-string -Path 'C:\temp\MOCK_DATA (3).csv' -Pattern "\bAnn\b" -AllMatches | Select-Object -ExpandProperty Matches).Count
That will return 5 since \b is for a word boundary. In essence it is only looking at what is between commas or beginning or end of each line. This omits results like "Anna" and "Annie" that you might have. Select-Object -ExpandProperty Matches is important to have if you have more than one match on a single line.
Small Caveat
It should not matter but in trying to keep the code simple it is possible that your header could match with the value you are looking for. Not likely which is why I don't account for it. If that is a possibility then we could use Get-Content instead with a Select -Skip 1.
Try cycling through properties like this:
(Import-Csv location | %{$record = $_; $record | Get-Member -MemberType Properties |
?{$record.$($_.Name) -eq 'a';}}).Count