I am having a difficult time with finding and separating certain regular expressions in Powershell.
I have an example of the following contents of the text file that I need to search in the link below. Beneath each question is the response.
I need to find the response to each question and just store it in an excel file. It is possible that the questions might be in a different order in the text file so I need to be able to extract the correct responses to their respective questions.
I can extract the questions by doing
$question1 = $file | Where-Object { $_.Contains("Name") }
$question2 = $file | Where-Object { $_.Contains("Are you feeling ok?") }
$question3 = $file | Where-Object { $_.Contains("Did you do your homework?") }
$question4 = $file | Where-Object { $_.Contains("List your favourite subjects?") }
I am having difficulty with extracting the response right underneath it (because the questions could be in any order. In the future there could be more questions in the text file).
Assuming a text file exaclty as you mention:
* Name
Jeff
* Are you feeling ok?
Yes
* Did you do your homework?
No
* What are your favourite subjects?
Maths, science
Then you could use Select-String to get the questions and answers like this:
Get-Content -Path .\test.txt |
Select-String -Pattern '^\* .*$' -Context 1 |
ForEach-Object {
[PsCustomObject]#{
Question = $_.Line -replace '\* ', ''
Answer = $_.Context.PostContext[0].Trim()
}
}
Example output:
Question Answer
-------- ------
Name Jeff
Are you feeling ok? Yes
Did you do your homework? No
What are your favourite subjects? Maths, science
The order of the questions in the file doesn't matter, though they will appear in the output in the order they are found. You can use Export-Csv to create a CSV file, which can be opened in Excel, or look on-line for technique of manipulating XLSX files from PowerShell.
Related
I have two files. One is called allper.csv
institutiongroup,studentid,iscomplete
institutionId=22343,123,FALSE
institutionId=22343,456,FALSE
institutionId=22343,789,FALSE
The other one is called actswithpersons.csv
abc,123;456
def,456
ghi,123
jkl,123;456
Note: The actswithpersons.csv does not have headers - they are going to be added in later via an excel power query so don't want them in there now. The actswithpersons csv columns are delimited with commas - there are only two columns, and the second one contains multiple personids - again Excel will deal with this later.
I want to remove all rows from allper.csv where the personid doesn't appear in actswithpersons.csv, and export them to another csv. So in the desired outcome, allper.csv would look like this
institutiongroup,studentid,iscomplete
institutionId=22343,123,FALSE
institutionId=22343,456,FALSE
and the export.csv would look like this
institutiongroup,studentid,iscomplete
institutionId=22343,789,FALSE
I've got as far as the below, which will put into the shell whether the personid is found in the actswithpersons.csv file.
$donestuff = (Get-Content .\ActsWithpersons.csv | ConvertFrom-Csv); $ids=(Import-Csv .\allper.csv);foreach($id in $ids.personid) {echo $id;if($donestuff -like "*$id*" )
{
echo 'Contains String'
}
else
{
echo 'Does not contain String'
}}
However, I'm not sure how to go the last step, and export & remove the unwanted rows from allper.csv
I've tried (among many things)
$donestuff = (Get-Content .\ActsWithpersons.csv | ConvertFrom-Csv);
Import-Csv .\allper.csv |
Where-Object {$donestuff -notlike $_.personid} |
Export-Csv -Path export.csv -NoTypeInformation
This took a really long time and left me with an empty csv. So, if you can give any guidance, please help.
Since your actswithpersons.csv doesn't have headers, in order for you to import as csv, you can specify the -Header parameter in either Import-Csv or ConvertFrom-Csv; with the former cmdlet being the better solution.
With that said, you can use any header name for those 2 columns then filter by the given column name (ID in this case) after your import of allper.csv using Where-Object:
$awp = (Import-Csv -Path '.\actswithpersons.csv' -Header 'blah','ID').ID.Split(';')
Import-Csv -Path '.\allper.csv' | Where-Object -Property 'Studentid' -notin $awp
This should give you:
institutiongroup studentid iscomplete
---------------- --------- ----------
institutionId=22343 789 FALSE
If you're looking to do it with Get-Content you can split by the delimiters of , and ;. This should give you just a single row of values which you can then compare the entirety of variable ($awp) using the same filter as above which will give you the same results:
$awp = (Get-Content -Path '.\actswithpersons.csv') -split ",|;"
Import-Csv -Path '.\allper.csv' | Where-Object -Property 'Studentid' -notin $awp
I am pretty sure there is a simple answer, but I cannot figure out how to ask it accurately enough as I am new to PowerShell.
Simple put, I am doing some API calls and then running through the results. The results include various properties that I converted from JSON into a Powershell Object.
A record looks simply like this:
id : 10123
path : /a/path/here/file.pptx
creationDate : 2019-06-28T09:37:32.000-04:00
modifiedDate : 2020-03-09T13:56:13.000-04:00
description : Record Description
creator : asmith
lastModifiedBy : jsmith
I then want to interact with the records, so I use a FOR loop on the IDs:
Foreach($SimpleID in $Records.id){
Write-Host $SimpleID + "`t|`t"+ $Records.path >> file.log
}
My question is, I want that output in the file to be:
10123 | /a/path/here/file.pptx
10124 | /next/path/file.txt
...etc
I know that the $Records.path is not correct there, but how do I filter it to only print the single path for the ID? I am sure there is a simple way, but I cannot figure out what to look up to start.
Any ideas on where to start?
You cannot use Write-Host to produce data output - see this post.
It sounds like you're looking for the following:
$Records | ForEach-Object { $_.id + "`t|`t"+ $_.path } > file.log
I would like to provide this alternative to mklement0's solution using Set-Content and Add-Content:
Set-Content -Path '.\log' -Value ''
$Records | ForEach-Object { Add-Content -Path '.\log' -Value "$_.id | $_.path" }
Loop over the Records objects and grab only what you need.
I need to overwrite the email value in userinfo.csv with the email address from email.csv
userid and u_user values are matching and unique in both csv's. The email address value in userinfo.csv in not good and needs to be overwritten with the email value from email.csv.
How do I match userid in both csv's and append email value?
No idea where to even start. Any help, please.
email.csv
userid,email
1234,user4#email.com
1235,user5#email.com
userinfo.csv
u_work,u_user,u_address,u_city,u_state,u_zip,u_email,u_phonehome
1234,here,there,everywhere,55555,1234#bad.org,555-555-5555
away,1235,there,here,everywhere,66666,1235#bad.com,666-666-6666
new.csv
u_work,u_user,u_address,u_city,u_state,u_zip,u_email,u_phonehome
1234,here,there,everywhere,55555,user4#email.com,555-555-5555
away,1235,there,here,everywhere,66666,user5#email.com,666-666-6666
Your CSVs as presented are not valid. The header row has 8 fields. Row 1 has 7 fields. That's not valid. I'm assuming that it should look like this:
userinfo.csv
u_work,u_user,u_address,u_city,u_state,u_zip,u_email,u_phone
home,1234,here,there,everywhere,55555,1234#bad.org,555-555-5555
away,1235,there,here,everywhere,66666,1235#bad.com,666-666-6666
In other words, that u_phonehome is actually u_phone and home is on the wrong row in your examples.
Your basic steps are:
A. Import email.csv into a hash table for quick lookup.
$emails = #{}
Import-Csv email.csv | ForEach-Object {
$email[$_.userid] = $_.email
}
B. Import userinfo.csv, and replace the email addresses where necessary.
$NewCSV = Import-Csv userinfo.csv | ForEach-Object {
if ($emails.ContainsKey($_.u_user)) {
$_.u_email = $emails[$_.u_user]
}
$_
}
C. Write the output file.
$NewCSV | Export-Csv new.csv -NoTypeInformation
You could also do step B with a Select-Object and a calculated property, but this is a bit easier to write.
You'd use Regex for the match and replace for the modification of specific stings. This is a common thing that is done, and there are many articles and posts on the topic. So, give the below resources a shot and come back with your effort if you need further assistance.
For example:
Windows PowerShell: Writing Regular Expressions
https://technet.microsoft.com/en-us/library/2007.11.powershell.aspx
Powershell: The many ways to use regex - Kevin Marquette
https://kevinmarquette.github.io/2017-07-31-Powershell-regex-regular-expression
"Hello. Yes, this is a cat." -replace 'cat','dog'
"Hello. Yes, this is a dog." -replace [regex]::Escape('.'),'!'
("Hello. Yes, this is a dog.").Replace('.','!')
PSTip A difference between the –replace operator and String.Replace method
https://www.powershellmagazine.com/2012/11/12/pstip-a-difference-between-the-replace-operator-and-string-replace-method/
(Get-Content C:\test\test.txt) |
Foreach-Object {$_ -replace "(?i)\b[A-Z0-9._%+-]+#[A-Z0-9.-]+\.[A-Z]{2,4}\b",'<$0>'} |
Set-Content C:\test\test.txt
Powershell Regex find emails and replace email with (<email>)
$txt='<p class=FillText><a name="InternetMail_P3"></a>First.Last#company-name.com</p>'
$re="[a-z0-9!#\$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#\$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?"
[regex]::MAtch($txt, $re, "IgnoreCase ")
Using Regex in Powershell to grab email
Parse .rtf file, output email addresses in .csv file?
I have an .rtf file containing a bunch of email addresses, I need this parsed so that I can compare a .csv file to active users in Active Directory.
Basically I want what is to the left of "#my.domain.com"
$finds = Select-String -Path "path\to\my.rtf" -Pattern "#my.domain.com" | ForEach-Object {$_.Matches}
$finds | Select-Object -First 1 | ft *
This of course gives me one result so that I don't have alot of output.
I only manage to get matches or the complete line.
I've tried adding something along the line of
$finds = Select-String -Path "path\to\my.rtf" -Pattern "\w.#my.domain.com"
This gives me the very two last letters in the addresses.
If I keep adding dots to the "wildcard"
-Pattern "\w.....#my.domain.com"
I also get a ton of numbers/characters (.rtf formatting) for addresses that contains fewer characters.
How do I do this?
EDIT: I will update the question as soon as I've found a solution. As of now I'm trying with regular expressions.
Example:
-Pattern "\w*?#my.domain.com"
$mPattern = "[a-zA-Z0-9._%+-]+#[a-zA-Z0-9.-]+(\.[a-zA-Z]{2,4})"
$lines = get-content "path\to\your.rtf"
foreach($line in $lines){
([regex]::MAtch($rtfInput, $mpattern, "IgnoreCase ")).value }
This code worked for me. My inital code but with a new search pattern.
$finds = Select-String -Path "path\to\my.rtf" -Pattern "[a-zA-Z0-9._%+-]+#[a-zA-Z0-9.-]+(\.[a-zA-Z]{2,4})" | ForEach-Object {$_.Matches}
$finds | Select-Object -First 10 | ft *
Thanks!
I have a fairly standard csv file with headers I want to add a new column & set all the rows to the same data.
Original:
column1, column2
1,b
2,c
3,5
After
column1, column2, column3
1,b, setvalue
2,c, setvalue
3,5, setvalue
I can't find anything on this if anybody could point me in the right direction that would be great. Sorry very new to Power Shell.
Here's one way to do that using Calculated Properties:
Import-Csv file.csv |
Select-Object *,#{Name='column3';Expression={'setvalue'}} |
Export-Csv file.csv -NoTypeInformation
You can find more on calculated properties here: http://technet.microsoft.com/en-us/library/ff730948.aspx.
In a nutshell, you import the file, pipe the content to the Select-Object cmdlet, select all exiting properties (e.g '*') then add a new one.
The ShayLevy's answer also works for me!
If you don't want to provide a value for each object yet the code is even easier...
Import-Csv file.csv |
Select-Object *,"column3" |
Export-Csv file.csv -NoTypeInformation
None of the scripts I've seen are dynamic in nature, so they're fairly limited in their scope & what you can do with them.. that's probably because most PS Users & even Power Users aren't programmers. You very rarely see the use of arrays in Powershell. I took Shay Levy's answer & improved upon it.
Note here: The Import needs to be consistent (two columns for instance), but it would be fairly easy to modify this to dynamically count the columns & generate headers that way too. For this particular question, that wasn't asked. Or simply don't generate a header unless it's needed.
Needless to say the below will pull in as many CSV files that exist in the folder, add a header, and then later strip it. The reason I add the header is for consistency in the data, it makes manipulating the columns later down the line fairly straight forward too (if you choose to do so). You can modify this to your hearts content, feel free to use it for other purposes too. This is generally the format I stick with for just about any of my Powershell needs. The use of a counter basically allows you to manipulate individual files, so there's a lot of possibilities here.
$chargeFiles = 'C:\YOURFOLDER\BLAHBLAH\'
$existingReturns = Get-ChildItem $chargeFiles
for ($i = 0; $i -lt $existingReturns.count; $i++)
{
$CSV = Import-Csv -Path $existingReturns[$i].FullName -Header Header1,Header2
$csv | select *, #{Name='Header3';Expression={'Header3 Static'}}
| select *, #{Name='Header4';Expression={'Header4 Static Tet'}}
| select *, #{Name='Header5';Expression={'Header5 Static Text'}}|
CONVERTTO-CSV -DELIMITER "," -NoTypeInformation |
SELECT-OBJECT -SKIP 1 | % {$_ -replace '"', ""} |
OUT-FILE -FilePath $existingReturns[$i].FullName -FORCE -ENCODING ASCII
}
You could also use Add-Member:
$csv = Import-Csv 'input.csv'
foreach ($row in $csv)
{
$row | Add-Member -NotePropertyName 'MyNewColumn' -NotePropertyValue 'MyNewValue'
}
$csv | Export-Csv 'output.csv' -NoTypeInformation
For some applications, I found that producing a hashtable and using the .values as the column to be good (it would allow for cross reference validation against another object that was being enumerated).
In this case, #powershell on freenode brought my attention to an ordered hashtable (since the column header must be used).
Here is an example without any validation the .values
$newcolumnobj = [ordered]#{}
#input data into a hash table so that we can more easily reference the `.values` as an object to be inserted in the CSV
$newcolumnobj.add("volume name", $currenttime)
#enumerate $deltas [this will be the object that contains the volume information `$volumedeltas`)
# add just the new deltas to the newcolumn object
foreach ($item in $deltas){
$newcolumnobj.add($item.volume,$item.delta)
}
$originalcsv = #(import-csv $targetdeltacsv)
#thanks to pscookiemonster in #powershell on freenode
for($i=0; $i -lt $originalcsv.count; $i++){
$originalcsv[$i] | Select-Object *, #{l="$currenttime"; e={$newcolumnobj.item($i)}}
}
Example is related to How can I perform arithmetic to find differences of values in two CSVs?
create a csv file with nothin in it
$csv >> "$PSScriptRoot/dpg.csv"
define the csv file's path. here $psscriptroot is the root of the script
$csv = "$PSScriptRoot/dpg.csv"
now add columns to it
$csv | select vds, protgroup, vlan, ports | Export-Csv $csv