Powershell - Splitting multiple lines of text into individual records - powershell

Fairly new to PowerShell and wondering if someone could provide a hand with the following. Basically I have a .txt document with a number of records separated by a special character ($)(e.g)
ID
Date Entered
Name
Title
Address
Phone
$
ID
Date Entered
Name
Title
Address
Phone
$
I want to split each item between the $ into individual "records" so that I can loop through each record. So for example I could check each record above and return the record ID for all records that didn't have a phone number entered.
Thanks in advance

If each record contains a fixed number of properties you can take this approach. It loops through creating custom objects while skipping the dollar sign line.
$d = Get-Content -Path C:\path\to\text\file.txt
for ($i = 0; $i -lt $d.Length; $i+=7) {
New-Object -TypeName PsObject -Property #{
'ID' = $d[$i]
'Date Entered' = $d[$i+1]
'Name' = $d[$i+2]
'Title' = $d[$i+3]
'Address' = $d[$i+4]
'Phone' = $d[$i+5]
}
}

I once had the same requirement... this is how I did it
$loglocation = "C:\test\dump.txt"
$reportlocation = "C:\test\dump.csv"
$linedelimiter = ":"
$blockdelimiter = "FileSize"
$file = Get-Content $loglocation
$report = #()
$block = #{}
foreach ($line in $file)
{
if($line.contains($linedelimiter))
{
$key = $line.substring(0,$line.indexof($linedelimiter)).trimend()
$value = $line.substring($line.indexof($linedelimiter)+1).trimstart()
$block.Add($key,$value)
if ($block.keys -contains $blockdelimiter)
{
$obj = new-object psobject -property $block
$report += $obj
$block = #{}
}
}
}
$report
$report | Export-Csv $reportlocation -NoTypeInformation
So you cycle through each line, define key and value and add the object to a hashtable. Once the keys contains the blockdelimiter a new object gets written to an array and the hashtable gets cleared.
In my case the linedelimiter was a colon and the blockdelimiter was a valid record so you will have to make some changes. Let me know if this approach suits your needs and you can't find what to do.
P.S. By default only the noteproperties of the first object in the array will be shown so you will have to pipe the array to Select-Object and add all properties needed.
Grts.

Related

Break Excel File by column value

I have an Excel data source file which contains some customer records.
Now I would like to break the large file into small batches.
I would like to break Excel file into four batches by the customer name column by not just evenly break it down.
I have a source file that have a column called "Customer Name", which I would like use as an indicator to break the source file. Currently I write a Power Shell script but I got stuck ,the current method I use is
Get the column value on the customer name column.
Unique the customer array on the customer name
Filter the excel by the customer name and break down into batches
Below is my script but I stuck on do not know how to filter the items by the company name array.
# Load the Microsoft Excel Com Object
$Excel = New-Object -ComObject Excel.Application
# Open the workbook
$Workbook = $Excel.Workbooks.Open("XXXXX.xlsx")
# Get the first worksheet
$Worksheet = $Workbook.Sheets.Item(1)
# Get the range of cells that contain the customer names
$Range = $Worksheet.Range("D1:D1000")
# Get the values of the cells and store them in a variable
$Values = $Range.Value2
# Sort the values and remove duplicates
$UniqueValues = ($Values | Sort-Object) | Select-Object -Unique
# Clear the original range of cells
$Range.Clear()
Write-Output $UniqueValues
$ArrayLength = $UniqueValues.Length
$numberofrecordperbatch=$UniqueValues.Length/4
Write-Output $numberofrecordperbatch
#--------Divided into 4 batches-------------
function DivideList {
param(
[object[]]$list,
[int]$chunkSize
)
$j=1
$batch= #()
for ($i = 0; $i -lt $list.Count; $i += $chunkSize) {
$j+=1
$batch =($list | select -Skip $i -First $chunkSize)
#$companynamearray|Export-Excel -Path "XXXX.xlsx"
Write-Output $i
Write-Output $j
Write-Output $chunkSize
}
}
DivideList -list $UniqueValues -chunkSize $numberofrecordperbatch| foreach { $_ -join ',' }
Write-Output "Start"
#---------Filter------------
# Get the first worksheet
# Get the range of cells that contain the customer names
$Range2 = $Worksheet.Range("A1:X1000")
# Get the values of the cells and store them in a variable
$Value2 = $Range2.Value2
Write-Output $Value2

Powershell to present 'Net View' data

happy Easter!
I am trying to write a script in Powershell that takes a list of hosts from a txt (or csv) and then for each does a "net view /all" on it, returning the presented shares in a csv.
I got something working but I need a column to show the host its looking at for each row otherwise I cant map them back.
Attempt 1 returns the data and the host but looks VERY messy and is proving difficult to dissect in Excel:
$InputFile = 'M:\Sources\Temp\net_view_list.txt'
$addresses = get-content $InputFile
foreach($address in $addresses) {
$sharedFolders = (NET.EXE VIEW $address /all)
foreach ($item in $sharedfolders)
{
$str_list = $address + "|" + $item
$obj_list = $str_list | select-object #{Name='Name';Expression={$_}}
$obj_list | export-csv -append M:\sources\temp\netview.csv -notype
}
}
Attempt 2 works better but cant get the hostname listed, plus the comments seem to appear in the "Used as" section (only using for one host to test the theory (didnt work!)):
$command = net view hostname #/all
$netview = $command -split '\n'
$comp = $netview[0].trim().split()[-1]
$result = $netview -match '\w' | foreach {
convertfrom-string $_.trim() -delim '\s{2,}' -propertynames 'Share','Type', 'Used as', 'Comment'
}
$result[0] = $null
$result | format-table 'Share', 'Type', 'Used as', 'Comment' -hidetableheaders
Also neither of these accounts for issues where the host either isn't accessible or has 0 shares.
I have literally spent all day on these - grateful for any guidance!
I will provide the way to get what you want in the your 1st example. The main reason it is not appearing like you are expecting it to is because you are not dealing with a PowerShell object. You are getting the raw output from an external command. What you need to do is take the data and create a PS Custom object then you can use it as you will. Below is the code that you should add after you have the $SharedFolder populated heavily commented to explain what each part is for.
# Create Array to hold PSCustom Object and variable to tell when the DO loop is done
$share_list = #()
$completed = $false
# Loop through each line in the output
for($x=0;$x -lt $sharedFolders.count;$x++){
$next_line = $x + 1
# If the line is a bunch of - then we know the next line contains the 1st share name.
if($sharedFolders[$x] -like "*-------*"){
# Here we will loop until we find the end of the list of shares
do {
# Take the line and split it in to an array. Note when you
# use -split vs variable.split allows you to use regular
# expressions. the '\s+' will consider x number of spaces as one
# the single quotes are important when using regex. Double
# quotes use variable expansion. Single quotes don't
$content = $sharedFolders[$next_line] -split '\s+'
$share_name = $content[0].Trim()
# Create a PS Custom Object. This is a bit over kill for one item
# but shows you how to create a custom Object. Note the Object last
# just one loop thus you create a new one each go round then add it to
# an Array before the loop starts over.
$custom_object = new-object PSObject
$custom_object | add-member -MemberType NoteProperty -name 'Share Name' -Value $share_name
# Add the Custom Object to the Array
$share_list += $custom_object
# This exits the Do loop by setting $completed to true
if($sharedFolders[$next_line] -like "*command completed*"){
$completed = $true
}
# Set to the next line
$next_line++
} until ($completed)
}
}
$share_list

When creating a dictionary in PowerShell are duplicate keys all accounted for?

I'm looping through some files and pulling values into a dictionary around a ":" delimiter.
The data in the txt files looks like this:
AD ID: 9999
Ad Placement: Computers
Landing Page: www.something.com
Interests: this and that and this
Interests: also this thing and one final thing
My script for creating a dictionary looks like the following:
$files = ls "*.txt"
$dictionary = #{}
[System.Collections.Generic.List[String]]$list = #()
foreach ($f in $files) {
$in = Get-Content -Raw $f
$in.Split([Environment]::NewLine) | ForEach-Object {
$key, $value = $_.Split(':')
$dictionary[$key] = $value
}
[void]$list.Add($dictionary['Ad ID'] + ',' + $dictionary['Ad Text'] + ',' +
$dictionary['Ad Landing Page'] + ',' + $dictionary['Interests'])
}
That's the basic idea at least. I've gotten unpredictable results when I come across a file that has a key twice, as is the case in the entry in the sample data above called "Interests."
What occurs when adding dictionary items to a list from a file?
In the above example, what is the value of $dictionary['interests'] as it goes through the script?
Since the data can contain duplicate keys, you cannot use the ConvertFrom-StringData cmdlet.
To get the data in a dictionary (hashtable) manually is not that hard to do and you can decide for yourself what to do with duplicate keys: either overwrite the values so the last entry found 'wins' or not:
# this decides which duplicate value you want to store in the hashtable
$allowOverwrite = $false
$hash = #{}
# get the content of the file as string array and loop through
Get-Content -Path 'THE FULL PATH AND FILENAME OF YOUR TEXTFILE' | ForEach-Object {
if ( -not [string]::IsNullOrWhiteSpace($_)) {
# split string to get the key and the value
$key, $value = $_ -split ':', 2 | ForEach-Object { $_.Trim() }
# if a key is found that already exists in the hashtable
if ($hash.ContainsKey($key)) {
# either overwrite the value 'Last-One-Wins'
# or do nothing 'First-One-Wins'
if ($allowOverwrite) { $hash[$key] = $value }
}
else {
$hash[$key] = $value
}
}
}
$hash["interests"]
shows "this and that and this" in case of $allowOverwrite = $false
shows "also this thing and one final thing" in case of $allowOverwrite = $true

putting foreach loop into a csv file

im stuck on how to output my foreach loop into a csv or excel file. this script just get all computers off a local network and then test to see if that computer has a certain KBPatch. The script works just like I said I need help trying to make it output to a csv file. any tips/help is appreciated
Code Below
$strCategory = "computer"
$objDomain = New-Object System.DirectoryServices.DirectoryEntry
$objSearcher = New-Object System.DirectoryServices.DirectorySearcher
$ObjSearcher.SearchRoot = $objDomain
$objSearcher.filter = ("(objectCategory=$strCategory)")
$colProplist = "name"
foreach($i in $colProplist)
{
$objSearcher.PropertiesToLoad.Add($i)
}
#Finds all operating systems and computer names
$colResults = $objSearcher.FindAll()
foreach($objResult in $colResults)
{
$objComputer = $objResult.Properties;
$names = $objComputer.name
# Define Hotfix to check
$CheckKBS = #(“patch#" , "patch#")
#Query the computers for the HotFix
foreach($name in $names)
{
foreach ($CheckKB in $CheckKBS) {
$HotFixQuery = Get-HotFix -ComputerName $name | Where-Object {$_.HotFixId -eq $CheckKB} | Select-Object -First 1;
if($HotFixQuery -eq $null)
{
Write-Host “Hotfix $CheckKB is not installed on $name”;
}
else
{
Write-Host “Hotfix $CheckKB was installed on $name by ” $($HotFixQuery.InstalledBy);
}
}}
}
To do this cleanly I usually use a custom object.
Before your foreach, instantiate an empty array:
$records = #()
and make an object template:
$tmpRecord = [PSCustomObject]#{
serverName = ''
missingKB = ''
}
Inside the foreach, clone the record obj:
$record = $tmpRecord.psobject.copy()
put your data into the record:
$record.serverName = $name
$record.missingKB = $CheckKb
Put the record into the array:
$records += $record
Then after the foreach, export to csv:
$records | export-csv yourcsv.csv
The need for an object template was confusing to me when I first learned this pattern. You need this because of the combination of scope and how objects are added to arrays (by reference).
If you try to get away with declaring an object inside the loop then that object will be scoped to the lifetime of the foreach loop. You'll then add a reference to that object to your $records array. After the foreach loop completes you will have an array full of references to objects that do not exist.

Import Excel data into PowerShell variables

I have an Excel File which has an unknown number of records in it, and these 3 columns:
Variable Name, Store Number, Email Address
I use this in QlikView to import data for certain stores and then create a separate report for each store in the list. I then need to email each report to each individual store (store number will be in the report file name).
So in PowerShell I would like to read the Excel File and set variables for each store:
$Store1 = The Store Number in Row 2 of the Excel File
$Store1Email = The Store Email in Row 2 of the Excel File
$Store2 = The Store Number in Row 3 of the Excel File
$Store2Email = The Store Email in Row 3 of the Excel File
etc. for each Storein the file (can be any number of stores).
Please note the "Variable Name" in the excel file must be ignored (that is for QLikView) and the PowerShell variables must be named as per my above examples, each time incrementing the number.
Check out my PowerShell Excel Module on Github. You can also grab it from the PowerShell Gallery.
$stores = Import-Excel C:\Temp\store.xlsx
$stores[2].Name
$stores[2].StoreNumber
$stores[2].EmailAddress
''
'All stores'
'----------'
$stores
Ok, first off if you are going to be working with actual .XLS or .XLSX or .XLSM files I would highly suggest using the Import-XLS function from the TechNet gallery (found here).
After that, just reference the object it imports to send the emails instead of making objects for each store. Such as:
$StoreList = Import-XLS <path to Excel file>
GC <report folder> | %{
$Current = $_
$Store = $StoreList|?{$_.StoreNumber -match $Current.BaseName}|Select -ExpandProperty StoreNumber
$Email = $StoreList|?{$_.StoreNumber -match $Current.BaseName}|Select -ExpandProperty StoreEmail
<code to send $Current to $Email>
}
My preference is to Save-As the Excel file to a '.csv' type. The comma separated value can easily be imported into PowerShell.
$csvFile = Import-Csv -Path c:\scripts\temp\excelFile.csv
#now the entire Excel '.csv' file is saved into csvFile variable
$csvFile |Get-Member
#look at the properties
Remember to study the greats so your PowerShell script looks great. Jeffery Snover, Jason Hicks, Don Jones, Ashley McGlone, and anyone on their friends list ha ha
The above answers usually work, but I just had a project with excel datasheets that caused some problems.
edit: Here's a much more advanced version that will pull it into an object, can handle blank and duplicate column names, and can skip human information at the beginning of the worksheet by looking for something in the header row. I've also included some example usages
Your example:
$file = New-Module -AsCustomObject -ScriptBlock $file_template
$file.from_excel("c:\folder\file.xls")
$Store1 = $file.data[0]."Store Number" #first row, column named "Store Number"
$Store1Email = $file.data[0]."Store Email" #first row, column named "Store Email"
foreach ($row in $file.data)
{
write-host "Store: $($row."Store Number")"
write-host "Store Email: $($row."Store Email")"
}
Example 1:
# Simplest example
$file = New-Module -AsCustomObject -ScriptBlock $file_template
$file.from_excel("c:\folder\file.xls")
$file.data[0]
Example 2:
#advanced usage
$file = New-Module -AsCustomObject -ScriptBlock $file_template
$file.header_contains="First Name" # if included it will drop everything before the first line that contains this, useful if there are instructions for humans in the worksheet
$file.indexer_column = 5 # Default: 1 (first column); This column's contents will set the minimum number of rows, use if there are blank rows in your file but more data after them
$file.worksheet_index = "January" # Default: 1; can be a sheet index or sheet name
$file.filename = "c:\folder\file.xls" #can set this independently, useful for validation and troubleshooting
$file.from_excel() #This is where we actually pull from excel
$collected = $file.data|ogv -pass thru #this is a neat way to select some rows you want
$file.headers.count # It stores an array of the headers here, useful for troubleshooting and advanced logic
Excel Reader pseudoclass
$file_template = {
# -- universal --
$filename = ""
$delimiter = ","
$headers = #()
$data = #()
# -- used by some functions --
# we put these here to allow assigning them before calling functions, which improves readability and auditability
$header_contains=""
$indexer_column=1
$worksheet_index=1
function from_excel(
$filename=$this.filename,
$worksheet_index=$this.worksheet_index
)`
{
$this.filename = $filename
$this.worksheet_index = $worksheet_index
$data_by_row = $this.from_excel_as_csv() # $data_by_row = $file.from_excel_as_csv($test_file)
$data_by_row = $data_by_row -split"`n"
#if ($this.headers.count -lt 1) {$this.headers = $data_by_row[0] -split $this.delimiter} #this would let us set headers elsewhere which is more flexible but less adaptive, Because columns change unpredicably we need something more adaptive
$temp_headers = $data_by_row[0] -split $this.delimiter
$temp_headers = $this.fix_blank_headers($temp_headers)
$this.headers = $this.dedupe_headers($temp_headers)
$this.data = $data_by_row|select -Skip 1|ConvertFrom-Csv -Header $this.headers -Delimiter $this.delimiter
}
function from_csv($filename=$this.filename)`
{
$this.filename = $filename
$this.headers = (Get-Content $this.filename -ReadCount 1|select -first 1) -split $this.delimiter
$this.data = Get-Content $this.filename|ConvertFrom-Csv -Delimiter $this.delimiter
}
function from_excel_as_csv(
$filename=$this.filename,
$worksheet_index=$this.worksheet_index
)`
{
$this.filename = $filename
$this.worksheet_index = $worksheet_index
#set up excel
Write-Host "Importing from excel, this may take a little while..."
$excel = New-Object -ComObject Excel.Application
$excel.DisplayAlerts = $false
$excel.Visible = $false
$workbook = $excel.workbooks.open($this.filename)
$worksheet = $workbook.Worksheets.Item($this.worksheet_index)
#import from excel
try{
$data_by_row = ""
$indexed_column = $worksheet.columns.item($this.indexer_column).value2 #we use this to work around some files having headers with blank space
$minimum_rows = (($indexed_column -join "◘").TrimEnd("◘") -split "◘").count # This Strips the million or so extra blank rows excel appends to get a realistic column length.
[bool]$header_found = 0
$i=1
do `
{
$row = $worksheet.rows.item($i).value2
$row_as_text = $row -join "◘" # ◘ (alt+8) is just a placeholder that's unlikely to show up in the text
$row_as_text = $row_as_text -replace $this.delimiter,"."
$row_as_text = $row_as_text.TrimEnd("◘")
$row_as_text = $row_as_text -replace "◘",$this.delimiter
if ($row_as_text -like "*$($this.header_contains)*"){[bool]$header_found=1}
if ($header_found) {$data_by_row+="$row_as_text`n"}
$i++
}
while ( ($row_as_text.Length -gt 1) -or ($i -lt $minimum_rows) )
}
catch {Write-Warning "ERROR Importing from excel"}
#close excel
$workbook.Close()
$excel.Quit()
write-host "Done importing from excel"
return $data_by_row
}
function dedupe_headers($headers){
$dupes = ($headers|group)|?{$_.count -gt 1}
if ($dupes.count -ge 1)
{
foreach ($dupe in $dupes)
{ #$dupe = $dupes[0]
$i=1
$new_headers = #()
foreach ($header in $headers)
{ #$header = $headers[0]
if ($header -eq $dupe.name)
{
$header = "$($header)_$($i)" # "header_#"
$i++
}
$new_headers += $header
}
}
}
else {$new_headers = $headers} # no duplicates found
return $new_headers
}
function fix_blank_headers($headers)
{
$replace_blanks_with = "_"
$new_headers = #()
foreach ($header in $headers)
{
if ($header -eq "") {$new_headers += $replace_blanks_with}
else {$new_headers += $header}
}
if ($new_headers.count -ne $header)
{
$error_json = #($headers),#($new_headers)|ConvertTo-Json -Compress
Write-Error "Error when fixing blank headers, original and new counts are different $($error_json)"
}
return $new_headers
}
<# function some_function($some_parameter){return $some_parameter} #>
Export-ModuleMember -Function * -Variable *
}
Forgive the ugliness here. I am not a programmer, so there are undoubtedly more optimized ways to do this, as well as better formatting. It will work, however, if I understand your requirements correctly.
$excelfile = import-csv "c:\myfile.csv"
$i = 1
$excelfile | ForEach-Object {
New-Variable "Store$i" $_."Store Number"
$iemail = $i.ToString() + "Email"
New-Variable "Store$iemail" $_."Email Address"
$i ++
}
edit: as per the reply to your original post, this works with a csv file. Just save it to csv first if necessary.
$excelfile = import-csv "C:\Temp\store.csv"
$i = 1 $excelfile | ForEach-Object {
$NA= $_."Name"
$SN= $_."StoreNumber"
Write-Output "row $i"
$NA
$SN
$i++ }