Issues importing csv column and replacing it from hash value - powershell

Please note that this data has been cleaned to prevent identifying information and considerable white space has been removed from between the commas in order to aid in readability. Lastly at the end of the TYPE column there is an additional line saying how many lines were exported which hopefully will be ignored by the script.
TYPE ,DATE ,TIME ,STREET ,CROSS-STREET ,X-COORD ,Y-COORD
459 ,2015-05-03 00:00:00.000,00:58:35,FOO DR ,A RD/B CT , 0.0, 0.0
488 ,2015-05-03 00:00:00.000,02:31:54,BAR AV ,C ST/D ST , 0.0, 0.0
I am attempting to import this CSV using Import-CSV, convert the TYPE numeric codes into different strings. An example would be 459 becomes Apple. 488 becomes Banana and so forth. I have created a hash with the TYPE numbers as the key and the value being what I want it changed to.
So my issue is really two-fold; I have been so far unable to get the TYPE CSV column to import into the script (I've been trying an array for the most part) and I am not sure the best way to build the logic to check the array data against my hash keys and replace it with the appropriate value.
# declare filename to modify
$strFileName="test.csv"
# import the type data into its own array
$imported_CSV = Import-Csv $strFileName
# populate hash
$conversion_Hash = #{
187 = Homicide;
211 = Robbery;
245 = Assault;
451 = Arson;
459 = Burglary;
484 = Larceny;
487 = Grand Theft;
488 = Petty Theft;
10851 = Stolen Vehicle;
HS = Drug;
}
# perform the conversion
foreach ($record in $imported_CSV)
{
$conversion_Hash[$record.Type]
}
This has no logic and just contains the code that was presented in the answer below. Note that I addressed that it doesn't work in the comments below.

I think this is an example of what you are looking for:
$hashTable = #{459= Apple; 488= Banana;}
$csv = import-csv <file>
foreach($record in $csv)
{
$hashTable[$record.Type] #returns hash value
}
Output:
Apple
Banana

So we have several little issues here. The two big ones are your source file and the your hashtable keys are integers and not strings.
# declare filename to modify
$strFileName="c:\temp\point.csv"
# import the type data into its own array
$imported_CSV = (Get-Content $strFileName) -replace "\s*,\s*","," | ConvertFrom-Csv
# populate hash
$conversion_Hash = #{
"187" = "Homicide";
"211" = "Robbery";
"245" = "Assault";
"451" = "Arson";
"459" = "Burglary";
"484" = "Larceny";
"487" = "Grand Theft";
"488" = "Petty Theft";
"10851" = "Stolen Vehicle";
"HS" = "Drug";
}
# perform the conversion
foreach ($record in $imported_CSV)
{
$conversion_Hash[$record.Type]
}
Output from naughty people
Burglary
Petty Theft
I don't know if your source file looks like it does in your question but there is a bunch of whitespace there that will be giving you a hassle. Namely you dont have a TYPE column but a "TYPE " (without the spaces). Same goes for the other columns. Data is affected as well. It's not 459 but "459 "(without the spaces).
To fix that I check the file and replace all space surrounding the commas with just the comma.
TYPE,DATE,TIME,STREET,CROSS-STREET,X-COORD,Y-COORD
459,2015-05-03 00:00:00.000,00:58:35,FOO DR,A RD/B CT,0.0,0.0
488,2015-05-03 00:00:00.000,02:31:54,BAR AV,C ST/D ST,0.0,0.0
If your data already looks like that then you need to be careful posting this stuff in your question. Onto the other issue with your comparison
You will see I have quoted almost everything in that hashtable. I had to for the values as they were being taken as commands otherwise. I also quoted the keys as the csv table contains string and not integers. I would have just casted to [int] to avoid the whole issue but one of your keys is called "HS" which does not look like a number to me :).
What I might have done
Just to play a little I might have added another note property to the list called TypeAsString which would add a column.
# perform the conversion
$imported_CSV | ForEach-Object{
$_ | Add-Member -MemberType NoteProperty -Name "TypeAsString" -Value $conversion_Hash[$_.Type] -PassThru
}
So the output from one item would look like this
TYPE : 459
DATE : 2015-05-03 00:00:00.000
TIME : 00:58:35
STREET : FOO DR
CROSS-STREET : A RD/B CT
X-COORD : 0.0
Y-COORD : 0.0
TypeAsString : Burglary
I could have made a more dynamic property like a script property, so that changes in $conversion_Hash are updated instantly, but this should suffice for what you need.

Related

How do I add a line feed (new line) in a csv file (powershell)

I have large csv file that should contain many records. However, for some reason, there are no line feeds or new record delimiters so as to be able to treat the various records separately (example by importing them to excel)*. Is there any way (eg with windows powershell) that I can add a line feed before a given field in the csv file? For example suppose we have an input csv file with contents :
data1,data2,data3,data4,data5,data6,data7,data8,data9,data10;data11;data12
The request is to get an output csv like this (so every record should contains 3 cells / fields....however this should be configurable):
data1,data2,data3
data4,data5,data6
data7,data8,data9,
data10,data11,data12
The above example is for illustration only. Consider that my real case contains a huge amount of data fields that I somehow need to organize.
Thank you very much in advance for every response
*Actually I have deliberately eliminated every new line feed from my source data. I did this to get rid of some unwanted newlines and other formatting characters (\t etc) that existed inside specific cells and totally messed up the structure of the data set. However, this way I lost the required newlines \n as well. Now I want to add them back, selecting the proper position they should be.
p.s. Since I am very new to powershell or scripting in general, sorry if I am making an obvious or trivial question.....
You can "wrap around" an arbitrary number of values in X columns by calculating the relative column offset with: $index % $columnWidth
I'd suggest writing a small function for this, something like:
function ConvertTo-TabularCollection {
param(
[Parameter(Mandatory, ValueFromPipeline)]
[string[]]$Data,
[Parameter(Mandatory)]
[string[]]$ColumnNames
)
begin {
# Calculate table width and prepare list to collect input data
$width = $columnNames.Length
$values = [System.Collections.Generic.List[string]]::new()
}
process {
# Copy any input to our `$values` list
$values.AddRange($Data)
}
end {
# Time to process all the input values we've collected
for($i = 0; $i -lt $values.Count; $i++){
# use the modulo operator to calculate the relative column offset
$offset = $i % $width
if($offset -eq 0){
# We're about to process the first column of a new row, create an empty dictionary to hold the column values
$properties = [ordered]#{}
}
# Pick the next available value and show it into the appropriate column
$properties[$columnNames[$offset]] = $values[$i]
if($offset -eq ($width - 1)){
# We've reached the last column, output object and clear previous column values collected
[pscustomobject]$properties
$properties = $null
}
}
# Test if there is a (partial) row trailing, output object
if($properties){
[pscustomobject]$properties
}
}
}
Now you can transform your data as needed:
PS ~> $data = 'data1,data2,data3,data4,data5,data6,data7,data8,data9,data10,data11,data12' -split ','
PS ~> $data |ConvertTo-TabularCollection -ColumnNames col1,col2,col3
col1 col2 col3
---- ---- ----
data1 data2 data3
data4 data5 data6
data7 data8 data9
data10 data11 data12

How to change each character in a string to the previous character in the alphabet

I am looking for a way to change each character in a string to the previous value in the alphabet. Essentially, I need to create a for each loop that will take every character in a string and increment it backwards by 1. For instance, I would like to change bcd234% to abc123$.
I have tried breaking the string into an array and subtracting 1 from each element.
$myString = "bcd234%"
$myArray = $myString.ToCharArray()
$myArray = $myArray | ForEach-Object { $_ - 1 }
$myArray
-join($myArray)
I would expect that it would iterate the value down 1 and then join all of the new values together.
What I would like to see is the new string:
abc123$
What it is actually doing is creating new values for each of the characters and joining them together instead.
The results I am getting are the new array:
97
98
99
49
50
51
36
And then it joins them together which looks like:
97989949505136
When you're subtracting an integer from a char PowerShell automatically converts the char to an integer for the calculation. Essentially $_ - 1 does the same as ([int]$_) - 1. For getting back a character you can simply cast the result of the operation back to a char.
$myArray = $myArray | ForEach-Object { [char]([int]$_ - 1) }
The [int] is redundant, as mentioned above, but I put it in for good measure, so it's more obvious what is happening there.

Replace string untill its length is less than limit with PowerShell

I try to update users AD accounts properties with values imported from csv file.
The problem is that some of the properties like department allow strings of length of max length 64 that is less than provided in the file which can be up to 110.
I have found and adopted solution provided by TroyBramley in this thread - How to replace multiple strings in a file using PowerShell (thank You Troy).
It works fine but... Well. After all replaces have place the text is less meaningful than originally.
For example, original text First Department of something1 something2 something3 something4 would result in 1st Dept of sth1 sth2 sth3 sth4
I'd like to have control over the process so I can stop it when the length of the string drops just under the limit alowed by AD property.
By the way. I'd like to have a choice which replacement takes first, second and so on, too.
I put elements in a hashtable alphabetically but it seems that they are not processed this way. I can't figure out the pattern.
I can see the resolution by replacing strings one by one, controlling length after each replacement. But with almost 70 strings it leds to huge portion of code. Maybe there is simpler way?
You can iterate the replacement list until the string reaches the MaxLength defined.
## Q:\Test\2018\06\26\SO_51042611.ps1
$Original = "First Department of something1 something2 something3 something4"
$list = New-Object System.Collections.Specialized.OrderedDictionary
$list.Add("First","1st")
$list.Add("Department","Dept")
$list.Add("something1","sth1")
$list.Add("something2","sth2")
$list.Add("something3","sth3")
$list.Add("something4","sth4")
$MaxLength = 40
ForEach ($Item in $list.GetEnumerator()){
$Original = $Original -Replace $Item.Key,$Item.Value
If ($Original.Length -le $MaxLength){Break}
}
"{0}: {1}" -f $Original.Length,$Original
Sample output with $MaxLength set to 40
37: 1st Dept of sth1 sth2 sth3 something4

finding index of key in an ordered dictionary in powershell

I am having a little bit of trouble with hashtables/dictionaries in powershell. The most recent roadblock is the ability to find the index of a key in an ordered dictionary.
I am looking for a solution that isn't simply iterating through the object.
(I already know how to do that)
Consider the following example:
$dictionary = [Ordered]#{
'a' = 'blue';
'b'='green';
'c'='red'
}
If this were a normal array I'd be able to look up the index of an entry by using IndexOf().
[array]::IndexOf($dictionary,'c').
That would return 2 under normal circumstances.
If I try that with an ordered dictionary, though, I get -1.
Any solutions?
Edit:
In case anyone reading over this is wondering what I'm talking about. What I was trying to use this for was to create an object to normalize property entries in a way that also has a numerical order.
I was trying to use this for the status of a process, for example:
$_processState = [Ordered]#{
'error' = 'error'
'none' = 'none'
'started' = 'started'
'paused' = 'paused'
'cleanup' = 'cleanup'
'complete' = 'complete'
}
If you were able to easily do this, the above object would give $_processState.error an index value of 0 and ascend through each entry, finally giving $_processState.complete an index value of 5. Then if you compared two properties, by "index value", you could see which one is further along by simple operators. For instance:
$thisObject.Status = $_processState.complete
If ($thisObject.Status -ge $_processState.cleanup) {Write-Host 'All done!'}
PS > All done!
^^that doesn't work as is, but that's the idea. It's what I was aiming for. Or maybe to find something like $_processState.complete.IndexNumber()
Having an object like this also lets you assign values by the index name, itself, while standardizing the options...
$thisObject.Status = $_processState.paused
$thisObject.Status
PS > paused
Not really sure this was the best approach at the time or if it still is the best approach with all the custom class options there are available in PS v5.
It can be simpler
It may not be any more efficient than the answer from Frode F., but perhaps more concise (inline) would be simply putting the hash table's keys collection in a sub expression ($()) then calling indexOf on the result.
For your hash table...
Your particular expression would be simply:
$($dictionary.keys).indexOf('c')
...which gives the value 2 as you expected. This also works just as well on a regular hashtable... unless the hashtable is modified in pretty much any way, of course... so it's probably not very useful in that case.
In other words
Using this hash table (which also shows many of the ways to encode 4...):
$hashtable = [ordered]#{
sample = 'hash table'
0 = 'hello'
1 = 'goodbye'
[char]'4' = 'the ansi character 4 (code 52)'
[char]4 = 'the ansi character code 4'
[int]4 = 'the integer 4'
'4' = 'a string containing only the character 4'
5 = "nothing of importance"
}
would yield the following expression/results pairs:
# Expression Result
#------------------------------------- -------------
$($hashtable.keys).indexof('5') -1
$($hashtable.keys).indexof(5) 7
$($hashtable.keys).indexof('4') 6
$($hashtable.keys).indexof([char]4) 4
$($hashtable.keys).indexof([int]4) 5
$($hashtable.keys).indexof([char]'4') 3
$($hashtable.keys).indexof([int][char]'4') -1
$($hashtable.keys).indexof('sample') 0
by the way:
[int][char]'4' equals [int]52
[char]'4' has a "value" (magnitude?) of 52, but is a character, so it's used as such
...gotta love the typing system, which, while flexible, can get really really bad at times, if you're not careful.
Dictionaries uses keys and not indexes. OrderedDictionary combines a hashtable and ArrayList to give you order/index-support in a dictionary, however it's still a dictionary (key-based) collection.
If you need to get the index of an object in a OrderedDictionary (or a hasthable) you need to use foreach-loop and a counter. Example (should be created as a function):
$hashTable = [Ordered]#{
'a' = 'blue';
'b'='green';
'c'='red'
}
$i = 0
foreach($key in $hashTable.Keys) {
if($key -eq "c") { $i; break }
else { $i++ }
}
That's how it works internaly too. You can verify this by reading the source code for OrderedDictionary's IndexOfKey method in .NET Reference Source
For the initial problem I was attempting to solve, a comparable process state, you can now use Enumerations starting with PowerShell v5.
You use the Enum keyword, set the Enumerators by name, and give them an integer value. The value can be anything, but I'm using ascending values starting with 0 in this example:
Enum _ProcessState{
Error = 0
None = 1
Started = 2
Paused = 3
Cleanup = 4
Complete = 5
Verified = 6
}
#the leading _ for the Enum is just cosmetic & not required
Once you've created the Enum, you can assign it to variables. The contents of the variable will return the text name of the Enum, and you can compare them as if they were integers.
$Item1_State = [_ProcessState]::Started
$Item2_State = [_ProcessState]::Cleanup
#return state of second variable
$Item2_state
#comparison
$Item1_State -gt $Item2_State
Will return:
Cleanup
False
If you wanted to compare and return the highest:
#sort the two objects, then return the first result (should return the item with the largest enum int)
$results = ($Item1_State,$Item2_State | Sort-Object -Descending)
$results[0]
Fun fact, you can also use arithmetic on them, for example:
$Item1_State + 1
$Item1_State + $Item2_State
Will return:
Paused
Verified
More info on Enum here:
https://blogs.technet.microsoft.com/heyscriptingguy/2015/08/26/new-powershell-5-feature-enumerations/
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_enum?view=powershell-6
https://psdevopsug.scot/post/working-with-enums-in-powershell/

Powershell - Use multidimention arrays for storing data - Need suggestion

I'm googling since a while, but I didn't find a solution to my problem.
I have to say I'm newbie in Powershell.
I would like to create the following array
$a = (A,B,C,D) where
A = 1 string (always)
B = 1 string (always)
C = undefined number of strings. I need to be able to add elements dynamically
D = undefined number of strings. I need to be able to add elements dynamically (same number as C)
Is this possible?
Example of 2 elements of the array
("WSTM0123456", "192.168.10.155",("WSTM8765421","WSTM9856454","WSTM1289765"),("192.36.36.36", "187.25.25.25","192.69.89.65"))
("WLDN1251254", "156.25.36.54", ("WLDN1234512", "WLDN9865323"), ("187.154.12.12","163.136.25.98"))
I don't know a priori how many elements will be in C and D and I'll have to append strings in position C and D with a for cycle.
Scope: group many strings (C & D) under the same string (A/B) which are in common.
Any help would be appreciated
Thanks,
Marco
You can do this, but it's probably quite painful as dealing with arrays is sometimes cumbersome in PowerShell due to lots of implicit flattening.
I'd suggest creating a custom type for this. Then you can also give the individual parts useful names (I don't know the purpose of what you're doing here, so I'm making up names here. Feel free to change):
$properties = #{
Name = 'WSTM0123456';
IP = [ipaddress]'192.168.10.155';
ListOfNames = #("WSTM8765421","WSTM9856454","WSTM1289765");
ListOfIPs = [ipaddress[]]#("192.36.36.36", "187.25.25.25","192.69.89.65")
}
$foo = New-Object PSObject -Property #properties
Then you can simply append new items like so:
$foo.ListOfNames += 'AnotherName'
I think this is pretty much the same idea. Use a hash table, and make two of the elements arrays. This is how you would create the arrays "on the fly" at runtime, without knowing what any of the contents were going to be in advance, taking $x and putting any item that starts with "t" in "C" , and everything else in "D":
$a = #{A = "Some string";B = "Some other string"}
$x = "one","two","three","four","five"
$x |% {
if ($_ -match "^t"){$a["C"] += #($_)}
else {$a["D"] += #($_)}
}
$a.a
Some string
$a.b
Some other string
$a.c
two
three
$a.d
one
four
five
$obj = new-object psobject -property $a