I am new to scripting, and I would like to ask you help in the following:
This script should be scheduled task, which is working with Veritas NetBackup, and it creates a backup register in CSV format.
I am generating two source files (.csv comma delimited):
One file contains: JobID, FinishDate, Policy, etc...
The second file contains: JobID, TapeID
It is possible that in the second file there are multiple same JobIDs with different TapeID-s.
I would like to reach that, the script for each line in source file 1 should check all of the source file 2 and if there is a JobID match, if yes, it should have the following output:
JobID,FinishDate,Policy,etc...,TapeID,TapeID....
I have tried it with the following logic, but sometimes I have no TapeID, or I have two same TapeID-s:
Contents of sourcefile 1 is in $BackupStatus
Contents of sourcefile 2 is in $TapesUsed
$FinalReport =
foreach ($FinalPart1 in $BackupStatus) {
write-output $FinalPart1
$MediaID =
foreach ($line in $TapesUsed){
write-output $line.split(",")[1] | where-object{$line.split(",")[0] -like $FinalPart1.split(",")[0]}
}
write-output $MediaID
}
If the CSV files are not huge, it is easier to use Import-Csv instead of splitting the files by hand:
$BackupStatus = Import-Csv "Sourcefile1.csv"
$TapesUsed = Import-Csv "Sourcefile2.csv"
This will generate a list of objects for each file. You can then compare these lists quite easily:
Foreach ($Entry in $BackupStatus) {
$Match = $TapesUsed | Where {$_.JobID -eq $Entry.JobID}
if ($Match) {
$Output = New-Object -TypeName PSCustomObject -Property #{"JobID" = $Entry.JobID ; [...] ; "TapeID" = $Match.TapeID # replace [...] with the properties you want to use
Export-Csv -InputObject $Output -Path <OUTPUTFILE.CSV> -Append -NoTypeInformation }
}
This is a relatively verbose variant, but I prefer it like this.
I am checking for each entry in the first file whether there is a matching entry in the second. If there is one I combine the required fields from the entry of the first list with the ones from the entry in the second list into one object that I can then export very comfortably using Export-Csv.
Related
I have a CSV file that generates everyday, and generates with data such as:
windows:NT:v:n:n:d:n:n:n:n:m:n:n
I should also mention that that example is one of 3,900+ lines, and not every line of data has the same number of "columns". What I'm trying to do is import just the first two "columns" of data into a variable. For this example, it would be "Windows" and "NT", nothing else.
How would I go about doing this? I've tried using -delimiter ':', and not much luck.
The number of lines shouldn't matter.
My approach from comment (to your previous question) should work,
if there is no header and you only want the first two columns,
just specify Header 1,2
> import-csv .\strange.csv -delim ':' -Header (1..2) |Where 2 -eq 'NT'
1 2
- -
windows NT
Example for building the entire array
$Splitted_List = #()
foreach($Line in Get-Content '.\myfilewithuseragents.txt'){
$Splitted = $Line -split ":"
$Splitted_Object = [PSCustomObject]#{
$part1 = $splitted[0]
$part2 = $Splitted[1]
}
$Splitted_List.Add($Splitted_Object) | Out-Null
}
For every line you'll just read the line and with the string from that line, you're easily able to split it
$useragent = "windows:NT:v:n:n:d:n:n:n:n:m:n:n"
Then the first part will be referenced to as $useragent.Split(":")[0], the second as $useragent.Split(":")[1], etc.
Including the for-loop that would be something like
foreach($useragent in Get-Content '.\myfilewithuseragents.txt') {
$splitted = $useragent.Split(":")
$part1 = $splitted[0]
}
I am currently trying to write a powershell script that can be run weekly on two CSV files, to check they both contain the same information. I want the script to output anything that appears in one file but not the other to a new file.
The script I have written so far compares the two but only adds <= and => to the values.
It also doesn't work all the time, because I manually checked the file and found results that existed in both.
Code below:
$NotPresents = compare-object -ReferenceObject $whatsup -DifferenceObject $vmservers -Property device
foreach ($NotPresent in $NotPresents)
{
Write-Host $NotPresent.device
}
$NotPresents | Out-File Filepath.txt
$NotPresents.count
Any ideas what I have done wrong?
In order to avoid having to iterate over one of the arrays more than once*, you may want to throw them each into a hashtable:
$whatsupTable = #{}
foreach($entry in $whatsup){
$whatsupTable[$entry.device] = $true
}
$vmserversTable = #{}
foreach($entry in $vmservers){
$vmserversTable[$entry.device] = $true
}
Now you can easily find the disjunction with a single loop and a lookup against the other table:
$NotInWhatsUp = $vmservers |Where { -not $whatsupTable[$_] }
$NotInVMServers = $whatsup |Where { -not $vmserversTable[$_] }
*) ok, technically we're looping through each twice, but still much better than nested looping
I'm trying to write a simple usage logger into my script that would store information about the time when user opened the script, finished using the script and the user name.
The first part of the logger where I gather the first two data works fine and adds two necessary columns with values to the CSV file. Yet when I run the second part of the logger it does not add a new column to my existing CSV file.
#Code I will add at the very beginning of my script
$FileNameDate = Get-Date -Format "MMM_yyyy"
$FilePath = "C:\Users\Username\Desktop\Script\Logs\${FileNameDate}_MonthlyLog.csv"
$TimeStamp = (Get-Date).toString("dd/MMM/yyyy HH:mm:ss")
$UserName = [string]($env:UserName)
$LogArray = #()
$LogArrayDetails = #{
Username = $UserName
StartDate = $TimeStamp
}
$LogArray += New-Object PSObject -Property $LogArrayDetails | Export-Csv $FilePath -Notypeinformation -Append
#Code I will add at the very end of my script
$logArrayFinishDetails = #{FinishDate = $TimeStamp}
$LogCsv = Import-Csv $FilePath | Select Username, StartDate, #{$LogArrayFinishDetails} | Export-Csv $FilePath -NoTypeInformation -Append
CSV file should look like this when the script is closed:
Username StartDate FinishDate
anyplane 08/Apr/2018 23:47:55 08/Apr/2018 23:48:55
Yet it looks like this:
StartDate Username
08/Apr/2018 23:47:55 anyplane
The other weird thing is that it puts the StartDate first while I clearly stated in $LogArrayDetails that Username goes first.
Assuming that you only ever want to record the most recent run [see bottom if you want to record multiple runs] (PSv3+):
# Log start of execution.
[pscustomobject] #{ Username = $env:USERNAME; StartDate = $TimeStamp } |
Export-Csv -Notypeinformation $FilePath
# Perform script actions...
# Log end of execution.
(Import-Csv $FilePath) |
Select-Object *, #{ n='FinishDate'; e={ (Get-Date).toString("dd/MMM/yyyy HH:mm:ss") } } |
Export-Csv -Notypeinformation $FilePath
As noted in boxdog's helpful answer, using -Append with Export-Csv won't add additional columns.
However, since you're seemingly attempting to rewrite the entire file, there is no need to use
-Append at all.
So as to ensure that the old version of the file has been read in full before you attempt to replace it with Export-Csv, be sure to enclose your Import-Csv $FilePath call in (...), however.
This is not strictly necessary with a 1-line file such as in this case, but a good habit to form for such rewrites; do note that this approach is somewhat brittle in general, as something could go wrong while rewriting the file, resulting in potential data loss.
#{ n='FinishDate'; e={ (Get-Date).toString("dd/MMM/yyyy HH:mm:ss") } is an example of a calculated property/column that is appended to the preexisting columns (*)
The other weird thing is that it puts the StartDate first while I clearly stated in $LogArrayDetails that Username goes first.
You've used a hashtable (#{ ... }) to declare the columns for the output CSV, but the order in which a hashtable's entries are enumerated is not guaranteed.
In PSv3+, you can use an ordered hashtable instead ([ordered] #{ ... }) to achieve predictable enumeration, which you also get if you convert the hashtable to a custom object by casting to [pscustomobject], as shown above.
If you do want to append to the existing file, you can use the following, but note that:
this approach does not scale well, because the entire log file is read into memory every time (and converted to objects), though limiting the entries to a month's worth should be fine.
as stated, the approach is brittle, as things can go wrong while rewriting the file; consider simply writing 2 rows per execution instead, which allows you to append to the file line by line.
there's no concurrency management, so the assumption is that only ever one instance of the script is run at a time.
$FilePath = './t.csv'
$TimeStamp = (Get-Date).toString("dd/MMM/yyyy HH:mm:ss")
$env:USERNAME = $env:USER
# Log start of execution. Note the empty 'FinishDate' property
# to ensure all rows ultimately have the same column structure.
[pscustomobject] #{ Username = $env:USERNAME; StartDate = $TimeStamp; FinishDate = '' } |
Export-Csv -Notypeinformation -Append $FilePath
# Perform script actions...
# Log end of execution:
# Read the entire existing file...
$logRows = Import-Csv $FilePath
# ... update the last row's .FinishDate property
$logRows[-1].FinishDate = (Get-Date).toString("dd/MMM/yyyy HH:mm:ss")
# ... and rewrite the entire file, keeping only the last 30 entries
$logRows[-30..-1] | Export-Csv -Notypeinformation $FilePath
Because your CSV already has a structure (i.e. defined headers), PowerShell honours this when appending and doesn't add additional columns. It is (sort of) explained in this excerpt from the Export-Csv help:
When you submit multiple objects to Export-CSV, Export-CSV organizes
the file based on the properties of the first object that you submit.
If the remaining objects do not have one of the specified properties,
the property value of that object is null, as represented by two
consecutive commas. If the remaining objects have additional
properties, those property values are not included in the file.
You could include the FinishDate property in the original file (even though it would be empty), but the best option might be to export your output to a different CSV at the end, perhaps deleting the original after import then recreating it with the additional data. In fact, just removing the -Append will likely give the result you want.
I have a CSV that contains a username, and then one or more values for the rest of the record. There are no headers in the file.
joe.user,Accounting-SG,CustomerService-SG,MidwestRegion-SG
frank.user,Accounting-SG,EastRegion-SG
I would like to read the file into a powershell object where the Username property is set to the first column, and the Membership property is set to either the remainder of the row (including the commas) or ideally, an array of strings with each element containing a single membership value.
Unfortunately, the following line only grabs the first membership and ignores the rest of the line.
$memberships = Import-Csv -Path C:\temp\values.csv -Header "username", "membership"
#{username=joe.user; membership=Accounting-SG}
#{username=frank.user; membership=Accounting-SG}
I'm looking for either of these outputs:
#{username=joe.user; membership=Accounting-SG,CustomerService-SG,MidwestRegion-SG}
#{username=frank.user; membership=Accounting-SG,EastRegion-SG}
or
#{username=joe.user; membership=string[]}
#{username=frank.user; membership=string[]}
I've been able to get the first result by enclosing the "rest" of the data in the csv file in quotes, but that doesn't really feel like the best answer:
joe.user,"Accounting-SG,CustomerService-SG,MidwestRegion-SG"
Well, the issue is that what you have isn't really a (proper) CSV. The CSV format doesn't support that notation.
You can "roll your own" and just process the file yourself, something like this:
$memberships = Get-Content -LiteralPath C:\temp\values.csv |
ForEach-Object -Process {
$user,$membership = $_.Split(',')
New-Object -TypeName PSObject -Property #{
username = $user
membership = $membership
}
}
You could do a half and half sort of thing. Using your modification, where the groups are all a single field in quotes, do this:
$memberships = Import-Csv -Path C:\temp\values.csv -Header "username", "membership" |
ForEach-Object -Process {
$_.membership = $_.membership.Split(',')
$_
}
The first example just reads the file line by line, splits on commas, then creates a new object with the properties you want.
The second example uses Import-Csv to create the object initially, then just resets the .membership property (it starts as a string, and we split the string so it's now an array).
The second way only makes sense if whatever is creating the "CSV" can create it that way in the first place. If you have to modify it yourself every time, just skip this and process it as it is.
I am very, very, very new to Powershell. I was wondering if any one an help me with the following script:
The idea is to have two excel spreadsheets.
1.csv
QCODE
PC1009
PC1009
PC1011
PC1012
2.csv
QCODE
PC1009
PC1009
PC1009
PC1012
I am trying to compare values between the two CSV documents. If the value in cell1 in 1.csv is equal to any cell in 2.csv the script must perform a certain action, once the action is finished it must loop over to cell2 in 1.csv and compare it again with all the values in 2.csv
This is about as far as I have managed yo get:
$CSV=Import-Csv C:\1.csv
$COMP=Import-Csv C:\2.csv
$count=0
$cnt=0
while($count -le $CSV.Count)
{
while($validator -eq $false)
{
if($CSV[$count].QCODE -eq $COMP[$cnt].QCODE)
{
Write-Host "Exiting"
$validator=$true
}
else{
$cnt++
}
}
$count++
}
It's a mess, I apologize. Any help would be greatly appreciated
Here is a solution for you. I have created two CSV files with matching headers. The column names are:
Prop1
Prop2
Prop3
Prop4
Prop5
When these lines are imported into PowerShell, it will automatically create a PSObject for each line. The property names on the PSObject will be the column headers. These two CSV files exist in the folder named c:\test.
NOTE: There is a single, mismatching value between the two files, in the dead middle. This will be our test.
The code looks like this. There are some in-line comments to help guide you. Basically, we're dynamically querying all of the property (column) names, getting the value of each one (the cell values), and comparing them. If they do not match, we throw a warning. Based on the single, mismatching "cell" in this example, the output I get is in a screenshot below. It seems to be working quite well in my testing.
NOTE: Even though it says that line #1 is mismatching, and you might think it's line #2, that's because arrays are zero-based. Therefore, in array terminology, #1 is actually #2, because it starts counting at zero.
# Import both CSV files
$Csv1 = Import-Csv -Path C:\test\csv1.csv;
$Csv2 = Import-Csv -Path C:\test\csv2.csv;
# For each line in CSV1 ...
foreach ($Line1 in $Csv1) {
$LineNumber = $Csv1.IndexOf($Line1);
# Get the same line from CSV2
$Line2 = $Csv2[$LineNumber];
# For each property (column) ...
foreach ($Property in (Get-Member -InputObject $Line1 -MemberType NoteProperty)) {
# Get the property's name
$PropertyName = $Property.Name;
# If the value of the property doesn't match each CSV file ..
if ($Line1.$PropertyName -ne $Line2.$PropertyName) {
# Warn the user
Write-Warning -Message ('Value of property {0} did not match for line # {1}' -f $PropertyName, $LineNumber);
# PERFORM SOME CUSTOM ACTION HERE
};
}
}