This is my sample code below:
foreach ($var1 in (gc 1.txt)){
// my code logic
}
Here 1.txt file contains list of values like abc, xyz, pqr etc..,like 100 lines in 1.txt file for processing one by one. After processing got completed for 100 lines, there is a new value altered in middle for processing and the count now is 101 lines. Now i need to restart the script and should process newly added value rather than starting from the scratch.
Actually for this requirement there are hundreds of lines are there in my project.
Can you please suggest me the best way to achieve this?
Thanks in advance
Try this out - saving a snapshot of the file and comparing it to the current list:
# Import values and list of completed updates
$ToUpdate = Get-Content list.txt
$Completed = Get-Content completed.txt
# Take a snapshot for next run
Copy-Item -Path list.txt -Destination completed.txt -Force
# use Compare to determine which updates are new
$Unprocessed = Compare-Object $ToUpdate $Completed |
where SideIndicator -EQ '<=' |
Select -ExpandProperty InputObject
Foreach ($var1 in $Unprocessed) {
# Do Stuff
}
Ideally, you want to check against your actual target instead of a log file. If you're updating a database/user directory/grocery list, query that instead, because otherwise you can run into issues. What if something else updates the target while you're not looking? What if your script errors out and doesn't actually complete all the updates? Something to keep in mind.
Related
I would like to remove last line of few files using PowerShell command. I saw the following which does the same thing for the first line of the files. How can I modify that to remove the last line of the files?
gci *.txt | % { (gc $_) | ? { (1) -notcontains $_.ReadCount } | sc -path $_ }
I will appreciate it if I also get the explanation for the commands.
Cheers,
Siavoush
I haven't try anything yet.
Undoubtedly there are a number of solutions for this.
The below one should perform quite fast:
# create a List object to store the contents of the files
$list = [System.Collections.Generic.List[string]]::new()
Get-ChildItem -Path 'X:\Where\The\Files\Are' -Filter '*.txt' -File | ForEach-Object {
$list.AddRange([System.IO.File]::ReadAllLines($_.FullName))
$list.RemoveAt($list.Count - 1)
[System.IO.File]::WriteAllLines($_.FullName, $list)
$list.Clear()
}
Some explanation to the code above:
Because an array is static, adding or removing elements from it is a time and memory consuming thing to do because the entire array would need to be recreated in memory.
This is why I chose to wrk with the System.Collections.Generic.List object that is optimized for adding or removing elements.
Instead of using Get-Content, I chose to use .Net [System.IO.File]::ReadAllLines() method because that has less overhead and performs faster when reading an entire text file as array of lines
$list.RemoveAt($list.Count - 1) takes away the last element (line) from the array of lines
[System.IO.File]::WriteAllLines($_.FullName, $list) rewrites the lines (minus the last because we removed that) back to the file, overwriting the original content
finally, $list.Clear() empties the List object to get it ready for the next file
first time poster here, but you helped me a lot before. I don´t know how to ask google this question.
I have a powershell script, where with foreach command i check every computer in .txt that contains computer names. (Short explanation is that in check bitlocker status, connection avalability etc.) Everything works fine, but since I fall in love with powershell recently and try to automate more and more thing, that i should upgrade this script little more.
I have foreach ($DestinationComputer in $DestinationComputers) and after i check everything i wanted, i want to delete that row in .txt file.
Can someone help? I am still learning this and got stuck.
Continuing from my comment, I suggest creating a list of computernames that did not process correctly, while discarding the ones that did not fail.
By doing so, you will effectively remove the items from the text file.
Something like this:
$DestinationComputers = Get-Content -Path 'X:\somewhere\computers.txt'
# create a list variable to store computernames in
$list = [System.Collections.Generic.List[string]]::new()
# loop over the computer names
foreach ($DestinationComputer in $DestinationComputers) {
# first check: is the machine available?
$succes = Test-Connection -ComputerName $DestinationComputer -Count 1 -Quiet
if ($succes) {
# do whatever you need to do with that $DestinationComputer
# if anything there fails, set variable $success to $false
<YOUR CODE HERE>
}
# test if we processed the computer successfully and if not,
# add the computername to the list. If all went OK, we do not
# add it to the list, thus removing it from the input text file
if (-not $success) {
$list.Add($DestinationComputer)
}
}
# now, write out the computernames we collected in $list
# conputernames that were processed OK will not be in there anymore.
# I'm using a new filename so we don't overwrite the original, but if that is
# what you want, you can set the same filename as the original here.
$list | Set-Content -Path 'X:\somewhere\computers_2.txt'
I have a Powershell script that keeps track of an inventory text file. I want to update the text file with the script. The following line will remove an item from the inventory as requested by the user:
Get-Content inventory | Where-Object {$_ -notmatch "$removed"} | Set-Content inventory.new
It works just fine, but I want to update the original inventory file, i.e. I don't want to make an "inventory.new," I just want to update "inventory." If I replace inventory.out with inventory I am presented with the following error:
Set-Content : The process cannot access the file 'path\inventory' because it is being used by another process.
The funny thing is that I am able to remove inventory, then rename inventory.out to inventory. Again, this works but isn't ideal. I felt silly just testing it. There must be a more elegant solution!
My understanding of it is that when you open the file with Get-Content, the file is in use until the end of that command, so it won't let you make changes until Get-Content lets go of the file.
An easy workaround might be:
$inventory = Get-Content inventory | Where-Object {$_ -notmatch "$removed"}
$inventory | Set-Content inventory
I'm finishing a script in PowerShell and this is what I must do:
Find and retrieve all .txt files inside a folder
Read their contents (there is a number inside that must be less than 50)
If any of these files has a number greater than 50, change a flag which will allow me to send a crit message to a monitoring server.
The piece of code below is what I already have, but it's probably wrong because I haven't given any argument to Get-Content, it's probably something very simple, but I'm still getting used to PowerShell. Any suggestions? Thanks a lot.
Get-ChildItem -Path C:\temp_erase\PID -Directory -Filter *.txt |
ForEach-Object{
$warning_counter = Get-Content
if ($warning_counter -gt '50')
{
$crit_counter = 1
Write-Host "CRITICAL: Failed to kill service more than 50 times!"
}
}
but it's probably wrong because I haven't given any argument to Get-Content
Yes. That is the first issue. Have a look at Get-Help <command> and or docs like TechNet when you are lost. For the core cmdlets you will always see examples.
Second, Get-Content, returns string arrays (by default), so if you are doing a numerical comparison you need to treat the value as such.
Thirdly you have a line break between foreach-object cmdlet and its opening brace. That will land you a parsing problem and PS will prompt for the missing process block. So changing just those mentioned ....
Get-ChildItem -Path C:\temp_erase\PID -Directory -Filter *.txt | ForEach-Object{
[int]$warning_counter = Get-Content $_.FullName
if ($warning_counter -gt '50')
{
$crit_counter = 1
Write-Host "CRITICAL: Failed to kill service more than 50 times!"
}
}
One obvious thing missing from this is you do not show which file triggered the message. You should update your notification/output process. You also have no logic validating file contents. The could easily fail, either procedural or programically, on files with non numerical contents.
I'm working with an application that creates a log file. Due to an error in the software itself, it keeps producing three errors I'm not interested in. Each line has a unique identifier so I can't just replace the line since each one is different.
I have two main issues with this: I need to save it with the same name, and while it works the file should be available (in case the logger needs to write something).
I can't hard-code the original app to prevent it from writing that part of the log.
I have tried so far:
Get-Content log.log | Where-Object {$_-notmatch 'ERROR1' -And $_-notmatch 'ERROR2' -And $_-notmatch 'ERROR3' } `|Set-Content log_stripped.log
^ It only works if the output file has a different name.
Get-Content error.log | foreach-object { Where-Object {$_-notmatch 'ERROR1' -And $_-notmatch 'ERROR2' -And $_-notmatch 'ERROR3' } } | Set-Content error.log
^ This one froze my PS session.
I also tried reading the file to a variable:
$logcontent = ${h:error.log}
but I got System.OutOfMemoryException.
Ideally, what I need is something that reads the log file, takes away all the lines I don't want, and then save it with its original name.
Ideas? (Keep in mind that the log file is +/- 900 MB with the unnecesary data and 45mb once I strip the data with the first method - but I need it to save the file with its original name)
You can't save the file back to the same name while you're still reading from it, which means you'd have to read the whole 900MB into memory before you start writing. Not a good idea.
Try this:
Remove-Item log_stripped.log
Get-Content log.log -ReadCount 1000 |
foreach {$_ -notmatch 'ERROR1|ERROR2|ERROR3' | Add-Content log_stripped.log }
Remove-item log.log
Rename-Item log_stripped.log log.log
I know you said you want to save to the same filename, but if the reason you want that is that you want the log to be continuously updated, then you could do the following:
Get-Content -Wait log.log |
? {$_ -notmatch 'ERROR1|ERROR2|ERROR3' } |
Out-File log_stripped.log
Note the -Wait on the Get-Content.
log_stripped.log will be continuously updated as log.log is updated.