Right now, i am looking to improve my code to use less space and be more intelligent, i need the below code to only backup files IF it's modified by the Find & Replace, right now i'm doing a backup of everything and overwriting old back-ups.
The next thing i would like would be to NOT overwrite the backups, but instead give them a number, so if there are 2 of the same backups in the "backup" folder it would look like this:
Filebackup.DCN3 -> Filebackup1.DCN3
So i always have the original file.
get-childitem -path "C:\Users\Administrator\Desktop\Eurocard\SEB" -filter *.* -recurse | copy-item -destination "C:\Users\Administrator\Desktop\Eurocard\Backup"
(Get-ChildItem "C:\Users\Administrator\Desktop\Eurocard\SEB\*.*" -recurse).FullName |
Foreach-Object {
(Get-Content $_ -Raw).
Replace('*','Æ').
Replace('"','Æ').
Replace('#','Æ').
Replace('¤','Æ').
Replace('&','Æ').
Replace('(','Æ').
Replace(')','Æ').
Replace('=','Æ').
Replace('?','Æ').
Replace('´','Æ').
Replace('`','Æ').
Replace('|','Æ').
Replace('#','Æ').
Replace('£','Æ').
Replace('$','Æ').
Replace('{','Æ').
Replace('[','Æ').
Replace(']','Æ').
Replace('}','Æ').
Replace('^','Æ').
Replace('~','Æ').
Replace('¨','Æ').
Replace('*','Æ').
Replace('<','Æ').
Replace('>','Æ').
Replace('\','Æ').
Replace('_','Æ').
Replace(';','Æ').
Replace('.','Æ').
Replace('!','Æ')|
Set-Content $_
}
Is there anyone who can help with this ?
Well, to start a large portion of your regex replaces probably aren't working, you need to escape most of them...for example "\". Anyways you can shorten up the whole replace to one expression like this:
-replace '[*"#¤&()=?´`|#£${\[\]}^~¨*<>\\_;.!]','Æ'
#query to show it working
'*"#¤&()=?´`|#£${[]}^~¨*<>\_;.!' -replace '[*"#¤&()=?´`|#£${\[\]}^~¨*<>\\_;.!]','Æ'
expanding on that here is how you would get it to only backup if you modify the file:
(Get-ChildItem "C:\Users\Administrator\Desktop\Eurocard\SEB\*.*" -recurse).FullName |
Foreach-Object {
$Content = (Get-Content $_ -Raw)
$Regex = '[*"#¤&()=?´`|#£${\[\]}^~¨*<>\\_;.!]'
If ($Content | Select-String $Regex -Quiet)
{
$Content -Replace $Regex,'Æ'
<#
rest of code block such as copies, backups, renames whatever would go here.
This way it is only taking place if the file has an unwanted character and is
modified
#>
}
}
Related
I have this code that works like a charm for small files. It just dumps the whole file into memory, replaces NUL and writes back to the same file. This is not really very practical for huge files when file size is larger than the available memory. Can someone help me convert it to a streaming model such that it won't choke for huge files.
Get-ChildItem -Path "Drive:\my\folder\path" -Depth 2 -Filter *.csv |
Foreach-Object {
$content = Get-Content $_.FullName
#Replace NUL and save content back to the original file
$content -replace "`0","" | Set-Content $_.FullName
}
The way you have this structured the entire file contents have to be read into memory. Note: That reading a file into memory uses 3-4x the file size in RAM, which's documented here.
Without getting into .Net classes, particularly [System.IO.StreamReader], Get-Content is actually very memory efficient, you just have to leverage the pipeline so you don't build up the data in memory.
Note: if you do decide to try StreamReader, the article will give you some syntax clues. Moreover, that topic has been covered by many others on the web.
Get-ChildItem -Path "C:\temp" -Depth 2 -Filter *.csv |
ForEach-Object{
$CurrentFile = $_
$TmpFilePath = Join-Path $CurrentFile.Directory.FullName ($CurrentFile.BaseName + "_New" + $CurrentFile.Extension)
Get-Content $CurrentFile.FullName |
ForEach-Object{ $_ -replace "`0","" } |
Add-Content $TmpFilePath
# Now that you've got the new file you can rename it & delete the original:
Remove-Item -Path $CurrentFile.FullName
Rename-Item -Path $TmpFilePath -NewName $CurrentFile.Name
}
This is a streaming model, Get-Content is streaming inside the outer ForEach-Object loop. There may be other ways to do it, but I chose this so I could keep track of the names and do the file swap at the end...
Note: Per the same article, in terms of speed Get-Content is quite slow. However, your original code was likely already suffering that burden. Moreover, you can speed it up a bit using the -ReadCount XXXX parameter. That will send some number of lines down the pipe at a time. That of course does use more memory, so you'd have to find a level that helps you say within the boundaries of your available RAM. Performance improvement with -ReadCount is mentioned in this answer's comments.
Update Based on Comments:
Here's an example of using StreamReader/Writer to perform the same operations from the previous example. This should be just as memory efficient as Get-Content, but should be much faster.
Get-ChildItem -Path "C:\temp" -Depth 2 -Filter *.csv |
ForEach-Object{
$CurrentFile = $_.FullName
$CurrentName = $_.Name
$TmpFilePath = Join-Path $_.Directory.FullName ($_.BaseName + "_New" + $_.Extension)
$StreamReader = [System.IO.StreamReader]::new( $CurrentFile )
$StreamWriter = [System.IO.StreamWriter]::new( $TmpFilePath )
While( !$StreamReader.EndOfStream )
{
$StreamWriter.WriteLine( ($StreamReader.ReadLine() -replace "`0","") )
}
$StreamReader.Close()
$StreamWriter.Close()
# Now that you've got the new file you can rename it & delete the original:
Remove-Item -Path $CurrentFile
Rename-Item -Path $TmpFilePath -NewName $CurrentName
}
Note: I have some sense this issue is rooted in encoding. The Stream constructors do accept an encoding enum as an argument.
Available Encodings:
[System.Text.Encoding]::BigEndianUnicode
[System.Text.Encoding]::Default
[System.Text.Encoding]::Unicode
[System.Text.Encoding]::UTF32
[System.Text.Encoding]::UTF7
[System.Text.Encoding]::UTF8
So if you wanted to instantiate the streams with, for example, UTF8:
$StreamReader = [System.IO.StreamReader]::new( $CurrentFile, [System.Text.Encoding]::UTF8 )
$StreamWriter = [System.IO.StreamWriter]::new( $TmpFilePath, [System.Text.Encoding]::UTF8 )
The streams do default to UTF8. I think the system default is typically code page Windows 1251.
This would be the simplest way using the least memory, one line at a time, to another file. But it needs double the disk space.
get-content file.txt | % { $_ -replace "`0" } | set-content file2.txt
I have written this script to find "Procedure division" and replace it with a null value. It seems to work, so that's a good this.
What I need to add is the actual file names that were changed. I have 12K files and only around 800 or so are supposedly needing the change. I need to know what ones were actually changed.
Is there a way to add the path and file name to be displayed in the below script?
$old = Read-Host 'PROCEDURE DIVISION.'
$new = Read-Host ''
Get-Children D:\temp *.cbl -recurse | ForEach {
(Get-Content $_ | ForEach {$_ -replace "$old", "$new"}) | Set-Content $_
}
I have a set of config files stored in each subfolder within a directory. These config files only contain a single string in the format XXX_YYYYMMDD where XXX is a number e.g. 006, 007 etc, so an example string would be 006_20150101. I want the powershell script to replace the XXX number with a new one in each of these config files. I'm using the below script to achieve that and it works fine. However, the issue is that it puts a new line character (ENTER) at the end of the string which I don't want. Any way to fix this?
$sourceDir = "C:\Users\001"
$configFiles = Get-ChildItem $sourceDir *.dat -rec
foreach ($file in $configFiles)
{
(Get-Content $file.PSPath) |
Foreach-Object { $_ -replace "006", "007" } |
Set-Content $file.PSPath
}
By default set-content ends with a newline, use -NoNewline to not have this behavior:
Set-Content -path $file.PSPath -NoNewline
I dont know if u can use this but you can use regex replace to match the first 3 digits in the string:
$regex = "^\d{3}"
# matches any 3 digits("\d{3}") at the beginning("^") of a string
"124_20201030" -replace $regex, "007"
I want to use PowerShell to generate a list of commands to move files from one location to another. (I'm sure PowersSell could actually do the moving, but I'd like to see the list of commands first ... and yes I know about -WhatIf).
The files are in a series of subfolders one layer down, and need moved to a corresponding series of subfolders on another host. The subfolders have 8-digit identifiers. I need a series of commands like
move c:\certs\40139686\22_05_2018_16_23_Tyre-Calligraphy.jpg \\vcintra2012\images\40139686\Import\22_05_2018_16_23_Tyre-Calligraphy.jpg
move c:\certs\40152609\19_02_2018_11_34_Express.JPG \\vcintra2012\images\40152609\Import\19_02_2018_11_34_Express.JPG
The file needs to go into the \Import subdirectory of the corresponding 8-digit-identifier folder.
The following Powershell will generate the data that I need
dir -Directory |
Select -ExpandProperty Name |
dir -File |
Select-Object -Property Name, #{N='Parent';E={$_.Directory -replace 'C:\\certs\\', ''}}
40139686 22_05_2018_16_23_Tyre-Calligraphy.jpg
40152609 19_02_2018_11_34_Express.JPG
40152609 Express.JPG
40180489 27_11_2018_11_09_Appointment tuesday 5th.jpg
but I am stuck on how to take that data and generate the concatenated string which in PHP would look like this
move c:\certs\$Parent\$Name \\vcintra2012\images\$Parent\Import\$Name
(OK, the backslashes would likely need escaped but hopefully it is clear what I want)
I just don't know to do this sort of concatenation of columnar output - any SO refs I look at e.g.
How do I concatenate strings and variables in PowerShell?
are not about how to do this.
I think I need to pipe the output to an expression that effects the concatenation, perhaps using -join, but I don't know how to refer to $Parent and $Name on the far side of the pipe?
Pipe your output into a ForEach-Object loop where you build the command strings using the format operator (-f):
... | ForEach-Object {
'move c:\certs\{0}\{1} \\vcintra2012\images\{0}\Import\{1}' -f $_.Parent, $_.Name
}
Another approach:
$source = 'C:\certs'
$destination = '\\vcintra2012\images'
Get-ChildItem -Path $source -Depth 1 -Recurse -File | ForEach-Object {
$targetPath = [System.IO.Path]::Combine($destination, $_.Directory.Name , 'Import')
if (!(Test-Path -Path $targetPath -PathType Container)) {
New-Item -Path $targetPath -ItemType Directory | Out-Null
}
$_ | Move-Item -Destination $targetPath
}
I have a problem with change extension of a file. I need to write a script which is replicating data, but data have two files. Filename is not a string, so we can't use normal -replace
I need to get from
filename.number.extension
this form
filename.number.otherextension
We try to use a split, but this command show us things like below
filename
number
otherextension
Thanks for any ideas,
[System.IO.Path]::ChangeExtension("test.old",".new")
You probably want something like the -replace operator:
'filename.number.extension' -replace 'extension$','otherextension'
The $ is regular expression syntax meaning end of line. This should ensure that the -replace does not match "extension" appearing elsewhere in the filename.
A simple Utility Function
<#
# Renames all files under the given path (recursively) whose extension matches $OldExtension.
# Changes the extension to $NewExtension
#>
function ChangeFileExtensions([string] $Path, [string] $OldExtension, [string] $NewExtension) {
Get-ChildItem -Path $Path -Filter "*.$OldExtension" -Recurse | ForEach-Object {
$Destination = Join-Path -Path $_.Directory.FullName -ChildPath $_.Name.Replace($OldExtension, $NewExtension)
Move-Item -Path $_.FullName -Destination $Destination -Force
}
}
Usage
ChangeFileExtensions -Path "c:\myfolder\mysubfolder" -OldExtension "extension" -NewExtension "otherextension"
But it can do more than just this. If you had the following files in the same folder as your script
example.sample.csv
example.txt
mysubfolder/
myfile.sample.csv
myfile.txt
this script would rename all the .sample.csv files to .txt files in the given folder and all subfolders and overwrite any existing files with those names.
# Replaces all .sample.csv files with .txt extensions in c:\myfolder and in c:\myfolder\mysubfolder
ChangeFileExtensions -Path "c:\myfolder" -OldExtension "sample.csv" -NewExtension "txt"
If you don't want it to be recursive (affecting subfolders) just change
"*.$OldExtension" -Recurse | ForEach-Object
to
"*.$OldExtension" | ForEach-Object
This could work:
Get-ChildItem 'C:\Users\Administrator\Downloads\text files\more text\*' *.txt | rename-item -newname { [io.path]::ChangeExtension($_.name, "doc") }
You can remove the last item with the the [0..-1] slice and add the new extension to that
(("filename.number.extension" -split "\.")[0..-1] -join '.') +".otherextension"