I need to update a file and remove any string that says "ENCRYPTED = YES", the below command works on a test file but the actual file is 250GB and it runs out of memory. Is there a way to go through part of the file at a time and make the update?
(Get-Content -path "X:\file1.sql") -replace "ENCRYPTED=YES;", ";" | Set-Content -Path X:\file_updated.sql
I'm using a tool called MobaXterm to open up SSH sessions to Linux Virtual Machines. I'm attempting to create an import file of hostnames from a script so I can dynamically create the list of VM's I want to connect to without adding them manually in the MobaXterm Gui. To that end I've created the following PowerShell script that reads in the Hostname and the IP address from a .csv file. The script is working in that an .mxtsessions file is being created and the file appears to be what is exported from my testing an export of sessions file from MobaXterm. Here is my working script:
$csvFilename = 'C:\mobaxterm\mobaXterm.csv'
$outfile = 'C:\mobaxterm\MobaXterm_Sessions.mxtsessions'
$csv = Import-Csv -Path $csvFilename -Delimiter ','
#'
[Bookmarks]
SubRep=
ImgNum=42
'# | Out-File -FilePath $outfile
$output = foreach ($line in $csv) {
"$($line.hostname)= #109#0%$($line.ip)%22%[loginuser]%%-1%-1%%%22%%0%0%0%%%-1%0%0%0%%1080%%0%0%1#MobaFont%10%0%0%0%15%236,236,236%0,0,0%180,180,192%0%-1%0%%xterm%-1%0%0,0,0%54,54,54%255,96,96%255,128,128%96,255,96%128,255,128%255,255,54%255,255,128%96,96,255%128,128,255%255,54,255%255,128,255%54,255,255%128,255,255%236,236,236%255,255,255%80%24%0%1%-1%<none>%%0#0#"
}
$output | Out-File -FilePath $outfile -Append
The import file is simply a .csv file of two columns where column one has the hostname and column 2 has the IP address of each hostname.
As I said my script appears to be working in that it's creating a file that appears to be valid...but when I try to import this .mxtsessions file into MobaXterm it won't load. No errors are shown. Perhaps there's a log I can view for why the import fails?
to further triage this issue I then manually added some machines to my MobaXterm manually and exported the file. I've compared the exported file to the file I've created with my PowerShell script. I'm not seeing any differences between both files. The properties on both files look identical (except for the name of course). The data within each file are identical from my compare.
Can anyone provide some pointers for me on why my generated .mxtsessions file won't load into MobaXterm? I've looked in the MobaXterm.log file and I'm not seeing any errors related to my import? Has anyone else created an import sessions file and successfully imported it into MobaXterm?
Any advice or pointers this forum can provide me would be greatly appreciated.
Thank you.
From just testing it, I think it's a character encoding issue. MobaXTerm import works if I save as ASCII or UTF8-sans-BOM, doesn't work otherwise.
If you only have ASCII characters, try adding an encoding parameter when writing:
'# | Out-File -FilePath $outfile -Encoding ASCII
$output | Out-File -FilePath $outfile -Append -Encoding ASCII
If you need Unicode, there's no way to write it without BOM from PowerShell 5.1 or earlier, so you'll need:
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($outfile, $allyourtextcontent, $Utf8NoBomEncoding)
I've written a script to make lots of regular expression modifications to a file and overwrite the file with the changes. However I have just noticed all the files are being written with UTF-16LE encoding. I realise that I can specify the encoding for the Out-File cmdlet, but I'm not confident that all the input files are the same encoding. Can I get the encoding from Get-Content somehow, or should I be using some other method of I/O? An outline of the script is below
Get-ChildItem *.aspx -Recurse| ForEach-Object {
MakeWritable($_)
(Get-Content -raw $_) | FixUp($_.FullName)
}
Filter FixUp($filename)
{
DoReplacements($_) | Out-File $filename
}
I search a way to do an automated task with Notepad++ from command line:
Open file
Change encoding to UTF-8
Save file
Is there any way to do it with some plugin or even with other program ?
Why do you want to use Notepad++ for that task? Which OS are you using?
Notepad++ got a Plugin-manager where you can install the Python Script plugin.
http://pw999.wordpress.com/2013/08/19/mass-convert-a-project-to-utf-8-using-notepad/
But if you want to convert files to UTF8 you can do that way easier with PowerShell on Windows or command line on Linux.
For Windows Power-Shell:
$yourfile = "C:\path\to\your\file.txt"
get-content -path $yourfile | out-file $yourfile -encoding utf8
For Linux use (e.g.) iconv:
iconv -f ISO-8859-15 -t UTF-8 source.txt > new-file.txt
Windows Powershell script to change all the files in the current folder (and in all subfolders):
foreach ($file in #(Get-ChildItem *.* -File -Recurse)) {
$content = get-content $file
out-file -filepath $file -inputobject $content -encoding utf8
}
If you want to change only specific files just change the *.* (in the first line).
Note: I tried the pipe (|) approach in Broco's answer and was not working (I got empty output files as Josh commented). I think is because we probably cannot read and write directly from and to the same file (while in my approach I put the content into a memory variable).
I am trying to do find and replace in a file using following approach.
Function Find-Replace ($FileFullpath, $FindString, $ReplacementString) {
Get-Content $FileFullpath |
Foreach-Object {$_ -replace $FindString, $ReplacementString } |
Set-Content $FileFullpath
}
Find-Replace "c:\program files (x86)\MyProj\web.config" $OldServiceName $NewServiceName
But i am always getting error.
Set-Content : The process cannot access the file 'c:\program files
(x86)\MyProj\web.config' because it is being used by another process.
The file is not opened any where. I think Get-content is yet to release the file.
Why it happens ? How to do find and replace in the same file without issue?
You can't read and write to the same file while it's open, Get-Content opens the file for reading and in the same time Set-Content tries to write to it. Put the Get-Conetnt call in parentheses, it will open the file, read it's content and close it.
(Get-Content $FileFullpath) | ...