I'm encountering an error when I try and run md5sum -c on a checksum file I generated. I should mention that I'm running this from PowerShell (as a script will be running this eventually) and this is the cygwin version of md5sum.
I have a test file, jira_defect.txt and I've created a checksum like this:
md5sum jira_defect.txt > result.md5
This gives a file with the following:
7d559b59459052f274e290b5e01a5485 *jira_defect.txt
But when I run
md5sum -c result.md5
I get the infamous error message
result.md5: no properly formatted MD5 checksum lines found
I've tried this again with the -t option, which removes the asterisk, but this hasn't made a difference.
Using the redirection operator to write the checksums to an output file causes the file to be created with the default encoding (Unicode). md5sum expects an ASCII file. Use Set-Content (or Out-File) to save the file with ASCII encoding:
md5sum jira_defect.txt | Set-Content result.md5 -Encoding ASCII
You can also work with Unicode files if you pipe their content into md5sum:
Get-Content result.md5 | md5sum -c
Demonstration:
PS C:\> md5sum .\test.ps1 > result.md5
PS C:\> md5sum -c .\result.md5
C:\md5sum.exe: .\result.md5: no properly formatted MD5 checksum lines found
PS C:\> Get-Content .\result.md5 | md5sum -c
.\test.ps1: OK
PS C:\> md5sum .\test.ps1 | Set-Content result.md5 -Encoding ASCII
PS C:\> md5sum -c .\result.md5
.\test.ps1: OK
Related
On my Windows 10 PC, there are three files, 10GB each, that I want to merge via cat file_name_prefix* >> some_file.zip. However, the output file grew as much as 38GB large before I aborted the operation via Ctrl+C. Is this expected behavior? If not, where am I making a mistake?
Cat is an alias of Get-Content which assumes text files by default - the output size is probably due to this conversion. You can try adding the -raw switch for binary files - this might work? (not sure)
Its definitely possible to "cat" binary files together with a CMD shell using the copy command like below.
copy /b part1.bin+part2.bin+part3.bin some_file.zip
(The 3 part*.bin are the files to be combined into some_file.zip).
PowerShell's cat A.K.A Get-Content reads text file content into an array of strings by default. It also reads the file and checks for the BOM to handle encodings properly if you don't specify a charset. That means it won't work with binary files
To combine binary files in PowerShell 6+ you need to use the -AsByteStream parameter
Get-Content -AsByteStream file_name_prefix* | `
Set-Content -AsByteStream some_file.zip # or
Get-Content -AsByteStream file1, file2, file3 | `
Set-Content -AsByteStream some_file.zip
Older PowerShell doesn't have that option so the only thing you can use is -Raw
Get-Content -Raw file_name_prefix* | Set-Content -Raw some_file.zip
However it'll be very slow because the input files are still treated as text files and read line-by-line. For speed you'll need to use other solutions, like calling Win32 APIs directly from PowerShell
Update:
As mentioned, there's only -Raw in Get-Content, not in Set-Content and it's unsuitable for binary content. You need to use -Encoding Byte
Get-Content -Encoding Byte file_name_prefix* | Set-Content -Encoding Byte some_file.zip
See
Fast and simple binary concatenate files in Powershell
Concatenate files using PowerShell
It is probably going in a loop, recursively concatenating all files including the result to the result file (with the glob wildcard).
You can add an extension in the glob, temporarily save it as another extension and move it to the correct one. (As suggested in: https://stackoverflow.com/a/53079166/12657997)
E.g. when you have 3 files:
a.txt with a inside
b.txt with b inside
c.txt with c inside
cat *.txt > res.csv ; mv res.csv res.txt
cat .\res.txt
a
b
c
Edit
This cat command (as shown above), in combination with the output redirect > will increase the result text file as #mklement0 points out.
According to the documentation (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-content?view=powershell-7.1):
-Encoding
Specifies the type of encoding for the target file. The default value is utf8NoBOM.
However the encoding with the output redirect changes the ecoding, as explained in this post: https://stackoverflow.com/a/40098904/12657997
To illustrate this I've converted the a.txt, b.txt and c.txt to zip files (now they are in a binary format).
cat -Encoding Byte *.zip > res.csv ; mv res.csv res2.txt
cat -Raw *.zip > res.csv ; mv res.csv res3.txt
ls .
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 15/03/2021 21:29 109 a.zip
-a---- 15/03/2021 21:29 109 b.zip
-a---- 15/03/2021 21:29 109 c.zip
-a---- 15/03/2021 21:39 2282 res2.txt
-a---- 15/03/2021 21:41 668 res3.txt
We can see that the output size doubles in size for res3.txt (for every utf-8 byte read utf-16 will output 2.
The -Encoding Byte output, in combination with the output redirect, will make it even worse.
I want to convert some tiff's into multipage tiff with powershell and nconvert.
I started script with
powershell script.ps1 \Path\to\tiffs
Content of script.ps1 is
$path=$args[0]
$File = dir $path\*.tif | foreach {$_.name}
$File | Out-File debug.txt
nconvert.exe -overwrite -o binder.tiff -multi -out tiff -c 7 -l $File
And get error only for the first first line of $File!
My hypotesis, the first line has an BOM. Now my question is How can I convert my $File variable into ansi, or utf8 without BOM?
I'm currently working on a script which will read a mail from a certain sender, who will be sending some commands for the scripts to run.
The Main Idea:
Sender sends a mail with a command for example: ipconfig /all > result.txt and the script running at the recipients side copies this command to a .bat file and running the .bat file to process the command.
The Code:
$junk = $routlook.GetDefaultFolder(23)
$MI = $junk.items
foreach($m in $MI)
{
if($m.SenderEmailAddress -eq '<sender-address>')
{
Echo "#ECHO OFF" > com.bat
Echo $MI.item(1).Body > com.bat
.\com.bat
}
break
}
The Error:
Your problem is that you are not specifying the encoding of your output file.
The easiest way to fix this is to use the Out-File cmdlet and specify the encoding yourself. Note that the 2nd and subsequent calls to Out-File must specify the -Append parameter or you will overwrite your file.
$> "#ECHO OFF" | Out-File -FilePath cmd.bat -Encoding ascii
$> "Echo hi" | Out-File -FilePath cmd.bat -Encoding ascii -Append
$> .\cmd.bat
I search a way to do an automated task with Notepad++ from command line:
Open file
Change encoding to UTF-8
Save file
Is there any way to do it with some plugin or even with other program ?
Why do you want to use Notepad++ for that task? Which OS are you using?
Notepad++ got a Plugin-manager where you can install the Python Script plugin.
http://pw999.wordpress.com/2013/08/19/mass-convert-a-project-to-utf-8-using-notepad/
But if you want to convert files to UTF8 you can do that way easier with PowerShell on Windows or command line on Linux.
For Windows Power-Shell:
$yourfile = "C:\path\to\your\file.txt"
get-content -path $yourfile | out-file $yourfile -encoding utf8
For Linux use (e.g.) iconv:
iconv -f ISO-8859-15 -t UTF-8 source.txt > new-file.txt
Windows Powershell script to change all the files in the current folder (and in all subfolders):
foreach ($file in #(Get-ChildItem *.* -File -Recurse)) {
$content = get-content $file
out-file -filepath $file -inputobject $content -encoding utf8
}
If you want to change only specific files just change the *.* (in the first line).
Note: I tried the pipe (|) approach in Broco's answer and was not working (I got empty output files as Josh commented). I think is because we probably cannot read and write directly from and to the same file (while in my approach I put the content into a memory variable).
I have a problem while trying to batch convert the encoding of some files from ISO-8859-1 to UTF-8 using iconv in a powershell script.
I have this bat file, that works ok:
for %%f in (*.txt) do (
echo %%f
C:\"Program Files"\GnuWin32\bin\iconv.exe -f iso-8859-1 -t utf-8 %%f > %%f.UTF_8_MSDOS
)
I need to convert all files on the directories structure, so I programmed this other script, this time using powershell:
Get-ChildItem -Recurse -Include *.java |
ForEach-Object {
$inFileName = $_.DirectoryName + '\' + $_.name
$outFileName = $inFileName + "_UTF_8"
Write-Host Convirtiendo $inFileName -> $outFileName
C:\"Program Files"\GnuWin32\bin\iconv.exe -f iso-8859-1 -t utf-8 $inFileName > $outFileName
}
And using this the result is the files be converted to UTF-16. I have no clue about what I am doing wrong.
Could anyone help me with this? Could be it some kind of problem with the encoding of powershell itself?
I am using W7 and WXP and LibIconv 1.9.2
> essentially is using the Out-File cmdlet who's default encoding is Unicode. Try:
iconv.exe ... | Out-File -Encoding Utf8
or with params:
& "C:\Program Files\GnuWin32\bin\iconv.exe" -f iso-8859-1 -t utf-8 $inFileName |
Out-File -Encoding Utf8 $outFileName
And since iconv.exe is outputting in UTF8, you have to tell the .NET console subsystem how to intrepret the stdin stream like so (execute this before iconv.exe):
[Console]::OutputEncoding = [Text.Encoding]::UTF8