I'm creating a zip file from a directory in two ways in Win10.
The first method using "Right-Click...Send To...Compressed (zipped) folder"
The second method using Powershell Compress-Archive. I wanted to automate the process using Powershell.
The zip file is a library to be imported into the Arduino IDE.
The first method will import into the Arduino IDE correctly. The second method does not import correctly.
Now before you say this is a Arduino problem please continue to read.
So to make sure the files are the same I used Powershell to generated MD5 checksums for both files. To my surprise they were different!!
The only difference I have found are the size of the created zip files. They are created from the same source files.
Why are they different? I can manually unzip both files by "Double-Clicking and Extract All" within File Explorer
Thank you for any advice/help. John.
Here is output of the powershell script...
Directory: C:\file1
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 05/16/2020 17:18 1948 t_OS.zip
Algorithm : MD5
Hash : 19249C2BB9B50E827A77F7A2952EFF6D
Path : C:\file1\t_OS.zip
Directory: C:\file2
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 05/16/2020 19:23 1724 t_OS.zip
Algorithm : MD5
Hash : 2BF7B111F54FC0555E6CA64719AD350F
Path : C:\file2\t_OS.zip
...
#
# sourceFiles are what I need to be zipped. It contains the
# directory t_OS and I need to create t_OS.zip
#
$sourceDir = "C:\sourceFiles"
$sourceFiles = "C:\sourceFiles\t_OS"
# file1 zip was created by right-clicking on the directory t_OS
# in the $sorceDir and selecting Send To, then
# Compressed (zipped) folder. This file was then moved to...
#
$file1 = "C:\file1\t_OS.zip"
# file2 zip is created by powershell using Compress-Archive below.
$file2 = "C:\file2\t_OS.zip"
Get-ChildItem -LiteralPath $sourceDir -Name -Recurse
If (Test-Path $file2) {Remove-Item $file2}
Compress-Archive -DestinationPath $file2 -CompressionLevel Fastest -Path $sourceFiles
Get-ItemProperty -LiteralPath $file1
Get-FileHash -Path $file1 -Algorithm MD5
Get-ItemProperty -LiteralPath $file2
Get-FileHash -Path $file2 -Algorithm MD5
...
The fact that the two zip files are different sizes can have a number of reasons. Here are a few
As already mentioned, the two zip files could be using different compression algorithms. Given the small size of the resultant zip files, it may be that one is storing the data compressed, while the other is storing it uncompressed.
There are a number of extensions that have been added to zip files over the years that have added things like more accurate timestamps. Those add more bytes the the resultant zip files, but they shouldn't prevent the file from being uncompressed. I believe a "right-click" created zip file on Windows 10 doesn't use any of these extensions. Not sure about Powershell.
Do you have access to a Linux setup? There are tools on it that can analyse the structure of zip files.
Also, can you share the Powershell command line you used to create the zip file?
Edit
On your Ubuntu setup, start by checking what unzip thinks about the two files
unzip -t t_OS.zip
If unzip finds an issue it will report it.
If that doesn't work run zipdetails against both zip files and update your question with the results.
zipdetais -v t_OS.zip
If you don't have zipdetails on your Ubuntu setup, get it from here
Related
I need a PS script that can copy specific files from one directory to another.
Main Goal: I want to copy all files from Month folder (November) to DirectoryX. HOWEVER,I only want to move the files with specific names from an .xlsx file with the column named FileName. So say there are 3,000 filesnames in the .xlsx file with unique filenames. This is a monthly report that is generated from SSMS.
Process: .xlsx file is created with data. The column for the filenames is FileName. I want to cross reference those filenames with the November folder and copy those files to a new directory to upload to the client.
Folder Structure:
Year: 2022
Month: 11
Day: 09
File Naming Convention: CA09a37ce4c69f31997c8656df274749c4.mp3.
Not sure the best way to do this. I have looked around on here and nothing that really suits what I need.
I really hope this makes sense and someone can guide me in the right direction. Thank you so much in advance.
We use an in-house application. I can add a PS script to it. I am brand new to Powershell so this is more complex than what I can handle.
Install the ImportExcel module, then this should work if you update the paths:
# import the FileNames column from excel file
$FileNames = Import-Excel 'C:\folder\file.xlsx' | Select -ExpandProperty FileName | Sort
# specify folder(s) to search
$Month = '11'
$SearchFolder = "C:\SourceFiles\2022\$Month" # all days in month
# get the list of existing source files, filtered by the imported file names
$SourceFiles = Get-ChildItem -Path $SearchFolder -Recurse -File |
Where Name -In $FileNames
# optionally check for missing files
$MissingFiles = Compare-Object $SourceFiles.Name $FileNames
If ($MissingFiles) {
Write-Warning "Could not find $($MissingFiles.Count) source files in $SearchFolder"
$MissingFiles.InputObject
}
# copy files to separate folder
$SourceFiles | Copy-Item -Destination "C:\Uploads\"
It's not the most efficient thing, but it should be easy enough to modify as needed
I receive files from the client and I need to make sure those files are healthy even after I process these files.
What I want to beware of is a situation of possible tampering with already processed files.
A customer error may occur for sending files with missing information and the customer wants to edit the original file to say the information was there.
That's why I had the idea of producing a report with the hash of these files that are in the reception directory.
I tried hard to make everything clearer and more intelligible, even though I didn't speak English.
Scene 1:
I have a directory with several text files and I need to export the SHA-256 and MD5 hash calculations from each file to a CSV somewhere.
Scene 2:
I have a directory with several subfolders with customer names and within these customers, other subfolders. How to extract SHA-256 & MD5 Hash from all files in these directories and subdirectories?
Something like the following should get you started. This finds the SHA256 hash for every file inside the current directory (not including subdirectories)
Get-ChildItem | Get-FileHash | Export-CSV -Path "C:\Temp\summary.csv"
PS C:\Users\Jan\Dropbox\py\advent_of_code> cat "C:\Temp\summary.csv"
#TYPE Microsoft.Powershell.Utility.FileHash
"Algorithm","Hash","Path"
"SHA256","8E8FF02948E62C54EF782373500CD0D97B8A2DA0F1655A6134B37284CF5BCE79","C:\Users\Jan\Dropbox\py\advent_of_code\20191201.in"
"SHA256","63E68322B7E8131CDDEFB77492EC7E1B8B8C46696772CE561850C854E4E8B6EA","C:\Users\Jan\Dropbox\py\advent_of_code\20191201.py"
"SHA256","FFF1D3F7F7FBDDC4CDC90E123D9BC7B0B7A450DC28F3A4F2D786701B4A9B279D","C:\Users\Jan\Dropbox\py\advent_of_code\20191204.py"
"SHA256","DE3662893F2779446AFC78B20E63F5250826D5F52206E5718E1A2713F876941E","C:\Users\Jan\Dropbox\py\advent_of_code\20191206.in"
"SHA256","B9381E11A442DDC8CF802F337F0487E9269800B145106FFDDB0E6472D8C6F129","C:\Users\Jan\Dropbox\py\advent_of_code\20191206.py"
If you really need both SHA256 and MD5:
$h = #(Get-ChildItem | Get-FileHash)
$h2 = ($h | Get-Item | Get-FileHash -Algorithm MD5)
for ($i=0; $i -lt $h.Length; $i++) {
$h[$i] = [PSCustomObject]#{Path=$h[$i].Path; SHA256=$h[$i].Hash; MD5=$h2[$i].Hash}
}
$h | Export-Csv "C:\Temp\expo.txt"
PS C:\Users\Jan\Dropbox\py\advent_of_code> cat "C:\Temp\expo.txt"
#TYPE System.Management.Automation.PSCustomObject
"Path","SHA256","MD5"
"C:\Users\Jan\Dropbox\py\advent_of_code\20191201.in","8E8FF02948E62C54EF782373500CD0D97B8A2DA0F1655A6134B37284CF5BCE79","11689BA3058C306DAA3651562621BE20"
"C:\Users\Jan\Dropbox\py\advent_of_code\20191201.py","63E68322B7E8131CDDEFB77492EC7E1B8B8C46696772CE561850C854E4E8B6EA","3F50E29C797ED4BD65122F8DA1208D4D"
"C:\Users\Jan\Dropbox\py\advent_of_code\20191204.py","FFF1D3F7F7FBDDC4CDC90E123D9BC7B0B7A450DC28F3A4F2D786701B4A9B279D","41094A3E067669F46C965D0E34EA5CA6"
"C:\Users\Jan\Dropbox\py\advent_of_code\20191206.in","DE3662893F2779446AFC78B20E63F5250826D5F52206E5718E1A2713F876941E","1A7571873E6B9430A2BFD846CA0B8AB7"
"C:\Users\Jan\Dropbox\py\advent_of_code\20191206.py","B9381E11A442DDC8CF802F337F0487E9269800B145106FFDDB0E6472D8C6F129","0CBB058D315944B7B5183E33FC780A4D"
I am using a PowerShell script to zip up files that are older than 60 days. Some of these files have really long filenames so I get the filename or extension is too long error.
I would rather not go into each file and change the names so I need a way to be able to apply something to all the files at the same time or have my script bypass the error somehow. This script is also going to be run on several computers so I would prefer not to download something on to each one.
This is the script:
#Set source and target
$Source = "D:\Testing\"
$Target = "$ENV:USERPROFILE\Desktop\TEST.zip"
#Set time parameters
$Days = 60
$LastWrite = (Get-Date).Date.AddDays(-$Days)
#Invoke 7-zip
if (-not (test-path "$env:ProgramFiles\7-Zip\7z.exe")) {throw
"$env:ProgramFiles\7-Zip\7z.exe needed"}
set-alias zip "$env:ProgramFiles\7-Zip\7z.exe"
$Target = Get-Childitem $Source -Recurse | Where-Object -FilterScript
{($_.LastWriteTime -ge $LastWrite)}
zip a -mx=9 $Target $Source
I am using 7-zip to zip up the files and I have PS version 5.1.
As mentioned in the comments, one way around long file names is to store relative paths. 7Zip will store relative paths if you specify an input file with the relative paths and they resolve to the files you want to archive, as described in this answer.
Intermediate files can be messy, so I've written a script that uses the ZipFileExtensions' CreateEntryFromFile method to store a relative path in a zip file.
You can specify -ParentFolder on the command line to store paths relative to the parent, including a UNC path if you want to archive files on another computer. If -ParentFolder is not specified it will choose the script's folder as the parent and store paths relative to the script.
Copy the code to a new script named ArchiveOldLogs.ps1 and run it with this command line:
.\ArchiveOldLogs.ps1 -ParentFolder "D:\Testing\" -FileSpecs #("*.*") -Filter { $_.LastWriteTime -lt (Get-Date).AddDays(-60)} -DeleteAfterArchiving:$false
That will get you 11 more characters at the end of the path to store, which should be enough to get around the 10 character difference between Windows and Zip path length limits. Try a deeper folder if you still get errors. The files that can't be archived, or are already archived will be skipped.
Remove -DeleteAfterArchiving:$false from the command line when you're comfortable that it's archiving only what you want.
I have a client with a large file server (on 2012r2). The client accesses the server using machines running Windows 10 Pro. The client is not using a file management system and so the files are just organized manually into folders and subfolders. The client wants to copy all files on the server that contain a particular string (in the file not in the filename) to a second location. The trouble is the files are in various directories and sub-directories and the files often have the same name. Is there a way to use xcopy or robocopy from the CLI or copy-item from powershell to recursively copy only those files that contain a particular string or that have a particular author to a new location while maintaining the folder structure?
The following script might what you want. Just edit the 3 variables at the beginning:
$sSrcDir = "U:\Data\SCRIPTS"
$sDestination = "F:\temp\t" # No terminating backslash!
$sPattern = "ffmpeg"
(Get-ChildItem -Recurse -File -LiteralPath "$sSrcDir" | select-string -Pattern "$sPattern" -List).path | ForEach-Object {
# Here, the string $_ holds the fullpathname of a file
$sDirToCreate = $_ | Split-Path -NoQualifier | Split-Path -Parent
mkdir -Force "$sDestination$sDirToCreate"
copy-item "$_" "$sDestination$sDirToCreate"
}
Edit: Thank you to mklement0. I updated the script to include his suggestions.
I want to extract all *.Z files in place in their current directories (7-Zip support LZW/UNIX compression).
I can't figure out why this is not working. I know how to use PowerShell to recursively get a full path of a filename and directory in which file resides. I also know how to use 7-zip to extract a file in place. But trying to put these two together is not working. Here's what I have:
Get-ChildItem -Recurse *.Z | foreach ($_) {7z.exe e $_.FullName -o$_.Directory}
I also tried:
Get-ChildItem -Recurse *.Z | foreach ($_) {7z.exe e $_.FullName -o$($_.Directory)}
and
Get-ChildItem -Recurse *.Z | foreach ($_) {7z.exe e $_.FullName -o${_.Directory}}
This is getting the files I want by FullName, and Directory is the correct path (no space between the o and directory is how 7-zip expects the output directory). However, it keeps trying to output the file to the .Z path rather than the directory. I know it is right if I do
Get-ChildItem -Recurse *.Z | Select Directory
that is where I want it to go.
I assume the issue was having no space but not sure why it would get same directory. I even manually defined a variable and passed that with no problems.
So the issue is 7-zip saying no files to process when I want to output to same directory but not sure why, and it seems Directory variable not being passed but file name again which makes no sense. Sample output per file:
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
Scanning the drive for archives:
1 file, 226 bytes (1 KiB)
Extracting archive: E:\Files\2014\more_cowbell\myfancyunixcompresseddocument.txt.Z
--
Path = E:\Files\2014\more_cowbell\myfancyunixcompresseddocument.txt.Z
Type = Z
No files to process
Everything is Ok
Files: 0
Size: 0
Compressed: 226
It works fine if I specify one specific output directory (no variable), but I want them extracted in place due to many subdirectories and conflicting file names between them.
I want to extract all *.Z files in place in their current directories (7-Zip support LZW/UNIX compression).
The -o parameter should be inside a doublequoted string with $($var) for variables:
Get-ChildItem -Recurse *.z | %{ 7z e $_.FullName "-o$($_.Directory)"}