How do I make powershell script transverse zip files and report based off select-string -pattern - powershell

I have the following that is working but I need to also have the ability to read the contents of compressed file (zip)
function Search-Files {
param ([string[]]$Servers, [string]$SearchPath, [string]$SearchItem, [string[]]$LogName)
ForEach ($Server in $Servers) {
if ($LogName -eq $null) {
dir -Path \\$server\$SearchPath -Recurse -Force -ErrorAction SilentlyContinue -WarningAction SilentlyContinue | Select-String -pattern $SearchItem -ErrorAction SilentlyContinue -WarningAction SilentlyContinue | Select-Object Filename, Path, Matches, LineNumber
}
Else {
dir -Path \\$server\$SearchPath -Recurse -Force -ErrorAction SilentlyContinue -WarningAction SilentlyContinue | ? {$_.Name -match $LogName} | Select-String -pattern $SearchItem -ErrorAction SilentlyContinue -WarningAction SilentlyContinue | Select-Object Filename, Path, Matches, LineNumber
}
}
}
Currently I am getting the following out put displayed which is what I would like to do for zip files as well
ip.ininlog \CO200197L\C$\Temp\Test\Test\ip\ip.ininlog {3030872954} 136594
I have found the following just not sure how to proceed to get them implemented
Grep File in Zip
List File in Zip
I need the ability to transverse all zip files that are store in a directory
Sample of Directory Structure
2014-07-01 - root
zip.zip
zip_1.zip
zip_2.zip
etc

In case you have NET 4.5 framework installed, you can use 4.5's built-in ZIP support to extract files to a temporary path and run the selection on the temporary file. If no 4.5 is available, I recommend using SharpCompress (https://sharpcompress.codeplex.com/) which works in a similar way.
The following code snippet demonstrates extracting a ZIP archive into a temporary file, running the selection process from your script and the cleanup after the extraction. You can significantly simplify the code by extracting the entire ZIP file at once (just use ExtractToDirectory() on the archive) if it contains only the files you are seeking.
# import .NET 4.5 compression utilities
Add-Type -As System.IO.Compression.FileSystem;
# the input archive
$archivePath = "C:\sample.zip";
# open archive for reading
$archive = [System.IO.Compression.ZipFile]::OpenRead($archivePath);
try
{
# enumerate all entries in the archive, which includes both files and directories
foreach($archiveEntry in $archive.Entries)
{
# if the entry is not a directory (which ends with /)
if($archiveEntry.FullName -notmatch '/$')
{
# get temporary file -- note that this will also create the file
$tempFile = [System.IO.Path]::GetTempFileName();
try
{
# extract to file system
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($archiveEntry, $tempFile, $true);
# create PowerShell backslash-friendly path from ZIP path with forward slashes
$windowsStyleArchiveEntryName = $archiveEntry.FullName.Replace('/', '\');
# run selection
Get-ChildItem $tempFile | Select-String -pattern "yourpattern" | Select-Object #{Name="Filename";Expression={$windowsStyleArchiveEntryName}}, #{Name="Path";Expression={Join-Path $archivePath (Split-Path $windowsStyleArchiveEntryName -Parent)}}, Matches, LineNumber
}
finally
{
Remove-Item $tempFile;
}
}
}
}
finally
{
# release archive object to prevent leaking resources
$archive.Dispose();
}
If you have multiple ZIP files in the directory, you can enumerate them as follows (using your example script):
$zipArchives = Get-ChildItem -Path \\$server\$SearchPath -Recurse "*.zip";
foreach($zipArchive in $zipArchives)
{
$archivePath = $zipArchive.FullName;
...
}
You can place the demo code in ... or move it to a PowerShell function.

Sometimes is not desirable to extract a zip entry as a file. Instead it may be preferable to work with the file in memory. Extracting a Zip entry containing XML or JSON text so it can be parsed in memory is an example.
Here is a technique that will allow you to do this. This example assumes there is a Zip entry with a name ending in .json and it is this file which is to be retrieved. Clearly the idea can be modified to handle different cases.
This code should work with version of the .NET Framework that includes the System.IO.Compression namespace.
try
{
# import .NET 4.5 compression utilities
Add-Type -As System.IO.Compression.FileSystem;
# A variable to hold the recovered JSON content
$json = $null
$zip = [IO.Compression.ZipFile]::OpenRead($zipFileName)
$zip.Entries |
Where-Object { $_.Name.EndsWith(".json") } |
ForEach-Object {
# Use a MemoryStream to hold the inflated file content
$memoryStream = New-Object System.IO.MemoryStream
# Read the entry
$file = $_.Open()
# Copying inflates the entry content
$file.CopyTo($memoryStream)
# Make sure the entry is closed
$file.Dispose()
# After copying, the cursor will be at the end of the stream
# so set the position to the beginning or there will be no output
$memoryStream.Position = 0
# Use a StreamReader because it allows the content to be
# read as a string in one go
$reader = New-Object System.IO.StreamReader($memoryStream)
# Read the content as a string
$json = $reader.ReadToEnd()
# Close the reader and memory stream
$reader.Dispose()
$memoryStream.Dispose()
}
# Finally close the zip file. This is necessary
# because the zip file does get closed automatically
$zip.Dispose()
# Do something with the JSON in memory
if ( $json -ne $null )
{
$objects = $json | ConvertFrom-Json
}
}
catch
{
# Report errors
}

Related

Powershell: Logging foreach changes

I have put together a script inspired from a number of sources. The purpose of the powershell script is to scan a directory for files (.SQL), copy all of it to a new directory (retain the original), and scan each file against a list file (CSV format - containing 2 columns: OldValue,NewValue), and replace any strings that matches. What works: moving, modifying, log creation.
What doesn't work:
Recording in the .log for the changes made by the script.
Sample usage: .\ConvertSQL.ps1 -List .\EVar.csv -Files \SQLFiles\Rel_1
Param (
[String]$List = "*.csv",
[String]$Files = "*.sql"
)
function Get-TimeStamp {
return "[{0:dd/MM/yyyy} {0:HH:mm:ss}]" -f (Get-Date)
}
$CustomFiles = "$Files\CUSTOMISED"
IF (-Not (Test-Path $CustomFiles))
{
MD -Path $CustomFiles
}
Copy-Item "$Files\*.sql" -Recurse -Destination "$CustomFiles"
$ReplacementList = Import-Csv $List;
Get-ChildItem $CustomFiles |
ForEach-Object {
$LogFile = "$CustomFiles\$_.$(Get-Date -Format dd_MM_yyyy).log"
Write-Output "$_ has been modified on $(Get-TimeStamp)." | Out-File "$LogFile"
$Content = Get-Content -Path $_.FullName;
foreach ($ReplacementItem in $ReplacementList)
{
$Content = $Content.Replace($ReplacementItem.OldValue, $ReplacementItem.NewValue)
}
Set-Content -Path $_.FullName -Value $Content
}
Thank you very much.
Edit: I've cleaned up a bit and removed my test logging files.
Here's the snippet of code that I've been testing with little success. I put the following right under $Content= Content.Replace($ReplacementItem.OldValue, $ReplacementItem.NewValue)
if ( $_.FullName -like '*TEST*' ) {
"This is a test." | Add-Content $LogFile
}
I've also tried to pipe out the Set-Content using Out-File. The outputs I end up with are either a full copy of the contents of my CSV file or the SQL file itself. I'll continue reading up on different methods. I simply want to, out of hundreds to a thousand or so lines, to be able to identify what variables in the SQL has been changed.
Instead of piping output to Add-Content, pipe the log output to: Out-File -Append
Edit: compare the content using the Compare-Object cmdlet and evaluate it's ouput to identify where the content in each string object differs.

PowerShell: How to copy only ".exe" file from ZIP folder to another folder

I have a "ZIP" file and when we extract this, we have 1 "EXE" file within 4-5 sub folder depth level.
I would like to grab that "EXE" file and copy into another folder. How to do it using PowerShell?
I tried below, but it will copy all the ZIP content,
$shell = New-Object -ComObject shell.application
$zip = $shell.NameSpace("Source Path")
foreach ($item in $zip.items()) {
$shell.Namespace("Destination Path").CopyHere($item)
}
Simple snippet should get your job done
#Sets the variable to the Source folder, recurse drills down to folders within
$Source = get-childitem "C:\Users" -recurse #"C:\Users" an example
#Filters by extension .exe
$List = $Source | where {$_.extension -eq ".exe"}
#Copies all the items to the specified destination
$List | Copy-Item -Destination "C:\Scripts" #"C:\Scripts" an example
The module above scans for every single .EXE files within C:\Users* and copies them to C:\Scripts
As it stands, Clint's answer did not work for me, but something based on Extract Specific Files from ZIP Archive does, with a variation to target a specifically named file. Will need a further tweak to handle multiple files sharing the same name.
Code:
# Set source zip path, target output directory and file name filter
$ZipPath = 'C:\temp\Test.zip'
$OutDir = 'C:\temp'
$Filter = 'MyExe.exe'
# Load compression methods
Add-Type -AssemblyName System.IO.Compression.FileSystem
# Open zip file for reading
$Zip = [System.IO.Compression.ZipFile]::OpenRead($Path)
# Copy selected items to the target directory
$Zip.Entries |
Where-Object { $_.FullName -eq $Filter } |
ForEach-Object {
# Extract the selected items from the zip archive
# and copy them to the out folder
$FileName = $_.Name
[System.IO.Compression.ZipFileExtensions]::ExtractToFile($_, "$OutDir\$FileName", $true)
}
# Close zip file
$Zip.Dispose()

Parse directory listing and pass to another script?

I am trying to write a PowerShell script that will loop through a directory in C:\ drive and parse the filenames with the file extension to another script to use.
Basically, the output of the directory listing should be accessible to be parsed to another script one by one. The script is a compiling script which expects an argument (parameter) to be parsed to it in order to compile the specific module (filename).
Code:
Clear-Host $Path = "C:\SandBox\"
Get-ChildItem $Path -recurse -force | ForEach { If ($_.extension -eq ".cob")
{
Write-Host $_.fullname
}
}
If ($_.extension -eq ".pco")
{
Write-Host $_.fullname }
}
You don't need to parse the output as text, that's deprecated.
Here's something that might work for you:
# getmyfiles.ps1
Param( [string])$Path = Get-Location )
dir $Path -Recurse -Force | where {
$_.Extension -in #('.cob', '.pco')
}
# this is another script that calls the above
. getmyfile.ps1 -Path c:\sandbox | foreach-object {
# $_ is a file object. I'm just printing its full path but u can do other stuff eith it
Write-host $_.Fullname
}
Clear-Host
$Path = "C:\Sandbox\"
$Items = Get-ChildItem $Path -recurse -Include "*.cob", "*.pco"
From your garbled code am guessing you want to return a list of files that have .cob and .pco file extensions. You could use the above code to gather those.
$File = $Items.name
$FullName = $items.fullname
Write-Host $Items.name
$File
$FullName
Adding the above lines will allow you to display them in various ways. You can pick the one that suites your needs then loop through them on a for-each.
As a rule its not a place for code to be writen for you, but you have tried to add some to the questions so I've taken a look. Sometimes you just want a nudge in the right direction.

How do I select files in a folder based on part of filename and zip them in Powershell?

I'm fairly new to Powershell(using Powershell 2.0 btw) and am trying to make a script that does several things(this is my 3rd script or so). I have most things in place but the last thing remaining is to group files of different types (xml, tfw and tif) in a folder, based on the first part of the filename(first three characters) and then zip these files into several zip-files with name like the first 3 characters, either in the same location or in a new one.
Sample of folder content:
001.tif
001.tfw
001.metadata.xml
002.tif
002.tfw
002.metadata.xml
003.tif
003.tfw
003.metadata.xml
003_svel.tif
003_svel.tfw
003_svel.metadata.xml
Wanted result:
001.zip containing 001.tif, 001.tfw, 001.metadata.xml
002.zip containing 002.tif, 002.tfw, 002.metadata.xml
003.zip containing 003.tif, 003.tfw, 003.metadata.xml, 003_svel.tif,
003_svel.tfw and 003_svel.metadata.xml
I have installed 7-zip to do the zipping and am using the commandline version. I've used 7-zip local on some testfiles and got it to work, but then it was only tif-files. I have a source folder where I search for the latest created folder and then process the files in it.
This is what I have so far(Powershell 2.0):
$dir_source = "c:\Test"
$new_folder = Get-ChildItem $dir_source -Recurse |
Where { $_.PSIsContainer} |
Sort-Object LastWriteTime -Descending |
Select-Object -ExpandProperty Fullname <-First 1
Get-ChildItem $new_folder -recurse -Exclude metafile.xml |
Group-Object {$_.Name.Substring(0,3)}
This gives me a list of grouped files in the lates created folder based on the first 3 characters in the filename. It also show what files are in each group.
Like below:
Count Name Group
----- ---- -----
3 003 {C:\Test\20150708 063255_B\003.metafile.xml, C:\Test\20150708 063255_B\003.tfw, C:\Test\20150708 063255_B\003.tif}
6 004 {C:\Test\20150708 063255_B\004.metafile.xml, C:\Test\20150708 063255_B\004.tfw, C:\Test\20150708 063255_B\004.tif,C:\Test...
6 009 {C:\Test\20150708 063255_B\009.metafile.xml, C:\Test\20150708 063255_B\009.tfw, C:\Test\20150708 063255_B\009.tif,C:\Test...
Now my next step ist to take these groups and zip them. Ideally create these zip-files in a different destination directory (I believe I can change this when setting the $directory- variable in the script below.)
foreach ($group in $dataset) {
$name = $file.name
$directory = $file.DirectoryName
$zipFile = $file.Name + ".zip"
sz a -t7z "$directory\$zipfile" "$directory\$name"
This last code is causing some trouble. I either get the message:
7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 Error:
c:\Test\Dest_test460.zip is not supported archive System error:
Incorrect function.
,or
WARNING: Cannot find 1 file 7-Zip (A) 9.20 Copyright (c) 1999-2010
Igor Pavlov 2010-11-18 Scanning \460: WARNING: The system cannot
find the file specified.
,or it starts zipping all files on my userprofile into a zip-file. Depending on changes I do to the $group-value. I believe there are one ore more basic errors in my script causing this, and this is where I'm asking for some help. It may be that I am approaching this the wrong way by first grouping the files I want and then try to zip them?
Anyone that can see my error or give me some hint to what I have to do?
Thanks for your time!
Lee Holmes New-ZipFile do the job, he has two versions one of them using the ICSharpCode.SharpZipLib.dll to compress, and the other not require it, i wrapped the 2nd one into a function:
Function New-ZipFile {
param(
## The name of the zip archive to create
$Path = $(throw "Specify a zip file name"),
## Switch to delete the zip archive if it already exists.
[Switch] $Force
)
Set-StrictMode -Version 3
## Create the Zip File
$zipName = $executionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath($Path)
## Check if the file exists already. If it does, check
## for -Force - generate an error if not specified.
if(Test-Path $zipName)
{
if($Force)
{
Remove-Item $zipName -Force
}
else
{
throw "Item with specified name $zipName already exists."
}
}
## Add the DLL that helps with file compression
Add-Type -Assembly System.IO.Compression.FileSystem
try
{
## Open the Zip archive
$archive = [System.IO.Compression.ZipFile]::Open($zipName, "Create")
## Go through each file in the input, adding it to the Zip file
## specified
foreach($file in $input)
{
## Skip the current file if it is the zip file itself
if($file.FullName -eq $zipName)
{
continue
}
## Skip directories
if($file.PSIsContainer)
{
continue
}
$item = $file | Get-Item
$null = [System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile(
$archive, $item.FullName, $item.Name)
}
}
finally
{
## Close the file
$archive.Dispose()
$archive = $null
}
}
To use it for example:
dir c:\folder -Recurse | New-ZipFile -Path c:\temp\folder.zip
The Source file(for the one that use the ICSharpCode): http://poshcode.org/2202
Use my Previous Answer Function New-ZipFile and use with this one:
$FolderName = "C:\temp"
$Files = dir $FolderName
$prfx = #()
foreach ($file in $files)
{
$prfx += $file.Name.Substring(0,3)
}
$prfx = $prfx | Group
foreach ($Prf in $prfx)
{
$prf = $prf.name.ToString()
dir $Files | ? {$_.Name -match "^$prf"} | New-ZipFile -Path "$foldername\$prf.zip"
}
According to your example It will output 3 zip files like you want,
it will always use the first 3 letters of the file, you can the change this in this line $prfx += $file.Name.Substring(0,3) and set it different if needed.
Good Luck
Could not get the suggested solution to work, had problems configuring ICSharpCode. I also wanted to use 7zip, since it is still under some updating regime.
Ended up copying my files to temp folders based on the filenames and then zip each folder. After that delete the tempfolders with files. Ugly code, but it does the job.
# Create folder based on filename and copy files into respective folder
Get-ChildItem $new_folder -Filter *.* | Where-Object {!$_.PSIsContainer} | Foreach-Object{
$dest = Join-Path $_.DirectoryName $_.Name.SubString(0,3)
if(!(Test-Path -Path $dest -PathType Container))
{
$null = md $dest
}
$_ | Copy-Item -Destination $dest -Force
}
# Create zip-file of each folder
dir $new_folder | Where-Object { $_.PSIsContainer } | ForEach-Object { sz a -t7z -mx9 "$dir_dest\$_.zip" $_.FullName }
# Delete temp-folders
dir $new_folder | Where-Object { $_.PSIsContainer } | Remove-Item -Recurse

How to modify my powershell script to search for all jpg/jpeg on a machine

I am working on building a PowerShell script that will find all the jpeg/jpg files on a machine. This is what I have so far-
# PowerShell script to list the DLL files under the C drive
$Dir = get-childitem C:\ -recurse
# $Dir |get-member
$List = $Dir | where {$_.extension -eq ".jpg"}
$List |ft fullname |out-file C:\Users\User1\Desktop\dll.txt
# List | format-table name
The only problem is that some of the files I am looking for don't have the extension jpg/jpeg. I know that you can look in the header of the file and if it says ÿØÿà then it is a jpeg/jpg but I don't know how to incorporate this into the script.
Any help would be appreciated. Thanks so much!
I'm not sure how to use powershell native commands to Look at file headers, I will do some research on it because it sounds fun. Until then I can suggest a shorter version of your initial command, reducing it to a one liner.
Get-ChildItem -Recurse -include *.jpg | Format-table -Property Fullname | Out-file C:\Users\User1\Desktop\Jpg.txt
or
ls -r -inc *.jpg | ft Fullname
EDITED: removed redundant code, thanks #nick.
I'll let you know what I find if I find anything at all.
Chris
The following will retrieve files with either a .jpg/.jpeg extension or that contain a JPEG header in the first four bytes:
[Byte[]] $jpegHeader = 255, 216, 255, 224;
function IsJpegFile([System.IO.FileSystemInfo] $file)
{
# Exclude directories
if ($file -isnot [System.IO.FileInfo])
{
return $false;
}
# Include files with either a .jpg or .jpeg extension, case insensitive
if ($file.Extension -match '^\.jpe?g$')
{
return $true;
}
# Read up to the first $jpegHeader.Length bytes from $file
[Byte[]] $fileHeader = #(
Get-Content -Path $file.FullName -Encoding Byte -ReadCount 0 -TotalCount $jpegHeader.Length
);
if ($fileHeader.Length -ne $jpegHeader.Length)
{
# The length of the file is less than the JPEG header length
return $false;
}
# Compare each byte in the file header to the JPEG header
for ($i = 0; $i -lt $fileHeader.Length; $i++)
{
if ($fileHeader[$i] -ne $jpegHeader[$i])
{
return $false;
}
}
return $true;
}
[System.IO.FileInfo[]] $jpegFiles = #(
Get-ChildItem -Path 'C:\' -Recurse `
| Where-Object { IsJpegFile $_; }
);
$jpegFiles | Format-Table 'FullName' | Out-File 'C:\Users\User1\Desktop\dll.txt';
Note that the -Encoding and -TotalCount parameters of the Get-Content cmdlet are used to read only the first four bytes of each file, not the entire file. This is an important optimization as it avoids basically reading every byte of file data on your C: drive.
This should give you all files starting with the sequence "ÿØÿà":
$ref = [byte[]]#(255, 216, 255, 224)
Get-ChildItem C:\ -Recurse | ? { -not $_.PSIsContainer } | % {
$header = [System.IO.File]::ReadAllBytes($_.FullName)[0..3]
if ( (compare $ref $header) -eq $null ) {
$_.FullName
}
} | Out-File "C:\Users\User1\Desktop\dll.txt"
To find if the header starts with ÿØÿà, use:
[System.String]$imgInfo = get-content $_ # where $_ is a .jpg file such as "pic.jpg"
if($imgInfo.StartsWith("ÿØÿà"))
{
#It's a jpeg, start processing...
}
Hope this helps
I would recommend querying the windows search index for the jpegs, rather than trying to sniff file contents. Searching the system index using filenames is insanely fast, the downside is that you must search indexed locations.
I wrote a windows search querying script using the windows sdk \samples\windowssearch\oledb, you would want to query using the imaging properties. However, I'm not certain off the top of my head if the search index uses the imaging api to look at unknown files or files without extensions. Explorer seems to know my jpeg thumbnails and metadata without jpg extensions, so I'm guessing the indexer is going to be as clever as explorer.
How about this one?
jp*g is used to match both jpg and jepg images.
$List = Get-ChildItem "C:\*.jp*g" -Recurse
$List |ft fullname |out-file C:\Users\User1\Desktop\dll.txt