Lets say i have 8 files in a folder, and I have a file.csv with 5 lines.
The 8 files start with the same like in the file.csv 0022***_something.csv. So I want to check if filename exist in file.csv
file.csv lines look like:
0022150;something;something;something
0022151;something;something;something
0022152;something;something;something
0022153;something;something;something
0022154;something;something;something
$FileCsv = Get-Content "\\sharedFolder\file.csv" | foreach {($_ -split ";")[0..0]} | Where {($_ -like "00*")}
$FolderPath = "\\SharedFolder\Path\"
Foreach ($file in $folderPath)
{
$file = $file.Substring(0,7)
if ($file exist in $FileCsv) #Not sure how I get this line right.
{
$file + "Exist"
}
else
{
$file + "Does not exist"
}
}
I would do something like this:
$FolderPath = "\\SharedFolder\Path\" #"# This is where the files are found
# read the CSV file, using a header 'Name' for the first column and get this column as an array of strings.
$NamesInCsv = (Import-Csv -Path "\\SharedFolder\file.csv" -Header Name -Delimiter ';').Name
# get a list if files with names that start with '022'
Get-ChildItem -Path $FolderPath -Filter '022*' -File | ForEach-Object {
# if the first 7 characters of the name are in the names from the CSV
if ($_.Name.Length -ge 7 -and $NamesInCsv -contains $_.Name.Substring(0,7)) {
Write-Host "File $($_.Name) exists in the Csv"
}
else {
Write-Host "File $($_.Name) does not exist in the Csv"
}
}
Hope that helps
Related
File has 2 fields, BinaryID and name. I download files by the BinaryID and need to rename them. The Write-Host example in the ForEach loop only echo's back the last line of the text file for each of the ID's in BinaryID's.
4468 TP_146_18.zip
4468 TP_146_18.zip
4468 TP_146_18.zip
4468 TP_146_18.zip
4468 TP_146_18.zip
$BinaryID = 3927,3988,4073,4151,4265
Get-Content -Path "D:\SIS\PS_Zips.txt" | ForEach-Object {
$BID = $_.Split(',')
}
ForEach ($ID in $BinaryID) {
if ( $ID -eq $BID[0] ) { Write-Host $BID[0] $BID[1] }
Write-Host $BID[0] $BID[1]
}
Thanks for explaining that the downloaded files do have an extension (.zip)
One way of doing this is to use Import-Csv with the -Header parameter like this:
$downloadFolder = 'X:\Path\to\where\you\have\downloaded\the\files'
$BinaryID = 3927,3988,4073,4151,4265
$data = Import-Csv -Path 'D:\SIS\PS_Zips.txt' -Header BinaryId, FileName
$files = Get-ChildItem -Path $downloadFolder -File -Filter '*.zip' |
Where-Object { $BinaryID -contains $_.BaseName }
foreach ($file in $files) {
$file | Rename-Item -NewName { ($data | Where-Object { $_.BinaryId -eq $file.BaseName }).FileName }
}
The Where-Object filters the files to collect so only files with a BaseName that is in your $BinaryID variable are passed through.
Next, all you need is to find the corresponding data row by its BinaryId field and use the FileName field to rename the file.
In a directory, there are files with the following filenames:
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
I want to iterate through the directory, and IF there is a filename that contains the string '_pn.mp3', I want to test if there is a similarly named file without the '_pn.mp3' in the same directory. If that file exists, I want to remove it.
In the above example, I'd want to remove:
ExampleFile.mp3
ExampleFile2.mp3
and I'd want to keep ExampleFile3.mp3
Here's what I have so far:
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path '$path' | Where-Object {! $_.PSIsContainer}
Foreach ($file in $files) {
If($file.Name -match $pattern){
# filename with _pn.mp3 exists
Write-Host $file.Name
# search in the current directory for the same filename without _pn
<# If(Test-Path $currentdir $filename without _pn.mp3) {
Remove-Item -Force}
#>
}
enter code here
You could use Group-Object to group all files by their BaseName (with the pattern removed), and then loop over the groups where there are more than one file. The result of grouping the files and filtering by count would look like this:
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1
Count Name Group
----- ---- -----
2 ExampleFile {ExampleFile.mp3, ExampleFile_pn.mp3}
2 ExampleFile2 {ExampleFile2.mp3, ExampleFile2_pn.mp3}
Then if we loop over these groups we can search for the files that do not end with the $pattern:
#'
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
'# -split '\r?\n' -as [System.IO.FileInfo[]] | Set-Variable files
$pattern = "_pn"
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$_.Group.Where({-not $_.BaseName.Endswith($pattern)})
}
This is how your code would look like, remove the -WhatIf switch if you consider the code is doing what you wanted.
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path -Filter *.mp3 -File
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$toRemove = $_.Group.Where({-not $_.BaseName.Endswith($pattern)})
Remove-Item $toRemove -WhatIf
}
I think you can get by here by adding file names into a hash map as you go. If you encounter a file with the ending you are interested in, check if a similar file name was added. If so, remove both the file and the similar match.
$ending = "_pn.mp3"
$files = Get-ChildItem -Path $path -File | Where-Object { ! $_.PSIsContainer }
$hash = #{}
Foreach ($file in $files) {
# Check if file has an ending we are interested in
If ($file.Name.EndsWith($ending)) {
$similar = $file.Name.Split($ending)[0] + ".mp3"
# Check if we have seen the similar file in the hashmap
If ($hash.Contains($similar)) {
Write-Host $file.Name
Write-Host $similar
Remove-Item -Force $file
Remove-Item -Force $hash[$similar]
# Remove similar from hashmap as it is removed and no longer of interest
$hash.Remove($similar)
}
}
else {
# Add entry for file name and reference to the file
$hash.Add($file.Name, $file)
}
}
Just get a list of the files with the _pn then process against the rest.
$pattern = "*_pn.mp3"
$files = Get-ChildItem -Path "$path" -File -filter "$pattern"
Foreach ($file in $files) {
$TestFN = $file.name -replace("_pn","")
If (Test-Path -Path $(Join-Path -Path $Path -ChildPath $TestFN)) {
$file | Remove-Item -force
}
} #End Foreach
I've a scenario where everyday I will received 2 csv files where the file naming is something like CMCS_{Timestamp}, example CMCS_02012016100101 and CMCS_02012016100102 . This 2 files are different files and have different structure, but because this 2 files will go into same folder where my ETL tools will pick it up and process. So I wrote a script where the script will based on the structure of the file to distinguish it whether is a file A or file B.
For File A, i tell a script to look at first line of the file and if line start with 'Name,Emp(Date).' then copy the file to folderA else if line start with 'Name,Group.' then copy the file to folderB else copy file to folder C
Here the code that i wrote, the powershell does not generate any errors but it does not produce any results too. I wonder what wrong in my script.
$fileDirectory = "D:\Data";
$output_path = "D:\Output\FileA";
$output_path2 = "D:\Output\FileB";
$output_path2 = "D:\Output\FileC";
foreach($file in Get-ChildItem $fileDirectory)
{
# the full file path.
$filePath = $fileDirectory + "\" + $file;
$getdata = Get-Content -path $filePath
$searchresults = $getdata | Select -Index 1 | Where-Object { $_ -like 'Name,Emp(Date).*' }
$searchresults2 = $getdata | Select -Index 1 | Where-Object { $_ -like 'Name,Group.*' }
if ($searchresults -ne $null) {
Copy-Item $filePath $output_path
}
if ($searchresults2 -ne $null) {
Copy-Item $filePath $output_path2
}
}
Your issue may be caused by the Select -Index 1, as Powershell uses 0 based indexing this will actually select the second line of the file. If you change this to 0 it should correctly get the header row.
On a separate note, instead of doing $filePath = $fileDirectory + "\" + $file; you can just use $file.FullName to get the file path.
EDIT:
I think this should do what you're after:
[string] $FileDirectory = "D:\Data";
[string] $OutputPath = "D:\Output\FileA";
[string] $OutputPath2 = "D:\Output\FileB";
[string] $OutputPath3 = "D:\Output\FileC";
foreach ($FilePath in Get-ChildItem $FileDirectory | Select-Object -ExpandProperty FullName)
{
[string] $Header = Get-Content $FilePath -First 1
if ($Header -match 'Name,Emp.*') {
Copy-Item $FilePath $OutputPath
}
elseif ($Header -match 'Name,Group.*') {
Copy-Item $FilePath $OutputPath2
}
else {
Copy-Item $FilePath $OutputPath3
}
}
I have a list of strings in a CSV file. The format is:
OldValue,NewValue
223134,875621
321321,876330
....
and the file contains a few hundred rows (each OldValue is unique). I need to process changes over a number of text files in a number of folders & subfolders. My best guess of the number of folders, files, and lines of text are - 15 folders, around 150 text files in each folder, with approximately 65,000 lines of text in each folder (between 400-500 lines per text file).
I will make 2 passes at the data, unless I can do it in one. First pass is to generate a text file I will use as a check list to review my changes. Second pass is to actually make the change in the file. Also, I only want to change the text files where the string occurs (not every file).
I'm using the following Powershell script to go through the files & produce a list of the changes needed. The script runs, but is beyond slow. I haven't worked on the replace logic yet, but I assume it will be similar to what I've got.
# replace a string in a file with powershell
[reflection.assembly]::loadwithpartialname("Microsoft.VisualBasic") | Out-Null
Function Search {
# Parameters $Path and $SearchString
param ([Parameter(Mandatory=$true, ValueFromPipeline = $true)][string]$Path,
[Parameter(Mandatory=$true)][string]$SearchString
)
try {
#.NET FindInFiles Method to Look for file
[Microsoft.VisualBasic.FileIO.FileSystem]::GetFiles(
$Path,
[Microsoft.VisualBasic.FileIO.SearchOption]::SearchAllSubDirectories,
$SearchString
)
} catch { $_ }
}
if (Test-Path "C:\Work\ListofAllFilenamesToSearch.txt") { # if file exists
Remove-Item "C:\Work\ListofAllFilenamesToSearch.txt"
}
if (Test-Path "C:\Work\FilesThatNeedToBeChanged.txt") { # if file exists
Remove-Item "C:\Work\FilesThatNeedToBeChanged.txt"
}
$filefolder1 = "C:\TestFolder\WorkFiles"
$ftype = "*.txt"
$filenames1 = Search $filefolder1 $ftype
$filenames1 | Out-File "C:\Work\ListofAllFilenamesToSearch.txt" -Width 2000
if (Test-Path "C:\Work\FilesThatNeedToBeChanged.txt") { # if file exists
Remove-Item "C:\Work\FilesThatNeedToBeChanged.txt"
}
(Get-Content "C:\Work\NumberXrefList.CSV" |where {$_.readcount -gt 1}) | foreach{
$OldFieldValue, $NewFieldValue = $_.Split("|")
$filenamelist = (Get-Content "C:\Work\ListofAllFilenamesToSearch.txt" -ReadCount 5) #|
foreach ($j in $filenamelist) {
#$testvar = (Get-Content $j )
#$testvar = (Get-Content $j -ReadCount 100)
$testvar = (Get-Content $j -Delimiter "\n")
Foreach ($i in $testvar)
{
if ($i -imatch $OldFieldValue) {
$j + "|" + $OldFieldValue + "|" + $NewFieldValue | Out-File "C:\Work\FilesThatNeedToBeChanged.txt" -Width 2000 -Append
}
}
}
}
$FileFolder = (Get-Content "C:\Work\FilesThatNeedToBeChanged.txt" -ReadCount 5)
Get-ChildItem $FileFolder -Recurse |
select -ExpandProperty fullname |
foreach {
if (Select-String -Path $_ -SimpleMatch $OldFieldValue -Debug -Quiet) {
(Get-Content $_) |
ForEach-Object {$_ -replace $OldFieldValue, $NewFieldValue }|
Set-Content $_ -WhatIf
}
}
In the code above, I've tried several things with Get-Content - default, with -ReadCount, and -Delimiter - in an attempt to avoid an out of memory error.
The only thing I have control over is the length of the old & new replacement strings file. Is there a way to do this in Powershell? Is there a better option/solution? I'm running Windows 7, Powershell version 3.0.
Your main problem is that you're reading the file over and over again to change each of the terms. You need to invert the looping of the replace terms and looping of the files. Also, pre-load the csv. Something like:
$filefolder1 = "C:\TestFolder\WorkFiles"
$ftype = "*.txt"
$filenames = gci -Path $filefolder1 -Filter $ftype -Recurse
$replaceValues = Import-Csv -Path "C:\Work\NumberXrefList.CSV"
foreach ($file in $filenames) {
$contents = Get-Content -Path $file
foreach ($replaceValue in $replaceValues) {
$contents = $contents -replace $replaceValue.OldValue, $replaceValue.NewValue
}
Copy-Item $file "$file.old"
Set-Content -Path $file -Value $contents
}
I have many file paths stored in a DB. I need to check if the files actually exist. I've done this before but lost the script for it and need some help.
I put all the paths in a text file and want to loop over them, and check if they exist. if they don't exist, I want to put the nonexistent path in a log file.
Something like this:
# ! equals -not
$log = "e:\pshell\notExists.log"
$log | out-file $log
$list = Get-Content "e:\pshell\files.txt"
Foreach ($file in $list)
{
CHECK IF FILE EXISTS
IF IT DOESNT then Write-Output $file
}
little help?
test-path?
$log = "e:\pshell\notExists.log" $log | out-file $log
$list = Get-Content "e:\pshell\files.txt"
Foreach ($file in $list)
{
If (!(test-path $file))
{
Write-Output $file
}
}
If you inputfile is one filepath per line, try:
$log = "e:\pshell\notExists.log"
Get-Content "e:\pshell\files.txt" | Where-Object {
#Keep only paths that does not exists
!(Test-Path $_)
} | Set-Content $log
$log = "e:\pshell\notExists.log"
Get-Content "e:\pshell\files.txt" |
where {!(test-path $_)} |
add-content $log