I'm in trouble here. newPath is a parameter of the function analyse which is written here. I have problems with my conditions...
The objective is to write every address in the .txt if it isn't there yet and if mSize (result from a working function) is greater than seuil (threshold). "seuil" equals 0 here. I'm matching the code in a folder, and every folder after this one is analysed.
Problem is, it works fine... For 2 addresses only.
I dont get it, when I launch the code, only two lines are written. With tests, I think "result" actually takes the null value and I don't know when and why.
$result = get-content -Path O:\folder\public\F*******\parcoursArborescence\logs.txt
$test = "start"
$test
cd $newPath
$newPath
$result
$dirSize = [math]::round($(getDirSize)/1048576, 2)
$mSize = [math]::round($(multimediaSize), 2)
if (($result -match [regex]::Escape($newPath + ";") -eq $FALSE)) {
$testR = $false
}
if (($result -match [regex]::Escape($newPath + ";") -eq $TRUE)) {
$testR = $true
}
$mSize
$testR
if ($mSize -gt $seuil -and $testR -eq $FALSE) {
$dataWriting = "writing..."
$dataWriting
$data = $newPath + ";" + $dirSize + ";" + $mSize + ";" + $(mmPercent) + ";" + $(getLastWriting) + ";" + $(getLastAccess)
$data | out-file -append O:\folder\public\F*******\parcoursArborescence\logs.txt -Encoding ASCII
}
dir |
foreach-object {
if ($_.PsISContainer -eq $True -and $result -notMatch [regex]::escape($_.fullname + ";")) {
$oldPath = $newPath
$newPath = $_.fullname
analyse $newPath
}
else {
$oldPath = cd ..\ | get-location
$newPath = $oldPath
}
}
Here are some lines of my textfile :
O:\folder\public\DataIntegrator;1030.95;812.7;79;08/13/2013 13:28:49;11/25/2015 09:47:28
O:\folder\public\DataIntegrator\package;988.99;810.93;82;08/13/2013 13:28:49;11/25/2015 09:47:28
My work is to classify differents types of files. I want the textfile to be full of adresses which guide to heavy folders (where length < "seuil"). The condition i'm working on is supposed not to allow adresses already in the .txt to be written again. These two lines are the only ones working atm ; after this, no address is written. I tried to put some tests values here and there ; the important one (I guess) is the "testR" one. At the beginning, it works and says its value is either true or false. But, I dont know why and when, it begins to say absolutely nothing, letting me guess it gets a null value.
Exemple of newPath : 0:\folder\public\DataIntegrator
-> Analyses DataIntegrator
EDIT : I modified the code a little. Here it is :
http://pastebin.com/zKaK2ZWX
For the problem, the only lines that matter are the 42 first. Even with all these conditions, testR doesnt change and stays false. It never confirm one of the 2 other conditions. If I switch testR to true, then it will stay true.
Okay, I think I got it.
I did more and more tests about the -match condition. It looks like the output isnt a boolean, but a String ; it sends me the lines of $result which contain the String $testPath, so the result never is either false or true.
But I have a new problem, then. How can I get a boolean out of it ? I need to know if $result contains $testPath.
Related
I use a simple function to download files and return the path to me when updating computers for simplicity.
I was stuck on why it was not working then realized that the proxy is appending a random number to the filename so instead of it being 12345.zip it is actually 8493830_12345.zip.
I have tried to find the file using the "_" as a split but while there are no errors, the file is not being returned and I have checked it is there manually.
function FileCheck {
$fileName.Split("_")[1]
$fileName = "{0}.zip" -f 12345
Download -ZipFileName $($fileName) -OutputDirectory $env:temp
$SleepTime = 300
$sleepElapsed = 0
$sleepInterval = 20
Start-Sleep $sleepInterval
$file = Get-ChildItem -Path $env:temp -Filter "$fileName*"
if ($file -ne $null) {
return $file[0].FullName
}
Start-Sleep($sleepInterval)
$sleepElapsed += $sleepInterval
if (($SleepTime) -le $sleepElapsed){
# Check for file with given prefix
$file = Get-ChildItem -Path $env:temp -Filter "$fileName*"
if ($file -eq $null) {
Write-Error 'file not found'
return $null
}
return $file[0].FullName
}
}
I am guessing the split is not working but googling and moving the filename.split has not worked for me. Any help is appreciated
Well, your split is doing nothing at all. You haven't defined $filename, but if you had, and it had an underscore, then $filename.split('_') would return two or more strings, depending on how many underscores were in the original string, but you never capture the result. I think the real problem here is the filter you are applying to Get-ChildItem later in your function.
$file = Get-ChildItem -Path $env:temp -Filter "$fileName*"
That will look for files beginning with $fileName, which you define on line 4 to be "12345.zip". That is exactly the opposite of what you want to be looking for. You need to move the asterisk to before $fileName, so it looks like this:
$file = Get-ChildItem -Path $env:temp -Filter "*$fileName"
That will return all files that end with "12345.zip", which would include things like:
myfuzzyhippo12345.zip
learn-to-count-12345.zip
8493830_12345.zip
Basically anything that ends in 12345.zip. Also, it appears that you are under the impression that executing a return $file[0].fullname or return $null will stop the function. That's a mistake. A function runs to completion unless exited early by something like a break command. Also, everything not explicitly captured or redirected will be passed back from the function, so reading through your function people are likely to get the output of your $filename.split('_') line, then possibly $null or $filename[0].fullname.
Lastly, it appears that you're trying to look for the file, if you don't find it to wait a bit, and try again, until $sleepElapsed is greater than $sleepTime. What you want here is a While or a Do/While loop. Here's what I'd do...
function FileCheck {
Param(
$fileName = '12345.zip',
$SleepTime = 300,
$sleepElapsed = 0,
$sleepInterval = 20
)
Download -ZipFileName $($fileName) -OutputDirectory $env:temp
Do{
Start-Sleep $sleepInterval
$sleepElapsed = $sleepElapsed + $sleepInterval
$file = Get-ChildItem -Path $env:temp -Filter "*$fileName"|Select -First 1
}While(!$file -and $sleepElapsed -le $sleepTime)
$file.FullName
}
That lets you define things like sleep settings at runtime if you want, or just let it default to what you were using, same with the file name. Then it downloads the file, and looks for it, pausing between attempts, until either it finds the file, or it runs out of time. Then it returns $file.FullName which is either the path to the file if it found one, or nothing if it didn't find a file.
Personally I'd have it return the file object, and just utilize the .FullName property if that's all I wanted later. Usually (not always, but usually) more info returned from a function is better than less info. Like what if the download fails and it's a zero byte file? Just returning only the path doesn't tell you that.
I've got a simple bit of code that looks for a string in a series of log files.
If it finds the string, it should exit the loop (nested inside another loop as part of a function) with $buildlogsuccess = 'True'
If it can't find the string, it should exit and return $buildlogsuccess = 'False'
The select-string statement itself works, however it looks like there's something wrong with the below code:
$logArr = gci C:\build\Logs | where {($_.name -like 'install*.log') -and (! $_.PSIsContainer)} | select -expand FullName
$count = ($logArr).count
Foreach ($log in $logArr) {
Do {
$count -= 1
$buildlogsuccess = [bool](select-string -path $log -simplematch $buildstring)
If (($buildlogsuccess)) {break}
} while ($count -gt '0')
}
When one of the logs has the string, the loop finishes and should return $buildlogsuccess as 'True'.
If I check $log it shows the file that I know has the string (in this instance C:\build\Logs\Installer1.log).
Strangely, at this point $count shows as having a value of -1?
If I take the string out of that file and run again it also exits and returns the correct variable value (and shows the $log variable as the last file in $logArr as expected), but this time $count shows as -24.
My code is also returning $buildlogsuccess as 'False' when the string is present in one of the log files.
Re-tested [bool](select-string -path $log -simplematch $buildstring) by manually populating $log (with a file that has that string) and $buildstring and get 'True' as expected when using
[bool](select-string -path $log -simplematch $buildstring)
Note: Variables it uses:
$buildstring = "Package
'F:\xxx\Bootstrap\apackage\Installsomething.xml' processed
successfully"
Any help identifying where I've gone wrong would be appreciated.
Your code can be greatly simplified:
$buildlogsuccess = Select-String -SimpleMatch -Quiet $buildstring C:\build\Logs\install*.log
The above assumes that there are no directories that match install*.log; if there's a chance of that, pipe the output of Get-ChildItem -File C:\build\Logs -Filter install*.log to Select-String instead.
Do-while will first do the thing, then check the while statement. You're iterating over n files. It doesn't check the value of $count before it executes that portion.
So let's say the first file does not contain the string you're looking for. It will (correctly) decrement the $count variable to zero, and then it moves on to the next $log in $logArr.
Now for each next file in the folder it will decrement $count, and then exit the loop when it sees that $count is not greater than 0.
I don't know why you're using the do-while loop at all here
Thanks Norsk
I over-complicated for myself.
This worked:
$logArr = gci C:\build\Logs | where {($_.name -like 'install*.log') -and (! $_.PSIsContainer)} | select -expand FullName
$count = ($logArr).count
Foreach ($log in $logArr) {
$buildlogsuccess = [bool](select-string -path $log -simplematch $buildstring)
If ($buildlogsuccess) {break}
}
I've been struggling with what I think is a really simple problem but I can't see it. I have a stack of 30-odd csv files of varying contents generated daily by different applications that I need to normalize before importing into a single reporting db. An Extract, Transform and Load (ETL) type of thing - global find and replace.
Looping through the files is no problem - not sure whether using ForEach-Object Fullname is the best way to go as outputting to an 'OUT' folder messes it up but using -Name means I have to include the path.
Basically, all 'True'/'False' text is to be replaced with 1/0, same with 'yes'/'no', poweredon/poweredoff, etc. Also we have 4 sites - each needs replacing with a ref. id, loads of stuff like that. I've tried modifying loads of scripts I've found on line - many in here. Tried using the replacement text in an array, pulling the CSV into a string, just can't see it. I've been doing the same thing for years with VBScript and it's easy. But I need to learn PowerShell so I'm going to persevere with it.
Ok, here is a quick search and replace function for you. It can read multiple CSV files and match\replace multiple values.
function Replace-CsvValue
{
[CmdletBinding()] # Enable pipeline support
Param
(
# Filename, mandatory, takes pipeline input
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
# Alias, allows to directly pipe Get-ChildItem output to this function
[Alias('FullName')]
[string]$File,
# Scriptblock, mandatory, does actual search and replace
[Parameter(Mandatory = $true, ValueFromPipelineByPropertyName = $true)]
[scriptblock]$ScriptBlock
)
Process
{
# Import CSV
$CsvFile = $File | Import-Csv
# Generate new filename
$NewFileName = Join-Path -Path (Split-Path -Path $File -Parent) -ChildPath ('Processed_' + (Split-Path -Path $File -Leaf))
# Iterate over each line in CSV
$CsvFile | ForEach-Object {
# Execute scritblock against record
& $ScriptBlock
}
# Export CSV
$CsvFile | Export-Csv -Path $NewFileName -NoTypeInformation
}
}
Usage:
Write scriptblock with required replace logic
Pipe filenames or Get-ChildItem output to the function and pass scriptblock
Example:
Original CSV file:
State, Active, Available
PoweredOn, True, Yes
Function call:
# Scriptblock with replace logic
$ReplaceRule = {
# Iterate over each item in CSV line
$Item = $_
$_.PSObject.Properties.Name | ForEach-Object {
# If item name matches...
switch ($_)
{
'State' {
# If item value matches...
if($Item.$_ -eq 'PoweredOn')
{
$Item.$_ = 'Online'
}
# Or if item value matches...
elseif($Item.$_ -eq 'PoweredOff')
{
$Item.$_ = 'Offline'
}
break
}
# More replace rules, you can add your own here...
'Active' {
if($Item.$_ -eq 'True')
{
$Item.$_ = '1'
}
elseif($Item.$_ -eq 'False')
{
$Item.$_ = '0'
}
break
}
'Available' {
if($Item.$_ -eq 'Yes')
{
$Item.$_ = '1'
}
elseif($Item.$_ -eq 'No')
{
$Item.$_ = '0'
}
break
}
}
}
}
# Get all CSV files that match wildcard and
# feed them to the Replace-CsvValue function
Get-ChildItem -Path '.\' -Filter '*Report*.csv' | Replace-CsvValue -ScriptBlock $ReplaceRule
Processed CSV file:
"State","Active","Available"
"Online","1","1"
I made this csv for testing with the help of Mockaroo. Notice someones first name is True. I have that in there as a check to be sure my logic is working.
Present Name Lunch State
------- ---- ----- -----
TRUE Jesse Daniels No Powered Off
FALSE Debra Cunningham Yes Powered Off
TRUE True Jones Yes Powered Off
TRUE George Fernandez Yes Powered Off
FALSE Lisa Cox No Powered On
For the purpose of this I think it would be simple to just ignore the fact that it is a CSV and just replace the text outright. The caveat we have to be careful for is partial matches. Using regex we should be able to account for that possibility.
From comments you already know that you can chain -replace. Lets add some regex magic in there to make the process easier.
$filename = "C:\temp\MOCK_DATA.csv"
$oneKeywordPattern = "Powered On","Yes","True" -join "|"
$zeroKeywordPattern = "Powered Off","No","False" -join "|"
(Get-Content $filename) -replace "(?<=^|,)$oneKeywordPattern(?=$|,)","1" -replace "(?<=^|,)$zeroKeywordPattern(?=$|,)","0" | Set-Content $filename
To make sure that the csv structure is accounted for we only replace if the element is at the start of the line or a comma followed the end of the line or comma (This is using a lookahead and lookbehind.). This also ensures that we only change full elements and True Jones is not affected.
We used $oneKeywordPattern so that you can add elements to the array that need to be changed to a 1. We join them with a pipe so that it is treated as a alternative regex pattern. Its counterpart $zeroKeywordPattern functions just the same.
Output
Present Name Lunch State
------- ---- ----- -----
1 Jesse Daniels 0 0
0 Debra Cunningham 1 0
1 True Jones 1 0
1 George Fernandez 1 0
0 Lisa Cox 0 1
You could likely have other patterns that do not need to be changed with this logic. Just chain another -replace and remember that it supports regex so watch out for special characters.
The two caveats here is that if the files are large it could take a while to load the file and process the regexes (especially if you add more.) Also if your text is enclosed in quotes we don't currently account for that but it would be easy.
Basically, all 'True'/'False' text is to be replaced with 1/0, same with 'yes'/'no', poweredon/poweredoff, etc. Also we have 4 sites - each needs replacing with a ref. id, loads of stuff like that. I've tried modifying loads of scripts I've found on line - many in here. Tried using the replacement text in an array, pulling the csv into a string, just can't see it. I've been doing the same thing for years with vbscript and it's easy. But I need to learn PShell so I'm going to persevere with it. I'd really appreciate some help here.
If it's that static, you can probably get away with:
$changes = #{
'true' = '1';
'false' = '0';
'poweredon' = '1';
'poweredoff' = '0'
}
$folder = "" # your folder here
$csvFiles = ls $folder *.csv
foreach ($file in $csvFiles) {
$csvData = import-csv $file
foreach ($row in $csvData) {
$cells = $row | `
gm | `
?{$_.MemberType -eq 'NoteProperty'} | `
select -exp Name
foreach ( $cell in $cells ) {
$val = $row."$cell"
$valueNeedsChanging = $changes.ContainsKey($val)
if ( $valueNeedsChanging ) {
$newValue = $changes[$val]
$row."$cell" = $newValue
}
}
}
cp $file.FullName "$($file.FullName).bak" # back it up before saving
$csvData | export-csv -Path $file.FullName -NoTypeInformation
}
I chose to use Import- and Export-CSV to preserve the structure of the CSV file for files that have a lot of advanced formatting.
I have a script that searches for a series of strings (stored in a txt file) in the contents of files in a directory. I would like to modify it to also list the text around the string found (these are regular strings, not regex expressions). I played around a lot and it seems like I need to use -Context, but I am not sure how to get the text from that.
Also, the files I am searching may not have linefeeds, so if it could just get the xx characters before and after the search term, that would be better.
Here's what I have so far (I omitted the looping though files parts):
$result = Get-Content $file.FullName | Select-String $control -quiet
If ($result -eq $True)
{
$match = $file.FullName
"Match on string : $control in file : $match" | Out-File $output -Append
Write-host "Match on string : $control in file : $match"
}
If it could write the context, that would be perfect. Seems like I need to use $_Matches, but not sure how.
If $control is just a regular string, can you turn it into a regular expression?
$n = 3
$re = "(.{0,$n})(" + [Regex]::Escape($control) + ")(.{0,$n})"
$result = (Get-Content $file.FullName) -match $re
With this, the $matches hashtable should give you access to the $n characters before and after the match:
if ($result.Length -gt 0) {
echo "Before: $($matches[1])"
echo "After: $($matches[3])"
}
Here is what I have now and it seems to work:
$regex = "[\s\S]{0,$ContextChars}$SearchTerm[\s\S]{0,$ContextChars}"
$results = Get-Content $file.FullName | Select-String -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value }
if ($results)
{
foreach($result in $results)
{
$display = $result
"File: $file Match ---$display---"
}
}
The only thing I wish I had but don't know how to get it is the line number the match is found on.
I'm new to PowerShell and been trying to get this script to work.
If ((Get-Date -UFormat %a) -eq "Mon") {$intSubtract = -3}
Else {$intSubtract = -1}
$datDate = (Get-Date).AddDays($intSubtract)
Write-Output "Find expected file --------------"
$strDate = ($datDate).ToString('yyyyMMdd')
Write-Host "strDate: $strDate"
$arrGetFile = Get-ChildItem -Path "\\Computer\Data\States\NorthDakota\Cities\*_Bismark_$strDate*.txt"
$strLocalFileName = $arrGetFile
If ($arrGetFile.count -ne 2)
{
Throw "No file or more than two files with today's date exists!"
}
Else {$strLocalFileName = $arrGetFile[0].Name}
Write-Output "Found file $strLocalFileName --------------"
#Encrypt each file
foreach ($arrGetFile in $strPath)
{
Write-Output "Start Encrypt --------------"
$strPath = "\\Computer\Data\States\NorthDakota\Cities\"
$FileAndPath = Join-Path $strPath $strLocalFileName
$Recipient = "0xA49B4B5D"
Import-Module \\JAMS\C$\PSM_PGP.psm1
Get-Module Encrypt
Encrypt $FileAndPath $Recipient
$strLocalFileNamePGP = $strLocalFileName + ".pgp"
Write-Output "End Encrypt --------------"
}
#Archive files
Write-Output "Archiving --------------"
move-item -path \\Computer\Data\States\NorthDakota\Cities\*_Bismark_$strDate*.txt -destination \\Computer\Data\States\NorthDakota\Cities\Archive
The Cities folder will contain two files. Example 2015_Bismark_20150626_183121.txt and 2015_Bismark_20150626_183121_Control.txt
I am trying to get both files encrypted however it is only finding and encrypting the file without _Control. It is archiving both files correctly.
Not sure what I am missing to also find the control file.
Your for loop is incorrect. You have foreach ($arrGetFile in $strPath), but $strPath doesn't seem to contain anything at that point.
Your for loop should be:
foreach ($LocalFile in $arrGetFile)
And you need to remove the following line:
$strLocalFileName = $arrGetFile
This is making $strLocalFileName an array of file objects, but later in the script you are treating it like a string. You may have more logical errors--you need to walk through the script very carefully and identify each variable and make sure it contains what you expect it to contain.
In general you seem to be treating arrays of non-string objects as if they are strings. Note that I changed your $strLocalFileName variable to $LocalFile. This is because it is a file object, not a string object.
Following is a sample that just shows that the for loop iterates through the both files.
If ((Get-Date -UFormat %a) -eq "Mon") {$intSubtract = -3}
Else {$intSubtract = -1}
$datDate = (Get-Date).AddDays($intSubtract)
Write-Output "Find expected file --------------"
$strDate = ($datDate).ToString('yyyyMMdd')
Write-Host "strDate: $strDate"
$arrGetFile = Get-ChildItem -Path "\\Computer\Data\States\NorthDakota\Cities\*_Bismark_$strDate*.txt"
If ($arrGetFile.count -ne 2)
{
Throw "No file or more than two files with today's date exists!"
}
Write-Output "Found files " ($arrGetFile | select Name | fl) "--------------"
#Process each file
foreach ($LocalFile in $arrGetFile)
{
$FileAndPath = Join-Path $LocalFile.DirectoryName $LocalFile
$FileAndPath
}
Start with this and then carefully add your encryption processing back into the loop.
Also, The line that assigns $FileAndPath could be removed. You can just use $LocalFile.FullName wherever you need the full path and filename.