I have a CSV file that gets created with long lists of images paths. E.x. ( del \servername\images\1.2.840.11....really long number.dic )
I want to run though this list and execute each of these lines. I tried to do some research and saw people using powershell for this. I wrote a small script but I am not sure how to get powerscript to execute each line of the CSV.
I am sure I just do not know powershell very well but I am open to try any method of going though a CSV and executing the lines.
$delList=IMPORT-CSV C:\application\imagesToDelete.csv
FOREACH ($Image in $delList) {
-- execute each line somehow
}
Thanks in advance, this is a one off issue I need to deal with but I have several CSV's containing tens of thousands of images to be deleted
Ok, let's make this simple and not use Import-CSV. We'll just use Get-Content and skip the header line. Then we use Remove-Item, get a substring of each line starting at the 4th character and going to the end of the line, and remove the result.
$Commands = GC $file | Select -Skip 1
$Counter=1
ForEach($Line in $Commands){
Write-Progress -Activity "Performing commands ($Counter/$($Commands.Count))" -PercentComplete ($Counter/$commands.count*100) -Status "Please wait..."
Remove-Item $Line.substring(4,$line.length-4)
$Counter++
}
Related
Good evening,
I'm hardly experienced in programming, but every now and then I try to build the things I need myself.
What do I want to achieve with the script?
This script should read a text file with words.
There is one new word per line. When reading the script should look if the word has between 3 and 16 letters. If it has less than 3 or more than 16, then the word is skipped. But if it is between the 3 and 16, then the word will be saved in a new Text_File. Again, I would love a new word every line.
Here is what I created.
Please don't judge my script too hard.
$regex = '[A-Z][a-z]{3,16}'
foreach ($line in Get-Content -Path C:\\Users\\Administrator\\Desktop\\namecheck\\words.txt)
{
if($line -match $regex)
{
Out-File -FilePath C:\\Users\\Administrator\\Desktop\\namecheck\\sorted.txt -Force
}
}
As mentioned above, the words are not written to a file. However, the file is created and the script also takes a little while to finish. So from my point of view something seems to happen.
'[A-Z][a-z]{3,16}' is only accounting for words that are of length 3+. That would be your first issue, and your second one is your export. Out-File isn't being told what you want to export. So, either provide it the a value via pipeline input, or using -InputObject:
$path = 'C:\Users\Administrator\Desktop\namecheck\'
$stream = [System.IO.StreamReader]::new("$path\words.txt")
while ($line = $stream.ReadLine())
{
if ($line.Length -gt 3 -and $line.Length -lt 16)
{
Out-File -FilePath "$path\sorted.txt" -InputObject $line -Append -Force
}
}
$stream.Close()
$stream.Dispose()
Although, a foreach loop is the fastest of the loops, when using Get-Content it still has to wait for the completion of the content gathering before it can move through the list. Only mentioning this since you said that the script takes quite a bit and without knowing the size of your words.txt file, im left to assume that's the cause.
With that said, using [System.IO.StreamReader] should speed up that reading of the file as you'll get to read and iterate through the file at the same time; with a while loop that is.
first time poster here, but you helped me a lot before. I don´t know how to ask google this question.
I have a powershell script, where with foreach command i check every computer in .txt that contains computer names. (Short explanation is that in check bitlocker status, connection avalability etc.) Everything works fine, but since I fall in love with powershell recently and try to automate more and more thing, that i should upgrade this script little more.
I have foreach ($DestinationComputer in $DestinationComputers) and after i check everything i wanted, i want to delete that row in .txt file.
Can someone help? I am still learning this and got stuck.
Continuing from my comment, I suggest creating a list of computernames that did not process correctly, while discarding the ones that did not fail.
By doing so, you will effectively remove the items from the text file.
Something like this:
$DestinationComputers = Get-Content -Path 'X:\somewhere\computers.txt'
# create a list variable to store computernames in
$list = [System.Collections.Generic.List[string]]::new()
# loop over the computer names
foreach ($DestinationComputer in $DestinationComputers) {
# first check: is the machine available?
$succes = Test-Connection -ComputerName $DestinationComputer -Count 1 -Quiet
if ($succes) {
# do whatever you need to do with that $DestinationComputer
# if anything there fails, set variable $success to $false
<YOUR CODE HERE>
}
# test if we processed the computer successfully and if not,
# add the computername to the list. If all went OK, we do not
# add it to the list, thus removing it from the input text file
if (-not $success) {
$list.Add($DestinationComputer)
}
}
# now, write out the computernames we collected in $list
# conputernames that were processed OK will not be in there anymore.
# I'm using a new filename so we don't overwrite the original, but if that is
# what you want, you can set the same filename as the original here.
$list | Set-Content -Path 'X:\somewhere\computers_2.txt'
This is my sample code below:
foreach ($var1 in (gc 1.txt)){
// my code logic
}
Here 1.txt file contains list of values like abc, xyz, pqr etc..,like 100 lines in 1.txt file for processing one by one. After processing got completed for 100 lines, there is a new value altered in middle for processing and the count now is 101 lines. Now i need to restart the script and should process newly added value rather than starting from the scratch.
Actually for this requirement there are hundreds of lines are there in my project.
Can you please suggest me the best way to achieve this?
Thanks in advance
Try this out - saving a snapshot of the file and comparing it to the current list:
# Import values and list of completed updates
$ToUpdate = Get-Content list.txt
$Completed = Get-Content completed.txt
# Take a snapshot for next run
Copy-Item -Path list.txt -Destination completed.txt -Force
# use Compare to determine which updates are new
$Unprocessed = Compare-Object $ToUpdate $Completed |
where SideIndicator -EQ '<=' |
Select -ExpandProperty InputObject
Foreach ($var1 in $Unprocessed) {
# Do Stuff
}
Ideally, you want to check against your actual target instead of a log file. If you're updating a database/user directory/grocery list, query that instead, because otherwise you can run into issues. What if something else updates the target while you're not looking? What if your script errors out and doesn't actually complete all the updates? Something to keep in mind.
We are copying a long list of files from their different directories into a single location (same server). Once there, I need to rename them.
I was able to move the files until I found out that there are duplicates in the list of file names to move (and rename). It would not allow me to copy the file multiple times into the same destination.
Here is the list of file names after the move:
"10.csv",
"11.csv",
"12.csv",
"13.csv",
"14.csv",
"15.csv",
"16.csv",
"17.csv",
"18.csv",
"19.csv",
"20.csv",
"Invoices_Export(16) - Copy.csv" (this one's name should be "Zebra.csv")
I wrote a couple of foreach loops, but it is not working exactly correctly.
The script moves the files just fine. It is the rename that is not working the way I want. The first file does not rename; the other files rename. However, they leave the moved file in place too.
This script requires a csv that has 3 columns:
Path of the file, including the file name (eg. c:\temp\smefile.txt)
Destination of the file, including the file name (eg. c:\temp\smefile.txt)
New name of the file. Just the name and extention.
# Variables
$Path = (import-csv C:\temp\Test-CSV.csv).Path
$Dest = (import-csv C:\temp\Test-CSV.csv).Destination
$NN = (import-csv C:\temp\Test-CSV.csv).NewName
#Script
foreach ($D in $Dest) {
$i -eq 0
Foreach ($P in $Path) {
Copy-Item $P -destination C:\Temp\TestDestination -force
}
rename-item -path "$D" -newname $NN[$i] -force
$i += 1
}
There were no error per se, just not the outcome that I expected.
Welcome to Stack Overflow!
There are a couple ways to approach the duplicate names situation:
Check if the file exists already in the destination with Test-Path. If it does, start a while loop that appends a number to the end of the name and check if that exists. Increment the number you append after each check with Test-Path. Keep looping until Test-Path comes back $false and then break out of the loop.
Write an error message and skip that row in the CSV.
I'm going to show a refactored version of your script with approach #2 above:
$csv = Import-Csv 'C:\temp\Test-CSV.csv'
foreach ($row in $csv)
{
$fullDestinationPath = Join-Path -Path $row.Destination -ChildPath $row.NewName
if (Test-Path $fullDestinationPath)
{
Write-Error ("The path '$fullDestinationPath' already exists. " +
"Skipping row for $($row.Path).")
continue
}
# You may also want to check if $row.Path exists before attempting to copy it
Copy-Item -Path $row.Path -Destination $fullDestinationPath
}
Now that your question is answered, here are some thoughts for improving your code:
Avoid using acronyms and abbreviations in identifiers (variable names, function names, etc.) when possible. Remember that code is written for humans and someone else has to be able to understand your code; make everything as obvious as possible. Someone else will have to read your code eventually, even if it's Future-You™!
Don't Repeat Yourself (called the "DRY" principle). As Lee_daily mentioned in the comments, you don't need to import the CSV file three times. Import it once into a variable and then use the variable to access the properties.
Try to be consistent. PowerShell is case-insensitive, but you should pick a style and stick to it (i.e. ForEach or foreach, Rename-Item or rename-item, etc.). I would recommend PascalCase as PowerShell cmdlets are all in PascalCase.
Wrap literal paths in single quotes (or double quotes if you need string interpolation). Paths can have spaces in them and without quotes, PowerShell interprets a space as you are passing another argument.
$i -eq 0 is not an assignment statement, it is a boolean expression. When you run $i -eq 0, PowerShell will return $true or $false because you are asking it if the value stored in $i is 0. To assign the value 0 to $i, you need to write it like this: $i = 0.
There's nothing wrong with $i += 1, but it could be shortened to $i++, if you want to.
When you can, try to check for common issues that may come up with your code. Always think about what can go wrong. "If I copy a file, what can go wrong? Does the source file or folder exist? Is the name pulled from the CSV a valid path name or does it contain characters that are invalid in a path (like :)?" This is called defensive programming and it will save you so so many headaches. As with anything in life, be careful not to go overboard. Only check for likely scenarios; rare edge-cases should just raise errors.
Write some decent logs so you can see what happened at runtime. PowerShell provides a pair of great cmdlets called Start-Transcript and Stop-Transcript. These cmdlets log all the output that was sent to the PowerShell console window, in addition to some system information like the version of PowerShell installed on the machine. Very handy!
I have a linux server that will be generating several files throughout the day that need to be inserted in to a database; using Putty I can sftp them off to a server running SQL 2008. Problem is is the structure of the file itself, it has a string of text that are to be placed in different columns, but bulk insert in sql tries to put it all in to one column instead of six. Powershell may not be the best method, but I have seen on several sites how it can find and replace or append to the end of the line, can it count and insert?
So the file looks like this: '18240087A +17135555555 3333333333', where 18, 24, 00, 87, A are different columns, then there is a blank space between the A and the +, that is character count 10-19 which is another column, then characters 20-30 are a column, characters 31-36 are a space which is new column and so on. So I want to insert a '|' or a ',' so that sql understands where the columns end. Is this possible for PowerShell to count randomly?
This may not be the way to respond to all who did answer, i apologize in advance. As this is my first PowerShell script, I appreciate the input from each of you. This is an Avaya SIP server that is generating CDR records, which I must pull from the server and insert in to SQL for later reports. The file exported looks like this:
18:47 10/15
18470214A +14434444444 3013777777 CME-SBC HHHH-CM 4 M00 0
At first I just thought to delete the first line and run a script against the output, which I modified from Kieranties post:
$test = Get-Content C:\Share\CDR\testCDR.txt
$pattern = "^(.{2})(.{2})(.{1})(.{2})(.{1})(.{1})\s*(.{15})(.{10})\s*(.{7})\s*(.{7})\s*(.{1})\s*(.{1})(.{1})(.{1})\s*(.*)$"
if($test -match $pattern){
$result = $matches.Values | select -first ($matches.Count-1)
[array]::Reverse($result, 0, $result.Length)
$result = $result -join "|"
$result | Out-File c:\Share\CDR\results1.txt
}
But then i realized I need that first line as it contains the date. I can try to work that out another way though.
I also now see that there are times when the file contains 2 or more lines of CDR info, such as:
18:24 10/15
18240087A +14434444444 3013777777 CME-SBC HRSA-CM 4 M00 0
18240096A +14434444445 3013777778 CME-SBC HRSA-CM 4 M00 0
Whereas the .ps1 file I made does not give the second string, so I tried adding in this:
foreach ($Data in $test)
{
$Data = $Data -split(',')
and it fails to run. How can I do multiple lines (and possibly that first line)? If you know of a tutorial that can help, that's greatly appreciated as well!
PowerShell is a great tool that I love and it can do many things. I see that you are using SQL Server 2008. Depending on the edition of SQL Server you have running on the server, it most likely has SQL Server Integration Services (SSIS), which is an Extract, Transform, and Load (ETL) tool designed to help migrate data in many scenarios, such as yours. The file you describe here is sounds like a fixed width file, which SSIS can easily handle and import and SQL Server has great ways to automate the loads if this is a recurring need (Which it sounds like), including the automation of the sftp task, and even running PowerShell scripts as part of the ETL (I've done that several times).
If your file truly is fixed width and you want to use PowerShell to transform it into a delimited file, the regex approach you have in your answer works well, or there are several approaches using the System.String methods, like .insert() which allows you to insert a delimiter character using a character index in your line (use Get-Content to read the file and create one String object per line, then loop through them using Foreach loop or Foreach-Object and the pipeline). A slightly more difficult approach would be to use the .Substring() method. You could build your new String line using Substring to extract each column and concatenating those values with a delimiter. That's probably a lot for someone new to PowerShell, but one of the best ways to learn and gain proficiency with it is to practice writing the same script multiple ways. You can learn new techniques that may solve other problems you might encounter in the future.
This is a way (really ugly IMO, I think it can better done):
$a = '18240087A +17135555555 3333333333'
$b = #( ($a[0..1] -join ''), ($a[2..3] -join ''), ($a[4..5] -join ''),
($a[6..7] -join ''), ($a[8] -join ''), ($A[10..19] -join ''),
($a[20..30] -join ''), ($a[31..36] -join ''))
$c = $b -join '|'
$c
18|24|00|87|A|+171355555|55 33333333|33
I don't know if is the rigth splitting you need, but changing the values in each [x..y] you can do what better fit your need. Remenber that character array are 0-based, then the first char is 0 and so on.
I don't quite follow the splitting rules. What kind of software writes the text file anyway? Maybe it can be instructed to change the structure?
That being said, inserting pipes is easy enough with .Insert()
$a= '18240087A +17135555555 3333333333'
$a.Substring(0, $a.IndexOf('+')).Insert(2, '|').insert(5,'|').insert(8, '|').insert(11, '|').insert(13, '|')
# Output: 18|24|00|87|A|
# Rest of the line:
$a.Substring($a.IndexOf('+')+1)
# Output: 17135555555 3333333333
From there you can proceed to splitting the rest of the row data.
I've improved my answer based on your response (note, it's probably best you update your actual question to include that information!)
The nice thing about Get-Content in Powershell is that it returns the content as an array split on the end of line characters. Couple that with allowing multiple assignment from an array and you end up with some neat code.
The following has a function to process each line based on your modified version of my original answer. It's then wrapped by a function which processes the file.
This reads the given file, setting the first line to $date and the rest of the content to $content. It then creates an output file adds the date to the output, then loops over the rest of the content performing the regex check and adding the parsed version of the content if the check is successful.
Function Parse-CDRFileLine {
Param(
[string]$line
)
$pattern = "^(.{2})(.{2})(.{1})(.{2})(.{1})(.{1})\s*(.{15})(.{10})\s*(.{7})\s*(.{7})\s*(.{1})\s*(.{1})(.{1})(.{1})\s*(.*)$"
if($line -match $pattern){
$result = $matches.Values | select -first ($matches.Count-1)
[array]::Reverse($result, 0, $result.Length)
$result = $result -join "|"
$result
}
}
Function Parse-CDRFile{
Param(
[string]$filepath
)
# Read content, setting first line to $date, the rest to $content
$date,$content = Get-Content $filepath
# Create the output file, overwrite if neccessary
$outputFile = New-Item "$filepath.out" -ItemType file -Force
# Add the date line
Set-Content $outputFile $date
# Process the rest of the content
$content |
? { -not([string]::IsNullOrEmpty($_)) } |
% { Add-Content $outputFile (Parse-CDRFileLine $_) }
}
Parse-CDRFile "C:\input.txt"
I used your sample input and the result I get is:
18:24 10/15
18|24|0|08|7|A|+14434444444 30|13777777 C|ME-SBC |HRSA-CM|4|M|0|0|0
18|24|0|09|6|A|+14434444445 30|13777778 C|ME-SBC |HRSA-CM|4|M|0|0|0
There are an incredible amount of resources out there but one I particularly suggest is Douglas Finkes Powershell for Developers It's short, concise and full of great info that will get you thinking in the right mindset with Powershell