Layering multiple 'ForEach' statements to loop through a directory and perform actions on each file found - powershell

Question

I've been working on this Powershell script for a good week now, and it almost works as expected.
Essentially, the script reaches into the specified directory which we have another script dropping .CSV files into, grabs the .CSV file(s) and pushes the information found into a Sharepoint list, well, that's the intention anyway. I've gotten the script to work perfectly if I manually specify the file, the issue I am having is actually getting all the .CSV files into a group, and then looping through each .CSV to pull the information out and push it into a Sharepoint list. Once done, it renames the file from .CSV to .ARCHIVED for another script to come in and re-locate after we're done with it.
I think I have, through selective (creative) troubleshooting, figured out what I am doing wrong, I just don't know how to proceed after identifying the issue.
I declare the string $Filecsv like so:
$Filecsv = get-childitem "Z:\" -recurse | where {$_.extension -eq ".csv"}
So, this reaches into my 'Z:\' directory, and pulls all the files with .CSV extension and combines them into a table...
ForEach ($items in $Filecsv) {
And this says for each item, perform logic...
foreach($row in $Filecsv)
The only problem is, when I call $Filecsv, it is returning the list of each .CSV file in the directory like such:
And as such, when I execute the bit of code that says 'put the information into my list', only the file name is added to my Sharepoint list....
Now, I can see what's going on here, it's pulling the 'Name' from the $Filecsv table, and pushing that up to Sharepoint, however, I am not sure how to re-construct my logic so that it operates as expected because as it exists now, it should (to me anyway) work as I think it does, but I am still new to Sharepoint and am certainly missing something here.
Below, is the full code, if it helps:
# Add SharePoint PowerShell Snapin which adds SharePoint specific cmdlets
Add-PSSnapin Microsoft.SharePoint.PowerShell -EA SilentlyContinue
#start the counter at 1 to track times script has looped
$iterations = 1
# set the location where the .CSV files will be pulled from and define the
# file extension we are concerned with
$filecsv = get-childitem "Z:\" -recurse | where {$_.extension -eq ".csv"}
# for each file found in the directory
ForEach ($items in $Filecsv) {
# check to see if files exist, if not exit cleanly
if ($Filecsv) {"File exists" + $Filecsv} else {exit}
# count the times we've looped through
"Iterations : $iterations"
# specify variables needed. The webURL should be the site URL, not including the list
# the listName should be the list name
$WebURL = "http://SHAREPOINTURL/"
$listName = "test"
# Get the SPWeb object and save it to a variable
$web = Get-SPWeb -identity $WebURL
# Get the SPList object to retrieve the list
$list = $web.Lists[$listName]
# START deletes all items. code shows the number of items in a list, then deletes all items
# If you don't want your script to delete items, then remove this
$site = new-object Microsoft.SharePoint.SPSite ( $WebURL )
$web = $site.OpenWeb()
"Web is : " + $web.Title
# Enter name of the List below instead of
$oList = $web.Lists["test"];
"List is :" + $oList.Title
"List Item Count: " + $oList.ItemCount
#delete existing contents and replace with new stuff
$collListItems = $oList.Items;
$count = $collListItems.Count - 1
for($intIndex = $count; $intIndex -gt -1; $intIndex--) {
"Deleting record: " + $intIndex
$collListItems.Delete($intIndex);
}
# END Deletes all items
# goes through the CSV file and performs action for each row
foreach($row in $Filecsv)
{
$newItem = $list.items.Add()
$item = $list.items.add()
# Check if cell value is not null in excel
if ($row."Name" -ne $null)
# Add item to sharepoint list. for this one, I had to use the internal column name.
#You don't always have to, but I had trouble with one SharePoint column, so I did
{$newItem["Name"] = $row."Name"}
else{$newItem["Name"] = $row."Not Provided"}
if ($row."Description" -ne $null)
{$newItem["Description"] = $row."Description"}
else{$newItem["Description"] = $row."No Description"}
if ($row."NetworkID" -ne $null)
{$newItem["Network ID"] = $row."NetworkID"}
else{$newItem["Network ID"] = $row."No NetworkID"}
if ($row."Nested" -ne $null)
{$newItem["Nested"] = $row."Nested"}
else{$newItem["Nested"] = $row."Not Nested"}
# Commit the update, then loop again until end of file
$newItem.Update()
}
# get the date and time from the system
$datetime = get-date -f MMddyy-hhmmtt
# rename the file
$NewName = $items.fullname -replace ".csv$","$datetime.csv.archived"
$Items.MoveTo($NewName)
# +1 the counter to count the number of files we've looped through
$iterations ++
}

Answer 1

a very cursory look would suggest that you need to use $items not $filecsv in your main loop.
essentially you are looping over the contents of the $filecsv collection, so you need to look at $items.

Answer 2

Your ForEach loops look redundant since they are both looping through a list of FileInfo objects. I think you want to find all the files, and for each file load it into memory and process it's contents. We'll go that route.
I have moved your SharePoint object creation out of the loop since I don't see any point to creating the object over and over for each file processed since it never references anything based on the file or it's contents. It simply makes the same object over, and over, and over.
# Add SharePoint PowerShell Snapin which adds SharePoint specific cmdlets
Add-PSSnapin Microsoft.SharePoint.PowerShell -EA SilentlyContinue
#start the counter at 1 to track times script has looped
$iterations = 1
# specify variables needed. The webURL should be the site URL, not including the list
# the listName should be the list name
#Setup SP object
$WebURL = "http://SHAREPOINTURL/"
$listName = "test"
# Get the SPWeb object and save it to a variable
$web = Get-SPWeb -identity $WebURL
# Get the SPList object to retrieve the list
$list = $web.Lists[$listName]
# START deletes all items. code shows the number of items in a list, then deletes all items
# If you don't want your script to delete items, then remove this
$site = new-object Microsoft.SharePoint.SPSite ( $WebURL )
$web = $site.OpenWeb()
"Web is : " + $web.Title
# Enter name of the List below instead of
$oList = $web.Lists["test"];
"List is : " + $oList.Title
"List Item Count: " + $oList.ItemCount
#delete existing contents and replace with new stuff
$collListItems = $oList.Items;
$count = $collListItems.Count - 1
for($intIndex = $count; $intIndex -gt -1; $intIndex--) {
"Deleting record: " + $intIndex
$collListItems.Delete($intIndex);
}
# END Deletes all items
Find all the CSV files, and start looping through the list of them. I removed the check to see if the file exists. You just pulled a directory listing to find these files, they really should exist.
# set the location where the .CSV files will be pulled from and define the
# file extension we are concerned with
$CSVList = get-childitem "Z:\" -recurse | where {$_.extension -eq ".csv"}
ForEach ($CSVFile in $CSVList) {
# count the times we've looped through
"Iterations : $iterations"
Now, this is different. It loads the CSV file, and processes each row in it as $row. I'm pretty sure this is what you intended to do from the start. I also changed it from If(Something -ne $null) to check for either null, or empty since either can actually exist and the later can cause you some issues. It's just a safer method in general.
foreach($row in (Import-CSV $CSVFile.FullName))
{
$newItem = $list.items.Add()
$item = $list.items.add()
# Check if cell value is not null in excel
if (![string]::IsNullOrEmpty($row."Name"))
# Add item to sharepoint list. for this one, I had to use the internal column name.
#You don't always have to, but I had trouble with one SharePoint column, so I did
{$newItem["Name"] = $row."Name"}
else{$newItem["Name"] = $row."Not Provided"}
if (![string]::IsNullOrEmpty($row."Description"))
{$newItem["Description"] = $row."Description"}
else{$newItem["Description"] = $row."No Description"}
if (![string]::IsNullOrEmpty($row."NetworkID"))
{$newItem["Network ID"] = $row."NetworkID"}
else{$newItem["Network ID"] = $row."No NetworkID"}
if (![string]::IsNullOrEmpty($row."Nested"))
{$newItem["Nested"] = $row."Nested"}
else{$newItem["Nested"] = $row."Not Nested"}
# Commit the update, then loop again until end of file
$newItem.Update()
}
I don't really understand why you are adding a new item twice, but if it works then more power to you. Then your bit to rename files when you're done with them (hey, this looks familiar):
# get the date and time from the system
$datetime = get-date -f MMddyy-hhmmtt
# rename the file
$NewName = $CSVFile.fullname -replace ".csv$","$datetime.csv.archived"
$CSVFile.MoveTo($NewName)
# +1 the counter to count the number of files we've looped through
$iterations ++
}
I did rename a few things to make them more indicative of what they represent ($Items to $CSVFile and what not). See if this works for you. If you have questions or concerns let me know.
Edit: Ok, to fix the loop trying to pull each item from the current folder we reference the FullName property of it. One line changed:
foreach($row in (Import-CSV $CSVFile.FullName))