How to increase List View threshold in PowerShell - powershell

I have a very large SharePoint folder with many nested subfolder and I'm trying to count how many items are in there before we migrate that folder to different location.
I've this simple script but I'm always getting this error when every I tried to count a folder that really large so just wondering how can I increase the threshold or fix this issue.
$SiteURL = "https://company-my.sharepoint.com/personal/hello_nam_corp_com"
$searchfor = "/Documents/Archive"
$folderpath = "Documents/Archive"
$CSVFile = "C:\Users\Desktop\Resource\FolderStats.csv"
#Connect to SharePoint Online
Connect-PnPOnline $SiteURL -useWebLogin
#Get the list
#$documents = Get-PnPList -Identity $ListName | Where-Object {$_.Title -eq 'Documents'}
$FolderItems = Get-PnpListItem -List $folderpath
$fieldvalues = $FolderItems.Fieldvalues
$result = #()
foreach ($field in $fieldvalues) {
$obj = New-object psobject -property $field
$result += $obj.fileref
}
$final = $result | where-object {$_ -match $searchfor}
$item = New-Object psobject -Property #{
FolderName = Split-Path -Path $searchfor -Leaf
URL = $searchfor
filesfoldercount = $final.count
}
$item
$item | Export-Csv -Path $CSVFile -NoTypeInformation
Get-PnpListItem : The attempted operation is prohibited because it exceeds the list view threshold.
At line:13 char:16
+ $FolderItems = Get-PnpListItem -List $folderpath

Try below PowerShell.
#Parameters
$SiteURL = "https://crescent.sharepoint.com/sites/marketing"
$ListName = "Branding"
$CSVFile = "C:\Temp\FolderStats.csv"
#Connect to SharePoint Online
Connect-PnPOnline $SiteURL -Interactive
#Get the list
$List = Get-PnPList -Identity $ListName
#Get Folders from the Library - with progress bar
$global:counter = 0
$FolderItems = Get-PnPListItem -List $ListName -PageSize 500 -Fields FileLeafRef -ScriptBlock { Param($items) $global:counter += $items.Count; Write-Progress -PercentComplete `
($global:Counter / ($List.ItemCount) * 100) -Activity "Getting Items from List:" -Status "Processing Items $global:Counter to $($List.ItemCount)";} | Where {$_.FileSystemObjectType -eq "Folder"}
Write-Progress -Activity "Completed Retrieving Folders from List $ListName" -Completed
$FolderStats = #()
#Get Files and Subfolders count on each folder in the library
ForEach($FolderItem in $FolderItems)
{
#Get Files and Folders of the Folder
Get-PnPProperty -ClientObject $FolderItem.Folder -Property Files, Folders | Out-Null
#Collect data
$Data = [PSCustomObject][ordered]#{
FolderName = $FolderItem.FieldValues.FileLeafRef
URL = $FolderItem.FieldValues.FileRef
FilesCount = $FolderItem.Folder.Files.Count
SubFolderCount = $FolderItem.Folder.Folders.Count
}
$Data
$FolderStats+= $Data
}
#Export the data to CSV
$FolderStats | Export-Csv -Path $CSVFile -NoTypeInformation
Script Link: https://www.sharepointdiary.com/2019/05/sharepoint-online-get-files-sub-folders-count-in-document-library-using-powershell.html

Related

ONLY get the first level folders (main folders) inside document library, seems "Get-PnPFolder -List $Library" retrieve all the sub-folders

I have this script to loop through all site collections >> and all document libraries >> then list the folders in this format "Site Name >> List Name >> Folder Name":-
$AdminUrl = "https://*****-admin.sharepoint.com/"
Connect-PnPOnline -Url $AdminUrl -Interactive
#sharepoint online get all site collections PowerShell
$SiteColl = Get-PnPTenantSite
#sharepoint online PowerShell iterate through all site collections
ForEach($Site in $SiteColl)
{
Connect-PnPOnline -Url $Site.Url -Interactive
$Webs = Get-PnPSubWeb -Recurse -IncludeRootWeb
#Get All document libraries
ForEach($Web in $Webs)
{
$DocumentLibraries = Get-PnPList -Web $Web | Where-Object {$_.BaseType -eq "DocumentLibrary" -and $_.Title -ne "Site Assets" -and $_.Hidden -eq $false}
#Iterate through each document library
ForEach($Library in $DocumentLibraries)
{
$f = Get-PnPFolder -List $Library
ForEach($folder in $f)
{
Write-Host $Site.Title " --> " $Library.Title " --> " $folder.Name
}
}
}
}
but the issue i am facing is that Get-PnPFolder -List $Library will get all the main folders and sub-folders.. so how i can restrict this command to only get the main folders (first level folders) without sub-folders?
thanks
EDIT here is the updated script:-
$AdminUrl = "https://***-admin.sharepoint.com/"
Connect-PnPOnline -Url $AdminUrl -Interactive
#sharepoint online get all site collections PowerShell
$SiteColl = Get-PnPTenantSite
$csvOutput = #()
#sharepoint online PowerShell iterate through all site collections
ForEach($Site in $SiteColl)
{
Connect-PnPOnline -Url $Site.Url -Interactive
$Webs = Get-PnPSubWeb -Recurse -IncludeRootWeb
#Get All document libraries
ForEach($Web in $Webs)
{
$DocumentLibraries = Get-PnPList -Web $Web | Where-Object {$_.BaseType -eq "DocumentLibrary" -and $_.Title -ne "Site Assets" -and $_.Hidden -eq $false}
#Iterate through each document library
ForEach($Library in $DocumentLibraries)
{
$f = Get-PnPFolder -List $Library
ForEach($folder in $f)
{
Write-Host $Site.Title " --> " $Library.Title " --> " $folder.Name " --> " $folder.ServerRelativeUrl
$csvOutput += [PsCustomObject]#{SiteTitle = $Site.Title; LibraryTitle = $Library.Title; Folder = $folder.Name; FolderPath = $folder.ServerRelativeUrl}
}
}
}
}
$csvOutput | Export-Csv -NoTypeInformation -Path "D:\export123.csv"
and the csvOutput will have values such as :-
NewRoot --> dv --> test123 --> /dv/test123
NewRoot --> dv --> P100 --> /dv/P100
NewRoot --> dv --> WIP --> /dv/P100/WIP
NewRoot --> Site Pages --> Templates --> /SitePages/Templates
NewRoot --> tagggg --> a1 --> /tagggg/a1
NewRoot --> tagggg --> a2 --> /tagggg/a1/a2
NewRoot --> testdcdc --> test --> /testdcdc/test
OK, based on our comments, I think what you want to do is this.
It iterates through the entire folder list, but uses a hashtable to make sure you only collect a unique list of top-level folders.
$AdminUrl = "https://***-admin.sharepoint.com/"
Connect-PnPOnline -Url $AdminUrl -Interactive
#sharepoint online get all site collections PowerShell
$SiteColl = Get-PnPTenantSite
# create a list to store the results
$csvOutput = [System.Collections.Generic.List[object]]::new()
# create a Hashtable to eliminate any duplicates
$hashUnique = #{}
#sharepoint online PowerShell iterate through all site collections
foreach($Site in $SiteColl) {
Connect-PnPOnline -Url $Site.Url -Interactive
$Webs = Get-PnPSubWeb -Recurse -IncludeRootWeb
#Get All document libraries
foreach($Web in $Webs) {
$DocumentLibraries = Get-PnPList -Web $Web |
Where-Object {$_.BaseType -eq "DocumentLibrary" -and
$_.Title -ne "Site Assets" -and
$_.Hidden -eq $false}
#Iterate through each document library
foreach($Library in $DocumentLibraries) {
Get-PnPFolder -List $Library | ForEach-Object {
# from the ServerRelativeUrl property, split out the 1st level folder name
$topFolder = ($_.ServerRelativeUrl -split '/')[1]
if (!$hashUnique.ContainsKey($topFolder)) {
# remember we have already seen this 1st level folder so we don't add it again
$hashUnique[$topFolder] = $true
Write-Host "$($Site.Title) --> $($Library.Title) --> $topFolder"
# add an object to the output list
$csvOutput.Add([PsCustomObject]#{
SiteTitle = $Site.Title
LibraryTitle = $Library.Title
Folder = $topFolder
})
}
}
}
}
}
# now write the csv
$csvOutput | Export-Csv -NoTypeInformation -Path "D:\FirstLevel Folders.csv"

How to get a full file path including extension from OneDrive using PowerShell

I've this PowerShell Script that get the folder name, Folder path and last modified from either SharePoint Online or OneDrive account but I'm just wondering how can I also get the the file name inside each folder, their path and latest modified date in the same report file.
I've stuck with this so any help or suggestion would be really appreciated.
Connect-PnPOnline $SiteURL -Credentials $credential
#Get the list
$List = Get-PnPList -Identity $ListName
#Get Folders from the Library - with progress bar
$global:counter = 0
$FolderItems = Get-PnPListItem -List $ListName -PageSize 500 -Fields FileLeafRef -ScriptBlock { Param($items) $global:counter += $items.Count; Write-Progress -PercentComplete `
($global:Counter / ($List.ItemCount) * 100) -Activity "Getting Items from List:" -Status "Processing Items $global:Counter to $($List.ItemCount)"; } | Where { $_.FileSystemObjectType -eq "Folder" }
Write-Progress -Activity "Completed Retrieving Folders from List $ListName" -Completed
$FolderStats = #()
ForEach ($FolderItem in $FolderItems) {
#Get Files and Folders of the Folder
Get-PnPProperty -ClientObject $FolderItem.Folder -Property Files, Folders | Out-Null
#Collect data
$Data = [PSCustomObject][ordered]#{
FolderName = $FolderItem.FieldValues.FileLeafRef
Path = $FolderItem.FieldValues.FileRef
ModifiedDate = $FolderItem.FieldValues.Modified
}
$Data
$FolderStats += $Data
}
$FolderStats | Export-Csv -Path $CSVFile -NoTypeInformation

Connection to OneDrive issue in PowerShell

I have this script running in VSCode but different $SiteURL. I open this script again in ISE and change the $SiteUR $searchfor and $folderpath. The issue I have is everytime I run in ISE and when it's doing Get-PnPListItem, it's getting an items from the path that I provided in the VSCode. Not sure what is going on so I would be really appreciated if I can get any help or suggestion.
$SiteURL = "https://companyName-my.sharepoint.com/personal/user_id"
$searchfor = "/personal/user_id/Documents/Pkmon"
$folderpath = "Documents/Pkmon"
$CSVFile = "C:\Users\user\Desktop\Resource\FolderStats.csv"
#Connect to SharePoint Online
Connect-PnPOnline $SiteURL -useWebLogin
$FolderItems = Get-PnpListItem -List $folderpath -PageSize 2000 -Fields ID, FileDirRef, FileRef -ScriptBlock `
{ Param($items) $global:counter += $items.Count; Write-Progress -PercentComplete ($global:Counter / ($List.ItemCount) * 100) -Activity `
"Getting Items from Folder '$FolderServerRelativeURL'" -Status "Getting Items $global:Counter of $($List.ItemCount)"; }
$fieldvalues = $FolderItems.Fieldvalues
$result = #()
foreach ($field in $fieldvalues) {
$obj = New-object psobject -property $field
$result += $obj.fileref
}
$final = $result | where-object {$_ -match $searchfor}
$item = New-Object psobject -Property #{
FolderName = Split-Path -Path $searchfor -Leaf
URL = $searchfor
filesfoldercount = $final.count
}
$item
$item | Export-Csv -Path $CSVFile -NoTypeInformation
You can capture the connection object in a variable like
$thisConnection = Connect-PnPOnline $SiteURL -useWebLogin -ReturnConnection
-ReturnConnection acts as what is normally called -PassThru. It makes the cmdlet return the connection object for use with the -Connection parameter on other cmdlets.
Then use that in parameter -Connection of the Get-PnpListItem cmdlet:
$FolderItems = Get-PnpListItem -List $folderpath -Connection $thisConnection ...
You may also need to specify the connection in the VSCode instance.
When done, disconnect using Disconnect-PnPOnline -Connection $thisConnection

How should I get file count for only one SharePoint folder in PowerShell

In my SharePoint site, I have two folders called "General" and "Testing2" under Document. I'm trying to get the count only for "General" but not sure why it's still also counting for Testing2 folder so I would be really appreciate if I can get any help or suggestion.
I tried like this, $ListName = "/Shared Documents/General" but still not working
#Parameters
$SiteURL = "https://comapny.sharepoint.com/sites/msteams_####/"
//I tried like this but still not working
$ListName = "/Shared Documents/General"
$CSVFile = "C:\Users\kek\Desktop\Resource\LitHold\FolderStats.csv"
#Connect to SharePoint Online
Connect-PnPOnline $SiteURL -useWebLogin
#Get the list
$List = Get-PnPList -Identity $ListName
#Get Folders from the Library - with progress bar
$global:counter = 0
$FolderItems = Get-PnPListItem -List $ListName -PageSize 500 -Fields FileLeafRef -ScriptBlock { Param($items) $global:counter += $items.Count; Write-Progress -PercentComplete `
($global:Counter / ($List.ItemCount) * 100) -Activity "Getting Items from List:" -Status "Processing Items $global:Counter to $($List.ItemCount)";} | Where {$_.FileSystemObjectType -eq "Folder"}
Write-Progress -Activity "Completed Retrieving Folders from List $ListName" -Completed
$FolderStats = #()
#Get Files and Subfolders count on each folder in the library
ForEach($FolderItem in $FolderItems)
{
#Get Files and Folders of the Folder
Get-PnPProperty -ClientObject $FolderItem.Folder -Property Files, Folders | Out-Null
#Collect data
$Data = [PSCustomObject][ordered]#{
FolderName = $FolderItem.FieldValues.FileLeafRef
URL = $FolderItem.FieldValues.FileRef
FilesCount = $FolderItem.Folder.Files.Count
SubFolderCount = $FolderItem.Folder.Folders.Count
}
$Data
$FolderStats+= $Data
}
#Export the data to CSV
$FolderStats | Export-Csv -Path $CSVFile -NoTypeInformation
I don't want to to print Testing 2

Run PowerShell script against each site from txt file

Looking for help with running a powershell script against sharepoint online. Basically currently I have is:
#Parameters
$SiteURL = "https://mysite.sharepoint.com/sites/site1"
$ListName= "Documents"
$ReportOutput = "C:\Temp\export.csv"
$Pagesize = 500
#Connect to SharePoint Online site
Connect-PnPOnline $SiteURL -UseWebLogin
#Delete the Output report file if exists
If (Test-Path $ReportOutput) { Remove-Item $ReportOutput}
#Array to store results
$Results = #()
#Get all Items from the document library with given fields
$ListItems = Get-PnPListItem -List $ListName -Fields "FileLeafRef", "SMTotalFileStreamSize", "FileDirRef","Author","Created","File_x0020_Type"
#Iterate through each item
Foreach ($Item in $ListItems)
{
#Filter Files only
If (($Item.FileSystemObjectType) -eq "File")
{
$Results += New-Object PSObject -Property ([ordered]#{
FileName = $Item["FileLeafRef"]
FileType = $Item["File_x0020_Type"]
RootFolder = $Item["FileDirRef"]
RelativeURL = $Item["FileRef"]
FileSize = ($Item["SMTotalFileStreamSize"])
CreatedBy = $Item["Author"].Email
CreatedOn = $Item["Created"]
Modified = $Item["Modified"]
ModifiedByEmail = $Item["Editor"].Email
})
}
}
#Export the results
$Results
$Results | Export-Csv -Path $ReportOutput -NoTypeInformation
However I have a list of sites in text file that I would like to run this against.
> https://mysite.sharepoint.com/sites/site1
> https://mysite.sharepoint.com/sites/site2
> https://mysite.sharepoint.com/sites/site3
Add the site url into a CSV:
Import and use the site url from CSV using Import-CSV command:
$contents = Import-Csv ".\Test.csv"
$ListName= "Documents"
$ReportOutput = "C:\Temp\export.csv"
$Pagesize = 500
foreach($row in $contents)
{
#Parameters
#$SiteURL = "https://mysite.sharepoint.com/sites/site1"
$SiteURL = $row.SiteUrl
#Connect to SharePoint Online site
Connect-PnPOnline $SiteURL -UseWebLogin
#Delete the Output report file if exists
If (Test-Path $ReportOutput) { Remove-Item $ReportOutput}
#Array to store results
$Results = #()
#Get all Items from the document library with given fields
$ListItems = Get-PnPListItem -List $ListName -Fields "FileLeafRef", "SMTotalFileStreamSize", "FileDirRef","Author","Created","File_x0020_Type"
#Iterate through each item
Foreach ($Item in $ListItems)
{
#Filter Files only
If (($Item.FileSystemObjectType) -eq "File")
{
$Results += New-Object PSObject -Property ([ordered]#{
FileName = $Item["FileLeafRef"]
FileType = $Item["File_x0020_Type"]
RootFolder = $Item["FileDirRef"]
RelativeURL = $Item["FileRef"]
FileSize = ($Item["SMTotalFileStreamSize"])
CreatedBy = $Item["Author"].Email
CreatedOn = $Item["Created"]
Modified = $Item["Modified"]
ModifiedByEmail = $Item["Editor"].Email
})
}
}
#Export the results
$Results
$Results | Export-Csv -Path $ReportOutput -NoTypeInformation
}