Active Directory transient error when bulk using ActiveDirectory cmdlet - powershell

Question

I have a system which synchronizes users from a data source. The data source consists of user information. When a new user is synchronized, a PowerShell task is triggered which creates or updates the user. This is all fine but when the amount of new/updated users becomes too big, some of the tasks fail with some interesting errors such as:
"The server has returned the following error: invalid enumeration
context."
or
"A connection to the directory on which to process the request was
unavailable. This is likely a transient condition."
When troubleshooting it seems obvious that the reason these errors occur are a lack of resources. It is because all the simultaneously triggered tasks are importing the module on their own PS session.
So I went trying some different things and measuring Import-Module speed etc. So I've concluded that it is faster to run Import-Module and then Get-ADUser for instance, then just Get-ADUser (which would also import the module).
Measure-Command {Import-Module ActiveDirectory}
Average time 340 ms
Measure-Command {Get-ADUser -Filter *}
Average time 420 ms
Get-ADUser after the module is imported
Average time 10 ms
But these marginal differences are not going to do anything to the issue. So I had to look further. I find that disabling the drive might help speed up the process, so I've added the following before importing the module:
$Env:ADPS_LoadDefaultDrive = 0
Measure-Command {Import-Module ActiveDirectory}
Average time 85 ms
4 times faster! But still the error persists at high amounts of users at the same time (e.g. 50 tasks). So I thought about polling the availability in the script, or make a do..while loop. Or maybe the system which fires the separate tasks needs to be redesigned, to have some sort of queue.
Does anyone recognize this situation? Or have some thoughts they'd like to share on this subject?

Answer 1

It is because all the simultaneously triggered tasks are importing the module on their own PS session.
Then you need to make sure this doesn't happen, or at least not so much that you run out of resources. So you have two options:
Limit the number of tasks that get run at any one time (maybe 5 at a time).
Make it one task that can work on several accounts. This way the module only gets loaded once.
I think option 2 is the better solution. For example, rather than triggering the script right away, your synchronization job could just write the username to a file (or in memory even) and once it's done finding all the users, it triggers the PowerShell script and passes the list (or the script could read the file that was written to). You have options there - whatever works best.
Update: All of .NET is available in PowerShell, so another option is to change the whole script to use .NET's DirectoryEntry instead of the ActiveDirectory module, which would use much less memory. There are even shortcuts for it in PowerShell. For example, [ADSI]"LDAP://$distinguishedName" would create a DirectoryEntry object for a user. It is a substantial rewrite, but the performance is much better (speed and memory consumption).
Here are some examples of searching with DirectorySearcher (using the [ADSISearcher] shortcut): https://blogs.technet.microsoft.com/heyscriptingguy/2010/08/23/use-the-directorysearcher-net-class-and-powershell-to-search-active-directory/
And here's an example of creating accounts with DirectoryEntry: https://www.petri.com/creating-active-directory-user-accounts-adsi-powershell