Loading million rows into partitioned Stateful service - azure-service-fabric

Question

I'm trying to load 20 million rows into partitioned stateful service ReliableDictionary. I partitioned stateful service into 10 partitions. Based on MSDN documentation, I understood that I need to use some hashing algorithm to find the correct partition and send data to it to load into IReliabledictionary. So I used the Hydra to get the partition number based on the value. All I'm storing is a List<long> in the IReliableDictionary.
So I created a Stateless service as wrapper,
which will fetch the rows from the SQL Server (20 million),
get the partition number using Hydra for each row,
group them by partition number
call the Stateful service for each partition using ServiceRemoting. However, I get fabric message too large exception if I send 1 million rows of data per each request so I chunked it into 100000 per request.
This is taking 74 minutes for it to complete. This is too long. Below is the code for uploading -
Please advise.
foreach (var itemKvp in ItemsDictionary)
{
var ulnv2Uri = new Uri("fabric:/TestApp/dataservice");
//Insert to the correct shard based on the hash algorithm
var dataService = _serviceProxyFactory.CreateServiceProxy<IDataService>(
dataStoreUri,
new ServicePartitionKey(itemKvp.Key), TargetReplicaSelector.PrimaryReplica, "dataServiceRemotingListener");
var itemsShard = itemKvp.Value;
//if the total records count is greater then 100000 then send it in chunks
if (itemsShard.Count > 1_000_000)
{
//var tasks = new List<Task>();
var totalCount = itemsShard.Count;
var pageSize = 100000;
var page = 1;
var skip = 0;
while (skip < totalCount)
{
await dataService.InsertData(itemsShard.Skip(skip).Take(pageSize).ToList());
page++;
skip = pageSize * (page - 1);
}
}
else
{
//otherwise send all together
await dataService.InsertData(itemsShard);
}
}

Answer 1

You can likely save some time here, by uploading to all partitions in parallel.
So create 10 service proxies (one for each partition) and use them simultaneously.