Upload files and folder into Azure Blob Storage - powershell

Question

I have created a storage account in Azure and created a container. I am trying to upload files stored in my Server the files are stored within 800 folders.
I have tried doing this with this Powershell script however it does not work with the subfolders.
$LocalFolder = "\\Server\Data"
Add-AzureAccount
# Set a default Azure subscription.
Select-AzureSubscription -SubscriptionName 'nameofsubscription' –Default
$Key = Get-AzureStorageKey -StorageAccountName mydatastorename
$Context = New-AzureStorageContext -StorageAccountKey $Key.Primary -StorageAccountName mydatastorename
foreach ($folder in Get-ChildItem $LocalFolder)
{
ls –Recurse –Path $LocalFolder |Set-AzureStorageBlobContent -Container nameofcontainer -Context $Context -BlobType Block
}
If set the $LocalFolder as "\Server\Data\subfolders001" the files in subfolder001 get uploaded to the container. But when I keep it as "\Server\Data" then it does not work.
I want the script to upload all the sub folders and files within into the storage container.
I have added the output I get when I run it
I don't get any error message, but one warning message each subfolder
WARNING: Can not upload the directory '\\Server\Data\subfolders001' to azure. If you want to upload directory, please use "ls -File -Recurse | Set-AzureStorageBlobContent -Container containerName".
Then noting happens after waiting for a while I have to stop the powershell script.

Answer 1

Was just solving this myself, see my solution below:
$StorageAccountName = "storageAccountName"
$StorageAccountKey = "StorageAccountKey"
$ContainerName = "ContainerName"
$sourceFileRootDirectory = "AbsolutePathToStartingDirectory" # i.e. D:\Docs
function Upload-FileToAzureStorageContainer {
[cmdletbinding()]
param(
$StorageAccountName,
$StorageAccountKey,
$ContainerName,
$sourceFileRootDirectory,
$Force
)
$ctx = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
$container = Get-AzureStorageContainer -Name $ContainerName -Context $ctx
$container.CloudBlobContainer.Uri.AbsoluteUri
if ($container) {
$filesToUpload = Get-ChildItem $sourceFileRootDirectory -Recurse -File
foreach ($x in $filesToUpload) {
$targetPath = ($x.fullname.Substring($sourceFileRootDirectory.Length + 1)).Replace("\", "/")
Write-Verbose "Uploading $("\" + $x.fullname.Substring($sourceFileRootDirectory.Length + 1)) to $($container.CloudBlobContainer.Uri.AbsoluteUri + "/" + $targetPath)"
Set-AzureStorageBlobContent -File $x.fullname -Container $container.Name -Blob $targetPath -Context $ctx -Force:$Force | Out-Null
}
}
}
Running:
Upload-FileToAzureStorageContainer -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey -ContainerName $ContainerName -sourceFileRootDirectory $sourceFileRootDirectory -Verbose
You should see that it prints the following output as it runs:
VERBOSE: Uploading testfile.json to https://storagecontainername.blob.core.windows.net/path/testfile.json
VERBOSE: Performing the operation "Set" on target "/path/testfile.json".
ICloudBlob : Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob
BlobType : BlockBlob
Length : 0
ContentType : application/octet-stream
LastModified : 23/03/2017 14:20:53 +00:00
SnapshotTime :
ContinuationToken :
Context : Microsoft.WindowsAzure.Commands.Common.Storage.AzureStorageContext
Name : /path/testfile.json
VERBOSE: Transfer Summary
--------------------------------
Total: 1.
Successful: 1.
Failed: 0.
I admit this isn't perfect but with a bit of effort you could extend it to cater for more complex scenario file uploads but it should do the trick if you wanted to upload the contents of a directory to a target storage container.
Azure Storage Containers are essentially directories that can't contain other storage containers. So in order to implement a folder structure in an Azure Storage Container, you need to prefix the file with the path up to the root folder you started searching from. I.e. If your root is :
D:\Files
And Files contains:
Folder 1
-> File 1.txt
File 2.txt
You need to set the target paths to be
$StorageContainerObject.CloudBlobContainer.Uri.AbsoluteUri + "\Files\Folder 1\File 1.txt"
$StorageContainerObject.CloudBlobContainer.Uri.AbsoluteUri + "\Files\File 2.txt"
If you view your storage container using a tool like Azure Storage Explorer then it recognises the file structures even though the files are all stored within one container.
Thanks,
Jamie

Answer 2

Azure Blob Storage has only one level of folders - containers.
Instead, you may use few options below
Use Azure File Services
https://learn.microsoft.com/en-us/azure/storage/storage-dotnet-how-to-use-files
Use workaround as described here https://www.codeproject.com/articles/597939/modelingplusaplusdirectoryplusstructureplusonplusa