Added more details at the bottom of the question.
We are testing deployment scenarios in Azure VM preview and have run into an issue.
Here is our scenario. We have a software stack that we use in all of our servers. We have created an image with all of that stack installed on an attached data drive. We have created a image of the VM that we can use as a template. Now what we want to do is to to create a VM based on that template and create a copy of the data drive and attach it to the newly created VM in an automated manner.
Our problem is that while we have found lots of information about creating drives, we can't find any guidance on how to copy the data drive using Azure for Powershell.
Any thoughts, code, or RTFMs happily accepted.
Cheers,
Terence
We have sucessfully created an operating system image that we can use to create VM's. But there is a data disk that holds our standard software stack that we want to reuse by copying it across VMs. The scenario that we are trying to implement is:
Create a VM from a standard VM image - PBIMaster
Attach a disk as F to that image called PBIMasterDisk
Install all of the software required for our app on F: (to big for the OS disk and besides sticking it on the OS disk seems messy)
Build an image from PBIMaster call it PBIMasterImage save it.
Create a new image from PBIMaster call it Node1
Copy PBIMasterDisk to a new Azure disk call it Node1Software disk
Attach Node1Softwaredisk to Node1 as F:
Since the image has the correct registry settings from the previous installs our stack is ready to go.
9 Add appropriate endpoints.
Rinse and repeat for each additional node.
Hopefully that makes our scenario clearer.
Thanks.
If I understood your objective correctly you already have uploaded two VHD in your subscription and you have also create a VM based on your OS Disk VHD1:
OS Disk (VHD1)
Data Disk (VHD2)
Now you want to copy VHD2 to VHD3 and then attach VHD3 to your VM (which is based on OS disk) via Powershell.
As of there is no powershell command which will let you copy DataDisk (VHD2) to another data disk (i.e VHD3)..
I haven't tried but you can use the following code to try copying your DataDisk:
http://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/12/introducing-asynchronous-cross-account-copy-blob.aspx
This method does copy blobs directly at cloud storage level so there is no bandwidth usage towards on-premise and potentially zero cost if you are in same DC. Trying using the same subscription and see if that solves your problem.
Related
I recently upgraded Fedora Server (F) v36 to v37. The F36 server had a volume group consisting of three physical drives combined to form a storage pool, which I named “BigDrive”. During the upgrade the logical volume information seems to have been lost and BigDrive didn’t appear or mount in the F37 server. I’ve been unable to find any backup logical volume information. At present the 3 drives are installed on the F37 server. I would welcome advise on how to recombine the three drives, recover the logical volume information, and access the data stored in the shared pool. Can anyone suggest a process to do that, or a utility that could rebuild the storage pool from the physical drives?
I haven't found any helpful information in the various Fedora documentation or usual websites that don't reference using the backed up logical volume information which somehow didn't survive the upgrade process. This is because the OS hard drive was wiped and repartitioned as part of the upgrade. The drives that formed the storage pool were not formatted, nor do they store any OS or application files. They were purely data storage.
I have a working VM instance that I'm trying to copy to allow redundancy behind google load balancer.
A test run with a dummy instance worked fine, creating a new instance from a snapshot of a running one.
Now, the real "original" instance have a persistent disk attached and this cause a problem in starting up the cloned instance because of the (obviously) missing persistent disk mount.
Logs from serial console output is as:
* Stopping cold plug devices[74G[ OK ]
* Stopping log initial device creation[74G[ OK ]
* Starting enable remaining boot-time encrypted block devices[74G[ OK ]
The disk drive for /mnt/XXXX-log is not ready yet or not present.
keys:Continue to wait, or Press S to skip mounting or M for manual recovery
As I understand there is no way to send any of this key strokes to the instance, is there any other way to overcome this issue? I know that I could unmount the disk before the snapshot, but the workflow I would like to instate is creating period snapshots of production servers, so un-mounting disks every time before performing it would require instance downtime (plus all the unnecessary risks of doing an action that would seem pointless).
Is there a way to boot this type of cloned instances successfully, and attach a new persistence disk afterwards?
Is this happening because the original persistent disk is in use, or the same problem would occur even if the original instance is offline (for example due to a failure in which case I would try to created a new instance from a snapshot)?
One workaround that I am using to get away from the same issue is that I dont't actually unmount the disk rather just comment out the the mount line in /etc/fstab and take the snapshot. This way my instance has no downtime or down disks while snapshoting. (I am using Ubuntu 14.04 as OS if that matters)
Later I fix and uncomment it when I use that snapshot on a new instance.
However you can also look into adding the nofail option in the commented line to get a better solution.
By the way I am doing a similar task building a load balanced setup with multiple webserver nodes. Each being cloned from the said snapshot with extra persistent disks mounted for eg uploads,data and logs etc
I'm a little unclear as to what you're trying to accomplish. It sounds like you're looking to periodically snapshot the data volumes of a production server so you can clone them later.
In all likelihood, you simply need to sync and fsfreeze to before you make your snapshot, rather than just unmounting/remounting it. The GCP documentation has a basic example of this in the Snapshots documentation.
My problem: I am learning Service Fabric, and doing simple tutorials, and the local cluster is filling up my C drive. I run the projects in Visual Studio. It first creates a cluster in a folder SfDevCluster. That takes up 842 MB of space. Then it deploys the services and web api sites. Remember, these are trivial tutorials with almost nothing in them. Now, I notice that I have a folder with a Size = 1.22 TB and Size on Disk of 9.4 GB. I'm not sure how to interpret that. But it consumes the remaining space on my C drive and sets off alarms.
I have other drives with lots of space. I would love to specify that those be used. Is there a way to do that with the service fabric cluster used by Visual Studio? Or is there a way to constrain the overly ambitious size allocations? And if you understand this, can you explain what these unusual folder sizes mean?
In the old days, I would have a hard drive with lots of space. But now, my developer machine has a much faster, but more expensive SSD drive, and space is at a premium. So I need more control of the cluster location.
You can set up a local cluster pointing to a non-system drive by running the DevClusterSetup script in PowerShell. You can find the script under %programfiles%\Microsoft SDKs\Service Fabric\ClusterSetup\. The command line you want is:
.\DevClusterSetup.ps1 -PathToClusterDataRoot <desired_app_and_data_location> -PathToClusterLogRoot <desired_tracelog_location>
If you already have a cluster running, this script will remove it and create a new one (note that this will delete any deployed apps and their data). Once you have the new cluster running, Visual Studio will automatically use that when you deploy locally.
As for the file sizes - this is mostly due to the log file used for replication of state stored in reliable collections. A large, sparse file is preallocated up-front, which is why you see a difference between size and size on disk. We are planning to make these values configurable so that they can be dialed down on local clusters.
In the Service Fabric SDK folder (C:\Program Files\Microsoft SDKs\ServiceFabric), you will find a ClusterSetup folder.
In there you will find ClusterManifestTemplate.json files for the different configurations of the local cluster. These are json configuration files used by the powershell scripts that create and manage the local service fabric cluster.
At the bottom of these files, in "fabricSettings" it is setting the value of the FabricDataRoot and FabricLogRoot, based on the "%systemDrive%". If you replace this by "D:" it should result in a local cluster on the D drive.
After making these changes, I stopped my local fabric, deleted the current fabric folders from my C drive, and rebooted my machine. When I then start a debug session in VS.2017, it creates the local dev fabric on my D drive and deploys the application to that location. (I do notice that some empty folders are created on my C drive but these are not used.)
What you also can do is resetting the local cluster once in a while.
Can be easily done using the Service Fabric Local Cluster Manager application:
I have created the Azure VM and taken successfully the backup of disk using powershell. Now i need to create another VM using that disk?
How to proceed with that. Also i wanted to do same using image also. My main goal is to take backup of vm using either disk or image and then able to create new VMs using those backups or update the existing vm using those backups
The process you need to follow is covered here:
http://blogs.msdn.com/b/amol/archive/2014/03/21/how-to-migrate-windows-azure-vm-from-one-region-to-another.aspx
Note that if you don't "sysprep" the source VM when you attempt to provision a new VM based on the VHD of the source VM it will provision but the Azure management portal will show Running (provisioning) for the host.
I've got a Web Role on Azure with 2 Applications and 1 Virtual Directory.
1 Application is a backend, where admins can upload files, which are stored in the virtual directory (which is accessed by both applications).
Everytime I deploy a new version to Azure, all the uploaded content in the virtual directory is deleted - this is what I don't want!
So how is it possible to publish a new version without deleting all my user generated files?
I've already managed to update the application with WebDeploy. But this is only possible for the "main" application, and not the 2nd application (which is configured as a Virtual Application).
Thanks
You can't. The web role is recreated on deployment. It may also occur on hardware failure, azure redeploys your system if an instance fails. Redeploys a clean virtual machine and then deploys your app to it. You should never store data you want to keep on a web role. You need to use blob storage etc to store files you want to persist.
Virtual directories are stored on "Application" partition which is recreated on each upgrade - see this for more information. So the virtual directory folder is not the right place to store stuff you want preserved across upgrades. BTW the "Application" partition only has 1 gigabyte of space and some of that is used for storing your application binary code so you may find yourself in a "disk full" situation at some moment.
If you want to store some data which you don't mind sacrificing on rare occasions - like cached results - you may use "local resources" disk for that which will survive in-place upgrades and reboots. However it is not guaranteed to be preserved if your VM crashes - for such level of preservation you have to use persistent storage like blob storage for example.
Since you are talking about virtual directories and using web deploy to update application outside of the usual Azure package deployment mechanism, it sounds like your architecture/application might be more suited to a persistent VM role rather than a Web role. These are available on Azure in preview only at the moment.
http://www.windowsazure.com/en-us/home/scenarios/virtual-machines/
They let you have persistent storage that will survive a recycle. The storage is actually backed by blob storage, but it looks like a normal disk from the PVM.