I have an EC2 Instance running an old AMI from AWS, which in my current setup is from WINDOWS_2016_BASE. Now I know that, but the Instance doesn't know that.
Using PowerShell I want the EC2 Instance to create more like itself, but not from an AMI I have created (as that would of course always be at the old version) and not "directly" from the actual AMI ImageId it was created from. As AWS don't allow that if it is not the latest version of it (hence I need the indirect route to getting that AMI ImageId, even if it turns out to to be the same.. i.e if it the latest version already).
So I need to find, using PowerShell, the latest version of an AMI I already know (which I got from the metadata) and I do not want to specify the name of the AMI e.g WINDOWS_2016_BASE since I would like the machine to create more like itself without me having to hardcode what it is in the script.
I have no idea how to do this since using Get-EC2ImageByName -Names WINDOWS_2016_BASE is not what I want to do since I don't want to hardcode that names parameter or pass it in through userdata.
Get-EC2Image -ImageId <an old imageid> returns null since the AMI is no longer current.
This will be somewhat difficult without either stashing this name somewhere that your script can cue off of (which you've stated that you don't want to do), or relying on some mapping between names/amis (that Get-EC2ImageByName already does for you).
AMI metadata is designed such that it is easy to retrieve a 'family tree' of related AMIs, under the assumption that their names are similar excepting for maybe minor differences such as versions or dates. It is a common pattern that prefixes of related AMIs will be consistent, which makes them easier to search for and aggregate.
This point makes your approach difficult, because your scripts will be missing that bit of data -- the AMI name prefix.
You could dynamically retrieve your instance's AMI ID from the metadata service and pass that into Get-EC2Image to get AMI details, but you couldn't go further from there without some sort of name prefix to match against and use to search for related AMIs that are newer.
Maybe reconsider this approach, and keep the name prefix stashed in a tag or on the instance via user-data? For example, I just checked and the English Windows Server 2016 Base AMIs all share this prefix: Windows_Server-2016-English-Full-Base. If you stored that on your instance or as one of your tags, your script could retrieve it and run the following powershell to get the latest Windows Server 2016 AMI:
#(Get-EC2Image -Owner amazon -Filter #{ Name="name"; Values="Windows_Server-2016-English-Full-Base*" } | Sort CreationDate -Desc)[0].ImageId
Example from the system log (which i got from aws console view)
2016/12/26 14:36:12Z: AMI Origin Name: Windows_Server-2016-English-Full-Base
The equivalent powershell is:
Get-EC2ConsoleOutput
So here below in full (i'm new to powershell but i am pretty sure someone can wrap that into one character somehow)
# read the system console log
$consoleLog = Get-EC2ConsoleOutput $currentInstanceObj.InstanceId
$consoleLogOutput = [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($consoleLog.Output));
# extract the lines that contain AMI Origin in them (there should be 2) - sort them so name i first and version is second
$originLines = $consoleLogOutput -split '[\r\n]' | Where-Object {$_.Contains("AMI Origin")} | Sort-Object
# get the running name
$originLine = $originLines[0]
$originLineParts = $originLine.Split(':')
$originName = $originLineParts[$originLineParts.Length - 1].Trim()
"The origin name is $originName"
# get the running version (slighly pointless since the code below doesn't care as we want the latest - but it's for verbosity)
$originLine = $originLines[1]
$originLineParts = $originLine.Split(':')
$originVersion = $originLineParts[$originLineParts.Length - 1].Trim()
"The origin version is $originVersion"
# concatenate to get the original origin name (note: amazon have a naming pattern here - (name-version)
$amiName = $originName + "-" + $originVersion
"The original origin ami name is $amiName"
#find the latest of the same name and report the difference
$latestOriginObj = (Get-EC2Image -Filter #{ Name="name"; Values=($originName + "*")} | Sort-Object CreationDate -Descending | Select-Object -First 1)
if($latestOriginObj.ImageId -ne $currentInstanceObj.ImageId)
{
"The ami has been upgraded from " + ($currentInstanceObj.ImageId) + " to " + ($latestOriginObj.ImageId)
}
#....so go ahead and use the $latestOriginObj.ImageId when you create a new instance
And the knowledge source for that springs from these amazons docs
Excerpt below:
The AWS Management Console provides details about the AMI that you use to create an Amazon EC2 instance. The AMI ID field on the Description tab contains information including the Windows Server SKU, the architecture (32-bit or 64-bit), the date the AMI was created, and an AMI ID.
If an AMI has been made private or replaced by later versions and is no longer listed in the catalog, the AMI ID field states, "Cannot load detail for ami-xxxxx. You may not be permitted to view it." To determine which AMI was used to create the instance, you must open the system log. In the EC2 console, choose an instance, and from the context-menu (right-click) choose Instance Settings and then choose Get System Log. The date the AMI was created and the SKU are listed in the AMI Origin Version and AMI Origin Name fields.
Related
Okay AWS Windows Powershell guys... here's a question for you. How do you add an ENI (Elastic Network Interface) to your Windows EC2? Seems simple enough as many examples show:
Add-EC2NetworkInterface -NetworkInterfaceId <your-eni-id> `
-InstanceId <your-ec2-id> `
-DeviceIndex 1 `
-Force
but in my past experiences DeviceId=1 stopped working and I switched to DeviceIndex=2. I went along my way and suddenly DeviceIndex=2 doesn't work, generating the exception
Instance 'i-xxxxxxxxxxxxxxxxx' already has an interface attached at device index '2'
and the ENI shows 'attaching' forever (and must be forcibly detached). However, today using DeviceIndex=1 does attach the ENI again. Now I recognize that I should programmatically determine which DeviceIndex is available and use that, but the closest I've seen to this value is "InterfaceIndex" in this blog:
https://blogs.technet.microsoft.com/heyscriptingguy/2014/01/15/using-powershell-to-find-connected-network-adapters/
with this command:
get-wmiobject win32_networkadapter | select netconnectionid, name, InterfaceIndex, netconnectionstatus
but after trying those values, it clearly isn't the value I'm looking for. Adding DeviceIndex=1 will get a network adapter at InterfaceIndex=29, for instance.
It's odd that this parameter is required in api when attaching an ENI from the console does not even have a place to enter this value (and works perfectly).
So to summarize, how do I determine the DeviceIndex to use for adding a new ENI to a Windows EC2?
I'm playing with smaller EC2 instances that only allow a max of two network devices, but I'd be willing to bet that this is the answer. If you look at the network interfaces of your ec2:
$instances = Get-Instances -InstanceId <your-ec2-id>
$instances.Instances[0].NetworkInterfaces.Attachment
you'll see the network devices that are attached along with the DeviceIndex property I've been looking for:
AttachmentId : eni-attach-aaaaaaaa
AttachTime : 5/15/2017 12:47:31 PM
DeleteOnTermination : True
DeviceIndex : 0
Status : attached
AttachmentId : eni-attach-bbbbbbbb
AttachTime : 5/17/2017 5:28:21 PM
DeleteOnTermination : False
DeviceIndex : 1
Status : attached
My guess is that the console finds the current and generates the next value for you, so you don't have to provide it, but since the Add-EC2NetworkInterface requires it, find the greatest value for DeviceIndex and attach using the next highest number.
Keep in mind that even if your call fails you still must detach the ENI (with Dismount-EC2NetworkInterface or from the AWS Console) because it will most likely be stuck in the 'attaching' state.
Hope this helps someone!
I'm tasked with automating the creation of Azure VM's, and naturally I do a number of more or less broken iterations of trying to deploy a VM image. As part of this, I automatically allocate serial hostnames, but there's a strange reason it's not working:
The code in the link above works very well, but the contents of my ResourceGroup is not as expected. Every time I deploy (successfully or not), a new entry is created in whatever list is returned by Get-AzureRmResourceGroupDeployment; however, in the Azure web interface I can only see a few of these entries. If, for instance, I omit a parameter for the JSON file, Azure cannot even begin to deploy something -- but the hostname is somehow reserved anyway.
Where is this list? How can I clean up after broken deployments?
Currently, Get-AzureRmResourceGroupDeployment returns:
azure-w10-tfs13
azure-w10-tfs12
azure-w10-tfs11
azure-w10-tfs10
azure-w10-tfs09
azure-w10-tfs08
azure-w10-tfs07
azure-w10-tfs06
azure-w10-tfs05
azure-w10-tfs02
azure-w7-tfs01
azure-w10-tfs19
azure-w10-tfs1
although the web interface only lists:
azure-w10-tfs12
azure-w10-tfs13
azure-w10-tfs09
azure-w10-tfs05
azure-w10-tfs02
Solved using the code $siblings = (Get-AzureRmResource).Name | Where-Object{$_ -match "^$hostname\d+$"}
(PS. If you have tips for better tags, please feel free to edit this question!)
If you create a VM in Azure Resource Management mode, it will have a deployment attached to it. In fact if you create any resource at all, it will have a resource deployment attached.
If you delete the resource you will still have the deployment record there, because you still deployed it at some stage. Consider deployments as part of the audit trail of what has happened within the account.
You can delete deployment records with Remove-AzureRmResourceGroupDeployment but there is very little point, since deployments have no bearing upon the operation of Azure. There is no cost associated they are just historical records.
Querying deployments with Get-AzureRmResourceGroupDeployment will yield you the following fields.
DeploymentName
Mode
Outputs
OutputsString
Parameters
ParametersString
ProvisioningState
ResourceGroupName
TemplateLink
TemplateLinkString
Timestamp
So you can know whether the deployment was successful via ProvisioningState know the templates you used with TemplateLink and TemplateLinkString and check the outputs of the deployment etc. This can be useful to figure out what template worked and what didn't.
If you want to see actual resources, that you are potentially being charged for, you can use Get-AzureRmResource
If you just want to retrieve a list of the names of VMs that exist within an Azure subscription, you can use
(Get-AzureRmVM).Name
I have read the Eucalyptus documentation ( https://www.eucalyptus.com/docs/euca2ools/3.0/euca2ools-guide/euca-run-instances.html#euca-run-instances ) as well as searched on Google as well as in this site to following question and could not find a solution.
Is there a way to set a human readable server name when setting up a new server (currently a lengthy UID is arbitrarily set as the server name)?
Situation: I am setting up a specific server and the name of the instance in our IAAS cloud, The assigned server name is a generic UID.
I would like to be able to specify the instance name for a number of valid reasons but am unable to find a way to do this.
Via Command Line (will convert to script for rollout to prod):
euca-run-instances -n 1 -g <nameofSecurityGroup> -k <NameofValidKey> -t <instanceType i.e. c1.medium> $<VariableHolding_InstanceID>
This runs successfully but the UID set as the Server Name is unhelpful for the users/clients/admins.
RESTATED: Is there any way to set a name for this new instance that is human friendly?
Please advise any reasonable thoughts or suggestions.
Thank you.
Jim
After creating an instance you can use euca-create-tags instance_id --tag Name=Server to change the name of the instance.
I am trying to start a google compute engine (GCE) instance with a pre-configured FQDN. We are intending to run an application that is licensed based on the contents of /etc/hosts.
I am starting the instances using the Google Cloud SDK utility - gcloud.
I have tried setting the "hostname" key using the metadata option like so:
gcloud compute instances create mynode (standard opts) --metadata hostname=mynode.example.com
Whenever I log into the developer console, under computer, instances, I can see hostname under "Custom metadata". This appears to be a new, custome key - it has no impact on what:
http://metadata.google.internal/computeMetadata/v1/instance/hostname
returns.
I have also tried setting "instance/hostname" like the below, which causes a parsing error when using gcloud.
--metadata instance/hostname=mynode.example.com
I have successfully used the startup scripts functionality of the metadata server to run a startup script that parses the new, internal IP address of the newly created instance, updated /etc/hosts. This appears to work but doesn't feel "like the google way".
Can I configure the FQDN (specifically, a domain name, as the instance name is always the hostname) of an instance, during instance creation, using the metaserver functionality?
try this:
Go to your GCE >> VM instances panel.
stop your gce instance.
clic on the instance name.
Edit your instance, adding this values on Custom metadata fields:
Key field: hostname / Value field: your.server.hostname
Key field: startup-script / Value field: sudo -s hostnamectl set-hostname your.server.hostname
setup-example-image.png
Finally, start your instance and test with a hostnamectl command.
regards!
According to this article 'hostname' is part of the default metadata entries that provide information about your instance and it is NOT possible to manually edit any of the default metadata pairs. You can also take a look at this video from the Google Team. Within the first few minutes it is mentioned that you cannot modify default metadata pairs. As such, it does not seem like you can specify the hostname upon instance creation other than through the use of a start-up script like you've done already. It is also worth mentioning that the hostname you've specified will get deleted and auto-synced by the metadata server upon reboot unless you're using a start-up script or something that would modify it every time.
If what you're currently doing works for what you're trying to accomplish, it might be the only workaround to your scenario.
Here is a patch for /usr/share/google/set-hostname to set FQDN to GCE instance.
https://gist.github.com/yuki-takeichi/3080521322f0f1d159ea6a343e2323e6
Before you use this patch, you must set your desired FQDN in your instance's metadata by specifying hostname key.
Hostname is set each time instance's IP address is renewed by dhclient. set-hostname is just a hook script which dhclient executes and serves new IP address and internal hostame to, and modifies /etc/hosts. This patch changes the source of hostname by querying instance's metadata from metadata server.
The original set-hostname script is here:
https://github.com/GoogleCloudPlatform/compute-image-packages/blob/master/google_config/bin/set_hostname.
Use this patch at your own risk.
When creating a VM, you can specify a custom FQDN hostname as an optional parameter. This feature is currently in Beta.
$ gcloud beta compute instances create INSTANCE_NAME --hostname example.hostname
This should work across OSes, and eliminate the need for workaround scripts.
More info in the docs.
-- Sirui (Product Manager, Google Compute Engine)
I've looked throughout this site to find answered questions and found a few things that work but with a couple solutions combined. This thread seems the place to answer.
1) echo example.com > /etc/hostname
2) add -- 127.0.1.1 example.com in /etc/hosts
3) add -- hostnamectl set-hostname
example.com -- command to /etc/rc.local script
4) uncomment /etc/dhcp/dhclient.conf line:
supersede domain-name "example.com";
5) profit.... Seems to stick after each reboot
(Note example.com is your domain name: fqdndomain.com - yourfqdndomain.org)
Also note this is for Ubuntu or Debian. Other Unix May slightly vary. I've tested this on Ubuntu 16.04
Always on the wording NOT possible to manually edit any of the default metadata pairs, how about the instant level default metadata "/scheduling"? we could set them manually as mentioned in this article
I am trying to connect a powershell script to an Auto Scaling Group via the .NET Amazon API.
I have checked the documentation here, but I am struggling to get an object that contains the IP addresses of the instances belonging to the Auto Scaling Group.
I am not sure which class to use, or which class contains my object.
I am currently using:
$request = New-Object -TypeName Amazon.AutoScaling.Model.DescribeAutoScalingInstancesRequest
Has anyone come across the same situation? Which class/object contains the IP addresses of the instances running in the AutoScaling group?
For anyone looking to do this via aws-cli
replace my autoscale group
aws --output text --query "Reservations[*].Instances[*].PublicIpAddress" ec2 describe-instances --instance-ids `aws --output text --query "AutoScalingGroups[0].Instances[*].InstanceId" autoscaling describe-auto-scaling-groups --auto-scaling-group-names "my autoscale group"`
Using the SDK approach you take the set of instance ids embedded in the response/result data returned from the DescribeAutoScalingInstances call and pass them to the DescribeInstances call for EC2 (using new-object again to get an EC2 client and request objects). This will net you a collection of Amazon.EC2.Model.Reservation objects (again inside the response/result data) from which the RunningInstance collection inside each reservation will get you the ip address(es) for the EC2 instance.
It is however much simpler to use the AWS Tools for Windows PowerShell like this:
Get-ASAutoScalingInstance | select -expandproperty InstanceId | Get-EC2Instance | select -expandproperty RunningInstance | ft InstanceId, IpAddress
Get-ASAutoScalingInstance maps to the request in your question; this yields the set of EC2 instances from which we extract the id of each instance with a select. We then request details for the instance using Get-EC2Instance; as noted above this yields an Amazon.EC2.Model.Reservation object, within with are the details of the instance (in the RunningInstance collection). We flatten this to pull out the instance id and associated ip address for the table.
The sample pipeline above assumes you've set credentials and region to use in the shell using Set-AWSCredentials and Set-DefaultAWSRegion.
The AWS Tools for Windows PowerShell are included with the download msi for the SDK and Visual Studio toolkit available here http://aws.amazon.com/net/.
Hope this helps.
Expanding on the answer above - you may want to include the name of the auto scaling group so you don't get every instance from every group. Also, if you're using VPC your instances may not have public IPs so you'll be after the private IPs like this
Get-ASAutoScalingInstance | ? {$_.AutoScalingGroupName -eq "web-autoscaler-group"} | select -ExpandProperty InstanceId | Get-EC2Instance | select -ExpandProperty RunningInstance | ft InstanceId, PrivateIpAddress