Adding An ENI (Elastic Network Interface) To A Windows EC2 - powershell

Okay AWS Windows Powershell guys... here's a question for you. How do you add an ENI (Elastic Network Interface) to your Windows EC2? Seems simple enough as many examples show:
Add-EC2NetworkInterface -NetworkInterfaceId <your-eni-id> `
-InstanceId <your-ec2-id> `
-DeviceIndex 1 `
-Force
but in my past experiences DeviceId=1 stopped working and I switched to DeviceIndex=2. I went along my way and suddenly DeviceIndex=2 doesn't work, generating the exception
Instance 'i-xxxxxxxxxxxxxxxxx' already has an interface attached at device index '2'
and the ENI shows 'attaching' forever (and must be forcibly detached). However, today using DeviceIndex=1 does attach the ENI again. Now I recognize that I should programmatically determine which DeviceIndex is available and use that, but the closest I've seen to this value is "InterfaceIndex" in this blog:
https://blogs.technet.microsoft.com/heyscriptingguy/2014/01/15/using-powershell-to-find-connected-network-adapters/
with this command:
get-wmiobject win32_networkadapter | select netconnectionid, name, InterfaceIndex, netconnectionstatus
but after trying those values, it clearly isn't the value I'm looking for. Adding DeviceIndex=1 will get a network adapter at InterfaceIndex=29, for instance.
It's odd that this parameter is required in api when attaching an ENI from the console does not even have a place to enter this value (and works perfectly).
So to summarize, how do I determine the DeviceIndex to use for adding a new ENI to a Windows EC2?

I'm playing with smaller EC2 instances that only allow a max of two network devices, but I'd be willing to bet that this is the answer. If you look at the network interfaces of your ec2:
$instances = Get-Instances -InstanceId <your-ec2-id>
$instances.Instances[0].NetworkInterfaces.Attachment
you'll see the network devices that are attached along with the DeviceIndex property I've been looking for:
AttachmentId : eni-attach-aaaaaaaa
AttachTime : 5/15/2017 12:47:31 PM
DeleteOnTermination : True
DeviceIndex : 0
Status : attached
AttachmentId : eni-attach-bbbbbbbb
AttachTime : 5/17/2017 5:28:21 PM
DeleteOnTermination : False
DeviceIndex : 1
Status : attached
My guess is that the console finds the current and generates the next value for you, so you don't have to provide it, but since the Add-EC2NetworkInterface requires it, find the greatest value for DeviceIndex and attach using the next highest number.
Keep in mind that even if your call fails you still must detach the ENI (with Dismount-EC2NetworkInterface or from the AWS Console) because it will most likely be stuck in the 'attaching' state.
Hope this helps someone!

Related

Converting REST API response to a table

When I do a GET on a REST API enabled system the return is formatted as a table and displayed as this:
problems
--------
{#{problemID=8004; description=vCenter Server 6853d04d-4ff0-4eba-b0a2-4a7dffcaf028 is not licensed.; severity=WARNING; category=SYSTEM; relevantSplitter=; cluster=; relevantRPAs=System.Object[]; relevantGlobalLinks=System.Object[]; relevantCopies=System.Object[]; relevantDevices=System.Object[]; relevantSplitterVolumesInfor…
If I pipe it to Format-List I get:
problems : {#{problemID=8004; description=vCenter Server 6853d04d-4ff0-4eba-b0a2-4a7dffcaf028 is not licensed.; severity=WARNING; category=SYSTEM; relevantSplitter=; cluster=; relevantRPAs=System.Object[]; relevantGlobalLinks=System.Object[]; relevantCopies=System.Object[]; relevantDevices=System.Object[];
relevantSplitterVolumesInformation=System.Object[]}, #{problemID=10100; description=At least one virtual RPA is running on the same ESX as the VM it is replicating. It is recommended not to have the virtual RPA that replicates a VM running on the same ESX as the replicated VM. Use vMotion to move one of them to
another ESX.; severity=WARNING; category=CONSISTENCY_GROUP; relevantSplitter=; cluster=; relevantRPAs=System.Object[]; relevantGlobalLinks=System.Object[]; relevantCopies=System.Object[]; relevantDevices=System.Object[]; relevantSplitterVolumesInformation=System.Object[]}, #{problemID=10100; description=At least
one virtual RPA is running on the same ESX as the VM it is replicating. It is recommended not to have the virtual RPA that replicates a VM running on the same ESX as the replicated VM. Use vMotion to move one of them to another ESX.; severity=WARNING; category=CONSISTENCY_GROUP; relevantSplitter=; cluster=;
relevantRPAs=System.Object[]; relevantGlobalLinks=System.Object[]; relevantCopies=System.Object[]; relevantDevices=System.Object[]; relevantSplitterVolumesInformation=System.Object[]}}
I am trying to make the leap to taking the value for problems and use it as an array of hash tables which it appears to be. I guess I just have not connected the dots as yet. Any help would be appreciated.
You can use ConvertFrom-Json on your output.

AWS latest ami imageId from an old imageid?

I have an EC2 Instance running an old AMI from AWS, which in my current setup is from WINDOWS_2016_BASE. Now I know that, but the Instance doesn't know that.
Using PowerShell I want the EC2 Instance to create more like itself, but not from an AMI I have created (as that would of course always be at the old version) and not "directly" from the actual AMI ImageId it was created from. As AWS don't allow that if it is not the latest version of it (hence I need the indirect route to getting that AMI ImageId, even if it turns out to to be the same.. i.e if it the latest version already).
So I need to find, using PowerShell, the latest version of an AMI I already know (which I got from the metadata) and I do not want to specify the name of the AMI e.g WINDOWS_2016_BASE since I would like the machine to create more like itself without me having to hardcode what it is in the script.
I have no idea how to do this since using Get-EC2ImageByName -Names WINDOWS_2016_BASE is not what I want to do since I don't want to hardcode that names parameter or pass it in through userdata.
Get-EC2Image -ImageId <an old imageid> returns null since the AMI is no longer current.
This will be somewhat difficult without either stashing this name somewhere that your script can cue off of (which you've stated that you don't want to do), or relying on some mapping between names/amis (that Get-EC2ImageByName already does for you).
AMI metadata is designed such that it is easy to retrieve a 'family tree' of related AMIs, under the assumption that their names are similar excepting for maybe minor differences such as versions or dates. It is a common pattern that prefixes of related AMIs will be consistent, which makes them easier to search for and aggregate.
This point makes your approach difficult, because your scripts will be missing that bit of data -- the AMI name prefix.
You could dynamically retrieve your instance's AMI ID from the metadata service and pass that into Get-EC2Image to get AMI details, but you couldn't go further from there without some sort of name prefix to match against and use to search for related AMIs that are newer.
Maybe reconsider this approach, and keep the name prefix stashed in a tag or on the instance via user-data? For example, I just checked and the English Windows Server 2016 Base AMIs all share this prefix: Windows_Server-2016-English-Full-Base. If you stored that on your instance or as one of your tags, your script could retrieve it and run the following powershell to get the latest Windows Server 2016 AMI:
#(Get-EC2Image -Owner amazon -Filter #{ Name="name"; Values="Windows_Server-2016-English-Full-Base*" } | Sort CreationDate -Desc)[0].ImageId
Example from the system log (which i got from aws console view)
2016/12/26 14:36:12Z: AMI Origin Name: Windows_Server-2016-English-Full-Base
The equivalent powershell is:
Get-EC2ConsoleOutput
So here below in full (i'm new to powershell but i am pretty sure someone can wrap that into one character somehow)
# read the system console log
$consoleLog = Get-EC2ConsoleOutput $currentInstanceObj.InstanceId
$consoleLogOutput = [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($consoleLog.Output));
# extract the lines that contain AMI Origin in them (there should be 2) - sort them so name i first and version is second
$originLines = $consoleLogOutput -split '[\r\n]' | Where-Object {$_.Contains("AMI Origin")} | Sort-Object
# get the running name
$originLine = $originLines[0]
$originLineParts = $originLine.Split(':')
$originName = $originLineParts[$originLineParts.Length - 1].Trim()
"The origin name is $originName"
# get the running version (slighly pointless since the code below doesn't care as we want the latest - but it's for verbosity)
$originLine = $originLines[1]
$originLineParts = $originLine.Split(':')
$originVersion = $originLineParts[$originLineParts.Length - 1].Trim()
"The origin version is $originVersion"
# concatenate to get the original origin name (note: amazon have a naming pattern here - (name-version)
$amiName = $originName + "-" + $originVersion
"The original origin ami name is $amiName"
#find the latest of the same name and report the difference
$latestOriginObj = (Get-EC2Image -Filter #{ Name="name"; Values=($originName + "*")} | Sort-Object CreationDate -Descending | Select-Object -First 1)
if($latestOriginObj.ImageId -ne $currentInstanceObj.ImageId)
{
"The ami has been upgraded from " + ($currentInstanceObj.ImageId) + " to " + ($latestOriginObj.ImageId)
}
#....so go ahead and use the $latestOriginObj.ImageId when you create a new instance
And the knowledge source for that springs from these amazons docs
Excerpt below:
The AWS Management Console provides details about the AMI that you use to create an Amazon EC2 instance. The AMI ID field on the Description tab contains information including the Windows Server SKU, the architecture (32-bit or 64-bit), the date the AMI was created, and an AMI ID.
If an AMI has been made private or replaced by later versions and is no longer listed in the catalog, the AMI ID field states, "Cannot load detail for ami-xxxxx. You may not be permitted to view it." To determine which AMI was used to create the instance, you must open the system log. In the EC2 console, choose an instance, and from the context-menu (right-click) choose Instance Settings and then choose Get System Log. The date the AMI was created and the SKU are listed in the AMI Origin Version and AMI Origin Name fields.

Move-AzureRmResource fails randomly

I am trying to move Service Bus Namespace to a different resource group using powershell commands. The code below sometimes works and sometimes fails.
$Resource = Find-AzureRmResource -ResourceType "Microsoft.ServiceBus/Namespaces" -ResourceNameContains $ServiceBusNamespace
Move-AzureRmResource -DestinationResourceGroupName $ResourceGroupName -ResourceId $Resource.ResourceId -Force
And here is a random error information:
Move-AzureRmResource : ResourceMoveFailed : Resources
'/subscriptions/f24b849a-ba33-4bd9-a87e-eca0df1cbcd2/resourceGroups/Default-ServiceBus-WestEurope/providers/Microsoft.ServiceBus/namespaces/cokolwiekNamespace'
could not be moved. The tracking Id is '64c
52d24-a471-490d-b18a-b7838966a8e0'
What does the tracking Id means? Can I find it in some logs and have more meanigfull information?
Thank you for your question.
I reproduce the same error in my lab, when I try to move my service bus to destination resource group, the destination resource group’s location is different from the source resource group. In fact, there are many reasons for this error. There are some important steps to perform before moving a resource. By verifying these conditions, you can avoid errors.
More information about move resource to new resource group, please refer to the link below:
https://azure.microsoft.com/en-us/documentation/articles/resource-group-move-resources/
The tracking id is a randomly generated, different operations have different tracking id.
If you want to see the logs about the PowerShell, we can find the logs in localhost, run “eventvwr”, and select the “Applications and Services logs” -> “Microsoft”-> “windows” “PowerShell”-> “Operational”. like:
The picture of powershell logs in localhost
If you still have questions, welcome to post back here. Thanks.

Configuring FQDN for GCE instance on startup

I am trying to start a google compute engine (GCE) instance with a pre-configured FQDN. We are intending to run an application that is licensed based on the contents of /etc/hosts.
I am starting the instances using the Google Cloud SDK utility - gcloud.
I have tried setting the "hostname" key using the metadata option like so:
gcloud compute instances create mynode (standard opts) --metadata hostname=mynode.example.com
Whenever I log into the developer console, under computer, instances, I can see hostname under "Custom metadata". This appears to be a new, custome key - it has no impact on what:
http://metadata.google.internal/computeMetadata/v1/instance/hostname
returns.
I have also tried setting "instance/hostname" like the below, which causes a parsing error when using gcloud.
--metadata instance/hostname=mynode.example.com
I have successfully used the startup scripts functionality of the metadata server to run a startup script that parses the new, internal IP address of the newly created instance, updated /etc/hosts. This appears to work but doesn't feel "like the google way".
Can I configure the FQDN (specifically, a domain name, as the instance name is always the hostname) of an instance, during instance creation, using the metaserver functionality?
try this:
Go to your GCE >> VM instances panel.
stop your gce instance.
clic on the instance name.
Edit your instance, adding this values on Custom metadata fields:
Key field: hostname / Value field: your.server.hostname
Key field: startup-script / Value field: sudo -s hostnamectl set-hostname your.server.hostname
setup-example-image.png
Finally, start your instance and test with a hostnamectl command.
regards!
According to this article 'hostname' is part of the default metadata entries that provide information about your instance and it is NOT possible to manually edit any of the default metadata pairs. You can also take a look at this video from the Google Team. Within the first few minutes it is mentioned that you cannot modify default metadata pairs. As such, it does not seem like you can specify the hostname upon instance creation other than through the use of a start-up script like you've done already. It is also worth mentioning that the hostname you've specified will get deleted and auto-synced by the metadata server upon reboot unless you're using a start-up script or something that would modify it every time.
If what you're currently doing works for what you're trying to accomplish, it might be the only workaround to your scenario.
Here is a patch for /usr/share/google/set-hostname to set FQDN to GCE instance.
https://gist.github.com/yuki-takeichi/3080521322f0f1d159ea6a343e2323e6
Before you use this patch, you must set your desired FQDN in your instance's metadata by specifying hostname key.
Hostname is set each time instance's IP address is renewed by dhclient. set-hostname is just a hook script which dhclient executes and serves new IP address and internal hostame to, and modifies /etc/hosts. This patch changes the source of hostname by querying instance's metadata from metadata server.
The original set-hostname script is here:
https://github.com/GoogleCloudPlatform/compute-image-packages/blob/master/google_config/bin/set_hostname.
Use this patch at your own risk.
When creating a VM, you can specify a custom FQDN hostname as an optional parameter. This feature is currently in Beta.
$ gcloud beta compute instances create INSTANCE_NAME --hostname example.hostname
This should work across OSes, and eliminate the need for workaround scripts.
More info in the docs.
-- Sirui (Product Manager, Google Compute Engine)
I've looked throughout this site to find answered questions and found a few things that work but with a couple solutions combined. This thread seems the place to answer.
1) echo example.com > /etc/hostname
2) add -- 127.0.1.1 example.com in /etc/hosts
3) add -- hostnamectl set-hostname
example.com -- command to /etc/rc.local script
4) uncomment /etc/dhcp/dhclient.conf line:
supersede domain-name "example.com";
5) profit.... Seems to stick after each reboot
(Note example.com is your domain name: fqdndomain.com - yourfqdndomain.org)
Also note this is for Ubuntu or Debian. Other Unix May slightly vary. I've tested this on Ubuntu 16.04
Always on the wording NOT possible to manually edit any of the default metadata pairs, how about the instant level default metadata "/scheduling"? we could set them manually as mentioned in this article

Getting AccessTokenRefreshError: invalid_grant in Google API fro service account

I am following this example
https://code.google.com/p/google-api-python-client/source/browse/samples/service_account/tasks.py
credentials = SignedJwtAssertionCredentials(
'141491975384#developer.gserviceaccount.com',
key,
scope='https://www.googleapis.com/auth/tasks')
http = httplib2.Http()
http = credentials.authorize(http)
service = build("tasks", "v1", http=http)
# List all the tasklists for the account.
lists = service.tasklists().list().execute(http=http)
pprint.pprint(lists)
The issue is , it works sometimes and i get the lists as JSON and after running program few more times i get error
File "/usr/local/lib/python2.7/site-packages/oauth2client/client.py", line 710, in _do_refresh_request
raise AccessTokenRefreshError(error_msg)
oauth2client.client.AccessTokenRefreshError: invalid_grant
I'm interface Google Drive but found the same error. In Issue 160 there is a report of setting the appropriate time on your local computer. Ever since I upgraded to Mac Mavericks I found that the I need to keep updating my system time. I was getting your reported error, set my system time back to current and I eliminated the error.
Are you running this code in a VM or sandboxed environment? If so, it could just be that your VM's clock isn't synchronised to your host machine. See a response to a similar question here.
I suffered the same (frustrating) problem and found that simply restarting my VM (ensuring the time was synchronised to the host machine (or at least set to the local timezone) fixed the problem.
Oauth service is highly dependent on the time, make sure your client uses NTP or another time syncing mechanism.
Test with curl -I https://docs.google.com/ ; date -u
You should see the same Date: (or whithin a few seconds) for this to work