Configuring FQDN for GCE instance on startup - metadata

I am trying to start a google compute engine (GCE) instance with a pre-configured FQDN. We are intending to run an application that is licensed based on the contents of /etc/hosts.
I am starting the instances using the Google Cloud SDK utility - gcloud.
I have tried setting the "hostname" key using the metadata option like so:
gcloud compute instances create mynode (standard opts) --metadata hostname=mynode.example.com
Whenever I log into the developer console, under computer, instances, I can see hostname under "Custom metadata". This appears to be a new, custome key - it has no impact on what:
http://metadata.google.internal/computeMetadata/v1/instance/hostname
returns.
I have also tried setting "instance/hostname" like the below, which causes a parsing error when using gcloud.
--metadata instance/hostname=mynode.example.com
I have successfully used the startup scripts functionality of the metadata server to run a startup script that parses the new, internal IP address of the newly created instance, updated /etc/hosts. This appears to work but doesn't feel "like the google way".
Can I configure the FQDN (specifically, a domain name, as the instance name is always the hostname) of an instance, during instance creation, using the metaserver functionality?

try this:
Go to your GCE >> VM instances panel.
stop your gce instance.
clic on the instance name.
Edit your instance, adding this values on Custom metadata fields:
Key field: hostname / Value field: your.server.hostname
Key field: startup-script / Value field: sudo -s hostnamectl set-hostname your.server.hostname
setup-example-image.png
Finally, start your instance and test with a hostnamectl command.
regards!

According to this article 'hostname' is part of the default metadata entries that provide information about your instance and it is NOT possible to manually edit any of the default metadata pairs. You can also take a look at this video from the Google Team. Within the first few minutes it is mentioned that you cannot modify default metadata pairs. As such, it does not seem like you can specify the hostname upon instance creation other than through the use of a start-up script like you've done already. It is also worth mentioning that the hostname you've specified will get deleted and auto-synced by the metadata server upon reboot unless you're using a start-up script or something that would modify it every time.
If what you're currently doing works for what you're trying to accomplish, it might be the only workaround to your scenario.

Here is a patch for /usr/share/google/set-hostname to set FQDN to GCE instance.
https://gist.github.com/yuki-takeichi/3080521322f0f1d159ea6a343e2323e6
Before you use this patch, you must set your desired FQDN in your instance's metadata by specifying hostname key.
Hostname is set each time instance's IP address is renewed by dhclient. set-hostname is just a hook script which dhclient executes and serves new IP address and internal hostame to, and modifies /etc/hosts. This patch changes the source of hostname by querying instance's metadata from metadata server.
The original set-hostname script is here:
https://github.com/GoogleCloudPlatform/compute-image-packages/blob/master/google_config/bin/set_hostname.
Use this patch at your own risk.

When creating a VM, you can specify a custom FQDN hostname as an optional parameter. This feature is currently in Beta.
$ gcloud beta compute instances create INSTANCE_NAME --hostname example.hostname
This should work across OSes, and eliminate the need for workaround scripts.
More info in the docs.
-- Sirui (Product Manager, Google Compute Engine)

I've looked throughout this site to find answered questions and found a few things that work but with a couple solutions combined. This thread seems the place to answer.
1) echo example.com > /etc/hostname
2) add -- 127.0.1.1 example.com in /etc/hosts
3) add -- hostnamectl set-hostname
example.com -- command to /etc/rc.local script
4) uncomment /etc/dhcp/dhclient.conf line:
supersede domain-name "example.com";
5) profit.... Seems to stick after each reboot
(Note example.com is your domain name: fqdndomain.com - yourfqdndomain.org)
Also note this is for Ubuntu or Debian. Other Unix May slightly vary. I've tested this on Ubuntu 16.04

Always on the wording NOT possible to manually edit any of the default metadata pairs, how about the instant level default metadata "/scheduling"? we could set them manually as mentioned in this article

Related

In docker-compose, what is the effect/purpose of `dns-search: .`

I am looking at the stackstorm docker-compose file, and within it almost all containers have a line dns_search: . According to docker-compose documentation, dns_search is for the purpose of configuring search domains.
I am used to seeing this in context of transparently adding a domain to unqualified short domains. For example if I add dns_search: mydomain.com, I would expect "host1" to transparently resolve as "host1.mydomain.com".
I have never seen this set as a single dot . before. What is the effect/purpose of doing this configuration?
I'm posting the answer from the Stackstorm Git project issue see comment/"dns_search: .". Paraphrasing: it was useful in old versions of Docker before 2017, before the ndots configuration was available. Nowadays that configuration has no impact, and in fact has been removed from the stackstorm docker-compose file.
I believe this is because all domain names end in . under the hood, but browsers and other software abstracts this out.
For example. under the hood www.google.com is actually www.google.com.
So, in the docker-compose file, this would essentially be saying "Find me any domain"
A bit more detail on why there's an extra dot, if you're interested:
Domain name resolution is heirachical, reading right to left, with each block, separated by a ., being a step in the process. A DNS resolver will first find a source of ., which will be able to return the address for a resolver for the next block, until it reaches the final block, where it returns the full DNS record.
Extending EdwardTeach's answer:
#ytjohn effectively said they did in the past because putting dns_search: . configures the DNS search domains to be only . instead of inheriting the host ones. I can't confirm that because I didn't test it.
Now, I tested what docker-compose does today, and in a container, cat /etc/resolve.conf returns:
nameserver 127.0.0.11
options ndots:0
Where options ndots:0 is (from resolv.conf docs):
ndots:n
Sets a threshold for the number of dots which must
appear in a name given to res_query(3) (see
resolver(3)) before an initial absolute query will
be made. The default for n is 1, meaning that if
there are any dots in a name, the name will be
tried first as an absolute name before any search
list elements are appended to it. The value for
this option is silently capped to 15.
With ndots:0, all domains will be attempted using the absolute name first, only then using the search list.
How I came to this conclusion
The Github comment:
If you don't set this dns_search: ., then whatever the host has in search in their /etc/resolv.conf will get put into your container's /etc/resolv.conf.
This doesn't happen. My host has search domain[0]: broadband (macOS command: scutil --dns), and in docker containers, it doesn't show broadband (linux command: cat /etc/resolv.conf). Instead, it says options ndots:0
dns_search docs:
dns defines custom DNS search domains to set on container network interface configuration. Can be a single value or a list.
What is a DNS search domain?
It is the DNS service used to resolve hostnames that are not fully qualified, e.g. hostname will try hostname.example.com then hostname.website.com if your search domains list was example.com, website.com. More information on https://superuser.com/a/184366
In another repo (crossdock), their dockerfile had the comment:
`dns_search: . # Ensures unified DNS config.`

Hyperledger Fabric CA - Storing the identity materials the correct way

Currently I have a VM running and installed the binaries needed for fabric-ca. I have a docker-compose file looking like this:
I have some questions regarding this:
the docker-compose file will create one container, if I want it for
more organizations, do I need to copy/paste this and change the port
number? (I don't want to use intermediate CAs).
When registering/enrolling an identity, it will override the default
materials because It will always put the materials from the new identity in /etc/hyperledger/fabric-ca-client. So when creating multiple
identities (orderer, peers, users etc..) how do I need to organize
them? What's the best practise?
In the image you can see that the server and clients are specified,
is this a good approach? Or should the client and the server be a
different container?
More than one CA in a Docker Compose file - you can look at the Build your first network tutorial in the Fabric Docs which has a 2 Org network and various configuration files including Docker Compose.
Combined client/server Container - This might be convenient for testing, but in a production scenario definitely not for Security and Operational Integrity reasons.
Overwriting Identities - the enroll command writes a tree of data to the location specified by the environment variable FABRIC_CA_CLIENT_HOME but you can use --home to redirect the tree to a different location:
fabric-ca-client enroll -u http://Jane:janepw#myca.example.com:7054 --home /home/test/Jane/

installing kubernetes on coreos with rkt and automated script

I'm trying to install kuberentes with rkt on my real (not virtual) coreos servers at home using the scripts at https://github.com/coreos/coreos-kubernetes/tree/master/multi-node/generic and I have some questions.
my etcd2 is using tls keys, I can't see anywhere in the script where I can define where the certificates are located.
can I supply a domain instead of IP for ADVERTISE_IP and CONTROLLER_ENDPOINT ?
when I tried to install kubernetes manually I needed start the rkt service api. it doesn't state in the documents that it needed here, does it mean that I don't need it if I use these scripts? or is it just something that's missing in the documents?
thanks!
update
Rob thank you so much for your response. I wasn't clear enough regarding etcd2. I already have etcd2 tls installed and properly configured on my coreos servers. so I configured my etcd servers in the controller-install.sh file:
export ETCD_ENDPOINTS="https://coreos-2.tux-in.com:2379,https://coreos-3.tux-in.com:2379"
but when I run the controller-install.sh script, it returns and repeat the following output:
Waiting for etcd...
Trying: https://coreos-2.tux-in.com:2379
Trying: https://coreos-3.tux-in.com:2379
Trying: https://coreos-2.tux-in.com:2379
Trying: https://coreos-3.tux-in.com:2379
...
so I was guessing it's because i didn't define etcd related tls certificates in the controller script and that is why it stuck in that faze.
on my macbook pro laptop I have the following alias configured:
alias myetcdctl="~/apps/etcd-v3.0.8-darwin-amd64/etcdctl --endpoint=https://coreos-2.tux-in.com:2379 --ca-file=/Users/ufk/Projects/coreos/tux-in/etcd/certs/certs-names/ca.pem --cert-file=/Users/ufk/Projects/coreos/tux-in/etcd/certs/certs-names/etcd1.pem --key-file=/Users/ufk/Projects/coreos/tux-in/etcd/certs/certs-names/etcd1-key.pem --timeout=10s"
so when I run myetcdctl member list I get:
8832ce6a269a7dac: name=ccff826d5f564c67abf35467306f80a0 peerURLs=https://coreos-3.tux-in.com:2380 clientURLs=https://coreos-3.tux-in.com:2379 isLeader=true
a2c0ac9708ef90fc: name=dc38bc8f20e64940b260d3f7b260430d peerURLs=https://coreos-2.tux-in.com:2380 clientURLs=https://coreos-2.tux-in.com:2379 isLeader=false
so I'm guessing that I don't really have a problem there.
any ideas?
thanks!
my etcd2 is using tls keys, I can't see anywhere in the script where I can define where the certificates are located.
These scripts don't start an etcd server. You will need to set one up manually and will be able to use TLS and as many nodes as you would like. This isn't clear in the current form of the document, I will attempt a PR to fix.
can I supply a domain instead of IP for ADVERTISE_IP and CONTROLLER_ENDPOINT ?
Only CONTROLLER_ENDPOINT be a domain name.
when I tried to install kubernetes manually I needed start the rkt service api. it doesn't state in the documents that it needed here, does it mean that I don't need it if I use these scripts? or is it just something that's missing in the documents?
These scripts include/start the rkt API service. As you can see below, it also has a Restart parameter set (source):
[Unit]
Before=kubelet.service
[Service]
ExecStart=/usr/bin/rkt api-service
Restart=always
RestartSec=10
[Install]
RequiredBy=kubelet.service

EUCALYPTUS Set machine name using euca-run-instances

I have read the Eucalyptus documentation ( https://www.eucalyptus.com/docs/euca2ools/3.0/euca2ools-guide/euca-run-instances.html#euca-run-instances ) as well as searched on Google as well as in this site to following question and could not find a solution.
Is there a way to set a human readable server name when setting up a new server (currently a lengthy UID is arbitrarily set as the server name)?
Situation: I am setting up a specific server and the name of the instance in our IAAS cloud, The assigned server name is a generic UID.
I would like to be able to specify the instance name for a number of valid reasons but am unable to find a way to do this.
Via Command Line (will convert to script for rollout to prod):
euca-run-instances -n 1 -g <nameofSecurityGroup> -k <NameofValidKey> -t <instanceType i.e. c1.medium> $<VariableHolding_InstanceID>
This runs successfully but the UID set as the Server Name is unhelpful for the users/clients/admins.
RESTATED: Is there any way to set a name for this new instance that is human friendly?
Please advise any reasonable thoughts or suggestions.
Thank you.
Jim
After creating an instance you can use euca-create-tags instance_id --tag Name=Server to change the name of the instance.

Query on DNS & connect to existing vm

In my current code base, when i create a VM, DNS name is being dynamically set as same as the instance name. For example, consider if my VM name is "anandInstance", DNS name of the name is being generated as "anandInstance.cloudapp.net". Is there a way to change the DNS name like "dns1.cloudapp.net" during the creation thru REST API??
"Connect to existing VM" , is it possible to achieve this option through REST call? In case "connect to existing.." option , we are getting a list of vms/services to choose and VM is getting created successfully. How to achieve the same using API.
Thanks
In my current code base, when i create a VM, DNS name is being
dynamically set as same as the instance name. For example, consider if
my VM name is "anandInstance", DNS name of the name is being generated
as "anandInstance.cloudapp.net". Is there a way to change the DNS name
like "dns1.cloudapp.net" during the creation thru REST API??
I don't think it is possible. Imagine what a nightmare in the portal would become if you were able to do so? How would you link a Cloud Service (whatever.cloudapp.net) to an actual deployment (MyDemoVm123). However you can use your own domain and have CNAME records pointing to your "want-to-change-for-some-reason.cloudapp.net" (frankly I surely think that soon we will use even longer names)
"Connect to existing VM" , is it possible to achieve this option
through REST call?
Connection to a VM is essentially opening a RDP session. If it a windows VM, you can try using the Download RDP file API call. Once you get the file, just start it with "process.start". If it is linux VM, just start SSH client on port 22 (or one you have defined) from the Cloud Service DNS name you have.
UPDATE
From the azure portal,for stand alone machineoption, we are able to give the dns name with deafult cloudoneapp.net. How to do the same
through the rest api call.any specfic paramter is there to specify the
same?
When you are using the REST API, you first create a Cloud Service (still named hosted service in the REST API) where your machine will be hosted. Here you give the name for that hosted service (the dns name with deafult cloudoneapp.net). Then you call the Create Virtual Machine Deployment API action.
In case "connect to existing.." option , we are getting a list of vms/services to choose and VM is getting created successfully. How to
achieve the same using API.
When you want to get list of all VMs, just get a list of all Hosted Services, then get properties of each and make a guess whether it is a VM or a Cloud Service (maybe by querying for Properties of each service). I don't see a direct access to the list of Virtual Machines. But as this feature being PREVIEW, things might change in the future.
Hope my answer is clear?