How can I provision Kubernetes with kops using multi document yaml file? - kubernetes

Question

I've got a multi document yaml file, how can I provision a Kubernetes cluster from this?
Using kops create -f only seems to import the cluster definition, and does not import the instance groups etc which are defined in additional 'documents' with the yaml:
...
masters: private
nodes: private
---
apiVersion: kops/v1alpha2
kind: InstanceGroup
...

Answer 1

According to kops source code, multiple section in one file is supported.
Perhaps you need to check your file for mistakes.
Here is the part responsible for parsing file and creating resources. To make code shorter, I've deleted all the error checks.
For investigation, please see original file:
func RunCreate(f *util.Factory, out io.Writer, c *CreateOptions) error {
clientset, err := f.Clientset()
// Codecs provides access to encoding and decoding for the scheme
codecs := kopscodecs.Codecs //serializer.NewCodecFactory(scheme)
codec := codecs.UniversalDecoder(kopsapi.SchemeGroupVersion)
var clusterName = ""
//var cSpec = false
var sb bytes.Buffer
fmt.Fprintf(&sb, "\n")
for _, f := range c.Filenames {
contents, err := vfs.Context.ReadFile(f)
// TODO: this does not support a JSON array
sections := bytes.Split(contents, []byte("\n---\n"))
for _, section := range sections {
defaults := &schema.GroupVersionKind{
Group: v1alpha1.SchemeGroupVersion.Group,
Version: v1alpha1.SchemeGroupVersion.Version,
}
o, gvk, err := codec.Decode(section, defaults, nil)
switch v := o.(type) {
case *kopsapi.Cluster:
// Adding a PerformAssignments() call here as the user might be trying to use
// the new `-f` feature, with an old cluster definition.
err = cloudup.PerformAssignments(v)
_, err = clientset.CreateCluster(v)
case *kopsapi.InstanceGroup:
clusterName = v.ObjectMeta.Labels[kopsapi.LabelClusterName]
cluster, err := clientset.GetCluster(clusterName)
_, err = clientset.InstanceGroupsFor(cluster).Create(v)
case *kopsapi.SSHCredential:
clusterName = v.ObjectMeta.Labels[kopsapi.LabelClusterName]
cluster, err := clientset.GetCluster(clusterName)
sshCredentialStore, err := clientset.SSHCredentialStore(cluster)
sshKeyArr := []byte(v.Spec.PublicKey)
err = sshCredentialStore.AddSSHPublicKey("admin", sshKeyArr)
default:
glog.V(2).Infof("Type of object was %T", v)
return fmt.Errorf("Unhandled kind %q in %s", gvk, f)
}
}
}
{
// If there is a value in this sb, this should mean that we have something to deploy
// so let's advise the user how to engage the cloud provider and deploy
if sb.String() != "" {
fmt.Fprintf(&sb, "\n")
fmt.Fprintf(&sb, "To deploy these resources, run: kops update cluster %s --yes\n", clusterName)
fmt.Fprintf(&sb, "\n")
}
_, err := out.Write(sb.Bytes())
}
return nil
}

Answer 2

This may not have been possible in the past, but it's possible now.
Here's how I'm currently IaC'ing my kops cluster.
I have 2 files:
devkube.mycompany.com.cluster.kops.yaml
instancegroups.kops.yaml
The 2nd file is a multi-document yaml file like you're requesting.
(with multiple instance.yaml's inside of it separated by ---).
Note: I've successfully been able to combine both of these .yaml files into a single multi-document .yaml file and have it work fine, I just do it this way so I can reuse the instancegroups.yaml on multiple clusters / keep the git repo dry.
I generated the files by using:
Bash# kops get cluster --name devkube.mycompany.com -o yaml > devkube.mycompany.com.cluster.kops.yaml
Bash# kops get ig --name devkube.mycompany.com -o yaml > instancegroups.kops.yaml
Here's how I use these files to provision a kops cluster from git:
export AWS_PROFILE=devkube-kops
export KOPS_STATE_STORE=s3://kops-state-devkube.mycompany.com
clustername="devkube.mycompany.com"
kops replace --name $clustername -f ../kops/qak8s.vibrenthealth.com.cluster.yaml --force
kops replace --name $clustername -f ../kops/instancegroups.yaml --force
kops create secret --name $clustername sshpublickey admin -i ~/.ssh/id_rsa.pub #so the person who provisions the cluster can ssh into the nodes
kops update cluster --name $clustername --yes