Unable to scale application - containercreating - kubernetes

Question

I am using this deployment template (is that what its called?). Two pods are running but two are stuck at containercreating. If I scale to 2 replicas, 1 is running and 1 is stuck at containercreating. How to have all 4 pods running?
This declaration creates a PV in AWS and attaches the pvc. Data is persistent. But unable to get past the containercreating issue.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
activemq-deployment-58cc677d85-497xt 1/1 Running 0 2m
activemq-deployment-58cc677d85-b4tpx 0/1 ContainerCreating 0 1m
activemq-deployment-58cc677d85-hprpv 1/1 Running 0 1m
activemq-deployment-58cc677d85-vdtcs 0/1 ContainerCreating 0 1m
Describe gives this:
$ kubectl describe deployments activemq-deployment
Name: activemq-deployment
Namespace: default
CreationTimestamp: Wed, 27 Feb 2019 21:49:11 -0800
Labels: app=activemq
Annotations: deployment.kubernetes.io/revision: 1
Selector: app=activemq
Replicas: 4 desired | 4 updated | 4 total | 2 available | 2 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=activemq
Containers:
activemq:
Image: activemq:1.0
Port: 8161/TCP
Host Port: 0/TCP
Environment: <none>
Mounts:
/opt/apache-activemq-5.15.6/data from activemq-data (rw)
Volumes:
activemq-data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: amq-pv-claim
ReadOnly: false
Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetAvailable
Available False MinimumReplicasUnavailable
OldReplicaSets: <none>
NewReplicaSet: activemq-deployment-58cc677d85 (4/4 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 3m8s deployment-controller Scaled up replica set activemq-deployment-58cc677d85 to 1
Normal ScalingReplicaSet 103s deployment-controller Scaled up replica set activemq-deployment-58cc677d85 to 4
Declaration:
apiVersion: apps/v1
kind: Deployment
metadata:
name: activemq-deployment
labels:
app: activemq
spec:
replicas: 1
selector:
matchLabels:
app: activemq
template:
metadata:
labels:
app: activemq
spec:
securityContext:
fsGroup: 2000
containers:
- name: activemq
image: activemq:1.0
ports:
- containerPort: 8161
volumeMounts:
- name: activemq-data
mountPath: /opt/apache-activemq-5.15.6/data
readOnly: false
volumes:
- name: activemq-data
persistentVolumeClaim:
claimName: amq-pv-claim
---
apiVersion: v1
kind: Service
metadata:
name: amq-nodeport-service
spec:
selector:
app: activemq
ports:
- port: 8161
targetPort: 8161
type: NodePort
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: amq-pv-claim
spec:
#storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 2Gi

Answer 1

Use StatefulSets for persisting the state of the container. Deployment is not recommended for running stateful containers