kubedns can't nslookup any service - kubernetes

version 1.6.1.I setup kubernetes with kubeadm.DNS cant resolve kubernetes service within heapster. But using ip of kubernetes is OK. Below is the container log of kubedns
[root#node-47 ~]# docker logs 624905719fa2
I0512 04:29:57.374905 1 dns.go:49] version: v1.5.2-beta.0+$Format:%h$
I0512 04:29:57.380148 1 server.go:70] Using configuration read from directory: /kube-dns-config%!(EXTRA time.Duration=10s)
I0512 04:29:57.380219 1 server.go:112] FLAG: --alsologtostderr="false"
I0512 04:29:57.380231 1 server.go:112] FLAG: --config-dir="/kube-dns-config"
I0512 04:29:57.380238 1 server.go:112] FLAG: --config-map=""
I0512 04:29:57.380242 1 server.go:112] FLAG: --config-map-namespace="kube-system"
I0512 04:29:57.380245 1 server.go:112] FLAG: --config-period="10s"
I0512 04:29:57.380268 1 server.go:112] FLAG: --dns-bind-address="0.0.0.0"
I0512 04:29:57.380272 1 server.go:112] FLAG: --dns-port="10053"
I0512 04:29:57.380278 1 server.go:112] FLAG: --domain="cluster.local."
I0512 04:29:57.380283 1 server.go:112] FLAG: --federations=""
I0512 04:29:57.380288 1 server.go:112] FLAG: --healthz-port="8081"
I0512 04:29:57.380292 1 server.go:112] FLAG: --initial-sync-timeout="1m0s"
I0512 04:29:57.380295 1 server.go:112] FLAG: --kube-master-url=""
I0512 04:29:57.380300 1 server.go:112] FLAG: --kubecfg-file=""
I0512 04:29:57.380303 1 server.go:112] FLAG: --log-backtrace-at=":0"
I0512 04:29:57.380309 1 server.go:112] FLAG: --log-dir=""
I0512 04:29:57.380315 1 server.go:112] FLAG: --log-flush-frequency="5s"
I0512 04:29:57.380318 1 server.go:112] FLAG: --logtostderr="true"
I0512 04:29:57.380323 1 server.go:112] FLAG: --nameservers=""
I0512 04:29:57.380326 1 server.go:112] FLAG: --stderrthreshold="2"
I0512 04:29:57.380330 1 server.go:112] FLAG: --v="2"
I0512 04:29:57.380334 1 server.go:112] FLAG: --version="false"
I0512 04:29:57.380340 1 server.go:112] FLAG: --vmodule=""
I0512 04:29:57.380386 1 server.go:175] Starting SkyDNS server (0.0.0.0:10053)
I0512 04:29:57.380816 1 server.go:197] Skydns metrics enabled (/metrics:10055)
I0512 04:29:57.380828 1 dns.go:147] Starting endpointsController
I0512 04:29:57.380832 1 dns.go:150] Starting serviceController
I0512 04:29:57.381406 1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0]
I0512 04:29:57.381417 1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0]
I0512 04:29:57.439494 1 dns.go:264] New service: monitoring-influxdb
I0512 04:29:57.439579 1 dns.go:462] Added SRV record &{Host:monitoring-influxdb.kube-system.svc.cluster.local. Port:8086 Priority:10 Weight:10 Text: Mail:false Ttl:30 TargetStrip:0 Group: Key:}
I0512 04:29:57.439601 1 dns.go:462] Added SRV record &{Host:monitoring-influxdb.kube-system.svc.cluster.local. Port:8083 Priority:10 Weight:10 Text: Mail:false Ttl:30 TargetStrip:0 Group: Key:}
I0512 04:29:57.439623 1 dns.go:264] New service: dai-1-daiym-context-46
I0512 04:29:57.439642 1 dns.go:264] New service: kubernetes
I0512 04:29:57.439722 1 dns.go:462] Added SRV record &{Host:kubernetes.default.svc.cluster.local. Port:443 Priority:10 Weight:10 Text: Mail:false Ttl:30 TargetStrip:0 Group: Key:}
I0512 04:29:57.439746 1 dns.go:264] New service: nginxservice
I0512 04:29:57.439766 1 dns.go:264] New service: heapster
I0512 04:29:57.439784 1 dns.go:264] New service: kube-dns
I0512 04:29:57.439803 1 dns.go:462] Added SRV record &{Host:kube-dns.kube-system.svc.cluster.local. Port:53 Priority:10 Weight:10 Text: Mail:false Ttl:30 TargetStrip:0 Group: Key:}
I0512 04:29:57.439818 1 dns.go:462] Added SRV record &{Host:kube-dns.kube-system.svc.cluster.local. Port:53 Priority:10 Weight:10 Text: Mail:false Ttl:30 TargetStrip:0 Group: Key:}
I0512 04:29:57.439834 1 dns.go:264] New service: kubernetes-dashboard
I0512 04:29:57.439851 1 dns.go:264] New service: monitoring-grafana
I0512 04:29:57.882534 1 dns.go:171] Initialized services and endpoints from apiserver
I0512 04:29:57.882995 1 server.go:128] Setting up Healthz Handler (/readiness)
I0512 04:29:57.883004 1 server.go:133] Setting up cache handler (/cache)
I0512 04:29:57.883009 1 server.go:119] Status HTTP port 8081
But I can't resolve any service within kubedns.
[root#node-47 ~]# docker exec -it 624905719fa2 sh
/ # nslookup nginxservice localhost
Server: 127.0.0.1
Address 1: 127.0.0.1 localhost
nslookup: can't resolve 'nginxservice': Try again
/ # nslookup heapster localhost
Server: 127.0.0.1
Address 1: 127.0.0.1 localhost
nslookup: can't resolve 'heapster': Try again

Please use full DNS name of service try again if service do not have the same namespace with DNS? Check https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/ to get full dns name.

Related

Grafana Timeout while querying large amount of logs from Loki

I have a Loki server running on AWS Graviton (arm, 4 vCPU, 8 GiB) configured as following:
common:
replication_factor: 1
ring:
kvstore:
store: etcd
etcd:
endpoints: ['127.0.0.1:2379']
storage_config:
boltdb_shipper:
active_index_directory: /opt/loki/index
cache_location: /opt/loki/index_cache
shared_store: s3
aws:
s3: s3://ap-south-1/bucket-name
limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 168h # 7d
ingestion_rate_mb: 10
ingestion_burst_size_mb: 20
per_stream_rate_limit: 8MB
ingester:
lifecycler:
join_after: 30s
chunk_block_size: 10485760
compactor:
working_directory: /opt/loki/compactor
shared_store: s3
compaction_interval: 5m
schema_config:
configs:
- from: 2022-01-01
store: boltdb-shipper
object_store: s3
schema: v11
index:
prefix: loki_
period: 24h
table_manager:
retention_period: 360h #15d
retention_deletes_enabled: true
index_tables_provisioning: # unused
provisioned_write_throughput: 500
provisioned_read_throughput: 100
inactive_write_throughput: 1
inactive_read_throughput: 100
Ingestion is working fine and I'm able to query logs for long durations from streams with less data sizes. I'm also able to query small durations of logs for streams with TiBs of data.
I see the following error in Loki when I try to query 24h of data from a large data stream and Grafana timeout after 5 mins:
Feb 11 08:27:32 loki-01 loki[19490]: level=error ts=2022-02-11T08:27:32.186137309Z caller=retry.go:73 org_id=fake msg="error processing request" try=2 err="context canceled"
Feb 11 08:27:32 loki-01 loki[19490]: level=info ts=2022-02-11T08:27:32.186304708Z caller=metrics.go:92 org_id=fake latency=fast query="{filename=\"/var/log/server.log\",host=\"web-199\",ip=\"192.168.20.239\",name=\"web\"} |= \"attachDriver\"" query_type=filter range_type=range length=24h0m0s step=1m0s duration=0s status=499 limit=1000 returned_lines=0 throughput=0B total_bytes=0B
Feb 11 08:27:32 loki-01 loki[19490]: level=info ts=2022-02-11T08:27:32.23882892Z caller=metrics.go:92 org_id=fake latency=slow query="{filename=\"/var/log/server.log\",host=\"web-199\",ip=\"192.168.20.239\",name=\"web\"} |= \"attachDriver\"" query_type=filter range_type=range length=24h0m0s step=1m0s duration=59.813829694s status=400 limit=1000 returned_lines=153 throughput=326MB total_bytes=20GB
Feb 11 08:27:32 loki-01 loki[19490]: level=error ts=2022-02-11T08:27:32.238959314Z caller=scheduler_processor.go:199 org_id=fake msg="error notifying frontend about finished query" err="rpc error: code = Canceled desc = context canceled" frontend=192.168.5.138:9095
Feb 11 08:27:32 loki-01 loki[19490]: level=error ts=2022-02-11T08:27:32.23898877Z caller=scheduler_processor.go:154 org_id=fake msg="error notifying scheduler about finished query" err=EOF addr=192.168.5.138:9095
Query: {filename="/var/log/server.log",host="web-199",ip="192.168.20.239",name="web"} |= "attachDriver"
Is there a way to stream the results instead of waiting for the response? Can I optimize Loki to process such queries better?

minikube service is failing to expose URL

F:\Udemy\GitRepo\Kubernetes-Tutorial>kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
my-app-deploy-68698d9757-wrs9z 1/1 Running 0 14m 172.17.0.3 minikube <none> <none>
F:\Udemy\GitRepo\Kubernetes-Tutorial>minikube service my-app-svc
|-----------|------------|-------------|-----------------------------|
| NAMESPACE | NAME | TARGET PORT | URL |
|-----------|------------|-------------|-----------------------------|
| default | my-app-svc | 80 | http://172.30.105.146:30365 |
|-----------|------------|-------------|-----------------------------|
* Opening service default/my-app-svc in default browser...
F:\Udemy\GitRepo\Kubernetes-Tutorial>kubectl describe service my-app-svc
Name: my-app-svc
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=my-app
Type: NodePort
IP: 10.98.9.115
Port: <unset> 80/TCP
TargetPort: 9001/TCP
NodePort: <unset> 30365/TCP
Endpoints: 172.17.0.3:9001
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
F:\Udemy\GitRepo\Kubernetes-Tutorial>kubectl logs my-app-deploy-68698d9757-wrs9z
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.1.RELEASE)
2021-08-21 13:37:21.046 INFO 1 --- [ main] c.d.d.DockerpublishApplication : Starting DockerpublishApplication v0.0.3 on my-app-deploy-68698d9757-wrs9z with PID 1 (/app.jar started by root in /)
2021-08-21 13:37:21.050 INFO 1 --- [ main] c.d.d.DockerpublishApplication : No active profile set, falling back to default profiles: default
2021-08-21 13:37:22.645 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 9091 (http)
2021-08-21 13:37:22.659 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2021-08-21 13:37:22.660 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.36]
2021-08-21 13:37:22.785 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2021-08-21 13:37:22.785 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1646 ms
2021-08-21 13:37:23.302 INFO 1 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
2021-08-21 13:37:23.496 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 9091 (http) with context path ''
2021-08-21 13:37:23.510 INFO 1 --- [ main] c.d.d.DockerpublishApplication : Started DockerpublishApplication in 3.279 seconds (JVM running for 4.077)
F:\Udemy\GitRepo\Kubernetes-Tutorial>
Everything seems to be good, but not working well.
Refused to connect issue comes as below for
minikube service my-app-svc
Your service or application is running on different port as you are getting connection refused.
Spring boot running on the 9091 : Tomcat started on port(s): 9091 (http) with context path ''
But your service is redirecting the traffic TargetPort: 9001/TCP
Your target port should be 9091 instead of 9001
Your will access the application over the node port Ip request, which will reach to the K8s service and be forwarded to the TargetPort: 9091/TCP on which the application is running.

fluent-bit parser not working. Field is not being parsed/filtered

Need help. I send logs from fluent-bit to grafana/loki but fluent-bit cannot parse logs properly. I use Helm charts
fluent-bit.conf: |-
[SERVICE]
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_PORT 2020
Flush 1
Daemon Off
Log_Level warn
Parsers_File parsers.conf
[INPUT]
Name tail
Tag kube.*
Path /var/log/containers/*.log
Parser cri
DB /run/fluent-bit/flb_kube.db
Mem_Buf_Limit 5MB
[FILTER]
Name kubernetes
Match kube.*
Kube_URL https://kubernetes.default.svc:443
Merge_Log On
K8S-Logging.Exclude On
K8S-Logging.Parser On
[Output]
Name grafana-loki
Match *
Url http://1.2.3.4:13100/loki/api/v1/push
TenantID ""
BatchWait 1
BatchSize 1048576
Labels {job="fluent-bit",env="dev"}
RemoveKeys kubernetes,stream
AutoKubernetesLabels false
LabelMapPath /fluent-bit/etc/labelmap.json
LineFormat json
LogLevel warn
parsers.conf: |-
[PARSER]
Name cri
Format regex
Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<log>.*)$
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L%z
[PARSER]
Format regex
Name nginx
Regex ^(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")
Time_Format %Y-%m-%d %H:%M:%S.%L
Time_Keep true
Time_Key apptime
log example
10.220.18.16 - - [02/Aug/2021:06:38:38 +0100] "GET /health HTTP/1.1" 200 4991 "-" "kube-probe/1.20" "-"
maybe smth wrong with loki configuration?

Heapster not fetching data from nodes

I have deployed kubernetes 1.2.4 with heapster, grafana and influxdb on ubuntu cluster.
kubectl cluster-info
Kubernetes master is running at http://192.168.56.10:8080
Heapster is running at http://192.168.56.10:8080/api/v1/proxy/namespaces/kube-system/services/heapster
KubeDNS is running at http://192.168.56.10:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at http://192.168.56.10:8080/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
Grafana is running at http://192.168.56.10:8080/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana
InfluxDB is running at http://192.168.56.10:8080/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb
grafna and heapster RestAPI both are listing node level details.
but when I am trying to get namespace level detail its not giving anything its giving empty value.
output of kubectl get pods --all-namespaces is:
NAMESPACE NAME READY STATUS RESTARTS AGE
default hazelcast-dymfs 1/1 Running 0 42m
kube-system heapster-v1.0.2-1238126759-jof6m 4/4 Running 0 1h
kube-system kube-dns-v11-simv4 4/4 Running 0 1d
kube-system kubernetes-dashboard-v1.0.1-abut5 1/1 Running 0 1d
kube-system monitoring-influxdb-grafana-v3-38exi 2/2 Running 0 1h
Below are the heapster container log
["kubectl logs heapster-v1.0.2-1238126759-jof6m --namespace=kube-system heapster"]<br>
`I0619 18:23:05.048147 1 manager.go:152] ScrapeMetrics: time: 47.671952ms size: 9
I0619 18:24:05.000197 1 manager.go:79] Scraping metrics start: 2016-06-19 18:23:00 +0000 UTC, end: 2016-06-19 18:24:00 +0000 UTC
I0619 18:24:05.032203 1 manager.go:152] ScrapeMetrics: time: 31.662441ms size: 9
I0619 18:25:05.022755 1 manager.go:79] Scraping metrics start: 2016-06-19 18:24:00 +0000 UTC, end: 2016-06-19 18:25:00 +0000 UTC
I0619 18:25:05.047227 1 manager.go:152] ScrapeMetrics: time: 24.379296ms size: 9
I0619 18:26:05.000176 1 manager.go:79] Scraping metrics start: 2016-06-19 18:25:00 +0000 UTC, end: 2016-06-19 18:26:00 +0000 UTC
I0619 18:26:05.030686 1 manager.go:152] ScrapeMetrics: time: 30.431612ms size: 9
I0619 18:27:05.000275 1 manager.go:79] Scraping metrics start: 2016-06-19 18:26:00 +0000 UTC, end: 2016-06-19 18:27:00 +0000 UTC
I0619 18:27:05.024293 1 manager.go:152] ScrapeMetrics: time: 24.012249ms size: 9
I0619 18:28:05.000178 1 manager.go:79] Scraping metrics start: 2016-06-19 18:27:00 +0000 UTC, end: 2016-06-19 18:28:00 +0000 UTC
I0619 18:28:05.020069 1 manager.go:152] ScrapeMetrics: time: 19.813727ms size: 9
I0619 18:29:05.000163 1 manager.go:79] Scraping metrics start: 2016-06-19 18:28:00 +0000 UTC, end: 2016-06-19 18:29:00 +0000 UTC
I0619 18:29:05.019929 1 manager.go:152] ScrapeMetrics: time: 19.689754ms size: 9
I0619 18:30:05.000152 1 manager.go:79] Scraping metrics start: 2016-06-19 18:29:00 +0000 UTC, end: 2016-06-19 18:30:00 +0000 UTC
I0619 18:30:05.122992 1 manager.go:152] ScrapeMetrics: time: 122.775222ms size: 9
I0619 18:31:05.000177 1 manager.go:79] Scraping metrics start: 2016-06-19 18:30:00 +0000 UTC, end: 2016-06-19 18:31:00 +0000 UTC
I0619 18:31:05.032408 1 manager.go:152] ScrapeMetrics: time: 32.15377ms size: 9
I0619 18:32:05.000176 1 manager.go:79] Scraping metrics start: 2016-06-19 18:31:00 +0000 UTC, end: 2016-06-19 18:32:00 +0000 UTC
I0619 18:32:05.026673 1 manager.go:152] ScrapeMetrics: time: 26.424226ms size: 9
I0619 18:33:05.000128 1 manager.go:79] Scraping metrics start: 2016-06-19 18:32:00 +0000 UTC, end: 2016-06-19 18:33:00 +0000 UTC
I0619 18:33:05.021314 1 manager.go:152] ScrapeMetrics: time: 21.115352ms size: 9
I0619 18:34:05.000159 1 manager.go:79] Scraping metrics start: 2016-06-19 18:33:00 +0000 UTC, end: 2016-06-19 18:34:00 +0000 UTC
I0619 18:34:05.028069 1 manager.go:152] ScrapeMetrics: time: 27.845914ms size: 9
I0619 18:35:05.000150 1 manager.go:79] Scraping metrics start: 2016-06-19 18:34:00 +0000 UTC, end: 2016-06-19 18:35:00 +0000 UTC
I0619 18:35:05.044087 1 manager.go:152] ScrapeMetrics: time: 43.861766ms size: 9
I0619 18:36:05.000183 1 manager.go:79] Scraping metrics start: 2016-06-19 18:35:00 +0000 UTC, end: 2016-06-19 18:36:00 +0000 UTC
I0619 18:36:05.021248 1 manager.go:152] ScrapeMetrics: time: 20.998928ms size: 9
I0619 18:37:05.000153 1 manager.go:79] Scraping metrics start: 2016-06-19 18:36:00 +0000 UTC, end: 2016-06-19 18:37:00 +0000 UTC
I0619 18:37:05.014427 1 manager.go:152] ScrapeMetrics: time: 14.204355ms size: 9
I0619 18:38:05.000182 1 manager.go:79] Scraping metrics start: 2016-06-19 18:37:00 +0000 UTC, end: 2016-06-19 18:38:00 +0000 UTC
I0619 18:38:05.029795 1 manager.go:152] ScrapeMetrics: time: 29.537191ms size: 9
I0619 18:39:05.000176 1 manager.go:79] Scraping metrics start: 2016-06-19 18:38:00 +0000 UTC, end: 2016-06-19 18:39:00 +0000 UTC
I0619 18:39:05.027009 1 manager.go:152] ScrapeMetrics: time: 26.75411ms size: 9
I0619 18:40:05.000167 1 manager.go:79] Scraping metrics start: 2016-06-19 18:39:00 +0000 UTC, end: 2016-06-19 18:40:00 +0000 UTC
I0619 18:40:05.033115 1 manager.go:152] ScrapeMetrics: time: 32.876629ms size: 9
I0619 18:41:05.000167 1 manager.go:79] Scraping metrics start: 2016-06-19 18:40:00 +0000 UTC, end: 2016-06-19 18:41:00 +0000 UTC
I0619 18:41:05.017157 1 manager.go:152] ScrapeMetrics: time: 16.926043ms size: 9
I0619 18:42:05.000156 1 manager.go:79] Scraping metrics start: 2016-06-19 18:41:00 +0000 UTC, end: 2016-06-19 18:42:00 +0000 UTC
I0619 18:42:05.028695 1 manager.go:152] ScrapeMetrics: time: 28.287187ms size: 9
I don't no where to look for to get more log and find out the error.
Any help will be greatly appreciated!
Edit-1
I have posted the output of "kubectl get pods --all-namespaces" above but when I checked on Kubedash, heapster is not listed there.

Cloud Foundry bosh Error 140003: unknown resource pool

I'm attempting to setup a service broker to add postgres to our Cloud Foundry installation. We're running our system on vmWare. I'm using this release in order to do that:
cf-contrib-release
I added the release in bosh:
#bosh releases
Acting as user 'director' on 'microbosh-ba846726bed7032f1fd4'
+-----------------------+----------------------+-------------+
| Name | Versions | Commit Hash |
+-----------------------+----------------------+-------------+
| cf | 208.12* | a0de569a+ |
| cf-autoscaling | 13* | 927bc7ed+ |
| cf-metrics | 34* | 22f7e1e1 |
| cf-mysql | 20* | caa23b3d+ |
| | 22* | af278086+ |
| cf-rabbitmq | 161* | 4d298aec |
| cf-riak-cs | 10* | 5e7e46c9+ |
| cf-services-contrib | 6* | 57fd2098+ |
| docker | 23* | 82346881+ |
| newrelic_broker | 1.3* | 1ce3471d+ |
| notifications-with-ui | 18* | 490b6446+ |
| postgresql-docker | 4* | a53c9333+ |
| push-console-release | console-du-jour-203* | d2d31585+ |
| spring-cloud-broker | 1.0.0* | efd69612 |
+-----------------------+----------------------+-------------+
(*) Currently deployed
(+) Uncommitted changes
Releases total: 13
I setup my resource pools and jobs in my yaml file according to this doumentation:
http://bosh.io/docs/vsphere-cpi.html#resource-pools
This is how our cluster looks:
vmware cluster
And here is what I put in the yaml file:
resource_pools:
- name: default
network: default
stemcell:
name: bosh-vsphere-esxi-ubuntu-trusty-go_agent
version: '2865.1'
cloud_properties:
cpu: 2
ram: 4096
disk: 10240
datacenters:
- name: 'Universal City'
clusters:
- USH_UCS_CLOUD_FOUNDRY_NONPROD_01: {resource_pool: 'USH_UCS_CLOUD_FOUNDRY_NONPROD_01_RP'}
jobs:
- name: gateways
release: cf-services-contrib
templates:
- name: postgresql_gateway_ng
instances: 1
resource_pool: 'USH_UCS_CLOUD_FOUNDRY_NONPROD_01_RP'
networks:
- name: default
default: [dns, gateway]
properties:
# Service credentials
uaa_client_id: "cf"
uaa_endpoint: http://uaa.devcloudwest.example.com
uaa_client_auth_credentials:
username: admin
password: secret
And I'm getting an error when I run 'bosh deploy' that says:
Error 140003: Job `gateways' references an unknown resource pool `USH_UCS_CLOUD_FOUNDRY_NONPROD_01_RP'
Here's my yaml file in it's entirety:
name: cf-22b9f4d62bb6f0563b71
director_uuid: fd713790-b1bc-401a-8ea1-b8209f1cc90c
releases:
- name: cf-services-contrib
version: 6
compilation:
workers: 3
network: default
reuse_compilation_vms: true
cloud_properties:
ram: 5120
disk: 10240
cpu: 2
update:
canaries: 1
canary_watch_time: 30000-60000
update_watch_time: 30000-60000
max_in_flight: 4
networks:
- name: default
type: manual
subnets:
- range: exam 10.114..130.0/24
gateway: exam 10.114..130.1
cloud_properties:
name: 'USH_UCS_CLOUD_FOUNDRY'
#resource_pools:
# - name: common
# network: default
# size: 8
# stemcell:
# name: bosh-vsphere-esxi-ubuntu-trusty-go_agent
# version: '2865.1'
resource_pools:
- name: default
network: default
stemcell:
name: bosh-vsphere-esxi-ubuntu-trusty-go_agent
version: '2865.1'
cloud_properties:
cpu: 2
ram: 4096
disk: 10240
datacenters:
- name: 'Universal City'
clusters:
- USH_UCS_CLOUD_FOUNDRY_NONPROD_01: {resource_pool: 'USH_UCS_CLOUD_FOUNDRY_NONPROD_01_RP'}
jobs:
- name: gateways
release: cf-services-contrib
templates:
- name: postgresql_gateway_ng
instances: 1
resource_pool: 'USH_UCS_CLOUD_FOUNDRY_NONPROD_01_RP'
networks:
- name: default
default: [dns, gateway]
properties:
# Service credentials
uaa_client_id: "cf"
uaa_endpoint: http://uaa.devcloudwest.example.com
uaa_client_auth_credentials:
username: admin
password: secret
- name: postgresql_service_node
release: cf-services-contrib
template: postgresql_node_ng
instances: 1
resource_pool: common
persistent_disk: 10000
properties:
postgresql_node:
plan: default
networks:
- name: default
default: [dns, gateway]
properties:
networks:
apps: default
management: default
cc:
srv_api_uri: http://api.devcloudwest.example.com
nats:
address: exam 10.114..130.11
port: 25555
user: nats #CHANGE
password: secret
authorization_timeout: 5
service_plans:
postgresql:
default:
description: "Developer, 250MB storage, 10 connections"
free: true
job_management:
high_water: 230
low_water: 20
configuration:
capacity: 125
max_clients: 10
quota_files: 4
quota_data_size: 240
enable_journaling: true
backup:
enable: false
lifecycle:
enable: false
serialization: enable
snapshot:
quota: 1
postgresql_gateway:
token: f75df200-4daf-45b5-b92a-cb7fa1a25660
default_plan: default
supported_versions: ["9.3"]
version_aliases:
current: "9.3"
cc_api_version: v2
postgresql_node:
supported_versions: ["9.3"]
default_version: "9.3"
max_tmp: 900
password: secret
And here's gist with the debug output from that error:
postgres_2423_debug.txt
The docs for the jobs blocks say:
resource_pool [String, required]: A valid resource pool name from the Resource Pools block. BOSH runs instances of this job in a VM from the named resource pool.
This needs to match the name of one of your resource_pools, namely default, not the name of the resource pool in vSphere.
The only sections that have direct references to the IaaS are things that say cloud_properties. Specific names of resources (like networks, clusters, or datacenters in your vSphere, or subnets, AZs, and instance types in AWS) only show up in places that say cloud_properties.
You use that data to define "networks" and "resource pools" at a higher level of abstraction that is IaaS-agnostic, e.g. except for cloud properties, the specifications you give for resource pools is the same whether you're deploying to vSphere, AWS, OpenStack, etc.
Then your jobs reference these networks, resource pools, etc. by the logical name you've given to the abstractions. In particular, jobs don't require any IaaS-specific configuration whatsoever, just references to a logical network(s) and a resource pool that you've defined elsewhere in your manifest.