Would you know if concourse is compatible with swarm mode?
I am able to spin up concourse web but not the worker when using swarm mode.
If docker-compose is used with the flag set to privileged: true, then it works.
Bug Report
Concourse version: fly-version is 3.2.1
fly -version
3.2.1
Deployment type (BOSH/Docker/binary):Docker
Infrastructure/IaaS:
Browser (if applicable):
Did this used to work?yes with docker-compose but not with in a docker swarm mode
ci_concourse-worker.1.sdwlru32wymn#node-ora | Exit trace for group:
ci_concourse-worker.1.sdwlru32wymn#node-ora | garden exited with error: Exit trace for group:
ci_concourse-worker.1.sdwlru32wymn#node-ora | garden-runc exited with error: bulk starter: setting up default chains: iptables: setup-global-chains: + set -o nounset
ci_concourse-worker.1.sdwlru32wymn#node-ora | + set -o errexit
ci_concourse-worker.1.sdwlru32wymn#node-ora | + shopt -s nullglob
ci_concourse-worker.1.sdwlru32wymn#node-ora | + filter_input_chain=w--input
ci_concourse-worker.1.sdwlru32wymn#node-ora | + filter_forward_chain=w--forward
ci_concourse-worker.1.sdwlru32wymn#node-ora | + filter_default_chain=w--default
ci_concourse-worker.1.sdwlru32wymn#node-ora | + filter_instance_prefix=w--instance-
ci_concourse-worker.1.sdwlru32wymn#node-ora | + nat_prerouting_chain=w--prerouting
ci_concourse-worker.1.sdwlru32wymn#node-ora | + nat_postrouting_chain=w--postrouting
ci_concourse-worker.1.sdwlru32wymn#node-ora | + nat_instance_prefix=w--instance-
ci_concourse-worker.1.sdwlru32wymn#node-ora | + iptables_bin=/worker-state/3.2.1/assets/iptables/sbin/iptables
ci_concourse-worker.1.sdwlru32wymn#node-ora | + case "${ACTION}" in
ci_concourse-worker.1.sdwlru32wymn#node-ora | + setup_filter
ci_concourse-worker.1.sdwlru32wymn#node-ora | + teardown_filter
ci_concourse-worker.1.sdwlru32wymn#node-ora | + teardown_deprecated_rules
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ /worker-state/3.2.1/assets/iptables/sbin/iptables -w -S INPUT
ci_concourse-worker.1.sdwlru32wymn#node-ora | + rules=
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | + echo ''
ci_concourse-worker.1.sdwlru32wymn#node-ora | + grep ' -j garden-dispatch'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + sed -e s/-A/-D/ -e 's/\s+$//'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + xargs --no-run-if-empty --max-lines=1 /worker-state/3.2.1/assets/iptables/sbin/iptables -w
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ /worker-state/3.2.1/assets/iptables/sbin/iptables -w -S FORWARD
ci_concourse-worker.1.sdwlru32wymn#node-ora | + rules=
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | + echo ''
ci_concourse-worker.1.sdwlru32wymn#node-ora | + grep ' -j garden-dispatch'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + sed -e s/-A/-D/ -e 's/\s+$//'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + xargs --no-run-if-empty --max-lines=1 /worker-state/3.2.1/assets/iptables/sbin/iptables -w
ci_concourse-worker.1.sdwlru32wymn#node-ora | + /worker-state/3.2.1/assets/iptables/sbin/iptables -w -F garden-dispatch
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | + /worker-state/3.2.1/assets/iptables/sbin/iptables -w -X garden-dispatch
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ /worker-state/3.2.1/assets/iptables/sbin/iptables -w -S w--forward
ci_concourse-worker.1.sdwlru32wymn#node-ora | + rules=
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | + echo ''
ci_concourse-worker.1.sdwlru32wymn#node-ora | + grep '-g w--instance-'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + sed -e s/-A/-D/ -e 's/\s+$//'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + xargs --no-run-if-empty --max-lines=1 /worker-state/3.2.1/assets/iptables/sbin/iptables -w
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ /worker-state/3.2.1/assets/iptables/sbin/iptables -w -S
ci_concourse-worker.1.sdwlru32wymn#node-ora | + rules=
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | + echo ''
ci_concourse-worker.1.sdwlru32wymn#node-ora | + grep '^-A w--instance-'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + sed -e s/-A/-D/ -e 's/\s+$//'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + xargs --no-run-if-empty --max-lines=1 /worker-state/3.2.1/assets/iptables/sbin/iptables -w
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ /worker-state/3.2.1/assets/iptables/sbin/iptables -w -S
ci_concourse-worker.1.sdwlru32wymn#node-ora | + rules=
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | + echo ''
ci_concourse-worker.1.sdwlru32wymn#node-ora | + grep '^-N w--instance-'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + sed -e s/-N/-X/ -e 's/\s+$//'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + xargs --no-run-if-empty --max-lines=1 /worker-state/3.2.1/assets/iptables/sbin/iptables -w
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ /worker-state/3.2.1/assets/iptables/sbin/iptables -w -S FORWARD
ci_concourse-worker.1.sdwlru32wymn#node-ora | + rules=
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | + echo ''
ci_concourse-worker.1.sdwlru32wymn#node-ora | + grep ' -j w--forward'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + sed -e s/-A/-D/ -e 's/\s+$//'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + xargs --no-run-if-empty --max-lines=1 /worker-state/3.2.1/assets/iptables/sbin/iptables -w
ci_concourse-worker.1.sdwlru32wymn#node-ora | + /worker-state/3.2.1/assets/iptables/sbin/iptables -w -F w--forward
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | + /worker-state/3.2.1/assets/iptables/sbin/iptables -w -F w--default
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ /worker-state/3.2.1/assets/iptables/sbin/iptables -w -S INPUT
ci_concourse-worker.1.sdwlru32wymn#node-ora | + rules=
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | + echo ''
ci_concourse-worker.1.sdwlru32wymn#node-ora | + grep ' -j w--input'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + sed -e s/-A/-D/ -e 's/\s+$//'
ci_concourse-worker.1.sdwlru32wymn#node-ora | + xargs --no-run-if-empty --max-lines=1 /worker-state/3.2.1/assets/iptables/sbin/iptables -w
ci_concourse-worker.1.sdwlru32wymn#node-ora | + /worker-state/3.2.1/assets/iptables/sbin/iptables -w -F w--input
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | + /worker-state/3.2.1/assets/iptables/sbin/iptables -w -X w--input
ci_concourse-worker.1.sdwlru32wymn#node-ora | + true
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ ip route show
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ grep default
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ cut '-d ' -f5
ci_concourse-worker.1.sdwlru32wymn#node-ora | ++ head -1
ci_concourse-worker.1.sdwlru32wymn#node-ora | + default_interface=eth1
ci_concourse-worker.1.sdwlru32wymn#node-ora | + /worker-state/3.2.1/assets/iptables/sbin/iptables -w -N w--input
ci_concourse-worker.1.sdwlru32wymn#node-ora | + /worker-state/3.2.1/assets/iptables/sbin/iptables -w -F w--input
ci_concourse-worker.1.sdwlru32wymn#node-ora | iptables v1.4.21: can't initialize iptables table `filter': Permission denied (you must be root)
ci_concourse-worker.1.sdwlru32wymn#node-ora | Perhaps iptables or your kernel needs to be upgraded.
ci_concourse-worker.1.sdwlru32wymn#node-ora |
ci_concourse-worker.1.sdwlru32wymn#node-ora | dns-proxy exited with nil
ci_concourse-worker.1.sdwlru32wymn#node-ora |
ci_concourse-worker.1.sdwlru32wymn#node-ora | baggageclaim exited with nil
ci_concourse-worker.1.sdwlru32wymn#node-ora | beacon exited with error: failed to dial: failed to connect to TSA: dial tcp: lookup concourse-web on 127.0.0.11:53: no such host
ci_concourse-worker.1.sdwlru32wymn#node-ora |
Short answer at the moment of writing: No
Privileged mode is not available for services deployed in a swarm, but a feature that will allow you to get similar permissions is in the works.
https://github.com/moby/moby/issues/32801
Related
print <<EOTEXT;
(`-') (`-') _<-. (`-')_ <-. (`-')
_(OO ) ( OO).-/ \( OO) ) .-> \(OO )_
,--.(_/,-.\(,------.,--./ ,--/ (`-')----. ,--./ ,-.)
\ \ / (_/ | .---'| \ | | ( OO).-. '| `.' |
\ / / (| '--. | . '| |)( _) | | || |'.'| |
_ \ /_) | .--' | |\ | \| |)| || | | |
\-'\ / | `---.| | \ | ' '-' '| | | |
`-' `------'`--' `--' `-----' `--' `--'
EOTEXT
This is my ascii art that id like to show up in console. How ever it seems that " \ " doesnt show up. Is there a way that i can make it appear.
In double-quoted string literals, \ is the start of an escape sequence. When followed by a non-word character, it causes that character to be produced. For example, \| and \␠ produce | and a space respectively. And of course, \\ produces \, so we can use \\ where we want \ in double-quote string literals.
Here docs (<< string literals) act as double-quoted string literals, unless the token that follows the << is single-quoted. Then the string produced matches the input exactly.
So we have the option of prepending \ to every special character (\, $ and #), or we can simply single-quote the token.
print <<'EOTEXT';
(`-') (`-') _<-. (`-')_ <-. (`-')
_(OO ) ( OO).-/ \( OO) ) .-> \(OO )_
,--.(_/,-.\(,------.,--./ ,--/ (`-')----. ,--./ ,-.)
\ \ / (_/ | .---'| \ | | ( OO).-. '| `.' |
\ / / (| '--. | . '| |)( _) | | || |'.'| |
_ \ /_) | .--' | |\ | \| |)| || | | |
\-'\ / | `---.| | \ | ' '-' '| | | |
`-' `------'`--' `--' `-----' `--' `--'
EOTEXT
We have the following line of text:
|  |  |  |
As you can see, the line of text simply consists of three similar phrases, which can be matched and changed (separately) using the following sed expression:
sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
If we had just one phrase (instead of the given three), the result would be the following:
$ echo '|  |' | sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
|  |
But when we have two or thee phrases, the result always points to the last matched phrase:
Here's an example with two matches:
$ echo '|  |  |' | sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
|  |
And here's an example with three matches:
$ echo '|  |  |  |' | sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
|  |
Why does this happening?
Is there a way to force sed to print the result only for the very first match?
The expected behavior? I though the following command would print something similar to this (just the first match):
$ echo '|  |  |  |' | sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
|  |
or this (all matches):
$ echo '|  |  |  |' | sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
|  |  |  |
What's happening is that | !\[.*\] matches the longest possible match. That is, the first phrase, up to the beginning of the last phrase. If you want to match only the first phrase you must be more specific. For instance with:
sed 's#| !\[\]\(([^.]*\.\([^.]*\)\.[^)]*)\) |.*#| ![\2]\1 |#'
I do not fully understand the question, but, you can try this sed
$ sed 's#\([^[]*.\)\([^\.]*.\([^\.]*\)[^)]*.\)#\1\3\2#' input_file
This will print all 3 matches but will only substitute into the first match
$ sed 's#\([^[]*.\)\([^\.]*.\([^\.]*\)[^)]*.\)#\1\3\2#' input_file
|  |  |  |
To target all 3, the g flag can be added
sed 's#\([^[]*.\)\([^\.]*.\([^\.]*\)[^)]*.\)#\1\3\2#g' input_file
|  |  |  |
You could also target just #2 for example
$ sed 's#\([^[]*.\)\([^\.]*.\([^\.]*\)[^)]*.\)#\1\3\2#2' input_file
|  |  |  |
I have deployed a docker container using the TimescaleDB extension to Postgres as a persistent store for metrics collected by Prometheus. This includes deploying the Prometheus Postgres adapter in a separate container. And then I deployed grafana in a third container.
Inside the TimescaleDB container I can execute the query:
SELECT time_bucket('5 minutes', time) AS five_min_bucket, name, avg(value)
FROM metrics
WHERE (name='node_load5' OR name='node_memory_Active_bytes') AND
time > NOW() - interval '1 day'
GROUP BY five_min_bucket,name
ORDER BY five_min_bucket;
and it returns these rows from the metrics table:
five_min_bucket | name | avg
------------------------+--------------------------+-------------------
2019-07-13 16:15:00+00 | node_load5 | 1.84263157894737
2019-07-13 16:15:00+00 | node_memory_Active_bytes | 1055776552.42105
2019-07-13 16:20:00+00 | node_load5 | 1.62533333333333
2019-07-13 16:20:00+00 | node_memory_Active_bytes | 1070225817.6
2019-07-13 16:25:00+00 | node_load5 | 1.06166666666667
2019-07-13 16:25:00+00 | node_memory_Active_bytes | 1084650018.13333
2019-07-13 16:30:00+00 | node_load5 | 0.593333333333333
2019-07-13 16:30:00+00 | node_memory_Active_bytes | 1093145668.26667
2019-07-13 16:35:00+00 | node_load5 | 0.322333333333333
2019-07-13 16:35:00+00 | node_memory_Active_bytes | 1108816145.06667
2019-07-13 16:40:00+00 | node_load5 | 0.180666666666667
2019-07-13 16:40:00+00 | node_memory_Active_bytes | 1125213388.8
2019-07-13 16:45:00+00 | node_load5 | 0.126
2019-07-13 16:45:00+00 | node_memory_Active_bytes | 1134488098.13333
2019-07-13 16:50:00+00 | node_load5 | 0.133333333333333
2019-07-13 16:50:00+00 | node_memory_Active_bytes | 1162440704
(16 rows)
The grafana container is run as:
docker run \
-d \
-p 3000:3000 \
--name=grafana \
-v grafana-storage:/var/lib/grafana \
-e "GF_INSTALL_PLUGINS=grafana-kubernetes-app" \
grafana/grafana
When I launch a browser to http://104.XX.XX.XX:3000 I can log into the web-ui and define the Datasource to postgres, but I received the error 'Metric request error'.
I looked for the grafana logs in its container and the path /var/log/grafana is empty. This is the log location indicated when I do a docker inspect command.
Using Chrome, I am able to inspect the Console and it reports the server responded with an internal error of 500.
On the host, I can confirm the docker-proxy is listening on port 5432.
sudo netstat -antp | grep "5432"
tcp 0 0 172.17.0.1:49486 172.17.0.2:5432 ESTABLISHED 60631/docker-proxy
tcp6 0 0 :::5432 :::* LISTEN 60631/docker-proxy
tcp6 0 0 10.0.0.4:5432 172.18.0.2:38568 ESTABLISHED 60631/docker-proxy
How do I get grafana to fetch metrics stored in the postgres database?
It appears that IoT-Agent isn't receiving measures, Agent's log says device group not found [MONGO-ALARM]. Does anyone understand what this error means or how to fix it?
Agent's log:
time=2018-12-10T16:48:24.824Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.COAPRouter | msg=Handling request with method [POST] on url [/rd/1] with messageId [24748]
time=2018-12-10T16:48:24.826Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.UpdateRegistration | msg=Handling update registration request
time=2018-12-10T16:48:24.826Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.COAPUtils | msg=Extracting query parameters from request
time=2018-12-10T16:48:24.827Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.UpdateRegistration | msg=Updating device register with lifetime [undefined] and address [193.136.33.222].
{"op":"IOTAgent.LWM2MHandlers","time":"2018-12-10T16:48:24.827Z","lvl":"DEBUG","msg":"Handling update registration of the device"}
time=2018-12-10T16:48:24.829Z | lvl=DEBUG | corr=54b6621d-65fd-43db-ac9b-fade34e4d947 | trans=54b6621d-65fd-43db-ac9b-fade34e4d947 | op=IoTAgentNGSI.MongoDBGroupRegister | srv=n/a | subsrv=n/a | msg=Looking for group params ["resource","apikey"] with queryObj {} | comp=IoTAgent
time=2018-12-10T16:48:24.835Z | lvl=DEBUG | corr=54b6621d-65fd-43db-ac9b-fade34e4d947 | trans=54b6621d-65fd-43db-ac9b-fade34e4d947 | op=IoTAgentNGSI.MongoDBGroupRegister | srv=n/a | subsrv=n/a | msg=Device group for fields [["resource","apikey"]] not found: [{}] | comp=IoTAgent
**time=2018-12-10T16:48:24.836Z | lvl=ERROR | corr=54b6621d-65fd-43db-ac9b-fade34e4d947 | trans=54b6621d-65fd-43db-ac9b-fade34e4d947 | op=IoTAgentNGSI.Alarms | srv=n/a | subsrv=n/a | msg=Raising [MONGO-ALARM]: {"name":"DEVICE_GROUP_NOT_FOUND","message":"Couldn\t find device group","code":404} | comp=IoTAgent**
time=2018-12-10T16:48:24.836Z | lvl=DEBUG | corr=54b6621d-65fd-43db-ac9b-fade34e4d947 | trans=54b6621d-65fd-43db-ac9b-fade34e4d947 | op=IoTAgentNGSI.MongoDBDeviceRegister | srv=n/a | subsrv=n/a | msg=Looking for device with id [raspiSensorTV]. | comp=IoTAgent
time=2018-12-10T16:48:24.843Z | lvl=ERROR | corr=54b6621d-65fd-43db-ac9b-fade34e4d947 | trans=54b6621d-65fd-43db-ac9b-fade34e4d947 | op=IoTAgentNGSI.Alarms | srv=n/a | subsrv=n/a | msg=Releasing [MONGO-ALARM] | comp=IoTAgent
{"op":"IOTAgent.LWM2MHandlers","time":"2018-12-10T16:48:24.843Z","lvl":"DEBUG","msg":"Preregistered device found."}
time=2018-12-10T16:48:24.844Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.UpdateRegistration | msg=Update registration request ended successfully
{"time":"2018-12-10T16:48:24.894Z","lvl":"DEBUG","msg":"Observers created successfully."}
The log above appears everytime the device signals to have sent a measure, but my query for provisioned devices returns empty list:
curl -X GET \
'http://localhost:4041/iot/devices' \
-H 'fiware-service: smartGondor' \
-H 'fiware-servicepath: /gardens'
{"count":0,"devices":[]}
Any idea please?
Your request is showing that no devices have been successfully provisioned yet.
It is likely that you have either not provisioned a service group for your IoT Agent or not provisioned the device itself. If a measurement is received before a service group has been provisioned then the IoT Agent will not know how devices identify themselves or where to persist the measurements.
You can provision a service group in multiple ways, but one method would be as follows:
curl -iX POST \
'http://iot-agent:4041/iot/services' \
-H 'Content-Type: application/json' \
-H 'fiware-service: <xxxxxxx>' \
-H 'fiware-servicepath: <yyyyy>' \
-d '{
"services": [
{
"apikey": "<api-key>",
"cbroker": "http://orion:1026",
"entity_type": "Thing",
...etc
}
]
}'
Thereafter any unknown device responding with the correct <api-key> will create an entity called Thing.
Of course if you want to be more specific you can provision the device as shown:
curl -iX POST \
'http://iot-agent:4041/iot/devices' \
-H 'Content-Type: application/json' \
-H 'fiware-service: <xxxxxxx>' \
-H 'fiware-servicepath: <yyyyy>' \
-d '{
"devices": [
{
"device_id": "motion001",
"entity_name": "urn:ngsd-ld:Motion:001",
"entity_type": "Motion",
"attributes": [
{ "object_id": "c", "name": "count", "type": "Integer" }
],
}
]
}
'
This ensures that a device identifying as motion001 will be stored as a Motion entity and various mappings will take place.
What is the difference between these two commands?
$ cat volcanoes.txt | wc > islands.txt
results:
$ cat islands.txt
17 26 204
second command:
$ cat volcanoes.txt | wc | cat > islands.txt
results:
$ cat islands.txt
17 26 204
We can see two results are same.
I am confused about the second commands.Why it has another cat in the commands.
Thank you for the help:)
The cat command just copies its input to its output verbatim.
So even
cat volcanoes.txt | cat | cat | wc | cat | cat | cat | cat > islands.txt
would lead to the same content of islands.txt.
Even without cat you can get the same result:
wc < volcanoes.txt > islands.txt
This one being the most efficient version, starting only one process.