I am using :
docker version: 1.11.1, build 5604cbe . I have made entries in /etc/default/docker as follows to configure REST API of docker for jenkins user group :
# Use DOCKER_OPTS to modify the daemon startup options.
#DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4"
DOCKER_OPTS="G- jenkins -H unix://var/run/docker.sock -H tcp://0.0.0.0:9090"
export DOCKER_HOST="tcp://0.0.0.0:9090"
ps- I have also tried with 127.0.0.1
Then I did sudo service docker restart
command $ ps aux|grep docker returned:
root 12385 0.0 0.2 421840 36016 ? Ssl 19:21 0:00 /usr/bin/docker daemon -H fd://
root 12391 0.0 0.0 294652 12188 ? Ssl 19:21 0:00 docker-containerd -l /var/run/docker/libcontainerd/docker-containerd.sock --runtime docker-runc
root 12654 0.0 0.0 21296 1028 pts/1 S+ 19:28 0:00 grep --color=auto docker
Seems like The REST API is not getting bind to port:9090.
Then I am using Jenkins Docker build step plugin to connect with docker REST API. It returns following :
Building in workspace /var/lib/jenkins/jobs/Telco_automated_build/workspace
[Docker] INFO: Pulling image registry.hub.docker.com/pratyush/product:latest
ERROR: Build step failed with exception
javax.ws.rs.ProcessingException: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:9090 [/127.0.0.1] failed: Connection refused
at org.glassfish.jersey.apache.connector.ApacheConnector.apply(ApacheConnector.java:513)
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:246)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:667)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:664)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:424)
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:664)
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:424)
at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:333)
at com.github.dockerjava.jaxrs.PullImageCmdExec.execute(PullImageCmdExec.java:37)
at com.github.dockerjava.jaxrs.PullImageCmdExec.execute(PullImageCmdExec.java:17)
at com.github.dockerjava.jaxrs.AbstrDockerCmdExec.exec(AbstrDockerCmdExec.java:57)
at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:29)
at com.github.dockerjava.core.command.PullImageCmdImpl.exec(PullImageCmdImpl.java:15)
at org.jenkinsci.plugins.dockerbuildstep.cmd.PullImageCommand.execute(PullImageCommand.java:75)
at org.jenkinsci.plugins.dockerbuildstep.DockerBuilder.perform(DockerBuilder.java:75)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
At the jenkins global setting when i hit test-connection it returns:
Something went wrong, cannot connect to http://127.0.0.1:9090/, cause: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:9090 [/127.0.0.1] failed: Connection refused
PS- I have restarted Jenkins server after changing global setting.
Any help, where am I missing ?
Ubuntu 16.04 uses systemd now I believe. In which case the docker daemon arguments are not set using /etc/default/docker. You can see they're not being picked up in the output of your $ ps aux|grep docker.
Instead you need to follow the instructions to set daemon args in systemd-based setups.
Related
I am trying to run this command:
mlflow server --backend-store-uri postgresql://aagmlflow:mlflow-#aagmlflow.cbh3397nepzq.us-east-1.rds.amazonaws.com/mlflow --default-artifact-root file:/root/mlruns -h 0.0.0.0 -p8000
Error is below here:
/usr/local/lib/python3.7/dist-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
""")
2022/10/27 16:16:03 WARNING mlflow.store.db.utils: SQLAlchemy engine could not be created. The following exception is caught.
(psycopg2.OperationalError) FATAL: password authentication failed for user "aagmlflow"
FATAL: password authentication failed for user "aagmlflow"
(Background on this error at: https://sqlalche.me/e/14/e3q8)
Operation will be retried in 0.1 seconds
2022/10/27 16:16:03 WARNING mlflow.store.db.utils: SQLAlchemy engine could not be created. The following exception is caught.
(psycopg2.OperationalError) FATAL: password authentication failed for user "aagmlflow"
FATAL: password authentication failed for user "aagmlflow"
(Background on this error at: https://sqlalche.me/e/14/e3q8)
Operation will be retried in 0.3 seconds
2022/10/27 16:16:04 WARNING mlflow.store.db.utils: SQLAlchemy engine could not be created. The following exception is caught.
(psycopg2.OperationalError) FATAL: password authentication failed for user "aagmlflow"
FATAL: password authentication failed for user "aagmlflow"
(Background on this error at: https://sqlalche.me/e/14/e3q8)
what could i be doing wrong here?
Is there any way i could troubleshoot this? Please i need help from the community.
this actually helped me resolved this issue.
mlflow server --backend-store-uri postgresql \
--default-artifact-root file:/tmp \
--host 0.0.0.0 \
--port 8000
I also ran a docker container with this too.
# Docker multistage build to reduce image size
FROM python:3.10 AS build
RUN python -m venv /opt/venv
# Make sure we use the virtualenv:
ENV PATH="/opt/venv/bin:$PATH"
RUN pip install mlflow
FROM python:3.10-slim
COPY --from=build /opt/venv /opt/venv
COPY --from=build /usr/lib /usr/lib
# Make sure we use the virtualenv:
ENV PATH="/opt/venv/bin:$PATH"
EXPOSE 8000/tcp
ENTRYPOINT [ "mlflow", "server", "--host", "0.0.0.0"]
CMD [ "--backend-store-uri", "/tmp"]
then build & run the container
docker build -t mlflow-tracking . && docker run -dp 80:8000 mlflow-tracking
I am following openflow tutorial on openflow tutorial. I am using mininet and ryu controller and openvswitch version 2.8.0. When I try to use command:
sudo ovs-ofctl show s1
It gives error as below:
asd#asd:~/ryu/ryu/mpls$ sudo ovs-ofctl show s1
ovs-ofctl: /var/run/openvswitch/s1.mgmt: failed to open socket (Connection refused)
As I am using sudo, I think there should not be any problem regarding permissions for opening socket connection. Not sure what is causing this error.
Help is appreciated.
The error message is saying that ovs-vswitchd did not open the Unix socket expected at /var/run/openvswitch/s1.mgmt. This usually means that ovs-vswitchd (the daemon process for Open vSwitch) is not running.
You can check that by running:
$ ps aux | grep ovs-vswitchd
root 77289 0.0 0.0 25632 4916 ? Ss 16:05 0:00 ovsdb-server --remote=punix:/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/var/log/openvswitch/ovs-vswitchd.log -vsyslog:info -vfile:info --pidfile --detach
root 77295 100 0.2 2608140 45820 ? Ssl 16:05 3:13 ovs-vswitchd unix:/var/run/openvswitch/db.sock --pidfile --detach --log-file=/var/log/openvswitch/ovs-vswitchd.log -vfile:info
paul 77447 0.0 0.0 14228 968 pts/2 S+ 16:08 0:00 grep --color=auto ovs-vswitchd
The first process, ovsdb-server, answers to OVSDB queries. The second process, ovs-vswitchd, answers to OpenFlow queries and processes packets.
While trying to run a puppet update form a node:
sudo /opt/puppetlabs/bin/puppet agent -t
I get an error:
Error: Could not retrieve catalog; skipping run
Error: Could not send report: Connection refused - connect(2) for "puppet" port 8140`
Elsewhere indicates this is likely a problem with the puppetserver service, and suggests to reboot the server. Restarting didn't help, and when I try to restart the service I get failure:
~$ sudo service puppetserver restart
Job for puppetserver.service failed because the control process exited with error code. See "systemctl status puppetserver.service" and "journalctl -xe" for details.
I've looked at these logs, and as a puppet/linux noob, I'm not sure what to do next.
systemctl status puppetserver.service
● puppetserver.service - puppetserver Service
Loaded: loaded (/lib/systemd/system/puppetserver.service; enabled; vendor preset: enabled)
Active: activating (start-post) since Fri 2016-09-02 15:54:26 PDT; 2s ago
Process: 22301 ExecStartPre=/usr/bin/install --directory --owner=puppet --group=puppet --mode=775 /var/run/puppetlabs/puppetserver (code=exited
Main PID: 22306 (java); : 22307 (bash)
Tasks: 17
Memory: 335.7M
CPU: 5.535s
CGroup: /system.slice/puppetserver.service
├─22306 /usr/bin/java -Xms6g -Xmx6g -XX:MaxPermSize=256m -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom -cp /opt/p
└─control
├─22307 /bin/bash /opt/puppetlabs/server/apps/puppetserver/ezbake-functions.sh wait_for_app
└─22331 sleep 1
Sep 02 15:54:26 puppet systemd[1]: Starting puppetserver Service...
Sep 02 15:54:26 puppet java[22306]: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
puppet version 4.6.1
The puppet master communicates with the other node using port number 8140.
I don't think a restart will help, since this looks like a connection issue between the server and the node.
please try the following -
first make sure that the puppet master is actually listening on port 8140. run the following command on the puppetmaster -
netstat -ntlp | grep 8140
this command should return something like this -
tcp 0 0 0.0.0.0:8140 0.0.0.0:* LISTEN 1783/puppetmaster
If you don't get the same output, your puppetmaster is not listening, and therefore can not compile catalogs for the node.
Try checking the puppet master log at /var/log/puppetmaster.log
check that the node can communicate with the puppetmaster on the relevant port. you can check this quickly with the telnet command. run this on your node -
telnet < puppetmaster ip address \ dns name> 8140
you should get something like -
Connected to <puppet-master-IP/DNS-name>
Escape character is '^]'.
if you don't get this output, this means that something is blocking you from accessing the puppetmaster. try opening the port in your firewall to access the puppetmaster.
if you're still stuck try using the --debug flag for verbose output and edit your question.
Could be 2 things: (1) in puppet.conf you have configured more memory than you have on your machine. Or (2) You installed both apt-get install puppetserver and apt-get install puppet.
If you get failed to start puppet.service: unit not found. error on slave machine while connecting to puppet.
Close the putty and then again open and connect it.The issue wont come while starting putty on slave.
The error occurs because there is not enough RAM and to fix the error, open the Puppet server configuration file:
sudo nano /etc/sysconfig/puppetserver
And reduce the amount of allocated RAM for the Puppet server (for example, I specified 512m instead of 2g):
JAVA_ARGS="-Xms512m -Xmx512m"
Now let’s start the Puppet server:
sudo systemctl start puppetserver
I have the following in my mup.json
// Install MongoDB in the server, does not destroy local MongoDB on future setup
"setupMongo": true,
// WARNING: Node.js is required! Only skip if you already have Node.js installed on server.
"setupNode": true,
// WARNING: If nodeVersion omitted will setup 0.10.36 by default. Do not use v, only version number.
"nodeVersion": "0.10.43",
// Install PhantomJS in the server
"setupPhantom": false,
// Show a progress bar during the upload of the bundle to the server.
// Might cause an error in some rare cases if set to true, for instance in Shippable CI
"enableUploadProgressBar": true,
// Application name (No spaces)
"appName": "myapp",
// Location of app (local directory)
"app": "/path/to/myapp",
// Configure environment
"env": {
"PORT": 5555,
"ROOT_URL": "http://myserver.com"
},
I got this in the deploy log
Started TaskList: Deploy app 'myapp' (linux)
[myserver.com] - Uploading bundle
[myserver.com] - Uploading bundle: SUCCESS
[myserver.com] - Setting up Environment Variables
[myserver.com] - Setting up Environment Variables: SUCCESS
[myserver.com] - Invoking deployment process
[myserver.com] x Invoking deployment process: FAILED
-----------------------------------STDERR-----------------------------------
eding commands with `sudo`, or if
npm WARN deprecated on Windows, run them from an Administrator prompt.)
npm WARN deprecated
npm WARN deprecated If you're running the version of npm bundled with
npm WARN deprecated Node.js 0.10 LTS, be aware that the next version of 0.10 LTS
npm WARN deprecated will be bundled with a version of npm#2, which has some small
npm WARN deprecated backwards-incompatible changes made to `npm run-script` and
npm WARN deprecated semver behavior.
npm WARN package.json meteor-dev-bundle#0.0.0 No description
npm WARN package.json meteor-dev-bundle#0.0.0 No repository field.
npm WARN package.json meteor-dev-bundle#0.0.0 No README data
js-bson: Failed to load c++ bson extension, using pure JS version
/usr/lib/node_modules/wait-for-mongo/bin/wait-for-mongo:14
throw err;
^
Error: TIMEOUTED_WAIT_FOR_MONGO
at null._onTimeout (/usr/lib/node_modules/wait-for-mongo/lib/waitForMongo.js:20:14)
at Timer.listOnTimeout [as ontimeout] (timers.js:121:15)
-----------------------------------STDOUT-----------------------------------
.1:27017]
wait-for-mongo: failed to connect to [127.0.0.1:27017]
wait-for-mongo: failed to connect to [127.0.0.1:27017]
wait-for-mongo: failed to connect to [127.0.0.1:27017]
wait-for-mongo: failed to connect to [127.0.0.1:27017]
when I ssh to the ec2 server and it looks like mongod is not started?
/opt/myapp$ ps -aux | grep mongod
ubuntu 9566 0.7 2.1 661524 22144 ? Sl 22:32 0:00 node /usr/bin/wait-for-mongo mongodb://127.0.0.1/myapp 300000
ubuntu 9569 0.0 0.0 10464 916 pts/0 S+ 22:33 0:00 grep --color=auto mongod
/opt/myapp$ mongo myapp
MongoDB shell version: 2.6.12
connecting to: myapp
2016-04-05T22:44:07.802+0000 warning: Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused
2016-04-05T22:44:07.803+0000 Error: couldn't connect to server 127.0.0.1:27017 (127.0.0.1), connection attempt failed at src/mongo/shell/mongo.js:146
exception: connect failed
I'm not sure how to gain access to mongo on the server, given that I "handed over" responsibility to meteor-up with "setup-mongo":true.
Any ideas would be appreciated.
update
switched to mupx, and set "deployCheckWaitTime": 300 mongo is now loading correctly. But now I am getting this error:
/bundle/bundle/programs/server/node_modules/fibers/future.js:278
throw(ex);
^
MongoError: driver is incompatible with this server version
at Object.Future.wait (/bundle/bundle/programs/server/node_modules/fibers/future.js:398:15)
at [object Object].MongoConnection._ensureIndex (packages/mongo/mongo_driver.js:790:1)
at [object Object].Mongo.Collection._ensureIndex (packages/mongo/collection.js:635:1)
It seems to be related to these issues:
https://github.com/arunoda/meteor-up/issues/841
https://github.com/meteor/meteor/issues/5809
SOLVED
I must had incompatible versions on my system from using mup and also meteor create; meteor mongo earlier on the ec2 server. I linked /opt to an empty folder in a different partition (had space problems) and ran mupx setup/deploy again from scratch. This time it worked fine (with Meteor 1.3).
Strangely, I noticed there was no /opt/nodejs folder, which was probably a leftover from my first attempt with mup.
also, the docker daemon doesn't seem to be running, but I can connect to my mongoDB from an ssh session the mup way
$ mongo myapp // works fine
$ docker exec -it mongodb mongo myapp
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
I downloaded PostgreSQL from the official website and ran the .dmg installer. After that I downloaded pgadmin3 and I am indeed able to connect to the database.
when I run 'psql' I get the following error:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
After hours of googling I read about some $PATH issues, so I put this into my .bashrc:
export PATH=/Library/PostgreSQL/9.1/bin:$PATH
However, this doesn't solve the error above at all. After some more hours of googling I tried to run 'psql -l localhost -U postgres'. This gives another error:
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (fe80::1) and accepting
TCP/IP connections on port 5432?
After some more googling I tried to edit /Library/PostgreSQL/9.1/data/pg_hba.conf and replaced all occurrences of 'md5' with 'trust'.
Then I changed user to postgres and executed 'pg_ctl stop' and 'pg_ctl start', switched back to my own user and tried to connect again, no luck.
Here are some more infos:
[~]$ which psql
/Library/PostgreSQL/9.1/bin/psql
ps aux | grep postgres
postgres 19022 0.0 0.0 2446096 484 ?? Ss 11:31PM 0:00.01 postgres: stats collector process
postgres 19021 0.0 0.0 2486532 1776 ?? Ss 11:31PM 0:00.01 postgres: autovacuum launcher process
postgres 19020 0.0 0.0 2486400 576 ?? Ss 11:31PM 0:00.03 postgres: wal writer process
postgres 19019 0.0 0.0 2486400 820 ?? Ss 11:31PM 0:00.05 postgres: writer process
postgres 19017 0.0 0.0 2446096 356 ?? Ss 11:31PM 0:00.01 postgres: logger process
postgres 19015 0.0 0.1 2486400 8216 s001 S 11:31PM 0:00.17 /Library/PostgreSQL/9.1/bin/postgres
sudo find / -name .s.PGSQL.5432
No file was found?!?
Update 1:
In /etc/sysctl.conf I added the values suggested by the installer's README:
kern.sysv.shmmax=1610612736
kern.sysv.shmall=393216
kern.sysv.shmmin=1
kern.sysv.shmmni=32
kern.sysv.shmseg=8
kern.maxprocperuid=512
Before these settings, the installer quits with an error, afterwards the wizard appears and installs postgres (and again, using pgadmin3 works, so I assume that the database is running fine).
Run lsof on the master process to all of this. In your case it's 19015 (shown with my PID):
> sudo lsof -p 286 | awk '$5 == "unix" && $NF ~ /\// { print $NF }'
/tmp/.s.PGSQL.5432
You can leave off the awk, but basically it's getting the UNIX socket on which postgres is listening. From there, you can use the -h option to psql (but only include the directory).
> psql -h /tmp template1
template1=# \q
If that doesn't work, you can check the lsof output to show you what TCP port it's actually listening on, and if it's not 5432, use the -p option to postgres
If you are able to connect to the database through pgadmin3, then your database is running.
(1) I'd verify by running netstat -an | grep 5432 on the command line, as a paranoia check. You are in business i.e. you actually installed Postgresql if you get this below:
tcp4 0 0 *.5432 . LISTEN
tcp6 0 0 *.5432 . LISTEN
ffffff80133bfed8 stream 0 0 ffffff8013be8000 0 0 0 /tmp/.s.PGSQL.5432
(2) Speaking for myself, I found that just running psql gets me your psql error message. I had to run
psql -h 127.0.0.1 -p 5432 [-d database] -U postgres
to connect with the database. In other words, I had to explicitly specify IP and port. Since you were able to run psql and get a non-connect error message, the OS understands that psql is a command and you don't have a pathing issue.