I wanted to know where buildbot is retrieving the PATH environment from?
My problem is the following:
I am compiling on MACOSX with zsh.
My master.cfg is well configured to use zsh since in the stdio I get :
SHELL=/bin/zsh
But my path is not the path configured in .zsh.
For now I hotfixed that by adding env={"PATH"=["PathIWant"]} but I find it a bit hacky and does not really like that solution.
I would prefer the buildbot slave to get the right path from the environment.
Since you mention master.cfg, keep in mind that the commands execute on the slave, not the master -- so the PATH on the master doesn't matter.
The environment on the slave is simply the environment in which the slave was started. For example,
export PATH=/path/i/want:$PATH
buildslave start /path/to/slave
would put /path/i/want in PATH for any commands executed by that slave instance.
Related
Let's say that for some reason I don't want to launch VSC to get a devcontainer shell running, but I still want all of that devcontainer goodness without rewriting all of the configuration files. There's a devcontainer CLI, but at the moment, the only options available are open (VSC, connected to the container) and build (which builds the image, in the use case that many people are sharing the same devcontainer environment).
Ideally, there'd be a third option devcontainer shell which does all the build, spin up and connection work that is done inside VSC, but the just execs to the running container.
The .devcontainer folder contains a devcontainer.json file. In it, if you're using docker-compose, there will be a dockerComposeFile key with an array of docker-compose files, loaded in order. You can do the same with a command such as docker-compose -f first-compose-file.yml -f second-compose-file.yml.
That same folder usually has its own docker-compose.yml file. You will notice it declares your main service and usually sets up a volume to share between the host and the container (useful to work inside the container).
There are other interesting keys in devcontainer.json such as forwardPorts, remoteUser or postCreateCommand. You should be able to set up most of them in your docker-compose file (dev stuff should go into the .devcontainer/ one). The post-create command can be run with docker compose exec SERVICENAME COMMAND.
I don't know if there's a command to detect .devcontainer files and pick up the right settings, but it should not be hard to write one.
I tried to run C:\Spark\spark-1.6.1-bin-hadoop2.6\sbin>start-master.sh
, but the following mistake appeared
I also noticed that there is a warn while running bin/spark-shell
16/04/24 23:14:41 WARN : Your hostname, Pavilion resolves to a loopback/non-reachable address: fe80:0:0:0:0:5efe:c0a8:867%net14, but we couldn't find any external IP address!
http://localhost/8080 is also unawaliable.
Can anybody please tell where is a mistake or did I miss any settings which are nessesary to run Master properly?
The problem is that this script is not designed to be executed on a Windows machine. Please refer to Official Spark Manual
Note: The launch scripts do not currently support Windows
The rule of thumb is that only script ending with .cmd will run on Windows. While scripts ending in .sh are designed for Linux and Mac OS. And while it should be possible to manually start Spark Master on Windows, it's probably better to just run a local[*] mode, unless you are creating a cluster of Windows machines. local[*] mode already fully utilizes power of the local machine.
On Compute Engine, using the click-to-deploy option for MEAN, how can we clone the repo of the sample app it locally creates so that we can start editing and pushing changes?
I tried gcloud init my-project however all it seems to do is initialize an empty repo. And indeed when I go to "source code" section for that project, there is nothing there.
How do I get the source code for this particular instance, setup a repo locally for it and then deploy changes to the same instance? Any help would be greatly appreciated.
OK, well I have made some progress. Once you click-to-deploy GCE will present you with a command to access your MEAN stack application through an SSH tunnel.
It will look something like this:
gcloud compute ssh --ssh-flag=-L3000:localhost:3000 --project=project-id --zone us-central1-f instance-name
You can change the port numbers as long as your firewall rules allow that specific port.
https://console.developers.google.com/project/your-project-id/firewalls/list
Once you SSH in, you will see the target directory, named the same as you told mean-io to use as the name of the application when you ran mean init
I first made a copy of this folder where mine was named "flow" cp -r flow flow-bck and then I removed some unnecessary directories with:
cd flow-bck && rm -rf node_modules bower_components .bower* .git
All of this to setup copying that folder to my local machine using gcloud compute copy-files availabe after installing Google Cloud SDK.
On my local machine, I ran the following:
gcloud compute copy-files my-instance-name:/remote/path/to/flow-bck /local/path/to/destination --zone the-instance-region
Above 'my-instance-name', '/remote/path/to', '/local/path/to', and 'the-instance-region' obviously need to changed to your deployment's info, etc.
This copied all the files from the remote instance to a folder called flow-bck on local found at the defined local path. I renamed this folder to what it is on remote flow and then did:
cd flow && npm install
This installed all the needed modules and stuff for MEAN io. Now the important part about this is you have to kill your remote ssh connection so that you can start running the local version of the app, because the ssh tunnel will be using that same port (3000) already, unless you changed it when you tunneled in.
Then in my local app directory flow I ran gulp to start the local version of the app on port 3000. So it loads up and runs just fine. I needed to create a new user as it's obviously not the same database.
Also I know this is basic stuff, but not too long ago I would have forgotten to start mongodb process by running mongod beforehand. In any case, mongo must be running before you can start the app locally.
Now the two things I haven't done yet, is editing and deploying a new version based on this... and the nagging question of whether this is all even necessary. That'd be great to find that this is all done with a few simple commands.
I have installed zookeeper on my Linux server ubuntu 12.04 in some folder like abc/zookeeper/zkserver/bin/zkserver.sh start. this works fine and starts the server as expected but when i put this zkserver.sh file in /etc/init.d folder and copy the file in rc2.d folder for the reason to start-up zookeeper on system reboot. and when i run this command /etc/init.d/zkserver.sh start this is giving errors like:
JMX enabled by default
Using config: /etc/init.d/../etc/zookeeper/zoo.cfg
grep: /etc/init.d/../etc/zookeeper/zoo.cfg: No such file or directory
mkdir: cannot create directory `': No such file or directory
Starting zookeeper ... STARTED
The zkServer.sh is dependent on a certain directory structure and certain files being present. It is not supposed to be moved in isolation like that. It is also not supposed to be used as an init script.
Check if your zk download comes with the init script. Try looking at src/packages/rpm/init.d/zookeeper or similar, and use that one instead.
here's my situation.
I'm running Cruise Control as a Windows Service and trying to get it to connect to a Mercurial Repository on BitBucket over SSH.
I'm pretty sure that everything's configured OK (PuttyGen, Pagaent, etc). I'm remoting onto the server using the same account that I am using to run the service and if I issue hg pull -b ssh://#bitbucket.org// from a command line everything works. I added -v to the ssh configuration in mercurial.ini and I can see all of the steps that are taken.
If I run CC.NET from a command prompt then it builds fine. In the console window I can see the same logging from the SSH operation.
However, if I run CC.NET as a service (using the same user account that I'm logged in on) the call to BitBucket times out. I can find no way to work out why either. The build log doesn't help and neither do ccnet.log or ccnet.trace in the temp directory. I was expecting one of them to contain the logging from the SSH operation, but they don't.
Can anyone help? Is it that running as a service prevents it from connecting to Pagaent (I've started Pagaent by adding it to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run). When I did the pull from the command line I had to OK a dialog, but only once. Is it waiting on the same dialog now that it's running as a service?
Getting close to my wits end here.
Thanks
I did get it working in the end. The trick was to create the public key without a passphrase. When running as a service the solution has to be completely non-interactive and the passphrase option with pagaent.exe just isn't.
Here are the steps:
Use PutTTygen to generate a secure key WITHOUT a passphrase. If you really do need one then you can add it to the mercurial.ini file, but defeats the point for me as it's in plain sight anyway.
Copy a mercurial.ini to two locations: C:\Windows\System32\config\systemprofile and C:\Windows\SysWOW64\config\systemprofile. Probably only one of these was really necessary, but I didn't have the time to experiment. The first is the home directory for the system user when running 64 bit apps, the SysWOW64 location for 32 bit. Make sure that if you do the same as me then keep both files in sync - or go one further and work out which is the correct location.
Add something like this line under the [ui] key in both files:
ssh = "D:\Program Files\TortoiseHg\TortoisePlink.exe" -ssh -2 -C -batch -v -i "[Path to your ppk file]"
Add the passphrase to the end of the command if one was created in step 1.
Make sure that TortoisePlink.exe is specified, not Plink.exe. They should both be in the same directory.
Download psexec from http://technet.microsoft.com/en-gb/sysinternals/bb842062.aspx
Run d:\PSTools\PsExec.exe -s -i cmd.exe. This will open a command line as the system account in interactive mode.
Now do an hg pull, or hg clone or whatever.
A dialog should pop up with a confirmation message. This is a one time thing and the reason that you have to do the PsExec step. OK the dialog.
Now cc.net should be able to be run as a service under the local system account using SSH!