BOSH working with Dynamic IP Addresses - cf-bosh

What's the best way to work with dynamic IP addresses with BOSH? Currently we're setting static IP addresses for each machine we want to use, but we only really care that one of those VMs has a static IP address.
Is there a way to get information about other VMs running in the BOSH network from within a BOSH VM? Or just get dynamic information about the deployment from within the VM? Such as which machines are currently running on which IP addresses?

It sounds like the recent introduction of "links" is worth a look for your use case.
Previously, if network communication was required between jobs, release authors had to add job properties to accept other job’s network addresses (e.g. a db_ips property). Operators then had to explicitly assign static IPs or DNS names for each instance group and fill out network address properties
This lets each job either expose or consume connections.
i.e. a DB exposes its connection
# Database job spec file.
name: database_job
# ...
provides:
- name: database_conn
type: conn
# Links always carry certain information, like its address and AZ.
# Optionally, the provider can specify other properties in the link.
properties:
- port
- adapter
- username
- password
- name
And a Application can consume it.
# Application job spec file.
name: application_job
# ...
consumes:
- name: database_conn
type: conn
The consuming job is provided with extra properties to use these addresses/info as needed, i.e.
#!/bin/bash
# Application's templated control script.
# ...
export DATABASE_HOST="<%= link('database_conn').instances[0].address %>"
export DATABASE_PORT="<%= link('database_conn').p('port') %>"

Related

Specify postgresql database name in cloud foundry manifest.yml

Is there a way to specify a postgresql database name to connect to in the cloud foundry manifest.yml file? I've been raking through the documentation and haven't yet found this specific information.
I'm imagining something like this:
applications:
- name: my-app
routes:
- route: my-app.mybluemix.net
services:
- postgres
dbname: database2
With that approach, a postgresql connection can be made by just the connection string provided by VCAP_SERVICES parsing modules (cfenv in the case of node).
If this is not possible, I will just set a dbname environment variable and build my own connection string.
There is nothing like that in a Cloud Foundry application manifest.yml.
The manifest.yml only takes a list of service instance names and the services with those names will be bound to your app. It does not allow you set other metadata.
https://docs.cloudfoundry.org/devguide/deploy-apps/manifest-attributes.html#services-block
I don't know if these will help, but when you cf bind-service directly there are two additional provisions you can make use of (these are not supported by manifest.yml as of me writing this):
Arbitrary bind parameters. These probably won't help unless your service broker supports them, but it's a way to pass additional info to the service broker. If your broker supported it, you could in theory say give me a database named XYZ by passing it some config this way.
Named service bindings. This provides what amounts to a second name. The intent is that you can create the service with a name of X, but your application can look for a service binding with name Y. You can use this to swap in differently named services, but still expose the same binding name to the application so it will always find the service.
If you are trying to pass in some other service instance related metadata to your application, you'd need to do it some other way. Like if you want to tell it the database name or the connection pool size, etc.. Using environment variables like you mentioned is one option. You could use a config file or cli arguments passed to your application. What you pick is probably a matter of preference/support in the library/framework you're using.
For what it's worth, most service brokers I've seen pass in and tell you a specific database name to use. If the broker said connect to db XYZ and you made your connection to myCoolDb, the connection would fail. Just wanted to mention this. Your mileage may vary.

Provide Proxy Information to Job

Wondering if anyone has come across this: Is it possible to provide proxy information to Concourse job? Something along lines of this:
- name: bosh-deploy-0
...
jobs:
- name: deploybosh
properties:
http_proxy_url: <http_proxy_url>:<http_proxy_port>
https_proxy_url: <https_proxy_url>:<http_proxy_port>
no_proxy:
- localhost
- 127.0.0.1
If anyone has a working example, I'd be very much appreaciative!!
You can only set these properties per worker. https://github.com/concourse/concourse-bosh-release/blob/v4.2.1/jobs/worker/spec#L142-L153.
If you want a job to run with specific proxy information set, you need to
Deploy a worker with those properties set, and with some worker tag.
Configure every step of the job with that same tag.
You could also set the proxy settings at the beginning of your job task (and optionally pass the proxy endpoint with parameters or a config server backend). That's maybe not the nicest way, however, it works quite well.

Adding a custom domain name with surge.sh

I'm a newbie in domain names, DNS etc.
I'm using surge.sh for deploying my app. Now I want to add a custom domain, that I registered using transIP, and I can't get it working. I set the IP address to 45.55.110.124, as they explain here. All together, I entered the following settings:
Name: *
TTL: 1 min
Type: A
Address: 45.55.110.124
And another one, exactly the same but then using name #:
Name: #
TTL: 1 min
Type: A
Address: 45.55.110.124
I created a test page that contains hello domain, inside a simple html file. Now, I deployed the page by moving to the folder that contains the html file and doing: surge ./ mydomain.io.
I waited over 5 minutes and nothing is changing.
Now, my questions are:
What am I doing wrong?
My domain provider suggests that I also use an IPv6 address, but which one should I use for Surge?
Why is there an option of setting TTL longer than 1 minute, who wants to wait longer before their deploy comes online?
For starters, you want to use the CNAME instead of A record if possible. The reason for this is that their IP address can possibly change out from under you when infrastructure changes / updates / re-deploys. If possible, remove the A records and create CNAME records pointing to na-west1.surge.sh. instead.
Next, assuming that they want you to point to the same IP as na-west1.surge.sh resolves to, that IP is different from the documentation (possible even due to my previous explanation). You can ping the domain or use the host utility to get the current IP address:
$ host na-west1.surge.sh
na-west1.surge.sh has address 138.197.235.123
Armed with this information, try changing to CNAME records first. If this isn't possible, then use the updated IP address that you get from resolving the their CNAME.

Windows ServiceBus 1.1 for Windows Server

I did move the databases from our ServiceBus test enviroment.
I started by leaving the farm with the single node, then I moved the databases.
After rejoining the farm I see that GatewayDBConnectionString is till pointing to the old one.
I can't find any valid PowerShell command to reconfigure the value in question.
Anyone know how to fix this?
Thank you in advance.
To answer this I will need you to understand this a bit more - and hence giving a high-level overview of Service Bus 1.1 Server farm configuration:
Service Bus Server 1.1 is a platform where users can create highly-durable distributed Pub-Sub (messaging Queues/Topics) entities. In simple words - the main job of this is to translate the Compute (your VMs) and Data (your MsgContainer databases) into messaging functionality Durable Queues and Topics. So, in short - the configuration wizard or the Powershell cmdlets used to configure ServiceBus 1.1 Server will try to take the VMs and Databases from you.
The Db SBManagementDB is considered to be the authoritative source of truth for any Farm level configuration -> like Nodes that are part of the Farm (Store.Nodes), Ports opened on each of the nodes, Gateway database connection string (Cluster Config) etc. Also pl. note that - as per the Windows Server product guidelines - any information that has to be securely persisted will be encrypted - so as the Gateway DB connection String.
a) when you did New-SBFarm (with a Gateway DB connection string) - you have essentially communicated to SBMgmtDB - the Gateway DB Server, database name etc.
b) when you do Add-SBHost - again you have communicated to SBMgmtDb that you want to add one Node to this Farm
Gateway db connection string is the one place for Truth for all Gateway Services to find any run-time info -> like Container Databases, entity to container mapping etc.
again, when you do New-SBMessageContainer PSCmdlet --> you communicated to SBGatewayDB that you are adding one db
Now, with this background - lets see how the action you did above will take into effect:
- When you moved all the Databases to a different Server - you changed the Gateway Database connection string - But the Gateway connection string you had communicated to the SBManagementDB (using the New-SBFarm cmdlet) was pointing to the Old Server.
- When you removed the Node from the Farm and again Joined back - you removed one node from the configuration and re-added it - no affect :)
The ANSWER
Use Restore-SBFarm PS Cmdlet to communicate to the SBManagementDB that you changed the GW db
and then Use Restore-SBMessageContainer PS Cmdlet to communicate to Gateway DB that you changed the Container databases.
Now, add the Nodes back to this restored farm.
HTH!
Sree

Is there a api for ganglia?

Hello I would like to enquire if there is an API that can be used to retrieve Ganglia stats for all clients from a single ganglia server?
The Ganglia gmetad component listens on ports 8651 and 8652 by default and replies with XML metric data. The XML data type definition can be seen on GitHub here.
Gmetad needs to be configured to allow XML replies to be sent to specific hosts or all hosts. By default only localhost is allowed. This can be changed in /etc/ganglia/gmetad.conf.
Connecting to port 8651 will get you a default XML report of all metrics as a response.
Port 8652 is the interactive port which allows for customized queries. Gmetad will recognize raw text queries sent to this port, i.e. not HTTP requests.
Here are examples of some queries:
/?filter=summary (returns a summary of the whole grid, i.e. all clusters)
/clusterName (returns raw data of a cluster called "clusterName")
/clusterName/hostName (returns raw data for host "hostName" in cluster "clusterName")
/clusterName?filter=summary (returns a summary of only cluster "clusterName")
The ?filter=summary parameter changes the output to contain the sum of each metric value over all hosts. The number of hosts is also provided for each metric so that the mean value may be calculated.
Yes, there's an API for Ganglia: https://github.com/guardian/ganglia-api
You should check this presentation from 2012 Velocity Europe - it was really a great talk: http://www.guardian.co.uk/info/developer-blog/2012/oct/04/winning-the-metrics-battle
There is also an API you can install from pypi with 'pip install gangliarest' and sets up a configurable API backed with a Redis cache and indexer to improve performance.
https://pypi.python.org/pypi/gangliarest