I am trying to find how the Beats Central Management console in Kibana deploys configuration changes to enrolled Beats.
I have searched the documentation of both Beats and Beats Central Management, but have seen no clear detail specifying how these changes are deployed. I know that when a Beat is enrolled it receives an "access token" for configuration polling, but does this mean that REST is used?
It was confirmed by an Elastic team member that the changes are deployed through HTTP calls.
Related
We have been tripped up twice recently as our development output has increased.
We have a; backend services, an Admin SPA site and a number of frontend applications including native apps. All in different repos
We also have a fully automated CI/CD pipelines for everything that works fantastically.
What has happened recently is the public applications have gotten ahead of the Admin SPA which is making the team look bad.
Has anyone seen a solution that requires minimum input for developers - the more I can rely on automation the better.
The goal is to keep feature deployments in concert
Tanks
So the plan is to go down versioning with Semantic versioning and a route on admin that returns a json response with the version number.
The build and deploy for admin takes in the version and returns it.
The deploy for reliant apps has a script that queries admin before starting.
There is still a bit of manual work for the developers but it is manageable.
Thanks #Bruno
i am interested in building a github app. reading through github Setting up your development environment to create a GitHub App documentation it explains that a github app is based on a http server which will handle webhooks.
yet, on every github app i installed, the app\installation did not require anything that involves hosting and/or creation of http server in order to deploy the app to my github accout.
for such github apps, which are installed directly through the github marketplace (you can take probot stale and rennovate as examples for such apps)
where do these application run? (e.g.; does github deploys the app on a dedicate (virtual) server?).
what are the resources limitations for such apps (amount of memory, cpu, etc.?)
how can the github app logs be accessed by the github account owner who installed such app?
links for reference and an answer will be great.
GitHub App is just another app that you create. GitHub apps are treated as first-class citizens when it comes to integrating with GitHub. One can use Nodejs , Ruby, etc to build the App. Once the app is ready it can be hosted on a Server just like any other server hosted apps. You register your app on GitHub by providing relevant details.
So, coming to your questions.
The Apps can run on any hosting service of your choice. It can be a Windows Server, Heroku, etc.
I believe it is only limited by the resource of your server or the hosting service provider that you chose. However you might be ineterset in erading more about the Rate Limit More on Rate limits here.
GitHub app logs are something which only the developer will be able to see. To the end user ,i.e. the repo owner who installed the GitHUb app on his repos, all that will be available are the checks , statuses and any other details that the developer of app decided to display.
A very handy guide on Deployment and other details : Probot Documentation. This documentation is great if you are planning to use the probot framework for developing your github apps, but most of the instructions still stand true in case you decide to pick up a different tech stack.
The most important thing to realise about a (so-called?) Github App is that the App itself does not run anywhere - or at least that is what I would argue. Basically Github Apps are two linked mechanisms, both a bit of infrastructure. The first of these mechanisms is access control, essentially replacing use of user PATs - you can give relatively fine grained access to repos that the App is installed in, rather than just giving access to all repos the user can access. The second mechanism is that of webhooks - generating events as requested.
What Github Apps do not directly provide is the bit between this - handling the webhooks and generating API calls using the App for access. Basically you are on your own and need to do it yourself. The plus, #asif-kamran-malick mentioned, is that you have freedom to implement it how you see fit.
One alternative possibility is that the App itself, rather than setting to handle ongoing Webhooks, runs on installation and looks to add Actions into the repo. Never done it, but some of the github examples seem to work this way. Of course, Actions are run within Github environments and are potentially subject to resource limits. Apart from this though, Actions are a completely separate "beast" and should not be confused.
We are using TFS 2015 for a quite of time. We have a lots of projects that running on the TFS solution. We use the Release Managements tools built-in to make the deployment and tracking more simple. But in our business context, we must deploy many different project in many different server (OnPremise, Azure, third party hosting). Some project use dependencies with other project.
The major problem we have actually is that we cannot track effectively where our different applications is deployed and wich have dependencies with other.
When is time to make maintenance on server or service, we have a hard time to detect all the dependencies and all projects that going to be affected.
We can check in every TFS project, on the Release Managements and edit every environments config, but this is not a solution as the number of projects grown.
What is the best strategy or guideline ?
Thanks
I'm afraid using the Release Management is the best choice for your situation. Even though you will met some performance or management limitation with the number of projects grown.
Release management retain full traceability (permanent audit trails)
Monitor the current status of the releases and deployments to all the environments.
Track the status of recent deployments in each of the environments.
Retain detailed audit history of all the activities performed on a release.
See the commits and work items that are associated with each release.
And since you are using it to make the deployment, is it not the best choice. You have said your team must deploy many different project in many different server (OnPremise, Azure, third party hosting). As you might imagine, which third-party tool or system will handle the such a complex cross-platform environment. And this tool should also tracking the source code, the deployment result through Release Management...
I have been playing with Spring Cloud Configuration. I like the simplicity of the solution and the fact that it uses git as it's default configuration store.
There are two aspects I need to figure out before pushing it as a solution for centralized configuration management.
The aspects are:
High availability
How to gradually roll out configuration changes (to support canary releases)
If you already implemented this in your data center or just playing with that please share your ideas!
Also I would like to hear from the creators, how they see the recommended deployment in single/cross data-center environments.
The Config Server itself is stateless, so you can spin up as many as these as you need and find them via eureka. Underneath the server itself, the git implementation you point to needs to be highly available as well. So if you point to github (private or public), then git is as available as github is. If the config server can't reach git it will continue to serve what it has checked out even if it is stale.
As far as gradual config changes, you could use a different branch and configure the canary to use that branch via spring.cloud.config.label and them merge the branch. You could also use profiles (eg application-<profilename>.properties) and configure the canary to use the specified profile.
I think the branch makes a little more sense, because you wouldn't have to reconfigure the non-canary nodes to use the new profile each time, just configure canary to use the branch.
Either way, the only time apps see config chages (when using spring cloud config client) is on startup or when you POST to /refresh on each node. You can also POST to /bus/refresh?destination=<servicename> if you use the Spring Cloud Bus to refresh all instances of a service at once.
I'm learning FluentMigrator. The thing that I like about FM is that it supports the idea of Forward and Back for migrations (aka Up/Down). I'm finding that it's not ideal about this; there are some holes. Still, it's good.
This leads me to wonder if there are any deployment tools (nant, msbuild or other) that support this idea of rolling forward and back. The scenario that I'm using it in is the deployment of a web app with a related database.
Ideally I'd like to set up my deployment so that, should any part of it fail, it will revert to the previous known working configuration. With FM, this is pretty easy to do (but there are rough spots), so that covers the db. How about the files that make up the web app? Do any deploy tools have support for this?
Deploying to a Windows Server. Assume that I can't make any changes to the server.
I don't know of any Microsoft-centric, automated provisioning/deployment tools like Capistrano. Here are some tools I've heard of, but never used:
MSDeploy, for deploying web application.
Microsoft Deployment Services, for managing operating system configuration
Microsoft's System Center Configuration Manager
BladeLogic
HP's Operations Center
Up until about three months ago, we did our deployment/provisioning using custom MSBuild scripts. After a server is provisioned, deploys happen automatically using Robocopy to copy files to a share on the application server, updating changed application binaries and markup files. We've never had a need to rollback any of our deployments, but since our scripts are custom, we could write the logic if we needed to.
MSBuild is a terrible deployment/provisioning language. For the past three months, we've been writing all new scripts in, and porting existing ones to, PowerShell. It is wonderful. With version 2, there is support for running commands on remote servers, like SSH. We haven't used that functionality yet, but I'm looking forward to pushing setup scripts to remote server to provision and deploy at the same time.
We have been using Git to do our deploys for the last 6 months.
Here is the whole process:
CI server build the project
CI server checks it in to a local git repository
CI server pushes the changes to the centralised git repository
User creates an empty repository on the live server
User adds the central git repository to the remotes
User pulls the latest version over https (no need to open any ports)
It is a lot to setup in the beginning but once setup it works great. Deploys take seconds as only changed files get copied.
Another great thing about this method is that git keeps history of changes so rolling back is pretty simple. You can also roll back a few revisions and it's done straight on the live server. If something goes wrong reverting is super fast.
Also you can save some time if you use a hosted git service (github) for your central repository.
This is a very brief description but I can give you more info if you want.
Of course! My favorite is Capistrano. This was originally built for Ruby but I've found that it works just as well for other languages.
https://github.com/capistrano/capistrano