We have a handful of Ruby (Rails/Sinatra) apps and are looking for an easy means of managing retention, search and analysis of our logs for these applications.
The initial problem was that every time we'd push a new version of our apps the logs would disappear.
We then started streaming our logs to a file via a terminal using:
cf logs AppName
however the logfiles get very big, very fast and quickly become a problem.
We know that the Bluemix Monitoring and Analytics service provides a lot of the function we need. We're looking that over but want to know if there are other recomended/proven options.
Thanks
We found several 3rd party apps that provide the functions we need.
To use any of these we first had to configure third party logging on Bluemix and used the steps below.
Any 3rd party logger that supports the syslog protocol can be used. The initial setup, registration and configuration of the log manegement service, is well covered at Configuring Selected Third-Party Log Management Services.
What will come out of the configuration step is a syslog URL which will be the destination for your logs.
Once the logging service is configured a user-provided service instance needs to be created to stream the logs to the logging service. We did this using:
cf create-user-provided-service <user-provided-service_name> -l <syslog_URL>
Last step is to bind the service instance to our Ruby apps.
cf bind-service AppName <user-provided-service_name>
For the changes to take effcet, we then had to restage our ruby apps:
cf restage AppName
There was a brief delay between when we'd see the logs generated and when they'd show up in the logger service but overall this is working out ok for us so far.
Related
I am using the standard environment of GAE / PY.
I'd like to confirm syslog, but I could not confirm it with stackdriver logging(only request_log and activity_log).
Can not you confirm syslog with App Engine standard environment?
You do not have access to the actual syslog of the systems on top of which your standard env GAE app instances are running. I suspect the user-provided app code might not even have write permissions to such logfile.
If you're looking for logs produced by your app (for example by using the logging facility, as recommended by ), they're bundled under the request_log entries for the requests that triggered the respective code execution, see Reading Application Logs on Google App Engine from Developer Console
I have a node-red flow in bluemix that uses dash-db nodes also. So each time some dash db maintenance or some other reason, this db connection gets lost and all writes fail. When i redeploy, everything is fine again. Bluemix shows only logs of last few hours hence I am finding it very difficult to debug. Meanwhile i was thinking of doing an automatic redeploy after i detect this issue to avoid losing writes.
Can this be done using GET /flows followed by POST /flows in the same node-red app itself?
it would be worth raising this as an issue with the dash-db nodes so the author can help address it - https://github.com/smchamberlin/node-red-nodes-cf-sqldb-dashdb
Yes, you can post back the flows. The full admin http api is documented here: http://nodered.org/docs/api/admin/ - have a look at the 'reload' option on /flows.
I want to monitor the console activities such as who login, Any new users got created and new hosts got added for my PCI Activity.
Unable to figure out in which logs the above mentioned info is present
I have looked into /var/log/ipaserver-install.log but this log does not give me the required information.
Please help.
All operations over IPA framework are visible in /var/log/httpd/error_log on each IPA master. This includes adding users, removing them, etc.
There is a prototyped demo on gathering all logs together and visualizing different flows available at https://www.freeipa.org/page/Centralized_Logging. It is not a complete solution but rather a sketch on how it would look like and it has a detailed description on which logs need to be gathered and how to configure a log forwarding specific to IPA on RHEL 7/CentOS 7: https://github.com/pschiffe/ipa-log-config
I'm building a hybrid app that speaks to a Bluemix app. I've got Mobile Application Security and Mobile Quality Assurance as a service. I'm using IBMLogger to send log messages, like so:
IBMBluemix.getLogger().info("Device successfully registered: "+ JSON.stringify(response));
I was under the impression that the MobileFirst-style services on Bluemix acted somewhat like the same components for a local MF install. I'm trying to find where I can see my log files in the BM console but have not been able to find them.
For a local MF install using the hybrid SDK, you have to specifically say you want your logs sent to the server, but I do not see that option when looking at the hybrid docs for BM/MF.
There is some doc here: https://www.ng.bluemix.net/docs/starters/mobile/mobilecloud/nodejsmobile.html#log
I suppose by default the log goes to the console, which you can retrieve through the loggregator, aka do cf logs <yourapp> --recent. You can also use the Monitoring & Analytics service to retain logs - see the above doc.
I've read some articles recently on setting up AWS infrastructure w/o enabling SSH on Ec2 instances. My web app requires a binary to run. So how can I deploy my application to an ec2 instance w/o using ssh?
This was the article in question.
http://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-started/
Although doable, like the article says, it requires to think about servers as ephemeral servers. A good example of this is web services that scale up and down depending on demand. If something goes wrong with one of the servers you can just terminate your server and spin up another one.
Generally, you can accomplish this using a pull model. For example at bootup pull your code from a git/mecurial repository and then execute scripts to setup your instance. The script will setup all the monitoring required to determine whether your server and application are up and running appropriately. You would still need an SSH client for this if you want to pull your code using ssh. (Although you could also do it through HTTPS)
You can also use configuration management tools that don't use ssh at all like Puppet or Chef. Essentially your node/server will pull all your application and server configuration from the Puppet master or the Chef server. The Puppet agent or Chef client would then perform all the configuration/deployment/monitoring changes for your application to run.
If you with this model I think one of the most critical components is monitoring. You need to know at all times if there's something wrong with one of your server and in the event something goes wrong discard the server and spin up a new one. (Even better if this whole process is automated)
Hope this helps.