I know that we can monitor the infrastructure and OS level metrics using Newrelic's integration with AWS. But how can we monitor the queries and DB level parameters using newrelic.
This feature was requested but it's not implemented by new relic yet, which basically pulling rds performance insights to newrelic
https://discuss.newrelic.com/t/add-rds-performance-insights-data-to-aws-integration/60821
Related
I am building my own webapp which requires a huge database. I want to build and manage my own Mongo database on AWS rather than using Mongo Atlas. Which will be more cost saving? And whether I should go for Mongo Atlas? What will be its advantage over my own database?
There are pros and cons for both approaches:
Running MongoDB on AWS
Pros:
Complete control over how you run the database and how resources are allocated on the server. This could even be together with an application server on the same EC2 instance depending on your traffic and load. This might help with cost saving if your database is huge but isn't likely to see much traffic.
Cons:
You will be responsible for ensuring database availability and applying security patches as and when they are available. You may also have to setup firewalls and protect the EC2 instance and database in other ways that would be trivial to do on a hosted service like Atlas.
Data sharding and clustering can be a real pain to manage by yourself.
Running on Atlas
Pros:
Completely managed service where you don't have to be concerned about performance optimization or scalability. You pay for the services and Mongodb takes care of the rest.
You can focus on building a great application instead of spending your time on administering the database and the EC2 instance on which the database runs.
Cons:
You will be constrained by the options offered by Atlas. For most use cases this should be fine, but if you really want a specific change, it would be difficult to implement it if Mongodb doesn't already support it as a part of Atlas.
Think running your application on EC2 vs buying a server on-premise and running your application on that.
Being a managed service, costs might also be higher if your database does not see much traffic.
HOSTING yourself: You can get one or more AWS ec2 instances(which are VMs) where you can install and run Mongo DB yourself and manage it like you wanted to, making sure that you spin up more instances when the workload becomes large and there are instances up and running at all times to enable high availability.
Cost (high) - Management responsibilities (lots) - Full MongoDB functionality
MongoDB Atlas is a managed service, you don't need to worry about management tasks like scaling of your database and high availability when a single/more instances die... You pay a very low cost for it - this is run by MongoDb themselves on AWS, Azure, Google cloud;
Cost (low) - Management responsibilities (some) - Full MongoDB functionality
Now AWS has its own Mongo compatible database called DocumentDB - this is also a managed database, so you don't need to worry about scalability, high availability etc. This is only available on AWS so super simple and convinient.
Cost (low) - Management responsibilities (minimal) - Limited MongoDB functionality
I am searching for tool, which should provide collections overview, queries to them, replicas configurations, instance performance dashboards.
You can look for zabbix as elementary monitoring. However for querying the DB for data you need to use custom shell scripts or metric reporting tools to get that information
Has anyone tried connecting to IBM bluemix using bosh-cli. I am seeing performance issues in my requests and was going through this article on cloud foundry. I am planning to login to ssh to gorouter and monitor go-router CPU utilization.
Can someone recommend any way to capture the following metrics from Bluemix:
CPU utilization
Latency
Requests per second
what do you mean by "connecting to IBM bluemix using bosh-cli"?
When you think about the public available IBM Cloud (formerly Bluemix) that's represented here https://console.bluemix.net/ it's not possible. The bosh cli is to maintain the platform, thus Cloudfoundry and potentially other deployments but not your apps.
If you have a private installation you might check the metrics that the system provides. Infos here https://docs.cloudfoundry.org/running/all_metrics.html
When you want to have metrics about your app I could think off your app is providing these metrics. Or you put something in place like the New Relic monitoring. The have a bunch of application performance monitoring (APM). Info here https://docs.newrelic.com/docs/agents
HP
can anyone please give me a high level difference between MongoDB Cloud Manager and Mongodb Atlas. My main aim is to monitor mongodb instances in AWS.
Thanks.
Cloud Manager is used when you want to manage your own infrastructure (you spin up the nodes where MongoDB runs) but still have the benefits of automated backups and monitoring.
Atlas goes one step further by automating everything for you including provisioning the infrastructure. It's a true database as a service fully managed by MongoDB. They hide the complexity of managing servers so all you have to worry about it using MongoDB. It's interesting to note they use AWS (with plans to support Azure and Google) to spin up nodes, perform monitoring, and backups.
The Major difference between Atlas and Cloud manager is that :
Cloud manager is used for monitoring your database deployment and providing the automated back ups in the self hosted environment.
While MongoDB Atlas is used when your deployments are hosted on the MongoDB Servers. So each and ever task is managed by the MongoDB staff. This is basically the database as a service. In case you encounter any issue all you need to open a case with the mongodb and they will help in the investigations of the issue occurred.
Here is an up-to-date answer to this question which explains differences between Atlas, Cloud Manager and also the Ops-Manager:
MongoDB Atlas handles all the complexity of deploying, managing, and healing your deployments on the cloud service provider of your choice (AWS, Azure, and GCP). Atlas pricing details are here 4.
Cloud Manager is a platform for managing MongoDB on the infrastructure of your choice. Cloud Manager pricing details are here 7.
Ops Manager automate, monitor, and back up your MongoDB infrastructure.
Here is the original article and additional resources in the MongoDB community forum: https://www.mongodb.com/community/forums/t/cloud-manager-vs-ops-manager-vs-atlas/42624
I'm working on trying to setup some monitoring on a Google Cloud SQL node and am not seeing how to do it. I was able to install the monitoring agent on my Google Compute Engine instances to monitor CPU, Network, etc. I have not been able to figure out how to do so on the Cloud SQL instance. I have access to these types of monitoring:
Storage Usage (GB)
Number of Read/Write operations
Egress Bytes
Active Connections
MySQL Queries
MySQL Questions
InnoDB Pages Read/Written (pages/sec)
InnoDB Data fsyncs (operations/sec)
InnoDB Log fsyncs (operations/sec)
I'm sure these are great options, but at this point all I want to pay attention to is if my node is performing on a CPU/RAM standpoint as they seem to first and foremost measures for performance.
If I'm missing something, or misunderstnading what I'm trying to do, any advice is appreciated.
Thanks!
Google has a Stackdriver which is for logging and monitoring Google and AWS cloud infrastructure. It can monitor every single thing present on GCP. You can create visualization to monitor your Cloud SQL instance in one dashboard. You just have to ---->
1. login to stackdriver and Go to any existing dashboard, If you dont have create one.---->
2. Add chart and select Cloud SQL in resource Name.---->
3. Select CPU Utilization from metric and save. You can also monitor memory, Disk I/o, Delta count of Queries or servers Up-time and many more.
if you want to monitor any other GCP Compute engine, App-Engine, Kubernetese Engine, storage bucket, Bigtable or pub/sub you just have to select appropriate resource name from list. Hope you got your answer.
You can view all of them directly from the "Overview" tab of the Cloud SQL console:
I have added this as a feature request as issue 110.
https://code.google.com/p/googlecloudsql/issues/detail?id=110