Overview: Log Management
The AWS Marketplace offers a large variety of commercial and open source offerings to augment log management within the AWS ecosystem. Sumo Logic’s AWS Marketscape looks at the pros and cons of the most common log management tools and solutions from the Marketplace.
Thanks to standardized utilities and well-understood software development guidelines, every resource in the modern computing environment now has the potential to generate enormous amounts of raw log data.
Whether spawned by the user-facing application or its underlying operating system, database engine, API, microservices or Web server—just to name just a few sources—these logs provide the building blocks to translate invaluable data into usage patterns and error conditions.
Logs may even supply intelligent guidance regarding where to expend efforts for new development. However, it’s impossible to fully realize these benefits without deploying finely tuned technologies designed to ingest and extract meaning from these raw logs. Amazon Web Services provides a few tools for log management, which can be combined with third party services for maximum value.
AWS Log Management Solutions and Tools
Amazon has always understood the value of producing, capturing, and analyzing detailed log information. Consequently, it developed AWS CloudWatch to serve as a centralized repository for logs from all major AWS technologies. AWS services rely on it as the shared destination for their own log details along with optional user-defined metrics.
Developers and administrators turn to the graphical dashboards and associated APIs provided by AWS CloudWatch Logs to understand the state of their AWS environment, including application performance, resource utilization, and responsiveness.
AWS CloudWatch may also be configured to take direct action when it detects predefined events. The resulting procedures may include provisioning new resources, terminating underutilized assets, and alerting system administrators to potential crises.
AWS Log Management Pricing
Amazon offers a free, basic AWS CloudWatch solution that furnishes useful dashboards and helpful metrics. For enterprises operating a more robust cloud computing portfolio, AWS CloudWatch logging expenditures will be driven by variables like the quantity of active dashboards, alarms, custom metrics, instances, API requests, and log transfer volume.
AWS Log Management Limitations
Although it permits developers to create and track their own custom parameters, AWS CloudWatch is primarily a log aggregation tool for Amazon-provided technologies. While this may suffice in certain circumstances, there are many other situations where developers will incorporate resources that aren’t candidates for this logging conduit.
This shortcoming, coupled with a lack of other advanced capabilities, is driving enterprises to evaluate enterprise-grade logging products.
Third Party Solutions for AWS Log Management
In response to the exponential growth and diversity of raw log data, a robust vendor ecosystem now supplies highly specialized logging products capable of consuming, storing, and organizing logs regardless of their origin—including AWS CloudWatch.
These comprehensive systems offer more flexible deployment and data retention options, and their rich user interfaces, deep analytics, and capacity for compiling log information makes them particularly compelling for enterprises fielding a diverse technology portfolio.
Third Party Log Management Features
|Centralized log repository||Support for heterogeneous log sources||Tight integration with AWS CloudWatch and CloudTrail|
|Search and sophisticated query language||Advanced analytics and out-of-the-box applications||Machine learning|
|Hybrid support regardless of whether the workload is on premise or in the cloud||Real time dashboards and reports||Real-time, event-driven notifications|
Third Party Log Management Software and Apps
|Sumo Logic||Sumo Logic offers a unified solution designed to troubleshoot modern applications using advanced analytics that analyzes logs and time-series metrics. At a glance, metrics graphs can help visualize quickly where issues lie and then facilitate drilling down into the specific logs to surface the needle in the haystack.|
|Splunk||Splunk offers makes it possible look closely at machine data and find insights that can help improve productivity, profitability, and security.|
|Elastic||The open source Elastic Stack — composed of Elasticsearch, Kibana, Logstash, and Beats — helps take data from any source, any format, and search, analyze, and visualize it in real time. And it can all be deployed on-premise or as a service using Elastic Cloud.|