Slides and descriptions from AWS Digital Training : CloudWatch Insights by Manbeen Kohli

Fully managed, highly scalable, log analytics capabilities

Alarms Operational Visabiligy in seconds ... 

Features : Works with any log sent to CloudWatch
AWS or on-premises applications 
Any being sent to CloudWatch 

Simple Powerful Querying

Writing Queries - Simple Query Language  
sample queries, common descriptions

Fields, FIlter, Java regex 

one or more log fields

numeric, string, datatype manipulation , conditional and mathematical operations

Filter : 1 or more log events    java style regex 

Stats : aggregation , Sort, Limit

Parse :  ephemeral field creation can be used later on in the query....  in this case "@severity " field

Progamatic access : to Logs Insights :   automated solutions

Example Insights Demostration : 
  • Open CloudWatch Console  and then click "insights" 

  • Select log you wish to query 
  • Specify timeframe 
  • enter a query / or use a sample query 
  • "Busiest" . IP address that has transferred the most/maximum amount of data over the last hour : 

    • fields bytes, srcAddr, dstAddr, @timestamp 

    • automatically discovers "fields" from AWS services such as ( Route53, Lambda, CloudTrail, VPC flowlogs , any json format ) list fields under Discovered fields . 

    • calculate over next 5 min and sort by desc , 

    • then filter by that IP address 

      stats sum(bytes) as mbytes by srcAddr, dstAddr, bin(5m) | sort mbytes desc | filter srcAddr='" 

  • Now max min and avg byte transfer for all IP over 5 min intervals  

    stats avg(bytes), min(bytes), max(bytes) by bin(5m)  

    • Or use Visualization to see graph of results .

    • History of queries 
    • Export to dashboard so you can see

  • CloudTrail logs ( using a sample query ) notice that "Discovered fields" from cloud trail logs 

Application logs : autodiscoverd the fields that were defined by system logs int his case sent to cloudwatch . looking for error, exception or failed 

  • And using parse command to create a field called "severity" and then use the count(*) 

Cost :  $0.005 per GB in US East (N. Virginia)

  • Pricing is based on the amount of ingested log data scanned for each query; you pay $0.005 per GB in US East (N. Virginia), with similar prices in the other regions. 

Custom Metrics to set alarms to accelerate troubleshooting 

Reference Links : 

  • No labels