Slides and descriptions from AWS Digital Training : CloudWatch Insights by Manbeen Kohli

Fully managed, highly scalable, log analytics capabilities


Alarms Operational Visabiligy in seconds ... 


Features : Works with any log sent to CloudWatch
AWS or on-premises applications 
Any being sent to CloudWatch 


Simple Powerful Querying


Writing Queries - Simple Query Language  
sample queries, common descriptions

Fields, FIlter, Java regex 



one or more log fields

numeric, string, datatype manipulation , conditional and mathematical operations

Filter : 1 or more log events    java style regex 

Stats : aggregation , Sort, Limit


Parse :  ephemeral field creation can be used later on in the query....  in this case "@severity " field



Progamatic access : to Logs Insights :   automated solutions


Example Insights Demostration : 
  • Open CloudWatch Console  and then click "insights" 


  • Select log you wish to query 
  • Specify timeframe 
  • enter a query / or use a sample query 
  • "Busiest" . IP address that has transferred the most/maximum amount of data over the last hour : 

    • fields bytes, srcAddr, dstAddr, @timestamp 



    • automatically discovers "fields" from AWS services such as ( Route53, Lambda, CloudTrail, VPC flowlogs , any json format ) list fields under Discovered fields . 

    • calculate over next 5 min and sort by desc , 



    • then filter by that IP address 

      stats sum(bytes) as mbytes by srcAddr, dstAddr, bin(5m) | sort mbytes desc | filter srcAddr='18.214.60.60" 




  • Now max min and avg byte transfer for all IP over 5 min intervals  

    stats avg(bytes), min(bytes), max(bytes) by bin(5m)  


    • Or use Visualization to see graph of results .


    • History of queries 
    • Export to dashboard so you can see


  • CloudTrail logs ( using a sample query ) notice that "Discovered fields" from cloud trail logs 


Application logs : autodiscoverd the fields that were defined by system logs int his case sent to cloudwatch . looking for error, exception or failed 

  • And using parse command to create a field called "severity" and then use the count(*) 


Cost :  $0.005 per GB in US East (N. Virginia)

  • Pricing is based on the amount of ingested log data scanned for each query; you pay $0.005 per GB in US East (N. Virginia), with similar prices in the other regions. 


Custom Metrics to set alarms to accelerate troubleshooting 


Reference Links : 


  • No labels