Monitoring Serverless Architectures in AWS
An article that discusses the methods for auditing and monitoring serverless architectures in AWS.
by Bo Bayles April 7, 2017 Technical Topics
The introduction of serverless architectures is a positive development from a security perspective – splitting up services into single-purpose functions with well-defined inputs and outputs helps reduce exposure to many types of threats. However, the security of serverless architectures is an under-studied topic; many practices and tools that are available in more traditional architectures don’t apply.
This article will discuss methods of auditing and monitoring of AWS Lambda functions (a key component of serverless architectures in AWS) and how the Observable Networks service puts these methods into practice.
Lambda security topics
We can break down questions about Lambda security into a few different areas:
- Access to the function: Who/what can modify and/or invoke a function?
- Access from the function: What can the function access when it is running?
- Unintended behavior: Can the function be made to do something unexpected?
For each area there is a question of auditing, or looking to see whether configurations match expectations, and monitoring, or keeping track of activities on an ongoing basis.
Auditing
The Lambda docs describe how to use the IAM API and Lambda API for auditing policies related to function access. It’s important to audit policies periodically to check:
- Whether some policies are outdated – they made sense in the past, but haven’t been updated to reflect changes (e.g., a user that has since moved to another team still has rights to change a Lambda function’s code).
- Whether some policies allow access to more resources than necessary (e.g., a Lambda function that can write to any S3 bucket).
The IAM API can be used to show which users (or roles) in an AWS account have the ability to change a Lambda function or its configuration. For example, we could query the IAM API to see who has the UpdateFunctionCode permission – this would show who is allowed to change the code a function executes.
The Lambda API can be used to show who (or what) is allowed to invoke a function. It’s common to allow other AWS services like S3 or API Gateway call a function. An example of something to look for would be cross-account access for S3 – the AWS docs provide this warning for this problem (reference):
If you add a permission for the Amazon S3 principal without providing the source ARN, any AWS account that creates a mapping to your function ARN can send events to invoke your Lambda function from Amazon S3.
The Lambda and IAM APIs together can show what a Lambda function is allowed to do. For example, to see what permissions a particular Lambda function is granted (e.g., writing to an S3 bucket) you can use the Lambda API’s GetFunction call to determine which role to look up with the IAM API.
Monitoring
The Lambda docs describe how to use CloudTrail and CloudWatch for monitoring.
CloudTrail tracks API activities. We can use CloudTrail logs to both look for changes to functions (e.g., the UpdateFunctionCode call described above) and changes made by functions. This helps keep track of access between audits.
CloudWatch Logs tracks function invocations and their output. This can be used for billing purposes (they include information about duration and memory usage, which are part of Lambda pricing) as well as for monitoring purposes. For example, a function can be made to write its input to the log – this could be used to help look for injection attacks.
CloudWatch Metrics also tracks function usage – the number of invocations, the duration of each run, and other metrics. Manual alarms can be set for things like excessive invocations. This can be used to monitor for unintended behavior and abuse – for example, if the function can be invoked from the outside, an alarm could show whether it is being abused and wasting resources.