Coolify: Exposing Environment Variables In Build Logs

by Alex Johnson 54 views

Introduction

In this article, we delve into a critical bug reported in Coolify, a self-hosted platform designed for easy deployment of web applications and services. The issue revolves around environment variables being exposed in build debug logs, which raises significant security concerns. This problem was observed during a Docker Compose deployment and affects how environment variables are treated, particularly those intended to be secrets. Understanding the nuances of this bug and its implications is crucial for anyone using Coolify to manage their deployments. The goal is to provide a comprehensive overview of the bug, its causes, steps to reproduce it, and potential solutions. By addressing this issue, we can ensure that Coolify remains a secure and reliable platform for developers.

Understanding the Bug: Environment Variable Exposure

The core of the problem lies in how Coolify handles environment variables during the build process. The user interface (UI) strongly suggests that environment variables should be treated as secrets. However, it does not explicitly state that they are only redacted from logs if they are marked as 'locked.' This ambiguity can lead to misunderstandings and potential security vulnerabilities. When deploying applications, Coolify generates a .env file containing these variables. The contents of this file are then exposed in the debug logs, defeating the purpose of treating these variables as secrets. Specifically, the debug logs reveal the environment variables in plain text, making them accessible to anyone who has access to these logs. This behavior is particularly concerning because it contradicts the expectation that sensitive information should be protected.

Root Cause Analysis

The exposure of environment variables can be traced back to the ApplicationDeploymentJob.php file within the Coolify codebase. Specifically, the script creates a .env file with runtime variables during the build phase. Subsequently, a command is executed to output the contents of this .env file to the logs. This command, found in the execute_remote_command function, uses a redaction process that only applies to variables marked as 'locked.' Consequently, any environment variables that are not explicitly locked are left unredacted and exposed in the logs. The issue is further compounded by the lack of clear documentation or UI indicators that highlight the importance of locking variables to ensure they are redacted from the build logs. This oversight creates a false sense of security, leading users to believe that all environment variables are protected when, in reality, only locked variables receive this protection. Therefore, the root cause of the problem is a combination of inadequate redaction logic and insufficient user guidance.

Detailed Error Analysis

To fully grasp the scope of the bug, it's essential to examine the specific commands and functions involved. The process begins with the creation of a .env file containing runtime variables for the build phase. This step is crucial for setting up the application environment but also introduces the risk of exposing sensitive information. The command docker exec is then used to execute a script that outputs the contents of the .env file. This command is particularly problematic because it directly reveals the values of all environment variables, regardless of their sensitivity. The redact_sensitive_info function is intended to prevent this exposure. However, it only redacts variables that are explicitly locked. The vulnerability lies in the fact that any unlocked variables are not subject to this redaction, resulting in their values being printed in plain text in the debug logs. This behavior contradicts the expectation that all environment variables should be treated as secrets, especially given the UI's suggestion that they are protected. The lack of comprehensive redaction makes the system vulnerable to unintentional exposure of sensitive data.

Steps to Reproduce the Bug

To replicate the bug, follow these steps:

  1. Set up a Repository: Create a repository with a Dockerfile and a docker-compose.yaml file. The docker-compose.yaml file should define services that use environment variables. Ensure that the application code in the app service builds successfully during deployment.
  2. Configure docker-compose.yaml: Define services in your docker-compose.yaml file, such as an app service and a db service (e.g., PostgreSQL). Use environment variables for configuration, such as database credentials.
  3. Create a Coolify Project: Create a new project in Coolify, linking it to your repository. Use "Docker Compose" as the build pack.
  4. Add Environment Variables: In the Coolify UI, navigate to the environment variables tab in the Configuration section. Add the necessary environment variables, ensuring that the "Available at runtime" option is checked. Importantly, do not lock these variables.
  5. Deploy the Application: Deploy the application through Coolify.
  6. Inspect Debug Logs: Once the deployment is complete, access the debug logs. Look for lines similar to:
    • [CMD]: docker exec <container id> bash -c 'cat /artifacts/build-time.env'
    • [CMD]: docker exec <container id> bash -c 'cat /artifacts/<container id>/.env'
  7. Verify Exposure: Examine the logs immediately following these lines. You should observe that all build-time and runtime environment variables are printed in plaintext.

By following these steps, you can consistently reproduce the bug and verify that environment variables are indeed exposed in the build logs when they are not locked.

Impact and Security Implications

The exposure of environment variables in build logs can have severe security implications. Environment variables often contain sensitive information such as API keys, database passwords, and other credentials necessary for the application to function correctly. When these variables are exposed, unauthorized individuals can gain access to critical systems and data. This can lead to data breaches, service disruptions, and other malicious activities. Furthermore, the exposure of environment variables can also violate compliance regulations, such as GDPR and HIPAA, which require organizations to protect sensitive data. Therefore, it is crucial to address this bug promptly to prevent potential security breaches and ensure the confidentiality of sensitive information. The risk is amplified by the fact that the UI suggests these variables are treated as secrets, leading users to a false sense of security.

Proposed Solutions and Mitigations

To address the bug and mitigate the risk of environment variable exposure, several solutions can be implemented:

  1. Comprehensive Redaction: Update the redact_sensitive_info function to redact all environment variable values, regardless of whether they are locked or not. This will ensure that no sensitive information is inadvertently exposed in the build logs.
  2. UI Enhancements: Improve the UI to clearly indicate that environment variables are only redacted from the logs if they are locked. Provide clear instructions on how to lock variables and the importance of doing so.
  3. Documentation Updates: Update the documentation to reflect the behavior of environment variable redaction accurately. Explain the importance of locking variables and the potential risks of not doing so.
  4. Secure Storage: Implement secure storage mechanisms for environment variables, such as using encrypted files or dedicated secret management tools. This will add an extra layer of protection against unauthorized access.
  5. Log Rotation and Access Control: Implement robust log rotation policies to ensure that old logs are regularly deleted. Restrict access to the logs to authorized personnel only.

By implementing these solutions, Coolify can significantly reduce the risk of environment variable exposure and enhance the security of the platform.

Additional Information

Coolify Version

v4.0.0-beta.442

Are you using Coolify Cloud?

No (self-hosted)

Operating System and Version (self-hosted)

Ubuntu 25.04 (ami-08451de661ab8fec8)

Conclusion

The bug involving the exposure of environment variables in Coolify's build debug logs poses a significant security risk. The current redaction logic, which only applies to locked variables, is insufficient to protect sensitive information. By implementing comprehensive redaction, enhancing the UI, updating documentation, and adopting secure storage mechanisms, Coolify can effectively mitigate this risk and ensure the confidentiality of environment variables. Addressing this issue is crucial for maintaining the integrity and trustworthiness of the platform. Remember to lock your sensitive environment variables in Coolify to ensure they are redacted from logs. For more information on securing environment variables, you can refer to external resources such as the OWASP guidelines on secure deployment strategies. By taking these steps, developers can confidently use Coolify to manage their deployments without compromising security.