Kibana Detection Rule Bug: Terraform Documents Field

Nov 14, 2025 by Alex Johnson 53 views

Introduction

This article addresses a specific bug encountered while using Terraform to manage Kibana security detection rules. Specifically, it focuses on the inability to properly configure the documents field within the actions.params block of a Kibana rule. This limitation stems from Terraform's insistence on treating the documents field as a string, whereas the Kibana API expects an object or a list of objects. This discrepancy prevents the correct automation of rule creation, leading to exported rules that deviate from the desired structure.

Describe the Bug

When creating a Kibana security detection rule with Terraform, it is impossible to properly configure the documents field in the actions.params block. Terraform forces it to be a string, but the API expects an object or list of objects. As a result, the exported rule does not match the expected structure, and we cannot automate rule creation correctly. The core issue lies in the type mismatch between what Terraform allows and what the Kibana API requires for the documents field. Terraform incorrectly interprets the documents field as a simple string, while the Kibana API expects it to be structured as either an object or, more commonly, a list of objects. This discrepancy arises when defining actions within a Kibana security detection rule using Terraform. Ideally, the documents field should accommodate complex data structures to accurately reflect the intended configuration of the rule. This limitation hinders the ability to fully automate the creation and management of Kibana rules through Terraform, as the resulting configurations are incomplete and do not align with the expected behavior. Consequently, any manual adjustments made outside of Terraform risk being overwritten or lost during subsequent Terraform deployments, leading to configuration inconsistencies. The problem underscores the need for the Terraform provider to accurately represent and support the data types and structures expected by the underlying Kibana API. Addressing this bug would significantly improve the usability and effectiveness of Terraform for managing Kibana security detection rules, ensuring seamless integration and accurate configuration management.

To Reproduce

To reproduce the behavior, follow these steps:

Use the following Terraform configuration snippet:

resource "elasticstack_kibana_security_detection_rule" "detection-rule" {
  name = "successful_ssh_connections_from_password"
  actions = [{
    id             = each.value.actions.id
    action_type_id = each.value.actions.action_type_id
    group          = each.value.actions.group
    params = {
      documents = ["{'test':'test'}"] # or "[{'test':'test'}]"
    }
  }]
}

Run terraform apply or tofu apply.

See the error:

Error: Incorrect attribute value type
element 0: attribute "params": element "documents": string required, but have object.

This error message highlights the core issue: Terraform expects a string value for the documents field, but the provided value is an object or a list of objects, which are the formats expected by the Kibana API. The configuration snippet demonstrates an attempt to define the documents field with a JSON-like string, which is a common workaround when Terraform incorrectly interprets the data type. However, this approach results in the Terraform provider throwing an error, as it strictly enforces a string type for the documents field. By running terraform apply or tofu apply, you trigger the Terraform deployment process, which then encounters the type mismatch and halts the execution. The error message provides clear feedback on the incorrect attribute value type, specifically pointing out that the documents field within the params attribute expects a string but receives an object. This discrepancy confirms the presence of the bug, as it demonstrates Terraform's inability to handle the complex data structures required for configuring Kibana security detection rules accurately. Resolving this issue requires updating the Terraform provider to correctly interpret and process the documents field as either an object or a list of objects, aligning with the expectations of the Kibana API.

Expected Behavior

Terraform should allow the documents field to accept an object or list of objects to match the expected structure of the action connector, not just a string. The applied rule should result in:

"params": {
  "documents": [
    {
      "test": "test"
    }
  ]
}

instead of forcing "documents": "[{'test':'test'}]" or stringified JSON. The expected behavior underscores the need for Terraform to accurately represent and support the data types and structures required by the Kibana API. Instead of restricting the documents field to a simple string, Terraform should recognize and accommodate objects or lists of objects. This would allow users to define complex configurations within the actions.params block, ensuring that the resulting Kibana rule matches the intended structure and behavior. When applying the Terraform configuration, the rule should be created or updated with the documents field containing a properly formatted list of objects, as shown in the example. This ensures that the action connector receives the correct data structure, enabling it to function as expected. By supporting objects and lists of objects, Terraform would provide a more seamless and intuitive experience for managing Kibana security detection rules. The discrepancy between the actual and expected behavior highlights the importance of aligning Terraform's type handling with the requirements of the underlying API. Addressing this issue would significantly improve the usability and effectiveness of Terraform for managing Kibana rules, ensuring accurate configuration and seamless integration.

Debug Output

Terraform logs when applying the resource:

resource "elasticstack_kibana_security_detection_rule" "detection-rule" {
  name = "successful_ssh_connections_from_password"
  actions = [{
    id             = each.value.actions.id
    action_type_id = each.value.actions.action_type_id
    group          = each.value.actions.group
    params = {
      documents = "[{'test':'test'}]"
    }
  }]
}

module.elastic-infomaniak.elasticstack_fleet_integration.integration["fim"]: Refreshing state...
module.elastic-infomaniak.elasticstack_kibana_security_detection_rule.detection-rule["successful_ssh_connections_from_password"]: Refreshing state...
module.elastic-infomaniak.elasticstack_kibana_security_detection_rule.detection-rule["successful_ssh_connections_from_password"]: Modifying... [id=default/969730e8-5aeb-4424-8717-a1830804b002]
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

The debug output provides valuable insights into how Terraform is interpreting and processing the resource configuration. Specifically, it shows the state of the elasticstack_kibana_security_detection_rule resource during the Terraform apply process. The configuration snippet within the debug output confirms that the documents field is being treated as a string, as indicated by the double quotes surrounding the JSON-like string value. This aligns with the reported bug, where Terraform incorrectly enforces a string type for the documents field, preventing the use of objects or lists of objects. The subsequent log messages indicate that Terraform is refreshing the state of the resource and attempting to modify it. However, due to the type mismatch, the modification process may not result in the desired configuration, as the documents field will not be properly formatted. The "Apply complete!" message suggests that the Terraform deployment has finished without any explicit errors. However, this does not guarantee that the resulting Kibana rule is configured correctly. The screenshot further illustrates the issue, potentially showing the incorrect representation of the documents field in the Kibana UI or API. Analyzing the debug output is crucial for understanding how Terraform interacts with the Kibana API and identifying any discrepancies between the intended configuration and the actual state of the resource. Addressing the bug requires modifying the Terraform provider to correctly interpret and process the documents field, ensuring that it aligns with the expected data structure of the Kibana API.

Versions

OS: Linux
Opentofu Version: 1.10.7
Provider version: elasticstack_v0.12.1
Elasticsearch Version: 8.18.2

The specified versions provide essential context for understanding the environment in which the bug was encountered. The OS (Linux) indicates the operating system on which Terraform and the Elasticsearch stack are running. This information is relevant because certain issues may be specific to certain operating systems or configurations. The Opentofu Version (1.10.7) specifies the version of the Opentofu infrastructure-as-code tool being used. This version is significant because bugs and features can vary between different versions of Opentofu. The Provider version (elasticstack_v0.12.1) indicates the version of the elasticstack Terraform provider, which is responsible for interacting with the Elasticsearch and Kibana APIs. This version is particularly important because the bug is suspected to reside within the provider itself. The Elasticsearch Version (8.18.2) specifies the version of the Elasticsearch cluster being managed by Terraform. This version is relevant because the Kibana API and data structures may differ between different versions of Elasticsearch. By providing these version numbers, it becomes easier to reproduce the bug and identify any known issues or compatibility problems related to the specific versions being used. This information is crucial for troubleshooting and resolving the bug effectively.

Additional Context

Tested various ways to pass the documents value: string, JSON string, list of strings. Only string is accepted, which prevents creating valid actions.
Manually created rules in Kibana have documents as a list of objects, e.g.:
```
"params": {
  "documents": [
    {"test": "test"}
  ]
}
```
This appears to be a bug in the Terraform provider.

The additional context provides further details and observations related to the bug. The first point highlights the attempts made to work around the issue by testing various ways to pass the documents value. Specifically, it mentions trying different data types such as string, JSON string, and list of strings. However, only the string type was accepted by Terraform, which prevented the creation of valid actions. This observation reinforces the suspicion that Terraform is incorrectly enforcing a string type for the documents field, despite the Kibana API expecting an object or list of objects. The second point compares the Terraform-managed rules with manually created rules in Kibana. It notes that manually created rules have the documents field configured as a list of objects, as expected by the Kibana API. This comparison further emphasizes the discrepancy between Terraform's behavior and the intended configuration. The example JSON snippet illustrates the correct structure of the documents field in a Kibana rule, where it contains a list of objects with key-value pairs. The final point concludes that the issue is likely a bug in the Terraform provider. This conclusion is based on the observations and attempts made to work around the issue, as well as the comparison with manually created rules in Kibana. By providing this additional context, it becomes clearer that the bug is not a result of incorrect configuration or usage, but rather a limitation within the Terraform provider itself. Addressing this bug requires updating the provider to correctly interpret and process the documents field, ensuring that it aligns with the expectations of the Kibana API.

Conclusion

In conclusion, the inability to set the documents field as an object in Kibana detection rules via Terraform is a bug that needs attention. The Terraform provider should be updated to allow objects or lists of objects for the documents field, aligning with Kibana's API requirements. This will enable users to automate the creation of valid Kibana security detection rules effectively.

For more information on Terraform and its providers, visit the Terraform documentation.This documentation provides detailed information about Terraform's features, usage, and available providers, including the Elastic Stack provider. You can explore the documentation to understand how to configure and manage various resources within the Elastic Stack using Terraform.