Fix #25: Support for sidecars and init containers. (#91)

* Fix #25: Support for sidecars and init containers. Signed-off-by: Travis Illig <[email protected]> * awsSigningSidecar is now an extraContainer. Signed-off-by: Travis Illig <[email protected]> * Corrected comment. Signed-off-by: Travis Illig <[email protected]> * Fixed comment typo. Signed-off-by: Travis Illig <[email protected]> * Updated README for v12. Signed-off-by: Travis Illig <[email protected]> * Fixed markdownlint line length issue. Signed-off-by: Travis Illig <[email protected]>
kokuwaio · Sep 20, 2021 · 0354ba5 · 0354ba5
1 parent 5b47ef4
commit 0354ba5
Show file tree

Hide file tree

Showing 4 changed files with 65 additions and 80 deletions.
diff --git a/charts/fluentd-elasticsearch/Chart.yaml b/charts/fluentd-elasticsearch/Chart.yaml
@@ -1,6 +1,6 @@
 apiVersion: v2
 name: fluentd-elasticsearch
-version: 11.15.0
+version: 12.0.0
 appVersion: 3.3.0
 type: application
 home: https://www.fluentd.org/

diff --git a/charts/fluentd-elasticsearch/README.md b/charts/fluentd-elasticsearch/README.md
@@ -63,15 +63,6 @@ The following table lists the configurable parameters of the Fluentd elasticsear
 | `configMaps.useDefaults.monitoringConf`              | Use default monitoring.conf                                                    | `true`                                             |
 | `configMaps.useDefaults.outputConf`                  | Use default output.conf                                                        | `true`                                             |
 | `extraConfigMaps`                                    | Add additional Configmap or overwrite disabled default                         | `{}`                                               |
-| `awsSigningSidecar.enabled`                          | Enable AWS request signing sidecar                                             | `false`                                            |
-| `awsSigningSidecar.extraEnvs`                        | List of env vars that are added to the AWS signing sidecar pods                | `[]`                                               |
-| `awsSigningSidecar.resources`                        | AWS Sidecar resources                                                          | `{}`                                               |
-| `awsSigningSidecar.network.port`                     | AWS Sidecar exposure port                                                      | `8080`                                             |
-| `awsSigningSidecar.network.address`                  | AWS Sidecar listen address                                                     | `localhost`                                        |
-| `awsSigningSidecar.network.remoteReadTimeoutSeconds` | AWS Sidecar socket read timeout when talking to ElasticSearch                  | `15`                                               |
-| `awsSigningSidecar.image.repository`                 | AWS signing sidecar repository image                                           | `abutaha/aws-es-proxy`                             |
-| `awsSigningSidecar.image.tag`                        | AWS signing sidecar repository tag                                             | `v1.0`                                             |
-| `awsSigningSidecar.args`                             | Additional command-line arguments for the AWS signing sidecar container        | `[]`                                               |
 | `elasticsearch.auth.enabled`                         | Elasticsearch Auth enabled                                                     | `false`                                            |
 | `elasticsearch.auth.user`                            | Elasticsearch Auth User                                                        | `null`                                             |
 | `elasticsearch.auth.password`                        | Elasticsearch Auth Password                                                    | `null`                                             |
@@ -127,6 +118,8 @@ The following table lists the configurable parameters of the Fluentd elasticsear
 | `fluentdArgs`                                        | Fluentd args                                                                   | `--no-supervisor -q`                               |
 | `fluentdLogFormat`                                   | Fluentd output log format in the default system.conf (either "text" or "json") | `text`                                             |
 | `secret`                                             | List of env vars that are set from secrets and added to the fluentd pods       | `[]`                                               |
+| `extraContainers`                                    | Add sidecar containers to each pod in the daemonset                            | `[]`                                               |
+| `extraInitContainers`                                | Add init containers to each pod in the daemonset                               | `[]`                                               |
 | `extraVolumeMounts`                                  | Mount extra volume, required to mount ssl certificates when ES has tls enabled | `[]`                                               |
 | `extraVolumes`                                       | Extra volume                                                                   | `[]`                                               |
 | `fluentConfDir`                                      | Specify where to mount fluentd location                                        | `/etc/fluent/config.d`                             |
@@ -209,7 +202,8 @@ extraVolumes: |
 
 ### AWS Elasticsearch Domains
 
-AWS Elasticsearch requires requests to upload data to be signed using [AWS Signature V4](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html). In order to support this, you can add `awsSigningSidecar: {enabled: true}` to your configuration. This results in a sidecar container being deployed that proxies all requests to your Elasticsearch domain and signs them appropriately.
+AWS Elasticsearch requires requests to upload data to be signed using [AWS Signature V4](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html). In order to support this, you can add a sidecar to the `extraContainers` configuration. An example is provided in `values.yaml`. This results in a sidecar container being deployed that proxies all requests to your Elasticsearch domain
+and signs them appropriately.
 
 ## Upgrading
 
@@ -354,3 +348,7 @@ In this version elasticsearch template in `output.conf` configmap was expanded t
 ### From a version < 10.0.0 to version => 11.0.0
 
 The chart requires now Helm >= 3.0.0 and Kubernetes >= 1.16.0
+
+### From a version < 11.0.0 to version => 12.0.0
+
+If you were using `awsSigningSidecar` to set up an AWS signing sidecar proxy, this has now moved to the `extraContainers` property. The example in the `values.yaml` shows the equivalent AWS signing sidecar configuration expressed now as `extraContainers`.
diff --git a/charts/fluentd-elasticsearch/templates/daemonset.yaml b/charts/fluentd-elasticsearch/templates/daemonset.yaml
@@ -44,11 +44,7 @@ spec:
           value: {{ .Values.fluentdArgs | quote }}
         {{- if .Values.elasticsearch.setOutputHostEnvVar }}
         - name: OUTPUT_HOSTS
-          {{- if .Values.awsSigningSidecar.enabled }}
-          value: "{{ .Values.awsSigningSidecar.network.address }}:{{ .Values.awsSigningSidecar.network.port }}"
-          {{- else }}
           value: "{{- join "," .Values.elasticsearch.hosts }}"
-          {{- end }}
         {{- end }}
         - name: OUTPUT_PATH
           value: {{ .Values.elasticsearch.path | quote }}
@@ -101,11 +97,7 @@ spec:
           value: {{ .Values.elasticsearch.template.useLegacy | quote }}
 {{- end }}
         - name: OUTPUT_SCHEME
-          {{- if .Values.awsSigningSidecar.enabled }}
-          value: 'http'
-          {{- else }}
           value: {{ .Values.elasticsearch.scheme | quote }}
-          {{- end }}
         - name: OUTPUT_TYPE
           value: {{ .Values.elasticsearch.outputType | quote }}
         - name: OUTPUT_SSL_VERIFY
@@ -210,33 +202,13 @@ spec:
             protocol: {{ $port.protocol }}
 {{- end }}
 {{- end }}
-      {{- if .Values.awsSigningSidecar.enabled }}
-      - name: {{ include "fluentd-elasticsearch.fullname" . }}-aws-es-proxy
-        image: {{ .Values.awsSigningSidecar.image.repository }}:{{ .Values.awsSigningSidecar.image.tag }}
-        imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
-        args:
-          - "-endpoint"
-          - "{{ .Values.elasticsearch.scheme }}://{{ index .Values.elasticsearch.hosts 0 }}"
-          - "-listen"
-          - "{{ .Values.awsSigningSidecar.network.address }}:{{ .Values.awsSigningSidecar.network.port }}"
-          - "-timeout"
-          - "{{ .Values.awsSigningSidecar.network.remoteReadTimeoutSeconds }}"
-{{- range $arg := .Values.awsSigningSidecar.args }}
-          - "{{ $arg }}"
+{{- if .Values.extraContainers }}
+{{ toYaml .Values.extraContainers | indent 6 }}
+{{- end }}
+{{- if .Values.extraInitContainers }}
+      initContainers:
+{{ toYaml .Values.extraInitContainers | indent 6 }}
 {{- end }}
-        env:
-        - name: PORT_NUM
-          value: {{ .Values.awsSigningSidecar.network.port | quote }}
-        {{- if .Values.awsSigningSidecar.extraEnvs }}
-        {{- range $env := .Values.awsSigningSidecar.extraEnvs }}
-        - name: {{ $env.name }}
-          value: {{ $env.value | quote }}
-        {{- end }}
-        {{- end }}
-        resources:
-{{ toYaml .Values.awsSigningSidecar.resources | indent 10 }}
-        volumeMounts:
-      {{- end }}
       terminationGracePeriodSeconds: 30
       volumes:
       - name: varlog

diff --git a/charts/fluentd-elasticsearch/values.yaml b/charts/fluentd-elasticsearch/values.yaml
@@ -12,33 +12,6 @@ image:
   # pullSecrets:
   #   - myRegistrKeySecretName
 
-## If using AWS Elasticsearch, all requests to ES need to be signed regardless of whether
-## one is using Cognito or not. By setting this to true, this chart will install a sidecar
-## proxy that takes care of signing all requests being sent to the AWS ES Domain.
-awsSigningSidecar:
-  enabled: false
-  # You can configure some features of AWS ES Proxy by passing specific environment
-  # variables. E.g. AWS EKS IRSA is supported by providing AWS_ROLE_ARN and
-  # AWS_WEB_IDENTITY_TOKEN_FILE
-  extraEnvs: []
-  # name: FOO
-  # value: BAR
-  resources: {}
-  # limits:
-  #   cpu: 100m
-  #   memory: 500Mi
-  # requests:
-  #   cpu: 100m
-  #   memory: 200Mi
-  network:
-    port: 8080
-    address: localhost
-    remoteReadTimeoutSeconds: 15
-  image:
-    repository: abutaha/aws-es-proxy
-    tag: v1.0
-  args: []
-
 # Specify to use specific priorityClass for pods
 # ref: https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
 # If a Pod cannot be scheduled, the scheduler tries to preempt (evict) lower priority
@@ -74,7 +47,9 @@ elasticsearch:
       key: null
   includeTagKey: true
   setOutputHostEnvVar: true
-  # If setOutputHostEnvVar is false this value is ignored
+  # If setOutputHostEnvVar is false the hosts value is ignored
+  # If you are configuring an AWS signing sidecar, hosts should be the network
+  # address of the sidecar, like "localhost:8080"
   hosts: ["elasticsearch-client:9200"]
   indexName: "fluentd"
   logstash:
@@ -124,6 +99,7 @@ elasticsearch:
       }
 
   path: ""
+  # If you are configuring an AWS signing sidecar, scheme should be "http"
   scheme: "http"
   sslVerify: true
   sslVersion: "TLSv1_2"
@@ -386,12 +362,51 @@ extraConfigMaps: {}
   #   </system>
 
 extraVolumes: []
-#   - name: es-certs
-#     secret:
-#       defaultMode: 420
-#       secretName: es-certs
+# - name: es-certs
+#   secret:
+#     defaultMode: 420
+#     secretName: es-certs
 
 extraVolumeMounts: []
-#   - name: es-certs
-#     mountPath: /certs
-#     readOnly: true
+# - name: es-certs
+#   mountPath: /certs
+#   readOnly: true
+
+## Sidecar containers
+##
+## If using AWS Elasticsearch, all requests to ES need to be signed regardless
+## of whether one is using Cognito or not. The example here shows how to
+## configure a sidecar proxy that takes care of signing all requests being sent
+## to the AWS ES Domain.
+extraContainers: []
+# - name: aws-es-proxy
+#   image: abutaha/aws-es-proxy:v1.0
+#   imagePullPolicy: IfNotPresent
+#   args:
+#   - "-endpoint"
+#   # Put your ElasticSearch host here as the endpoint and set...
+#   # elasticsearch.hosts location to the sidecar address, e.g. localhost:8080
+#   # elasticsearch.scheme to http
+#   - "http://elasticsearch-client:9200"
+#   - "-listen"
+#   - "localhost:8080"
+#   - "-timeout"
+#   - "15"
+#   # You can configure some features of AWS ES Proxy by passing specific
+#   # environment variables. E.g. AWS EKS IRSA is supported by providing
+#   # AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE
+#   env:
+#   - name: PORT_NUM
+#     value: "8080"
+#   resources:
+#     limits:
+#       cpu: 100m
+#       memory: 500Mi
+#     requests:
+#       cpu: 100m
+#       memory: 200Mi
+
+extraInitContainers: []
+# - name: do-something
+#   image: busybox
+#   command: ['do', 'something']