Merge pull request #360 from tsloughter/readme-updates

Readme updates
open-telemetry · Feb 15, 2022 · feb17ea · feb17ea
2 parents ea7649c + f1a9daf
commit feb17ea
Show file tree

Hide file tree

Showing 12 changed files with 238 additions and 82 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### [SDK]
+
+#### Added
+
+- [Simpler configuration of span processors](https://github.com/open-telemetry/opentelemetry-erlang/pull/357)
+
 ## 1.0.1 - 2022-02-03
 
 ### [API]

diff --git a/apps/opentelemetry/README.md b/apps/opentelemetry/README.md
@@ -0,0 +1,170 @@
+# Erlang/Elixir OpenTelemetry SDK
+
+[![Hex.pm](https://img.shields.io/hexpm/v/opentelemetry?label=SDK&style=for-the-badge)](https://hex.pm/packages/opentelemetry)
+[![EEF Observability WG
+project](https://img.shields.io/badge/EEF-Observability-black?style=for-the-badge)](https://github.com/erlef/eef-observability-wg)
+[![GitHub Workflow Status](https://img.shields.io/github/workflow/status/open-telemetry/opentelemetry-erlang/Erlang?style=for-the-badge)](https://github.com/open-telemetry/opentelemetry-erlang/actions)
+
+The [SDK](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.8.0/specification/trace/sdk.md) is an implementation of the [OpenTelemetry
+API](https://hex.pm/packages/opentelemetry_api) and should be included in your
+final deployable artifact (usually an OTP Release).
+
+## Configuration
+
+The SDK starts up its supervision tree on boot, so initial configuration must be
+done through the Application or [OS environment
+variables](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.8.0/specification/sdk-configuration.md).
+The following example configurations show configuring the SDK to use the batch
+span processor which then exports through the [OpenTelemetry
+Protocol](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.8.0/specification/protocol/otlp.md)
+over HTTP to `http://localhost:4318`, encoding the Spans with protobufs.
+
+``` erlang
+[
+ {opentelemetry,
+  [{span_processor, batch},
+   {exporter, otlp}]},
+
+ {opentelemetry_exporter, 
+  [{otlp_protocol, http_protobuf},
+   {otlp_endpoint, "http://localhost:4318"}]}]}
+].
+```
+
+``` elixir
+config :opentelemetry, 
+  span_processor: :batch,
+  exporter: :otlp
+
+config :opentelemetry_exporter,
+  otlp_protocol: :http_protobuf
+  otlp_endpoint: "http://localhost:4318"
+```
+
+The following sections detail the available SDK options and how to configure
+them through either the OS or Application environment.
+
+### Automatically Created Tracers
+
+Spans are created by Tracers and each Tracer is associated with a [name and
+version](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.8.0/specification/trace/api.md#get-a-tracer).
+The name and version can be anything, but the macros provided by the API will always use
+a Tracer with name/version of the OTP Application the macro is used in. To
+facilitate this a Tracer is created for each available Application when the SDK
+boots. It relies on mapping modules to Applications, so if there is a huge
+number of modules in a Release the time to create all these Tracers can become
+more than you want to wait on boot.
+
+| OS                              | Application                | Default | Type    |
+|:--------------------------------|:---------------------------|:--------|:--------|
+| OTEL_CREATE_APPLICATION_TRACERS | create_application_tracers | true    | boolean |
+
+Disabling the creation of Tracers at start-up means when the macros are used to
+start Spans the name and version of the Tracer creating those Spans will be
+undefined, but everything will continue to work as before.
+
+### Propagators
+
+Propagators define how to inject and extract context in requests to or from
+external services. A propagator that works on key/value pairs of text, like HTTP
+headers, can be any module that implements the `otel_propagator_text_map`
+behaviour. There are 4 built-in propagators that are part of the API:
+
+- [tracecontext](https://w3c.github.io/trace-context/): Encodes the Trace
+  context as headers `traceparent` and `tracestate`.
+- [baggage](https://w3c.github.io/baggage/): Encodes
+  [baggage](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.8.0/specification/baggage/api.md)
+  elements as a comma delimited list under header key `baggage`.
+- [b3multi](https://github.com/openzipkin/b3-propagation#multiple-headers): The
+  propagation format from [Zipkin](https://zipkin.io/) that breaks the context across multiple
+  headers, `X-B3-TraceId`, `X-B3-SpanId` and `X-B3-Sampled`.
+- [b3](https://github.com/openzipkin/b3-propagation#single-header):  The
+  propagation format from Zipkin that encodes to a single header `b3`.
+
+| OS               | Application          | Default                | Type                                                             |
+|:-----------------|:---------------------|:-----------------------|:-----------------------------------------------------------------|
+| OTEL_PROPAGATORS | text_map_propagators | [tracecontext,baggage] | List of propagators (`tracecontext`, `baggage`, `b3multi`, `b3`) |
+
+
+### Batch Span Processor
+
+[Span
+processors](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#span-processor)
+are invoked when a Span is started and when it ends. The default processor is
+the batch processor (`otel_batch_processor`) -- also available is the simple
+processor (`otel_simple_processor`) which blocks on every finished Span until it
+is exported.
+
+The batch processor waits `bsp_scheduled_delay_ms` milliseconds or until the buffer of
+finished Spans reaches `bsp_max_queue_size` and then exports the spans together.
+The export is canceled if it takes longer than `bsp_exporting_timeout_ms`.
+
+| OS                             | Application              | Default | Type    |
+|:-------------------------------|:-------------------------|:--------|:--------|
+| OTEL_BSP_SCHEDULE_DELAY_MILLIS | bsp_scheduled_delay_ms   | 5000    | integer |
+| OTEL_BSP_EXPORT_TIMEOUT_MILLIS | bsp_exporting_timeout_ms | 30000   | integer |
+| OTEL_BSP_MAX_QUEUE_SIZE        | bsp_max_queue_size       | 2048    | integer |
+
+### Resource Detectors
+
+[Resources](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.8.0/specification/overview.md#resources)
+capture information about the environment the program is running in. For
+example, a process producing telemetry that is running in a container on
+Kubernetes has a Pod name, it is in a namespace and possibly is part of a
+Deployment which also has a name. All three of these attributes can be included
+in the Resource.
+
+| OS                             | Application               | Default                                        | Type            |
+|:-------------------------------|:--------------------------|:-----------------------------------------------|:----------------|
+| OTEL_RESOURCE_DETECTORS        | resource_detectors        | [otel_resource_env_var, otel_resource_app_env] | list of modules |
+| OTEL_RESOURCE_DETECTOR_TIMEOUT | resource_detector_timeout | 5000                                           | integer         |
+|                                |                           |                                                |                 |
+
+The default detectors read resource attributes from the OS environment variable
+`OTEL_RESOURCE_ATTRIBUTES` and Application environment variable `resource`.
+
+### Span Limits
+
+The number of Attributes, Events and Links on a Span are limited, as well as the
+length of an Attribute's value. When the limit is reached any additional
+Attributes, Events or Links are dropped and Attribute values larger than the
+length limit are truncated.
+
+| OS                                     | Application                  | Default  | Type                    |
+|:---------------------------------------|:-----------------------------|:---------|:------------------------|
+| OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT        | attribute_count_limit        | 128      | integer                 |
+| OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT | attribute_value_length_limit | infinity | integer &#124; infinity |
+| OTEL_SPAN_EVENT_COUNT_LIMIT            | event_count_limit            | 128      | integer                 |
+| OTEL_SPAN_LINK_COUNT_LIMIT             | link_count_limit             | 128      | integer                 |
+| OTEL_EVENT_ATTRIBUTE_COUNT_LIMIT       | attribute_per_event_limit    | 128      | integer                 |
+| OTEL_LINK_ATTRIBUTE_COUNT_LIMIT        | attribute_per_link_limit     | 128      | integer                 
+
+Read more in the specification about [Span
+limits](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#span-limits)
+and [Attribute
+limits](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/common/common.md#attribute-limits).
+
+### Span Sweeper
+
+Because of Erlang's "Let It Crash" philosophy it is possible that at some point
+a Span will never have `end_span` called. To handle these cases the SDK includes
+a process that will periodically check the active Span table for Spans that are
+older than the `span_ttl` and will drop or end the Span depending on the
+configured `strategy`. The `failed_attribute_and_end_span` strategy will add the
+attribute `finished_by_sweeper` with value `true` to the Span before calling
+`end_span`.
+
+The sweeper's configuration is a map under environment key `sweeper`.
+
+| OS                             | Application  | Default | Type                                                                 |
+|:-------------------------------|:-------------|:---------|:-----------------------------------------------------------------------------|
+| OTEL_SPAN_SWEEPER_INTERVAL     | interval     | 600000   | integer &#124; infinity                                                      |
+| OTEL_SPAN_SWEEPER_STRATEGY     | strategy     | drop     | drop &#124; end_span &#124; failed_attribute_and_end_span &#124; fun/1 |
+| OTEL_SPAN_SWEEPER_SPAN_TTL     | span_ttl     | 1800000  | integer &#124; infinity                                                      |
+| OTEL_SPAN_SWEEPER_STORAGE_SIZE | storage_size | infinity | integer &#124; infinity                                                      |
+
+## Contributing
+
+Read OpenTelemetry project [contributing
+guide](https://github.com/open-telemetry/community/blob/main/CONTRIBUTING.md)
+for general information about the project.
diff --git a/docs.config → apps/opentelemetry/docs.config b/docs.config → apps/opentelemetry/docs.config
@@ -1,4 +1,4 @@
 {source_url, <<"https://github.com/open-telemetry/opentelemetry-erlang">>}.
-{extras, [<<"README.md">>, <<"LICENSE">>, <<"VERSIONING.md">>]}.
+{extras, [<<"apps/opentelemetry/README.md">>, <<"apps/opentelemetry/LICENSE">>, <<"VERSIONING.md">>]}.
 {main, <<"readme">>}.
 {proglang, erlang}.
diff --git a/apps/opentelemetry_api/README.md b/apps/opentelemetry_api/README.md
@@ -68,4 +68,6 @@ and on [hex.pm](https://hex.pm) under the [OpenTelemetry organization](https://h
 
 ## Contributing
 
-See the [contributing file](CONTRIBUTING.md).
+Read OpenTelemetry project [contributing
+guide](https://github.com/open-telemetry/community/blob/main/CONTRIBUTING.md)
+for general information about the project.
diff --git a/apps/opentelemetry_api/docs.config b/apps/opentelemetry_api/docs.config
@@ -1,5 +1,4 @@
 {source_url, <<"https://github.com/open-telemetry/opentelemetry-erlang">>}.
-{extras, [<<"apps/opentelemetry_api/README.md">>,
-<<"apps/opentelemetry_api/LICENSE">>]}.
+{extras, [<<"apps/opentelemetry_api/README.md">>, <<"apps/opentelemetry_api/LICENSE">>, <<"VERSIONING.md">>]}.
 {main, <<"readme">>}.
 {proglang, erlang}.
diff --git a/apps/opentelemetry_api/mix.exs b/apps/opentelemetry_api/mix.exs
@@ -19,10 +19,8 @@ defmodule OpenTelemetry.MixProject do
       # homepage_url: "http://YOUR_PROJECT_HOMEPAGE",
       test_coverage: [tool: :covertool],
       docs: [
-        markdown_processor: ExDoc.Markdown.Cmark,
-        main: "OpenTelemetry",
+        main: "readme"
         # logo: "path/to/logo.png",
-        extras: erlang_docs()
       ],
       aliases: [
         # when build docs first build edocs with rebar3
@@ -52,7 +50,6 @@ defmodule OpenTelemetry.MixProject do
       dep when is_atom(dep) -> {dep, ">= 0.0.0"}
     end)
     |> Enum.concat([
-      {:cmark, "~> 0.7", only: :dev, runtime: false},
       {:ex_doc, "0.21.0", only: :dev, runtime: false},
       {:dialyxir, "~> 1.0", only: [:dev], runtime: false},
       {:covertool, ">= 0.0.0", only: :test}
@@ -72,15 +69,6 @@ defmodule OpenTelemetry.MixProject do
     ]
   end
 
-  def erlang_docs() do
-    files =
-      for file <- Path.wildcard("edoc/*.md"),
-          file != "edoc/README.md",
-          do: {String.to_atom(file), [title: Path.basename(file, ".md")]}
-
-    [{:"README.md", [title: "Overview"]} | files]
-  end
-
   defp load_config do
     {:ok, config} = :file.consult('rebar.config')
 

diff --git a/apps/opentelemetry_api/rebar.config b/apps/opentelemetry_api/rebar.config
@@ -2,8 +2,7 @@
 {deps, []}.
 
 {profiles,
- [{docs, [{deps, [edown]},
-          {edoc_opts,
+ [{docs, [{edoc_opts,
            [
             {doclet, edoc_doclet_chunks},
             {layout, edoc_layout_chunks},

diff --git a/apps/opentelemetry_exporter/README.md b/apps/opentelemetry_exporter/README.md
@@ -10,56 +10,9 @@ Currently only supports the Tracer protocol using either GRPC or Protobuffers ov
 
 ## Configuration
 
-### Options to span processor
-
-Exporter configuration can be done as arguments to the span processor (simple or batch) in the OpenTelemetry application environment.
-
-For an Erlang release in `sys.config`:
-
-``` erlang
-{opentelemetry,
-  [{processors,
-    [{otel_batch_processor,
-        #{exporter => {opentelemetry_exporter, #{endpoints =>
-        ["http://localhost:9090"],
-            headers => [{"x-honeycomb-dataset", "experiments"}]}}}}]}]}
-```
-
-The default protocol is `http_protobuf`, to override this and use grpc add
-`protocol` to the config map:
-
-``` erlang
-{opentelemetry,
-  [{processors,
-    [{otel_simple_processor,
-        #{exporter => {opentelemetry_exporter, #{protocol => grpc,
-                                                 endpoints => ["http://localhost:9090"],
-                                                 headers => [{"x-honeycomb-dataset", "experiments"}]}}}}]}]}
-```
-
-In Elixir, you can use `config.exs` or `runtime.exs`:
-
-``` elixir
-config :opentelemetry, :processors,
-  otel_batch_processor: %{
-    exporter: {:opentelemetry_exporter, %{endpoints: ["http://localhost:9090"],
-                                          headers: [{"x-honeycomb-dataset", "experiments"}]}}
-  }
-```
-
-### The configuration map
-
-The second element of the configuration tuple is a configuration map. It can contain the following keys:
-
-- `protocol` - one of: `http_protobuf`, `grpc` or `http_json`. Defaults to `http_protobuf`. `http_json` is not implemented yet.
-- `endpoints` - A list of endpoints to send traces to. Can take one of the forms described below. By default, exporter sends data to `http://localhost:4318`.
-- `headers` - a list of headers to send to the collector (i.e `[{<<"x-access-key">> <<"secret">>}]`). Defaults to an empty list.
-- `compression` - an atom. Setting it to `gzip` enables gzip compression.
-- `ssl_options` - a list of SSL options. See Erlang's [SSL docs](https://www.erlang.org/doc/man/ssl.html#TLS/DTLS%20OPTION%20DESCRIPTIONS%20-%20CLIENT) for what options are available.
-
-### Application Environment
+By default the exporter will use HTTP to export protobuf encoded Spans to
+`http://localhost:4138/v1/traces`. 
 
-Alternatively the `opentelemetry_exporter` Application can be configured itself.
 Available configuration keys:
 
 - `otlp_endpoint`: The URL to send traces and metrics to, for traces the path `v1/traces` is appended to the path in the URL.
@@ -124,6 +77,56 @@ OTEL_EXPORTER_OTLP_TRACES_COMPRESSION=gzip
 OTEL_EXPORTER_OTLP_TRACES_HEADERS=x-honeycomb-team=<HONEYCOMB API TOKEN>,x-honeycomb-dataset=experiments
 ```
 
+### Options to span processor
+
+In addition to using the environment variables the exporter accepts a map of
+arguments. The argument to the exporter's `init` function can be configured as
+part of the Span Processor (simple or batch) in the OpenTelemetry application
+environment.
+
+For an Erlang release in `sys.config`:
+
+``` erlang
+{opentelemetry,
+  [{processors,
+    [{otel_batch_processor,
+        #{exporter => {opentelemetry_exporter, #{endpoints =>
+        ["http://localhost:9090"],
+            headers => [{"x-honeycomb-dataset", "experiments"}]}}}}]}]}
+```
+
+The default protocol is `http_protobuf`, to override this and use grpc add
+`protocol` to the config map:
+
+``` erlang
+{opentelemetry,
+  [{processors,
+    [{otel_simple_processor,
+        #{exporter => {opentelemetry_exporter, #{protocol => grpc,
+                                                 endpoints => ["http://localhost:9090"],
+                                                 headers => [{"x-honeycomb-dataset", "experiments"}]}}}}]}]}
+```
+
+In Elixir, you can use `config.exs` or `runtime.exs`:
+
+``` elixir
+config :opentelemetry, :processors,
+  otel_batch_processor: %{
+    exporter: {:opentelemetry_exporter, %{endpoints: ["http://localhost:9090"],
+                                          headers: [{"x-honeycomb-dataset", "experiments"}]}}
+  }
+```
+
+#### The configuration map
+
+The second element of the configuration tuple is a configuration map. It can contain the following keys:
+
+- `protocol` - one of: `http_protobuf`, `grpc` or `http_json`. Defaults to `http_protobuf`. `http_json` is not implemented yet.
+- `endpoints` - A list of endpoints to send traces to. Can take one of the forms described below. By default, exporter sends data to `http://localhost:4318`.
+- `headers` - a list of headers to send to the collector (i.e `[{<<"x-access-key">> <<"secret">>}]`). Defaults to an empty list.
+- `compression` - an atom. Setting it to `gzip` enables gzip compression.
+- `ssl_options` - a list of SSL options. See Erlang's [SSL docs](https://www.erlang.org/doc/man/ssl.html#TLS/DTLS%20OPTION%20DESCRIPTIONS%20-%20CLIENT) for what options are available.
+
 ## Contributing
 
 This project uses a submodule during development, it is not needed if the application is being used as a dependency, so be sure to clone with the option `recurse-submodules`:

diff --git a/apps/opentelemetry_exporter/docs.config b/apps/opentelemetry_exporter/docs.config
@@ -1,5 +1,4 @@
 {source_url, <<"https://github.com/open-telemetry/opentelemetry-erlang">>}.
-{extras, [<<"apps/opentelemetry_exporter/README.md">>,
-<<"apps/opentelemetry_exporter/LICENSE">>]}.
+{extras, [<<"apps/opentelemetry_exporter/README.md">>, <<"apps/opentelemetry_exporter/LICENSE">>]}.
 {main, <<"readme">>}.
 {proglang, erlang}.
diff --git a/apps/opentelemetry_exporter/rebar.config b/apps/opentelemetry_exporter/rebar.config
@@ -10,8 +10,7 @@
                     {i, "apps/opentelemetry_exporter/opentelemetry-proto/"}]}]}.
 
 {profiles,
- [{docs, [{deps, [edown]},
-          {edoc_opts,
+ [{docs, [{edoc_opts,
            [
             {doclet, edoc_doclet_chunks},
             {layout, edoc_layout_chunks},

diff --git a/docs.sh b/docs.sh
@@ -13,7 +13,7 @@ version=1.0.1
 
 ex_doc "opentelemetry" $version "_build/default/lib/opentelemetry/ebin" \
   --source-ref v${version} \
-  --config docs.config $@ \
+  --config apps/opentelemetry/docs.config $@ \
   --output "apps/opentelemetry/doc"
 
 ex_doc "opentelemetry_exporter" $version "_build/default/lib/opentelemetry_exporter/ebin" \