Skip to content

Commit

Permalink
[Feature] inject otel context (#83)
Browse files Browse the repository at this point in the history 
* Add `addOpentelemetryContext` option

* Update dependencies

* Add `openTelemetryContext` unit tests

* docs + changelog

* docs

* remove commented code

* Use `logzio-sender-version` instead of static version

* Update readme dep versions

* Update dependencies
  • Loading branch information
@yotamloe
yotamloe authored Jan 6, 2025
1 parent 23274e0 commit 2c20dfa
Show file tree
Hide file tree
Showing 4 changed files with 177 additions and 36 deletions.
77 changes: 51 additions & 26 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ JDK 11 and above:
<dependency>
<groupId>io.logz.log4j2</groupId>
<artifactId>logzio-log4j2-appender</artifactId>
<version>2.0.1</version>
<version>2.2.0</version>
</dependency>
```

Expand All @@ -32,7 +32,7 @@ The appender also requires a logger implementation, for example:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.15.0</version>
<version>2.24.1</version>
</dependency>
```

Expand All @@ -54,35 +54,36 @@ The appender also requires a logger implementation, for example:
```

### Parameters
| Parameter | Default | Explained |
| ------------------ | ------------------------------------ | ----- |
| **logzioToken** | *None* | Your Logz.io token, which can be found under "settings" in your account, If the value begins with `$` then the appender looks for an environment variable or system property with the name specified. For example: `$LOGZIO_TOKEN` will look for environment variable named `LOGZIO_TOKEN` |
| **logzioType** | *java* | The [log type](https://support.logz.io/hc/en-us/articles/209486049-What-is-Type-) for that appender, it must not contain any spaces |
| **logzioUrl** | *https://listener.logz.io:8071* | The url that the appender sends to. If your account is in the EU you must use https://listener-eu.logz.io:8071 |
| **drainTimeoutSec** | *5* | How often the appender should drain the queue (in seconds) |
| **socketTimeoutMs** | *10 * 1000* | The socket timeout during log shipment |
| **connectTimeoutMs** | *10 * 1000* | The connection timeout during log shipment |
| **addHostname** | *false* | Optional. If true, then a field named 'hostname' will be added holding the host name of the machine. If from some reason there's no defined hostname, this field won't be added |
| **additionalFields** | *None* | Optional. Allows to add additional fields to the JSON message sent. The format is "fieldName1=fieldValue1;fieldName2=fieldValue2". You can optionally inject an environment variable value using the following format: "fieldName1=fieldValue1;fieldName2=$ENV_VAR_NAME". In that case, the environment variable should be the only value. In case the environment variable can't be resolved, the field will be omitted. |
| **debug** | *false* | Print some debug messages to stdout to help to diagnose issues |
| **compressRequests** | *false* | Boolean. `true` if logs are compressed in gzip format before sending. `false` if logs are sent uncompressed. |
| **exceedMaxSizeAction** | *"cut"* | String. cut to truncate the message field or drop to drop log that exceed the allowed maximum size for logzio. If the log size exceeding the maximum size allowed after truncating the message field, the log will be dropped. |
| Parameter | Default | Explained |
|-----------------------------|---------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **logzioToken** | *None* | Your Logz.io token, which can be found under "settings" in your account, If the value begins with `$` then the appender looks for an environment variable or system property with the name specified. For example: `$LOGZIO_TOKEN` will look for environment variable named `LOGZIO_TOKEN` |
| **logzioType** | *java* | The [log type](https://support.logz.io/hc/en-us/articles/209486049-What-is-Type-) for that appender, it must not contain any spaces |
| **logzioUrl** | *https://listener.logz.io:8071* | The url that the appender sends to. If your account is in the EU you must use https://listener-eu.logz.io:8071 |
| **drainTimeoutSec** | *5* | How often the appender should drain the queue (in seconds) |
| **socketTimeoutMs** | *10 * 1000* | The socket timeout during log shipment |
| **connectTimeoutMs** | *10 * 1000* | The connection timeout during log shipment |
| **addHostname** | *false* | Optional. If true, then a field named 'hostname' will be added holding the host name of the machine. If from some reason there's no defined hostname, this field won't be added |
| **additionalFields** | *None* | Optional. Allows to add additional fields to the JSON message sent. The format is "fieldName1=fieldValue1;fieldName2=fieldValue2". You can optionally inject an environment variable value using the following format: "fieldName1=fieldValue1;fieldName2=$ENV_VAR_NAME". In that case, the environment variable should be the only value. In case the environment variable can't be resolved, the field will be omitted. |
| **addOpentelemetryContext** | *true* | Optional. Add `trace_id`, `span_id`, `service_name` fields to logs when opentelemetry context is available. |
| **debug** | *false* | Print some debug messages to stdout to help to diagnose issues |
| **compressRequests** | *false* | Boolean. `true` if logs are compressed in gzip format before sending. `false` if logs are sent uncompressed. |
| **exceedMaxSizeAction** | *"cut"* | String. cut to truncate the message field or drop to drop log that exceed the allowed maximum size for logzio. If the log size exceeding the maximum size allowed after truncating the message field, the log will be dropped. |

#### Parameters for in-memory queue
| Parameter | Default | Explained |
| ------------------ | ------------------------------------ | ----- |
| **inMemoryQueueCapacityBytes** | *1024 * 1024 * 100* | The amount of memory(bytes) we are allowed to use for the memory queue. If the value is -1 the sender will not limit the queue size.|
| **inMemoryLogsCountCapacity** | *-1* | Number of logs we are allowed to have in the queue before dropping logs. If the value is -1 the sender will not limit the number of logs allowed.|
| **inMemoryQueue** | *false* | Set to true if the appender uses in memory queue. By default the appender uses disk queue|
| Parameter | Default | Explained |
|--------------------------------|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|
| **inMemoryQueueCapacityBytes** | *1024 * 1024 * 100* | The amount of memory(bytes) we are allowed to use for the memory queue. If the value is -1 the sender will not limit the queue size. |
| **inMemoryLogsCountCapacity** | *-1* | Number of logs we are allowed to have in the queue before dropping logs. If the value is -1 the sender will not limit the number of logs allowed. |
| **inMemoryQueue** | *false* | Set to true if the appender uses in memory queue. By default the appender uses disk queue |


#### Parameters for disk queue
| Parameter | Default | Explained |
| ------------------ | ------------------------------------ | ----- |
| **fileSystemFullPercentThreshold** | *98* | The percent of used file system space at which the sender will stop queueing. When we will reach that percentage, the file system in which the queue is stored will drop all new logs until the percentage of used space drops below that threshold. Set to -1 to never stop processing new logs |
| **gcPersistedQueueFilesIntervalSeconds** | *30* | How often the disk queue should clean sent logs from disk |
| **bufferDir**(deprecated, use queueDir) | *System.getProperty("java.io.tmpdir")* | Where the appender should store the queue |
| **queueDir** | *System.getProperty("java.io.tmpdir")* | Where the appender should store the queue |
| Parameter | Default | Explained |
|------------------------------------------|----------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **fileSystemFullPercentThreshold** | *98* | The percent of used file system space at which the sender will stop queueing. When we will reach that percentage, the file system in which the queue is stored will drop all new logs until the percentage of used space drops below that threshold. Set to -1 to never stop processing new logs |
| **gcPersistedQueueFilesIntervalSeconds** | *30* | How often the disk queue should clean sent logs from disk |
| **bufferDir**(deprecated, use queueDir) | *System.getProperty("java.io.tmpdir")* | Where the appender should store the queue |
| **queueDir** | *System.getProperty("java.io.tmpdir")* | Where the appender should store the queue |



Expand Down Expand Up @@ -156,6 +157,27 @@ Will send a log to Logz.io that looks like this:
}
```

## Add opentelemetry context
If you're sending traces with OpenTelemetry instrumentation (auto or manual), you can correlate your logs with the trace context. That way, your logs will have traces data in it, such as service name, span id and trace id (version >= `2.2.0`). This feature is enabled by default, To disable it, set the `addOpentelemetryContext` option in your configuration to `false`, like in this example:

```xml
<Appenders>
<LogzioAppender name="Logzio">
<logzioToken>your-logzio-personal-token-from-settings</logzioToken>
<logzioType>myAwesomeType</logzioType>
<logzioUrl>https://listener.logz.io:8071</logzioUrl>
<addOpentelemetryContext>false</addOpentelemetryContext>
</LogzioAppender>

</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="Logzio"/>
</Root>
</Loggers>
```


## Build and test locally
1. Clone the repository:
```bash
Expand All @@ -169,6 +191,9 @@ Will send a log to Logz.io that looks like this:
```

### Release notes
- 2.2.0
- Updated LogzioSender version to `2.2.0`
- Add `addOpentelemetryContext` option, to add `trace_id`, `span_id`, `service_name` fields to logs when opentelemetry context is available.
- 2.1.0
- Updated LogzioSender version to `2.1.0`
- Upgrade packages version
Expand Down
14 changes: 7 additions & 7 deletions pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

<groupId>io.logz.log4j2</groupId>
<artifactId>logzio-log4j2-appender</artifactId>
<version>2.1.0</version>
<version>2.2.0</version>

<packaging>jar</packaging>
<name>Logz.io Log4j2 Appender</name>
Expand Down Expand Up @@ -36,7 +36,7 @@
</scm>

<properties>
<logzio-sender-version>2.1.0</logzio-sender-version>
<logzio-sender-version>2.2.0</logzio-sender-version>
</properties>

<build>
Expand Down Expand Up @@ -66,7 +66,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>3.10.0</version>
<version>3.11.2</version>
<executions>
<execution>
<id>attach-javadocs</id>
Expand All @@ -79,7 +79,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>3.2.4</version>
<version>3.2.7</version>
<executions>
<execution>
<id>sign-artifacts</id>
Expand All @@ -104,7 +104,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<version>3.6.0</version>
<executions>
<execution>
<phase>package</phase>
Expand Down Expand Up @@ -150,12 +150,12 @@
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.24.0</version>
<version>2.24.3</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.24.1</version>
<version>2.24.3</version>
</dependency>
<dependency>
<groupId>io.logz.sender</groupId>
Expand Down
Loading

0 comments on commit 2c20dfa

Please sign in to comment.