Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow different label names injected by an ExtractLabeler #190

Open
mutefiRe opened this issue Feb 14, 2024 · 2 comments
Open

Allow different label names injected by an ExtractLabeler #190

mutefiRe opened this issue Feb 14, 2024 · 2 comments
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@mutefiRe
Copy link

mutefiRe commented Feb 14, 2024

In a larger system, maintaining consistent tenant identifiers across all metrics isn't always practical. For example, some metrics might use tenant_id while others rely on page_id, depending on the reporting service. Therefore, the proxy needs to decide which label to enforce for each metric.

We currently have a working prototype that modifies vendor files. Instead of using the label parameter in the NewRoutes function, we inject the label name in the same way as the label values. As a result, the ExtractLabeler handles determining which label name to enforce.

Since we don't require the full functionality of the proxy, such as rules, silences, and alerts, which might be challenging to support, I wanted to see if you're interested in and supportive of this idea. If so, we're open to investing more effort in refining it for a potential upstream pull request for a possible direction. Anyways, we can provide a fork once it's ready.

@jotak
Copy link

jotak commented Jun 13, 2024

Hi @mutefiRe
I'm also interested in exactly this (if I read your suggestion correctly) : being able to override the label to use for injection.
I think for security reasons, the list of labels allowed for override should be restricted by config (and perhaps enforced to be tied to some specific metrics only)

.. I was working on a PoC that I haven't had the opportunity to test yet ..

Does that sound aligned with you use case?

@mutefiRe
Copy link
Author

mutefiRe commented Jun 14, 2024

Hi @mutefiRe I'm also interested in exactly this (if I read your suggestion correctly) : being able to override the label to use for injection. I think for security reasons, the list of labels allowed for override should be restricted by config (and perhaps enforced to be tied to some specific metrics only)

.. I was working on a PoC that I haven't had the opportunity to test yet ..

Does that sound aligned with you use case?

Hey @jotak,

We've switched to allowing multiple key-value(s) pairs using a map. The proxy ensures that each existing matcher is duplicated for every different label key. Additionally, we have a configuration for a custom Enforcer that matches regex patterns with the incoming metric name to avoid injecting every possible combination into each upstream request. However, this isn't feasible for endpoints like /api/v1/label/__name__/values, where the metric names are unknown in advance. As a result, every combination must be injected, potentially leading to quite large requests depending on the number of IDs you want to inject.
In the previous version, if nothing matched our rules, we also used to fall back to the default rule or inject an artificial label-value combination that surely doesn't exist. This ensured the API remained 100% compatible with the upstream API instead of returning errors.

You can check out our fork as well, but note that we simplified the code for our use case, removing all regex enforcement, as well as alerts and silences.

@simonpasquier simonpasquier added the kind/feature Categorizes issue or PR as related to a new feature. label Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

3 participants