Skip to content

Conversation

@aknuds1
Copy link

@aknuds1 aknuds1 commented Jan 12, 2026

Summary

Fixes #2884

The docker-compose.yml uses the syntax - OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE which passes the variable from the shell environment, not from the .env file. When the shell doesn't have this variable set, Docker Compose doesn't apply the .env value - instead, the value baked into the container images (delta) takes precedence. See reference on how environment variable interpolation works with .env files.

This causes Prometheus to reject metrics with "invalid temporality and type combination" errors, affecting HTTP/RPC server metrics from most services.

Changes

Changed all occurrences from:

- OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE

To explicit interpolation:

- OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=${OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE}

Test plan

  1. make start
  2. Verify docker exec cart env | grep TEMPORALITY shows cumulative
  3. Verify docker logs prometheus 2>&1 | grep "invalid temporality" shows no errors
  4. Check APM dashboard shows metrics for cart, checkout, ad, etc.

Use explicit variable interpolation for OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE
so the value from .env is correctly applied. Without this, Docker Compose passes the
variable from the shell environment rather than the .env file, causing services to use
delta temporality (baked into images) instead of cumulative, which Prometheus rejects.

Fixes open-telemetry#2884
@aknuds1 aknuds1 requested a review from a team as a code owner January 12, 2026 17:00
@github-actions github-actions bot added the helm-update-required Requires an update to the Helm chart when released label Jan 12, 2026
@julianocosta89
Copy link
Member

This change shouldn't be necessary.
All the env vars come from the .env file, and when the env var has the same name as whatever is set in the .env file, the value from the .env file is used.

@aknuds1 Could you elaborate further on the issue?
Most OTLP endpoints and all resource attributes are being set without ${value} being explicitly set.
Everything comes from .env file.

@aknuds1
Copy link
Author

aknuds1 commented Jan 12, 2026

I will try again tomorrow.

@aknuds1
Copy link
Author

aknuds1 commented Jan 13, 2026

@julianocosta89 I tried to reproduce yesterday's problem (delta temporality being configured in the demo stack) this morning, but couldn't. I've inferred that the probable cause was OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta being exported in my shell for some reason unknown to me (I didn't set it knowingly). I got confused by the Docker Compose documentation making it look like there's a behavioral difference between the syntaxes. Sorry for the noise!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

helm-update-required Requires an update to the Helm chart when released

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE from .env not applied to services

3 participants