Defining A Common Observability Query Language and Other observability TAG Updates

• Original Link
• https://kccnceu2023.sched.com/event/1Ipz2/defining-a-common-observability-query-language-and-other-observability-tag-updates-alolita-sharma-matt-young-apple?iframe=no&w=100%&sidebar=yes&bg=no

CNCF Observability Project Landscape

• Graduated Projectes:

  • fluentd
  • jeager
  • prometheus

• Incubating Projects:

  • chaos mesh
  • openmetrics
  • opentelemetry
  • cortex
  • litmux
  • thanos

• Sandbox Projects:

  • chaosblade
  • Pixie
  • fonio
  • skooner
  • kuberhealthy
  • trickster
  • opencost

OTAG Initiatives, Workgroups, Activities

• Initiatives:

  • Observability query language standardization
  • Continuous cost measurement and optimization
  • Profilling in Open Telemetry
  • Graphs in Observability
  • Exceptions as another telemetry data type
  • Correlation across telemetry data signals

• https://opentelemetry.io/blog/2022/why-and-how-ebay-pivoted-to-opentelemetry/

• https://px.dev/

Profiling support in OpenTelemetry

• Profiling support in Opentelemetry:
  • Update by Ryan Perry (Jan 2023)

Observability Query Language Standardization

• Started as a request at Open Telemetry Community Meeting at Kubecon NA 2022 in Detroit:
  • OpenTelemetry project recommendation to continue query specification discussions in OTAG This discussion has been requestsed by several end-users( e.g. EBay, Netflix, Apple)
• Workgroup charter discussed and finalkized in CNCF Observability TAG
• Workgroup creation request for approval from TOC is currently under consideration
• Get Involved. See the following links:
  • https://github.com/cncf/tag-observability/blob/main/working-groups/query-standardization.md

Observability Query Language Stadnardization

• Benefits:
  • Portability for end users
  • Correlate various telemetry data types
  • Reduce developer and operations toil
  • Federated veiw regardless of data location
  • Let vendors focus on unique features
  • Lower migration and acquisition costs
• Goals:
  • Compile a list of end user use cases
  • Research existing QLs
  • Recommend semantics, models and behaviors for a standard:
    • Follow-up groups/projects would implement recommendations
• Help Needed:
  • Provides uses cases Document languages and semantics Work together towards a standard

Q&A

• There are various types in tsdb, how can I define how to sort the results when querying:
  • Details about the implementation, there are various languages right now, and there are some that are not open source, so we still have to come up with a proposal to know.
  • (Personal thought) I thought the question was ambiguous to answer, so the answer is bound to be ambiguous accordingly. I wonder if it was a question with too standard content and an answer with standard content.

Personal Impression

• There are a lot of links in the presentation. But, there is no link to access it. It made me spend a lot of time to find the links.
• It is just a good video to follow the recent observability’s trend.:
  • Many observability tools are separated.
  • And too many developers are suffered by repeated implementation and makes a mistake again.
  • So, observability query languages should be defined nowadays.
  • IMO, promql is a de-facto standard for metrics.
    • There are not a standard for log & trace.
    • But, cncf & opentelemetry are very closed to grafana.
    • logql & traceql are strong candidates because of it.
    • I believe they are not kind to user, so I worry.