Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Traceql support #366

Merged
merged 32 commits into from
Oct 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
b2e906e
#feat: bun support
akvlad Oct 6, 2023
228e17d
#fix: correct buffer send; add node 20 to the ci
akvlad Oct 6, 2023
f4bc4d3
#fix: pure virtual method called issue;
akvlad Oct 6, 2023
9e83eff
#fix: Dockerfile for bun env
akvlad Oct 6, 2023
7f337ed
#fix: custom host/port to listen
akvlad Oct 6, 2023
fee92d3
Merge branch 'master' into bun_support
lmangani Oct 6, 2023
37fc338
#feat: prom support
akvlad Oct 13, 2023
a17aabc
Merge branch 'master' into bun_support
lmangani Oct 14, 2023
31de334
#feat: prom series support
akvlad Oct 16, 2023
c1c5034
#fix: error return
akvlad Oct 16, 2023
ab59344
#fix: rm outdated unit tests
akvlad Oct 16, 2023
4f876aa
#fix: fix prom instance recreation
akvlad Oct 16, 2023
d4b8456
Merge branch 'master' into prom_support
lmangani Oct 23, 2023
410e983
#fix: minor fixes
akvlad Oct 24, 2023
095972e
Update README.md
lmangani Oct 24, 2023
90c8568
Merge branch 'master' into bun_support
lmangani Oct 24, 2023
51d09ec
Merge branch 'master' into prom_support
lmangani Oct 24, 2023
8559c5a
Update Release Action
lmangani Oct 24, 2023
4336b4e
Delete .github/workflows/bump_version_beta.yml
lmangani Oct 24, 2023
80357f1
Update README.md
lmangani Oct 24, 2023
d7b32da
Update README.md
lmangani Oct 24, 2023
a073f19
Update README.md
lmangani Oct 24, 2023
eb1c52c
Update README.md
lmangani Oct 24, 2023
5768021
#feat: traceql support init
akvlad Oct 26, 2023
3ce6360
Update README.md
lmangani Oct 26, 2023
abedcbe
Update README.md
lmangani Oct 27, 2023
addc20d
Update README.md
lmangani Oct 27, 2023
56c1d70
Merge branch '3.0' into bun_support
akvlad Oct 27, 2023
79c374c
Merge pull request #363 from metrico/bun_support
akvlad Oct 27, 2023
5e4dc2e
Merge remote-tracking branch 'origin/3.0' into prom_support
akvlad Oct 27, 2023
334a03b
Merge pull request #364 from metrico/prom_support
akvlad Oct 27, 2023
19228bf
Merge branch '3.0' into traceql_support
akvlad Oct 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 16 additions & 30 deletions .github/workflows/bump_version.yml
Original file line number Diff line number Diff line change
@@ -1,39 +1,25 @@
name: 'Build Multi-Arch'

on:
push:
branches:
- 'master'
paths-ignore:
- '**.md'
- '**.yml'
- '**.yaml'
release:
types: [created]

jobs:
bump-version:
name: 'Bump Version on master & Publish'
build:
name: 'Build & Publish'
runs-on: ubuntu-latest

steps:
- name: 'Checkout source code'
uses: 'actions/checkout@v2'
uses: 'actions/checkout@v3'
with:
ref: ${{ github.ref }}
- name: 'cat package.json'
run: cat ./package.json
- name: 'Automated Version Bump'
id: version-bump
uses: 'phips28/gh-action-bump-version@master'
with:
tag-prefix: 'v'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: 'cat package.json'
run: cat ./package.json
- name: 'Output Step'
env:
NEW_TAG: ${{ steps.version-bump.outputs.newTag }}
run: echo "new tag $NEW_TAG"
- uses: MYXOMOPX/modify-pkg-json@master
id: setcmnver
with:
target: ./package.json
action: "set_version"
argument: "${{ github.ref_name }}"
- name: Check NPM secret presence
id: checksecrets
shell: bash
Expand All @@ -45,10 +31,10 @@ jobs:
fi
env:
SECRET: ${{ secrets.NPM_TOKEN }}
- uses: actions/setup-node@v1
- uses: actions/setup-node@v4.0.0
if: ${{ steps.checksecrets.outputs.secretspresent }}
with:
node-version: 14
node-version: 18
- name: Publish to NPM
if: ${{ steps.checksecrets.outputs.secretspresent }}
run: |
Expand Down Expand Up @@ -93,9 +79,9 @@ jobs:
push: true
tags: |
qxip/qryn:latest
qxip/qryn:${{ steps.version-bump.outputs.newTag }}
qxip/qryn:${{ github.ref_name }}
qxip/cloki:latest
qxip/cloki:${{ steps.version-bump.outputs.newTag }}
qxip/cloki:${{ github.ref_name }}

- name: Log in to the GHCR registry
uses: docker/[email protected]
Expand All @@ -111,4 +97,4 @@ jobs:
push: true
tags: |
ghcr.io/metrico/qryn:latest
ghcr.io/metrico/qryn:${{ steps.version-bump.outputs.newTag }}
ghcr.io/metrico/qryn:${{ github.ref_name }}
95 changes: 0 additions & 95 deletions .github/workflows/bump_version_beta.yml

This file was deleted.

4 changes: 2 additions & 2 deletions .github/workflows/node-clickhouse.js.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:

strategy:
matrix:
node-version: [18, 16.x]
node-version: [18, 16.x, 20]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/

steps:
Expand All @@ -43,4 +43,4 @@ jobs:
CLICKHOUSE_TSDB: loki
INTEGRATION_E2E: 1
CLOKI_EXT_URL: 127.0.0.1:3100
run: node qryn.js >/dev/stdout & npm run test --forceExit
run: node qryn.mjs >/dev/stdout & npm run test --forceExit
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,8 @@ node_modules
/test/e2e/
/lib/influx/.idea/
/lib/influx/influx.iml
/wasm_parts/_vendor.zip
/wasm_parts/.idea/
/wasm_parts/vendor/
/wasm_parts/main.wasm
/wasm_parts/wasm_parts.iml
15 changes: 15 additions & 0 deletions Dockerfile_bun
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Qryn
FROM oven/bun:1-alpine

# BUILD FORCE
ENV BUILD 703030
ENV PORT 3100

COPY . /app
WORKDIR /app
RUN bun install

# Expose Ports
EXPOSE 3100

CMD [ "bun", "qryn.mjs" ]
101 changes: 66 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,53 +4,60 @@

[![Build Status](https://github.com/metrico/qryn/actions/workflows/bump_version.yml/badge.svg)](https://github.com/metrico/qryn/actions/workflows/bump_version.yml)
![CodeQL](https://github.com/lmangani/cLoki/workflows/CodeQL/badge.svg)
<a href="https://matrix.to/#/#qryn:matrix.org">
<img src="https://img.shields.io/badge/Join%20Matrix-Chat-green" alt="Matrix">
</a>
[![Stand With Ukraine](https://raw.githubusercontent.com/vshymanskyy/StandWithUkraine/main/badges/StandWithUkraine.svg)](https://stand-with-ukraine.pp.ua)


# [qryn.dev](https://qryn.dev) :cloud: [qryn.cloud](https://qryn.cloud) :heart:
> ... it's pronounced /ˈkwɪr..ɪŋ/ or just querying
# [qryn 3.x](https://qryn.dev) :cloud: [qryn.cloud](https://qryn.cloud)
> ... it's pronounced /ˈkwɪr..ɪŋ/ or just _querying_

![image](https://user-images.githubusercontent.com/1423657/232089970-c4536f16-5967-4051-85a5-8ad94fcde67c.png)


:rocket: **qryn** is a _drop-in Grafana compatible_ **polyglot observability** framework<br/>
- **Logs, Metrics and Traces** living happily together. Drop-in compatible with multiple vendors formats.
- Native [LogQL/PromQL/TempoQL APIs](https://qryn.cloud) support for [querying](https://github.com/lmangani/qryn/wiki/LogQL-for-Beginners), [processing](https://github.com/lmangani/qryn/wiki/LogQL-Supported-Queries), [tracing](https://github.com/lmangani/qryn/wiki/Tempo-Tracing) and [alerting](https://github.com/lmangani/qryn/wiki/Ruler---Alerts) [^2] in [Grafana](http://docs.grafana.org/features/explore/) [^3]
- Search, filter and extract metrics from _logs, events, spans and traces_ using familiar languages. _SQL Optional_.
- Ingestion [APIs](https://qryn.metrico.in/#/support) transparently compatible with [Opentelemetry, Loki, Prometheus, InfluxDB, Elastic](https://qryn.dev) _and [more](https://github.com/metrico/otel-collector)_
- Ready to use with popular Agents such as [Promtail, Grafana-Agent, Vector, Logstash, Telegraf](https://qryn.metrico.in/#/ingestion) _and more_
- Built in [Explore UI](https://github.com/metrico/cloki-view) and [CLI](https://github.com/lmangani/vLogQL) for querying supported datasources
- Designed for edge _(js/bun/wasm)_ and core/backend deployments _(golang/rust)_.
- Total data control. Compatible with [ClickHouse](https://clickhouse.com/) or [InfluxDB IOx](https://influxdata.com) with S3 object storage.

:rocket: **qryn.cloud** is the _supercharged_ **qryn** version developed in _go_ with additional _functionality, speed and features!_<br/>
:rocket: _lighweight, multi-standard drop-in compatible_ **polyglot observability** framework for _**Logs, Metrics and Traces**_.<br/>

- **Polyglot**: Use **LogQL, PromQL**, and **TempoQL** languages to query, process and alert _any data_.
- **Voracious**: Ingest anything compatible with **Opentelemetry, Loki, Prometheus, Influx, Datadog, Elastic** _& more_.
- **Versatile**: Explore data with qryn's built-in **Explore UI** and **CLI** or _native_ **Grafana** compatibility.
- **Secure**: Keep control of your data, using **ClickHouse** or **InfluxDB IOx** on top of disposable **S3** storage.
- **Unmetered**: Unlimited FOSS deployments or **qryn.cloud** option with advanced features and high performance.
- **Effective**: Do more with less, replace lock-in competitors for a fractions of the cost and complexity.
<br>



## 🚀 [Get Started](https://qryn.metrico.in/#/installation)

* Deploy qryn OSS using the [documentation](https://qryn.metrico.in/#/installation) or get help in our [Matrix room](https://matrix.to/#/#qryn:matrix.org) :octocat:
* Create a free account on [qryn.cloud](https://qryn.cloud) and get polyglot without moving a finger ☁️

<a href="https://qryn.cloud" target="_blank">
<img src="https://user-images.githubusercontent.com/1423657/218818279-3efff74f-0191-498a-bdc4-f2650c9d3b49.gif">
</a>

<br>

## 🚀 [Get Started](https://qryn.metrico.in/#/installation)
<br>

:octocat: Get qryn OSS up and running on-prem in no time using the [Documentation](https://qryn.metrico.in/#/installation) or join our [Matrix Room](https://matrix.to/#/#qryn:matrix.org)
## Features

☁️ Create a free account on [qryn.cloud](https://qryn.cloud) and go straight to production at any scale with **polyglot confidence**.
💡 _**qryn** independently implements popular observability standards, protocols and query languages:_

<br>

### 📚 OpenTelemetry

⚡ **qryn** is officially integrated with [opentelemetry](https://github.com/metrico/otel-collector) supports _any log, trace or metric format_

Ingested data can be queried using any of the avialable qryn APIs _(LogQL, PromQL, TraceQL)_

<br>

## Supported Features
### 📚 Loki + LogQL

### 📚 OpenTelemetry
qryn fully supports opentelemetry and comes with a powerful [otel-collector](https://github.com/metrico/otel-collector) distribution supporting _any log, trace or metric format_ and writing directly to ClickHouse _qryn tables_ ready to be consumed through any query API.
> Any Loki compatible client or application can be used with qryn out of the box

⚡ **qryn** implements the [Loki API](https://github.com/lmangani/qryn/wiki/LogQL-Supported-Queries) for transparent compatibility with **[LogQL](https://grafana.com/docs/loki/latest/query/)** clients<br>

### 📚 LogQL
qryn implements a complete [LogQL API](https://github.com/lmangani/qryn/wiki/LogQL-Supported-Queries) to provide transparent compatibility with Loki clients<br>
The Grafana Loki datasource can be used to natively browse and query _logs_ and display extracted _timeseries_<br>

<a href="https://qryn.metrico.in/#/logs/query" target="_blank">
Expand All @@ -59,23 +66,33 @@ The Grafana Loki datasource can be used to natively browse and query _logs_ and

:tada: _No plugins needed_


<br>

### 📈 Prometheus
qryn implements a complete [Prometheus API](https://github.com/lmangani/qryn/wiki/LogQL-Supported-Queries) to provide transparent compatibility with Prometheus clients<br>
The Grafana Prometheus datasource can be used to natively browse and query _metrics_ and display extracted _timeseries_<br>
### 📈 Prometheus + PromQL

> Any Prometheus compatible client or application can be used with qryn out of the box

⚡ **qryn** implements the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) for transparent **[PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/)** compatibility using WASM 🏆<br>

The Grafana Prometheus datasource can be used to natively to query _metrics_ and display _timeseries_<br>

<a href="https://qryn.metrico.in/#/metrics/query" target="_blank">
<img src="https://user-images.githubusercontent.com/1423657/196654084-1f1d8a62-3fd2-4420-a2fa-57ac2872938c.gif" width=700 class=border />
</a>

:tada: _No plugins needed_


<br>

### 🕛 Tempo
qryn implements the [Tempo API](https://github.com/lmangani/qryn/wiki/LogQL-Supported-Queries) to provide transparent compatibility with Tempo/OTLP clients.<br>
The Tempo datasource can be used to natively query _traces_ including _beta search_ and _service graphs_<br>
### 🕛 Tempo + TraceQL

⚡ **qryn** implements the [Tempo API](https://github.com/lmangani/qryn/wiki/LogQL-Supported-Queries) for transparent compatibility with **[TraceQL](https://grafana.com/docs/tempo/latest/traceql/)** clients.<br>

> Any Tempo/Opentelemetry compatible client or application can be used with qryn out of the box

The Tempo datasource can be used to natively query _traces_ including _**TraceQL**_ and supporting _service graphs_<br>

<a href="https://qryn.metrico.in/#/telemetry/query" target="_blank">
<img src="https://user-images.githubusercontent.com/1423657/196654097-8a235253-bf5d-4937-9e78-fddf12819d44.gif" width=700 class=border />
Expand All @@ -85,16 +102,30 @@ The Tempo datasource can be used to natively query _traces_ including _beta sear

<br>

### ↔️ Correlation
Data correlation made simple with dynamic **links** between _logs, metrics and traces_
### 📚 Other Vendors

**qryn** can ingest data using the [InfluxDB, DataDog, Elastic](https://qryn.metrico.in/#/support) and other vendors.


<br>

With **qryn** and **grafana** everything _just works_ right out of the box:

- Native datasource support without any plugin or extension
- Advanced Correlation between Logs, Metrics and Traces
- Service Graphs and Service Status Panels, and all the cool features

<br>

<a href="https://qryn.dev" target="_blank">
<img src="https://user-images.githubusercontent.com/1423657/184538094-13c11500-24ef-4468-9f33-dc9d564238e3.gif" width=700 class=border />
</a>

<br>

### :eye: View
<br>

### :eye: Explore View

No Grafana? No Problem. **qryn** ships with **view** - it's own lightweight data exploration tool

Expand All @@ -121,9 +152,9 @@ Whether it's code, documentation or grammar, we ❤️ all contributions. Not su

&nbsp;&nbsp;&nbsp;&nbsp;[![Contributors for @metrico/qryn](https://contributors-img.web.app/image?repo=lmangani/cloki)](https://github.com/metrico/qryn/graphs/contributors)

[![Stargazers repo roster for @metrico/qryn](https://reporoster.com/stars/metrico/qryn)](https://github.com/metrico/qryn/stargazers)
[![Stargazers repo roster for @metrico/qryn](https://bytecrank.com/nastyox/reporoster/php/stargazersSVG.php?user=metrico&repo=qryn)](https://github.com/metrico/qryn/stargazers)

[![Forkers repo roster for @metrico/qryn](https://reporoster.com/forks/metrico/qryn)](https://github.com/metrico/qryn/network/members)
[![Forkers repo roster for @metrico/qryn](https://bytecrank.com/nastyox/reporoster/php/forkersSVG.php?user=metrico&repo=qryn)](https://github.com/metrico/qryn/network/members)


#### License
Expand Down
Loading
Loading