diff --git a/docs/docs.logflare.com/docs/concepts/ingestion/index.mdx b/docs/docs.logflare.com/docs/concepts/ingestion/index.mdx index 97e99f662..338e6acc8 100644 --- a/docs/docs.logflare.com/docs/concepts/ingestion/index.mdx +++ b/docs/docs.logflare.com/docs/concepts/ingestion/index.mdx @@ -104,7 +104,7 @@ metadata: { :::note On high ingestion volume, Logflare will sample incoming events instead of checking each event. The sample rate decreases as the ingestion rate increases. Ingestion rates are compared only on an individual local server that is performing the ingestion. -From 10-100 events per second, sample rate is 0.2. From 100-1000 events per second, sample rate is 0.1. Above 1000 events per second, sample rate is 0.01. +From 10-100 events per second, sample rate is 0.2. From 100-500 events per second, sample rate is 0.1. From 500-1000 events per second, sample rate is 0.05. Above 1000 events per second, sample rate is 0.01. ::: ### Key Transformation diff --git a/lib/logflare/source/bigquery/pipeline.ex b/lib/logflare/source/bigquery/pipeline.ex index c31d705f6..2b818ffbe 100644 --- a/lib/logflare/source/bigquery/pipeline.ex +++ b/lib/logflare/source/bigquery/pipeline.ex @@ -195,6 +195,7 @@ defmodule Logflare.Source.BigQuery.Pipeline do probability = case PubSubRates.Cache.get_local_rates(source.token) do %{average_rate: avg} when avg > 1000 -> 0.01 + %{average_rate: avg} when avg > 500 -> 0.05 %{average_rate: avg} when avg > 100 -> 0.1 %{average_rate: avg} when avg > 10 -> 0.2 _ -> 1