docs: add concurent compaction docs

[317brian]

Description

• Adds the docs for concurrent compaction
• Refactors the page to be more structured:

General compaction info as the parent w/ manual compaction and auto compaction as its children:

Build preview: https://druid-git-concurrent-compaction-317brian.vercel.app/docs/latest/data-management/compaction

---

This PR has:

• been self-reviewed.

[AmatyaAvadhanula]

The UI for compaction says "Allow concurrent compactions (experimental)"

[AmatyaAvadhanula]

Could we please explicitly add that a streaming supervisor spec must always have an APPEND lock when using concurrent append and replace?

[317brian]

Added below in the append section that talks about lock type

[kfaraz]

Thanks for the changes, @317brian ! I have left some suggestions.

Given that this is an experimental feature, we can proceed with the docs that we have so far.

As the feature evolves and is hardened, we will enrich these docs to include some more technical details.

[kfaraz]

I don't think this heading aligns with the rest of headings.

Also, the type of compaction is not really much of a choice as say how partioning type is a choice (range or hashed or dynamic, where we are choosing three different paths that give you 3 different results).

We should just call this Ways to run compaction or something in a similar vein.

[kfaraz]

This is not exactly correct. It doesn't make a lot of sense to "update a datasource" unless you mean adding data to a datasource.

Moreover, we shouldn't even look at this as a two step process, rather as an opt-in behaviour. Any ingestion job that wants to run concurrently with other ingestion jobs needs to use the correct lock types.

Please see the other suggestion.

[kfaraz]

Suggested change

	If you enable automatic compaction, you can use concurrent append and replace to concurrently compact data as you ingest it for streaming and legacy JSON-based batch ingestion.
	This feature allows you to safely replace the existing data in an interval of a datasource while new data is being appended to that interval. One of the most common applications of this is appending new data (using say streaming ingestion) to an interval while compaction of that interval is already in progress.

[kfaraz]

Suggested change

	Setting up concurrent append and replace is a two-step process. The first is to update your datasource and the second is to update your ingestion job.
	Using concurrent append and replace in the following scenarios can be beneficial:
	- If the job with an `APPEND` task and the job with a `REPLACE` task have the same segment granularity. For example, when a datasource and its streaming ingestion job have the same granularity.
	- If the job with an `APPEND` task has a finer segment granularity than the replacing job.
	You can enable concurrent append and replace by ensuring the following:
	- The append task (with `appendToExisting` set to `true`) has `taskLockType` set to `APPEND` in the task context.
	- The replace task (with `appendToExisting` set to `false`) has `taskLockType` set to `REPLACE` in the task context.
	- The segment granularity of the append task is equal to or finer than the segment granularity of the replace task.

[kfaraz]

This point should be in a note or warning block.

Two more points to call out are that:

At any point in time
- There can only be a single task that holds a `REPLACE` lock on a given interval of a datasource.
- There may be multiple tasks that hold `APPEND` locks on a given interval of a datasource and append data to that interval simultaneously.

[317brian]

renders as

[317brian]

@kfaraz and @AmatyaAvadhanula I think I got all your comments. I manually made the changes for the comment suggestions since I already had the page open

[kfaraz]

Minor change, otherwise LGTM.