qa: increase the http.maxRequestBuffer to 100MB and enable the git debug logs

Reviewed-by: Venky Shankar <[email protected]>

mds: add an asok command to dump export states

Reviewed-by: Venky Shankar <[email protected]>

mgr/dashboard: Replacing bootstrap modal service with cds modal service

Reviewed-by: Afreen Misbah <[email protected]>

…ecs-add-supported-squid-distros

doc/start: add supported Squid distros

Reviewed-by: Anthony D'Atri <[email protected]>

Fixes: https://tracker.ceph.com/issues/68026
Signed-off-by: Milind Changire <[email protected]>

fallocate now returns EOPNOTSUPP for mode 0

Signed-off-by: Milind Changire <[email protected]>

Signed-off-by: Milind Changire <[email protected]>

Signed-off-by: Milind Changire <[email protected]>

…wners

CODEOWNERS: let orchestration cover all of python-common

Reviewed-by: Adam King <[email protected]>
Reviewed-by: Ernesto Puerta <[email protected]>
Reviewed-by: Redouane Kachach <[email protected]>

mgr/cephadm: Change NVME image version to 1.3.3.

We want to use this location for files to be shared between
the cephadm binary and cephadm mgr module so it must be included
as part of the zipapp

Signed-off-by: Adam King <[email protected]>

To be shared between the binary and mgr module.

Signed-off-by: Adam King <[email protected]>

Otherwise the unit tests will fail trying to import from here.

Signed-off-by: Adam King <[email protected]>

Additionally, remove the image defaults from the two previous locations
and update the check for the number of instances of "quay" and "docker"
in these areas respectively.

Fixes: https://tracker.ceph.com/issues/68120

Signed-off-by: Adam King <[email protected]>

mgr/nfs: add NFSv3 protocol to default protocols

Reviewed-by: John Mulligan <[email protected]>

osd: add clear_shards_repaired command

Reviewed-by: Ronen Friedman <[email protected]>

librados: dont duplicate class names in member definitions

Reviewed-by: Nitzan Mordechai <[email protected]>

…norelist

qa: Add ignorelist entries for reef-x tests

Reviewed-by: Laura Flores <[email protected]>
Reviewed-by: Yuri Weinstein <[email protected]>

src/include/uuid.h: fix for boost 1.86.0

Reviewed-by: Casey Bodley <[email protected]>

os: remove unused btrfs_ioctl.h and tests

Reviewed-by: Patrick Donnelly <[email protected]>
Reviewed-by: Radoslaw Zarzynski <[email protected]>
Reviewed-by: Anthony D'Atri <[email protected]>

…c-network

cephadm: make bootstrap default to "global" section for public_network setting

Reviewed-by: Redouane Kachach <[email protected]>

mgr/cephadm: offline host handling for maintenance exit command

Reviewed-by: John Mulligan <[email protected]>

…date

smb: prevent switching between clustering modes

Reviewed-by: Adam King <[email protected]>
Reviewed-by: Avan Thakkar <[email protected]>

cephadm: use shared spot for default container image names

Reviewed-by: John Mulligan <[email protected]>
Reviewed-by: Redouane Kachach <[email protected]>

mgr/dashboard: rm nvmeof conf based on its daemon name

Reviewed-by: Adam King <[email protected]>
Reviewed-by: Afreen Misbah <[email protected]>

cephadm/smb: convert config object to use a dataclass

Reviewed-by: Adam King <[email protected]>

…mary_bi is empty

Fixes: https://tracker.ceph.com/issues/68787
Signed-off-by: Xuehan Xu <[email protected]>

osd: revert PR#57582

Reviewed-by: Samuel Just <[email protected]>

…-wrt-snap-schedule

doc/cephfs: add note to remind user about snap-schedule during volume delete

Change the subordinating conjunction "Whereas" followed by a comma to
the less grammatically-incorrect "Although". I've been meaning to do
this since 22 Mar 2023.

Signed-off-by: Zac Dover <[email protected]>

doc: s/Whereas,/Although/

We disabled dwz in 2017, then re-enabled it in 2021. In my tests I observe dwz
adding about 50% to build times in exchange for 33% space-saving.

Signed-off-by: Zack Cerza <[email protected]>

…mplates

The current documentation lacks a lot of other available templates as the source code here:

https://github.com/ceph/ceph/tree/main/src/pybind/mgr/cephadm/templates/services

Signed-off-by: Alexon Oliveira <[email protected]>

The current documentation lacks a lot of other available templates as
the source code here:
https://github.com/ceph/ceph/tree/main/src/pybind/mgr/cephadm/templates/services

v2 - This updated commit fixes the j2 extensions I left in the previous one wrongly.
v3 - This updated commit changes the title as per PR's comment
orientation

Signed-off-by: Alexon Oliveira <[email protected]>

Before, we were updating the balancer status by
iterating through all pg upmap entires. This was
affecting the loading time of other mgr modules
on clusters with a large number of pgs (600+).

This can be optimized by simply pulling from
the incremental.

Fixes: https://tracker.ceph.com/issues/68657
Signed-off-by: Laura Flores <[email protected]>

This addition gives users the option of enabling/disabling this feature.

They can do so by running:
ceph config set mgr mgr/balancer/update_pg_upmap_activity <True|False>

The feature is off by default.

Signed-off-by: Laura Flores <[email protected]>

Signed-off-by: Laura Flores <[email protected]>

condition

Signed-off-by: Leonid Chernin <[email protected]>

This reverts commit a9de8be.

Signed-off-by: Matan Breizman <[email protected]>

crimson/osd/pg: trigger PG::wait_for_active_blocker only when the pg is going active

Reviewed-by: Samuel Just <[email protected]>
Reviewed-by: Matan Breizman <[email protected]>

crimson/osd/shard_services: fix dangling reference caused by rvalue reference of ShardSercies::dispatch_context()

Reviewed-by: Matan Breizman <[email protected]>

crimson/osd/backfill_state: discard CancelBackfill events when the backfill is done

Reviewed-by: Matan Breizman <[email protected]>

crimson/osd/osd_operations/client_request: no need to hold head's obc lock throughout the recovery of clones

Reviewed-by: Matan Breizman <[email protected]>

crimson/osd/pg: make sure ops are sent to backfill targets if and only if the object has been pushed.

Reviewed-by: Samuel Just <[email protected]>

* refs/pull/60236/head:
	MDS/CDir: return as early as possible from CDir::should_split_fast()

Reviewed-by: Patrick Donnelly <[email protected]>
Reviewed-by: Dhairya Parmar <[email protected]>
Reviewed-by: Venky Shankar <[email protected]>

…d hostname

Fixes: https://tracker.ceph.com/issues/68681

Signed-off-by: Shweta Bhosale <[email protected]>

Not only in the mClock docs, but also in the reference
of the options that are disabled.

Otherwise users are bound to miss it, and surprised
why their options are ignored or reset.

Signed-off-by: Niklas Hambüchen <[email protected]>

os/bluestore: log txc details in slow op notification on committed_kv

…t-buffer-size

test/erasure-code: increase ceph_erasure_code_benchmark default buffe…

…PP-for-mode-0

client: return EOPNOTSUPP for fallocate with mode 0

Reviewed-by: Venky Shankar <[email protected]>

I have a new script added for starting the kcli cluster called
quick-bootstrap.sh

The goal is to use that script to download the ceph image on local
(rather than inside vm) and then copy them over to all the vms that is
being spawned by the kcli. This way all the hosts will get the ceph
image which will make the deployment loads faster.

Another thing I added is to add some dnf.conf to improve
parallel_downlaods and get the fastest server to install deps

eg:
```
╰─$ ./quick-bootstrap.sh -h                                                                                    255 ↵
+ set +x
Usage: ./quick-bootstrap.sh [OPTIONS]

Options:
  -u, --use-cached-image     Uses the existing podman image in local. Only use this if there is such an image present.
  -dir, --ceph-dir             Use this to provide the local ceph directory. eg. --ceph-dir=/path/to/ceph
  -e, --expanded-cluster     To add all the hosts and deploy OSDs on top of it.
  -h, --help             Display this help message.

Example:
  ./quick-bootstrap.sh --use-cached-image

```

```
./quick-bootstrap.sh -u --ceph-dir=/home/nia/projects/ceph
```

Signed-off-by: Nizamudeen A <[email protected]>

also remove obsolete admin commands

Fixes: https://tracker.ceph.com/issues/68818

Signed-off-by: Yuval Lifshitz <[email protected]>

doc/rgw/notification: add missing admin commands

Reviewed-by: Anthony D'Atri <[email protected]>

…_to_drain_non_existing_host_must_fail

mgr/cephadm: ceph orch host drain command to return error for invalid hostname

Reviewed-by: Adam King <[email protected]>
Reviewed-by: Anuradha Gadge <[email protected]>

…-read

osd,crimson/osd: rework of replica read and related state

Reviewed-by: Matan Breizman <[email protected]>

Add whitespace to match typical python formatting as flake8 cares about
the formatting of the embedded expressions now.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Fix various errors found by running mypy with python 3.12 on the
python-common subtree. Uses a Protocol as a stand-in for actual file
system integration objects.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Fixes `AttributeError: 'called_with' is not a valid assertion` errors
that appear when running the unit tests with python3.12.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

flake8 flagged unused imports in this file due to the use of old style
type comments. This file was easy to convert to new style annotations so
that was done.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Tell flake8 not to flag unused imports when those imports are part of an
old-style type comment. This was done rather than updating the type annotations
to the latest style because I am less familiar with this file and there are
many type comments.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Tell flake8 not to flag unused imports when those imports are part of an
old-style type comment. This was done rather than updating the type annotations
to the latest style because I am less familiar with this file and there are
many type comments.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Tell flake8 not to flag unused imports when those imports are part of an
old-style type comment. This was done rather than updating the type annotations
to the latest style because I am less familiar with this file and there are
many type comments.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Tell flake8 not to flag unused imports when those imports are part of an
old-style type comment. This was done rather than updating the type annotations
to the latest style because I am less familiar with this file and there are
many type comments.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Rename a single letter variable name that was flagged by flake8. The
unique name `lbl` was chosen as it was short and didn't conflict with
any existing uses of the variable `label` in this file.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Fix a string that should not be an f-string by removing the f-string
(f) prefix. Error found by newer python/flake8 version.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Tell flake8 not to flag unused imports when those imports are part of an
old-style type comment. This was done rather than updating the type
annotations to the latest style because I am less familiar with this
file and there are many type comments.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Update some function typing from the old comment based style to the current
type annotations style. Not only does this modernize the code but it fixes
issues found by newer versions of flake8 that were flagging types only
referenced in type comments as unused imports.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Update the tox.ini environments that lint the sources by splitting the
old lint env into one that just runs flake8 and another that runs the
rstcheck tool. I strongly dislike envs that do more than one thing and
make it hard to check specific files.

Additionally, the main point of this change was to drop the old
`requirements-lint.txt` requirements file that specified an extremely
old version of flake8 that did not work correctly with python 3.12.  The
flake8 of src/pybind/mgr was not pinned so we now match that behavior in
src/python-common, which is heavily used by the mgr modules.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

The file is no longer used for anything.

Signed-off-by: John Mulligan <[email protected]>

Match recent(ish) changes to CMakeLists.txt for src/pybind/mgr and
stop hard-coding a list of tox envs to run when testing is triggered
via cmake.
Instead, automatically reuse the defaults in tox.ini. This is good
because we recently changed those defaults and this change makes
'make check' run the same set of tests and checks again.

Signed-off-by: John Mulligan <[email protected]>

Update some function typing from the old comment based style to the current
type annotations style. Not only does this modernize the code but it fixes
issues found by newer versions of flake8 that were flagging types only
referenced in type comments as unused imports.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Update some function typing from the old comment based style to the current
type annotations style. Not only does this modernize the code but it fixes
issues found by newer versions of flake8 that were flagging types only
referenced in type comments as unused imports.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Fix extra white space found in a f-string that was flagged by a newer
version of flake8.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Update some function typing from the old comment based style to the current
type annotations style. Not only does this modernize the code but it fixes
issues found by newer versions of flake8 that were flagging types only
referenced in type comments as unused imports.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Update some function typing from the old comment based style to the current
type annotations style. Not only does this modernize the code but it fixes
issues found by newer versions of flake8 that were flagging types only
referenced in type comments as unused imports.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Update some function typing from the old comment based style to the current
type annotations style. Not only does this modernize the code but it fixes
issues found by newer versions of flake8 that were flagging types only
referenced in type comments as unused imports.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Update some function typing from the old comment based style to the current
type annotations style. Not only does this modernize the code but it fixes
issues found by newer versions of flake8 that were flagging types only
referenced in type comments as unused imports.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Update some function typing from the old comment based style to the current
type annotations style. Not only does this modernize the code but it fixes
issues found by newer versions of flake8 that were flagging types only
referenced in type comments as unused imports.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Update some function typing from the old comment based style to the current
type annotations style. Not only does this modernize the code but it fixes
issues found by newer versions of flake8 that were flagging types only
referenced in type comments as unused imports.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Fixes `AttributeError: 'called_with' is not a valid assertion` errors
that appear when running the unit tests with python3.12.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Add a context manager that wraps the normal mock ctx context manager -
this one disables bootstrap rollback by default. Use it in the tests
that call command_bootstrap. The unit tests are older than the boostrap
rollback behavior and therefore don't need it. However, this was not
previously a problem. In python 3.12 (or 3.11?) changes to the shutil module
prevent shutil.rmtree from working properly with the fake file system
that cephadm tests rely upon heavily.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Mock out calls to shutil.rmtree in test_deploy_and_rm_iscsi. The test
never asserts that things in the file system are removed, so making the
call a no-op is safe. Calling shutil.rmtree with the fake file system
enabled recently became a problem.  In python 3.12 (or 3.11?) changes to
the shutil module function shutil.rmtree are incompatible with the fake
file system that cephadm tests rely upon heavily.

Part of an effort to get ceph tox environments passing on Python 3.12.

Signed-off-by: John Mulligan <[email protected]>

Allow the test env to use the latest flake8. This gets us a working
version of flak8 on python3.12 and matches the tox configuration in
src/pybind/mgr.

Signed-off-by: John Mulligan <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

Signed-off-by: Max Kellermann <[email protected]>

This header uses `std::string`, but not `std::system_error`.

Signed-off-by: Max Kellermann <[email protected]>