chore: use ruff in the project #87

PatrykWyzgowski · 2024-10-11T07:32:03Z

No description provided.

…eginning of docstrigns.

# Conflicts: # .pre-commit-config.yaml # packages/ragbits-core/src/ragbits/core/embeddings/litellm.py # packages/ragbits-core/src/ragbits/core/llms/base.py # packages/ragbits-core/src/ragbits/core/llms/clients/base.py # packages/ragbits-core/src/ragbits/core/prompt/base.py # packages/ragbits-core/src/ragbits/core/prompt/lab/app.py # packages/ragbits-core/src/ragbits/core/prompt/parsers.py # packages/ragbits-core/src/ragbits/core/prompt/prompt.py # packages/ragbits-core/src/ragbits/core/vector_store/base.py # packages/ragbits-core/src/ragbits/core/vector_store/chromadb_store.py # packages/ragbits-document-search/src/ragbits/document_search/_main.py # packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py # packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured.py # packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/noop.py # pyproject.toml # scripts/update_ragbits_package.py

# Conflicts: # packages/ragbits-core/examples/chromadb_example.py # packages/ragbits-core/src/ragbits/core/embeddings/litellm.py # packages/ragbits-core/src/ragbits/core/prompt/lab/app.py # packages/ragbits-core/src/ragbits/core/prompt/prompt.py # packages/ragbits-core/src/ragbits/core/prompt/promptfoo.py # packages/ragbits-core/tests/unit/prompts/test_prompt.py # packages/ragbits-document-search/tests/unit/test_providers.py

github-actions · 2024-10-11T08:59:15Z

Code Coverage Summary

Filename                                                                                                     Stmts    Miss  Cover    Missing
---------------------------------------------------------------------------------------------------------  -------  ------  -------  ----------------------------------------
packages/ragbits-core/src/ragbits/core/__init__.py                                                               0       0  100.00%
packages/ragbits-core/src/ragbits/core/config.py                                                                 6       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/__init__.py                                                   11       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/base.py                                                        4       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/exceptions.py                                                 14       6  57.14%   7-8, 17, 26-27, 36
packages/ragbits-core/src/ragbits/core/embeddings/litellm.py                                                    27      19  29.63%   5-6, 45-53, 70-86
packages/ragbits-core/src/ragbits/core/embeddings/local.py                                                      39      25  35.90%   9-10, 34-44, 56-72, 76-78, 82-83
packages/ragbits-core/src/ragbits/core/embeddings/noop.py                                                        4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/__init__.py                                                         15       8  46.67%   28-41
packages/ragbits-core/src/ragbits/core/llms/base.py                                                             31       2  93.55%   33, 52
packages/ragbits-core/src/ragbits/core/llms/factory.py                                                          18       2  88.89%   47, 60
packages/ragbits-core/src/ragbits/core/llms/litellm.py                                                          24       4  83.33%   7-8, 53, 84
packages/ragbits-core/src/ragbits/core/llms/local.py                                                            23      10  56.52%   7-8, 41-46, 56, 68-69
packages/ragbits-core/src/ragbits/core/llms/types.py                                                             9       2  77.78%   25, 29
packages/ragbits-core/src/ragbits/core/llms/clients/__init__.py                                                  4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/clients/base.py                                                     23       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/clients/exceptions.py                                               14       6  57.14%   7-8, 17, 26-27, 36
packages/ragbits-core/src/ragbits/core/llms/clients/litellm.py                                                  49      10  79.59%   9-10, 69, 107, 121-126
packages/ragbits-core/src/ragbits/core/llms/clients/local.py                                                    36      12  66.67%   10-11, 61-69, 90-101
packages/ragbits-core/src/ragbits/core/prompt/__init__.py                                                        2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/base.py                                                           18       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/parsers.py                                                        35       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/prompt.py                                                        110       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/__init__.py                                              2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/prompt_discovery.py                                     33       2  93.94%   54-55
packages/ragbits-core/src/ragbits/core/utils/_pyproject.py                                                      28       0  100.00%
packages/ragbits-core/src/ragbits/core/utils/config_handling.py                                                  9       0  100.00%
packages/ragbits-core/src/ragbits/core/utils/decorators.py                                                      29       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_store/__init__.py                                                 13       1  92.31%   28
packages/ragbits-core/src/ragbits/core/vector_store/base.py                                                     14       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_store/chromadb_store.py                                           71       7  90.14%   11-12, 64-71, 88, 131
packages/ragbits-core/src/ragbits/core/vector_store/in_memory.py                                                28       0  100.00%
packages/ragbits-core/tests/unit/__init__.py                                                                     0       0  100.00%
packages/ragbits-core/tests/unit/llms/__init__.py                                                                0       0  100.00%
packages/ragbits-core/tests/unit/llms/test_litellm.py                                                           64       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/__init__.py                                                        3       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/test_get_default_llm.py                                            9       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/test_get_llm_from_factory.py                                       8       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/test_has_default_llm.py                                            9       0  100.00%
packages/ragbits-core/tests/unit/prompts/__init__.py                                                             0       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_parsers.py                                                        65       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_prompt.py                                                        143       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/__init__.py                                                   0       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/prompt_classes_for_tests.py                                  30       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/test_prompt_discovery.py                                     18       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/__init__.py                    2       1  50.00%   3
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/__init__.py            3       2  33.33%   2-4
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt1.py       14       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt2.py       14       0  100.00%
packages/ragbits-core/tests/unit/utils/test_decorators.py                                                       26       2  92.31%   17, 39
packages/ragbits-core/tests/unit/utils/pyproject/test_find.py                                                   13       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_config.py                                              9       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_instace.py                                            27       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_chromadb_store.py                                           81       4  95.06%   32, 35, 40, 47
packages/ragbits-core/tests/unit/vector_stores/test_simple_vector_store.py                                      72       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/__init__.py                                         2       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/_main.py                                           69       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/__init__.py                               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/document.py                              60       3  95.00%   55, 98, 145
packages/ragbits-document-search/src/ragbits/document_search/documents/element.py                               37       2  94.59%   34, 107
packages/ragbits-document-search/src/ragbits/document_search/documents/exceptions.py                            11       5  54.55%   7-8, 17, 26-27
packages/ragbits-document-search/src/ragbits/document_search/documents/sources.py                               93      16  82.80%   13-14, 63, 76, 160-165, 203-206, 210-211
packages/ragbits-document-search/src/ragbits/document_search/ingestion/__init__.py                               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py                    28       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/__init__.py                    13       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/base.py                        14       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/dummy.py                       11       1  90.91%   33
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/__init__.py        0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/default.py        44       4  90.91%   97, 102-103, 135
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/images.py         38      15  60.53%   61-68, 75-83, 95, 108
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/pdf.py            19       6  68.42%   23, 35-43
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/utils.py          32      10  68.75%   49, 60-61, 76-79, 103-118
packages/ragbits-document-search/src/ragbits/document_search/retrieval/__init__.py                               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/__init__.py                   15       4  73.33%   39-44
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/base.py                        7       1  85.71%   32
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/llm.py                        20       9  55.00%   27-28, 45-48, 65-67
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/noop.py                        4       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/prompts.py                    16       4  75.00%   49-54
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/__init__.py                    12       1  91.67%   25
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/base.py                         6       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/noop.py                         6       0  100.00%
packages/ragbits-document-search/tests/__init__.py                                                               0       0  100.00%
packages/ragbits-document-search/tests/helpers.py                                                                3       0  100.00%
packages/ragbits-document-search/tests/integration/__init__.py                                                   0       0  100.00%
packages/ragbits-document-search/tests/integration/test_sources.py                                              23      10  56.52%   22-32, 40-45
packages/ragbits-document-search/tests/integration/test_unstructured.py                                         47      10  78.72%   46-52, 65-71
packages/ragbits-document-search/tests/unit/__init__.py                                                          0       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_processor.py                                          17       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_search.py                                             75       0  100.00%
packages/ragbits-document-search/tests/unit/test_documents.py                                                   13       0  100.00%
packages/ragbits-document-search/tests/unit/test_elements.py                                                    15       0  100.00%
packages/ragbits-document-search/tests/unit/test_local_file_source.py                                           13       0  100.00%
packages/ragbits-document-search/tests/unit/test_providers.py                                                   31       0  100.00%
packages/ragbits-document-search/tests/unit/test_sources.py                                                     25       0  100.00%
TOTAL                                                                                                         2172     226  89.59%

Diff against main

Filename                                                                                                    Stmts    Miss  Cover
--------------------------------------------------------------------------------------------------------  -------  ------  --------
packages/ragbits-core/src/ragbits/core/embeddings/litellm.py                                                   -1       0  -2.51%
packages/ragbits-core/src/ragbits/core/llms/base.py                                                            -2      -2  +5.67%
packages/ragbits-core/src/ragbits/core/llms/litellm.py                                                         -1       0  -0.67%
packages/ragbits-core/src/ragbits/core/llms/local.py                                                           -1       0  -1.81%
packages/ragbits-core/src/ragbits/core/llms/types.py                                                           +1       0  +2.78%
packages/ragbits-core/src/ragbits/core/llms/clients/litellm.py                                                 -1       0  -0.41%
packages/ragbits-core/src/ragbits/core/llms/clients/local.py                                                   -1       0  -0.90%
packages/ragbits-core/src/ragbits/core/prompt/parsers.py                                                       +1       0  +100.00%
packages/ragbits-core/src/ragbits/core/prompt/prompt.py                                                         0      -2  +1.82%
packages/ragbits-core/src/ragbits/core/utils/_pyproject.py                                                     +3       0  +100.00%
packages/ragbits-core/src/ragbits/core/utils/decorators.py                                                     +1       0  +100.00%
packages/ragbits-core/src/ragbits/core/vector_store/chromadb_store.py                                          +2       0  +0.28%
packages/ragbits-core/tests/unit/llms/test_litellm.py                                                          +1       0  +100.00%
packages/ragbits-core/tests/unit/llms/factory/test_get_default_llm.py                                          +1       0  +100.00%
packages/ragbits-core/tests/unit/llms/factory/test_has_default_llm.py                                          +1       0  +100.00%
packages/ragbits-document-search/src/ragbits/document_search/_main.py                                          +2       0  +100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py                   -1       0  +100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/default.py       -1       0  -0.20%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/images.py        -1       0  -1.01%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/pdf.py           -1       0  -1.58%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/utils.py         -1       0  -0.95%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/__init__.py                   -1       0  -0.64%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/noop.py                       -1       0  +100.00%
TOTAL                                                                                                          -1      -4  +0.17%

Results for commit: f29285d

Minimum allowed coverage is 60%

♻️ This comment has been updated with latest results

github-actions · 2024-10-11T12:22:21Z

Trivy scanning results.

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: Slack (slack-web-hook)
════════════════════════════════════════
Slack Webhook
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/proxy/_types.py:1288
────────────────────────────────────────
1286 alert_to_webhook_url: Optional[Dict] = Field(
1287 None,
1288 [ bhook_url: {'budget_alerts': '*****************************************************************************'}`",
1289 )
────────────────────────────────────────

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: JWT (jwt-token)
════════════════════════════════════════
JWT token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA:80
────────────────────────────────────────
78 >>> encoded = jwt.encode({"some": "payload"}, "secret", algorithm="HS256")
79 >>> print(encoded)
80 [ *********************************************************************************************************
81 >>> jwt.decode(encoded, "secret", algorithms=["HS256"])
────────────────────────────────────────

.venv/lib/python3.10/site-packages/litellm/llms/huggingface_llms_metadata/hf_text_generation_models.txt (secrets)

Total: 1 (MEDIUM: 0, HIGH: 0, CRITICAL: 1)

CRITICAL: HuggingFace (hugging-face-access-token)
════════════════════════════════════════
Hugging Face Access Token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/llms/huggingface_llms_metadata/hf_text_generation_models.txt:36162
────────────────────────────────────────
36160 mncai/Llama2-7B-Active_3rd-floor-LoRA-dim64_epoch4
36161 ajcdp/CM
36162 [ Nagharjun17/*************************************
36163 BigSalmon/InformalToFormalLincoln114Paraphrase
────────────────────────────────────────

PatrykWyzgowski · 2024-10-11T12:24:27Z

Please mind that too-many-newlines-at-end-of-file (W391) is still in preview mode.

packages/ragbits-core/src/ragbits/core/prompt/promptfoo.py

packages/ragbits-core/src/ragbits/core/llms/clients/local.py

packages/ragbits-core/src/ragbits/core/prompt/parsers.py

packages/ragbits-core/tests/unit/prompts/test_prompt.py

micpst · 2024-10-14T08:32:00Z

In general, most of the code should remain unchanged, please fix the formatting problems.

packages/ragbits-core/src/ragbits/core/llms/clients/base.py

packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py

There seem to be more formatting issues than I initially noticed

.pre-commit-config.yaml

# Conflicts: # packages/ragbits-core/src/ragbits/core/prompt/lab/app.py # packages/ragbits-core/src/ragbits/core/vector_store/__init__.py # packages/ragbits-document-search/src/ragbits/document_search/_main.py

…format`

# Conflicts: # packages/ragbits-cli/src/ragbits/cli/__init__.py # packages/ragbits-core/src/ragbits/core/prompt/promptfoo.py # packages/ragbits-core/src/ragbits/core/vector_store/chromadb_store.py # packages/ragbits-document-search/src/ragbits/document_search/_main.py # packages/ragbits-document-search/src/ragbits/document_search/documents/document.py # packages/ragbits-document-search/src/ragbits/document_search/documents/sources.py

PatrykWyzgowski added 8 commits October 8, 2024 09:48

Applying ruff automatic changes to files.

f9b562e

Adding config options to support Ruff. Disabling no new line at the b…

4209360

…eginning of docstrigns.

One line docs are still pretty.

5bedf86

Applying ruff with isort (with isort).

5fcf55b

Switching to just ruff and mypy.

de78074

Switching off automatic fixes. Ruff ignores D200.

a76ab51

PatrykWyzgowski linked an issue Oct 11, 2024 that may be closed by this pull request

chore: use ruff in the project #68

Closed

PatrykWyzgowski added 5 commits October 11, 2024 10:02

Manually fixed one-line docstrings to comply with our standards.

71029fe

And one more file.

a962e5c

Fixing fixable ruff lint errors.

17b6b09

Adding reasonable fixes from --unsafe-fixes.

802188f

Mypy issues resolved.

dda1287

PatrykWyzgowski added 2 commits October 11, 2024 12:29

Fixed vast majority of ruff's alerts.

4d2d9a0

Fixed the rest of ruff's claims.

c98bda1

PatrykWyzgowski marked this pull request as ready for review October 11, 2024 12:27

PatrykWyzgowski self-assigned this Oct 11, 2024

akonarski-ds reviewed Oct 14, 2024

View reviewed changes

packages/ragbits-core/src/ragbits/core/prompt/promptfoo.py Outdated Show resolved Hide resolved

akonarski-ds previously approved these changes Oct 14, 2024

View reviewed changes

micpst reviewed Oct 14, 2024

View reviewed changes

packages/ragbits-core/src/ragbits/core/llms/clients/local.py Outdated Show resolved Hide resolved

packages/ragbits-core/src/ragbits/core/prompt/parsers.py Outdated Show resolved Hide resolved

packages/ragbits-core/tests/unit/prompts/test_prompt.py Outdated Show resolved Hide resolved

Aligning to micpst's comments.

57ef338

micpst added the chore Changes to the build process or repo setup label Oct 14, 2024

PatrykWyzgowski added 3 commits October 14, 2024 14:17

Unifying default dir argument assignment.

1246038

Unnecessary modification of first entries of prompt chat.

3d0e547

Missing corrections.

72e6989

micpst reviewed Oct 15, 2024

View reviewed changes

packages/ragbits-core/src/ragbits/core/llms/clients/base.py Show resolved Hide resolved

ludwiktrammer reviewed Oct 15, 2024

View reviewed changes

packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py Outdated Show resolved Hide resolved

PatrykWyzgowski added 2 commits October 15, 2024 16:02

Back to ABC for LLMOptions.

26b8f0b

Excluding tests from formatting.

7f40eb4

akonarski-ds reviewed Oct 16, 2024

View reviewed changes

.pre-commit-config.yaml Outdated Show resolved Hide resolved

PatrykWyzgowski and others added 7 commits October 16, 2024 10:59

Merge branch 'refs/heads/main' into 68-feat-use-ruff-in-the-project

c4140f5

# Conflicts: # packages/ragbits-core/src/ragbits/core/prompt/lab/app.py # packages/ragbits-core/src/ragbits/core/vector_store/__init__.py # packages/ragbits-document-search/src/ragbits/document_search/_main.py

After-update pre-commit run.

e5fc913

Last error deruffed.

38dd2c5

This works as intended. Let's have tests that are formatted by `ruff-…

a903dd6

…format`

Merge branch 'main' into 68-feat-use-ruff-in-the-project

f5e081f

Fix mypy and ruff issues

6a43980

akonarski-ds linked an issue Oct 24, 2024 that may be closed by this pull request

chore: run mypy outside of pre-commit #124

Closed

akonarski-ds added 5 commits October 24, 2024 13:26

Update contributing guide

5fe6eb6

Fix ci

7d23532

Fix ci

f809b4d

Add more repositories for trivy

8cf70aa

Remove unnecessary empty line

f29285d

ludwiktrammer approved these changes Oct 24, 2024

View reviewed changes

akonarski-ds merged commit d79ef6d into main Oct 24, 2024
6 checks passed

akonarski-ds mentioned this pull request Oct 24, 2024

chore: run ruff and mypy githooks #147

Closed

mhordynski deleted the 68-feat-use-ruff-in-the-project branch November 21, 2024 21:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: use ruff in the project #87

chore: use ruff in the project #87

PatrykWyzgowski commented Oct 11, 2024

github-actions bot commented Oct 11, 2024 •

edited

Loading

github-actions bot commented Oct 11, 2024 •

edited

Loading

PatrykWyzgowski commented Oct 11, 2024

micpst commented Oct 14, 2024

chore: use ruff in the project #87

chore: use ruff in the project #87

Conversation

PatrykWyzgowski commented Oct 11, 2024

github-actions bot commented Oct 11, 2024 • edited Loading

Code Coverage Summary

Diff against main

github-actions bot commented Oct 11, 2024 • edited Loading

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

.venv/lib/python3.10/site-packages/litellm/llms/huggingface_llms_metadata/hf_text_generation_models.txt (secrets)

PatrykWyzgowski commented Oct 11, 2024

micpst commented Oct 14, 2024

github-actions bot commented Oct 11, 2024 •

edited

Loading

github-actions bot commented Oct 11, 2024 •

edited

Loading