Skip to content

Commit

Permalink
revert cookbook customizations
Browse files Browse the repository at this point in the history
  • Loading branch information
pnadolny13 committed Jan 18, 2024
1 parent b915e28 commit 226df1f
Showing 1 changed file with 3 additions and 5 deletions.
8 changes: 3 additions & 5 deletions map_gpt_embeddings/cookbook.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@
Example command to call script:
```
python /Users/pnadolny/Documents/Git/GitHub/MeltanoLabs/map-gpt-embeddings/map_gpt_embeddings/cookbook.py \
--requests_filepath /Users/pnadolny/Documents/Git/GitHub/MeltanoLabs/map-gpt-embeddings/.secrets/requests.json \
--save_filepath /Users/pnadolny/Documents/Git/GitHub/MeltanoLabs/map-gpt-embeddings/.secrets/requests_output.jsonl \
python examples/api_request_parallel_processor.py \
--requests_filepath examples/data/example_requests_to_parallel_process.jsonl \
--save_filepath examples/data/example_requests_to_parallel_process_results.jsonl \
--request_url https://api.openai.com/v1/embeddings \
--max_requests_per_minute 1500 \
--max_tokens_per_minute 6250000 \
Expand Down Expand Up @@ -119,7 +119,6 @@ async def process_api_requests_from_file(
token_encoding_name: str,
max_attempts: int,
logging_level: int,
# should_stop_processing,
):
"""Processes API requests in parallel, throttling to stay under rate limits."""
# constants
Expand Down Expand Up @@ -237,7 +236,6 @@ async def process_api_requests_from_file(

# if all tasks are finished, break
if status_tracker.num_tasks_in_progress == 0:
# and should_stop_processing.is_set():
break

# main loop sleeps briefly so concurrent tasks can run
Expand Down

0 comments on commit 226df1f

Please sign in to comment.