Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sample notebook: model alignment #75

Merged
merged 1 commit into from
Jul 24, 2024
Merged

Sample notebook: model alignment #75

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
370 changes: 370 additions & 0 deletions notebooks/model_alignment.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,370 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Aligning a model on Arcee Cloud\n",
"\n",
"In this notebook, you will learn how to align a model with Supervised Fine Tuning on Arcee Cloud.\n",
"\n",
"In order to run this demo, you need a Starter account on Arcee Cloud. Please see our [pricing](https://www.arcee.ai/pricing) page for details.\n",
"\n",
"The Arcee documentation is available at [docs.arcee.ai](https://docs.arcee.ai/deployment/start-deployment)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prerequisites\n",
"\n",
"Please [sign up](https://app.arcee.ai/account/signup) to Arcee Cloud and create an [API key](https://docs.arcee.ai/getting-arcee-api-key/getting-arcee-api-key).\n",
"\n",
"Then, please update the cell below with your API key. Remember to keep this key safe, and **DON'T COMMIT IT to one of your repositories**."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%env ARCEE_API_KEY=YOUR_API_KEY"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a new Python environment (optional but recommended) and install [arcee-python](https://github.com/arcee-ai/arcee-python)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment the next three lines to create a virtual environment\n",
"#!pip install -q virtualenv\n",
"#!virtualenv -q arcee-cloud\n",
"#!source arcee-cloud/bin/activate\n",
"\n",
"%pip install -qU arcee-py"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will also need the Hugging Face [`datasets`](https://huggingface.co/datasets) library to download an alignment model from the Hugging Face hub."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -q datasets"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import arcee\n",
"import pprint\n",
"import pandas as pd\n",
"from datasets import load_dataset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Preparing our dataset\n",
"\n",
"Model alignment requires a dataset with questions and answers. At the moment, Arcee Cloud requires this dataset to be in the CSV format, with one column for questions and one for answers.\n",
"\n",
"Let's download the [`reasoning-sharegpt`](https://huggingface.co/datasets/arcee-ai/reasoning-sharegpt) dataset from the Hugging Face hub. \n",
"\n",
"This dataset contains close to 30,000 single-turn question-answers pairs focused on step-by-step reasoning."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dataset_name = \"arcee-ai/reasoning-sharegpt\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dataset = load_dataset(dataset_name)\n",
"print(dataset)\n",
"pprint.pprint(dataset['train'][0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, let's format it in two columns with just the questions and the answers, then save it to the CSV format."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"data = [{\n",
" \"questions\": item['conversations'][0]['value'],\n",
" \"answers\": item['conversations'][1]['value']\n",
"} for item in dataset['train']]\n",
"\n",
"df = pd.DataFrame(data)\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The last preparation step is to save the dataset in CSV format."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"csv_name = \"reasoning-sharegpt.csv\"\n",
"\n",
"df.to_csv(csv_name, index=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Uploading our dataset\n",
"\n",
"Now that we've prepared the dataset, we can upload it to Arcee Cloud with the `upload_qa_pairs_from_csv()` API."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"help(arcee.upload_qa_pairs_from_csv)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"qa_set=\"reasoning-share-gpt\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"arcee.upload_qa_pairs_from_csv(\n",
" qa_set=qa_set,\n",
" csv_path=f\"./{csv_name}\",\n",
" question_column=\"questions\",\n",
" answer_column=\"answers\"\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Aligning the model\n",
"\n",
"We can now use the dataset for model alignment. \n",
"\n",
"Here, we will align the [Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) model. We could pick any model available on the Hugging Face hub, or a model we've already worked with on Arcee Cloud.\n",
"\n",
"Let's launch the alignment job with the `start_alignment()` API."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"help(arcee.start_alignment)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"alignment_name = \"llama-3-8B-reasoning-share-gpt\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response=arcee.start_alignment(alignment_name=alignment_name,\n",
" qa_set=qa_set,\n",
" hf_model=\"meta-llama/Meta-Llama-3-8B\"\n",
")\n",
"print(response)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from time import sleep\n",
"\n",
"while True:\n",
" response = arcee.alignment_status(alignment_name)\n",
" if response[\"processing_state\"] == \"processing\":\n",
" print(\"Alignment is in progress. Waiting 5 minutes before checking again.\")\n",
" sleep(300)\n",
" else:\n",
" print(response)\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deploying our aligned model\n",
"\n",
"Once alignment is complete, we can deploy and test the aligned model. As part of the Arcee Cloud free tier, this is free of charge and the endpoint will be automatically shut down after 2 hours.\n",
"\n",
"Deployment should take 5-7 minutes. Please see the model deployment sample notebook for details."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"deployment_name = \"llama-3-8B-reasoning-share-gpt\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response = arcee.start_deployment(deployment_name=deployment_name, alignment=alignment_name)\n",
"\n",
"while True:\n",
" response = arcee.deployment_status(deployment_name)\n",
" if response[\"deployment_processing_state\"] == \"pending\":\n",
" print(\"Deployment is in progress. Waiting 30 seconds before checking again.\")\n",
" sleep(30)\n",
" else:\n",
" print(response)\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once the model endpoint is up and running, we can prompt the model with a domain-specific question."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"query = \"Give me instructions to change the oil on my Harley-Davidson.\"\n",
"\n",
"response = arcee.generate(deployment_name=deployment_name, query=query)\n",
"print(response[\"text\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Stopping our deployment\n",
"\n",
"When we're done working with our model, we should stop the deployment to save resources and avoid unwanted charges.\n",
"\n",
"The `stop_deployment()` API only requires the deployment name."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"arcee.stop_deployment(deployment_name=deployment_name)\n",
"arcee.deployment_status(deployment_name)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This concludes the model alignment demonstration. Thank you for your time!\n",
"\n",
"If you'd like to know more about using Arcee Cloud in your organization, please visit the [Arcee website](https://www.arcee.ai), or contact [[email protected]](mailto:[email protected]).\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.4"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading