Skip to content

Commit

Permalink
Initial version (#74)
Browse files Browse the repository at this point in the history 
Co-authored-by: Julien Simon <[email protected]>
Co-authored-by: Jacobsolawetz <[email protected]>
  • Loading branch information
@juliensimon @Jacobsolawetz
3 people authored Jul 22, 2024
1 parent 0101469 commit f54fc09
Showing 1 changed file with 294 additions and 0 deletions.
294 changes: 294 additions & 0 deletions notebooks/model_merging.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,294 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Merging a model on Arcee Cloud\n",
"\n",
"In this notebook, you will learn how to merge a model on Arcee Cloud.\n",
"\n",
"You can run this demo for free thanks to the Arcee free tier, which allows unlimited base merges.\n",
"\n",
"The Arcee documentation is available at [docs.arcee.ai](https://docs.arcee.ai/deployment/start-deployment)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prerequisites\n",
"\n",
"Please [sign up](https://app.arcee.ai/account/signup) to Arcee Cloud and create an [API key](https://docs.arcee.ai/getting-arcee-api-key/getting-arcee-api-key).\n",
"\n",
"Then, please update the cell below with your API key. Remember to keep this key safe, and **DON'T COMMIT IT to one of your repositories**."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%env ARCEE_API_KEY=YOUR_API_KEY"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a new Python environment (optional but recommended) and install [arcee-python](https://github.com/arcee-ai/arcee-python)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment the next three lines to create a virtual environment\n",
"#!pip install -q virtualenv\n",
"#!virtualenv -q arcee-cloud\n",
"#!source arcee-cloud/bin/activate\n",
"\n",
"%pip install -q arcee-py"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import arcee"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Merging a model\n",
"\n",
"In Arcee Cloud, merging is based on the [`mergekit`](https://github.com/arcee-ai/mergekit) open-source library. The merging operation must be defined in a configuration file: models to merge, merging algorithm and parameters."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here's an example where we merge two variants of the [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) model hosted on the Hugging Face hub:\n",
"* The instruction-fine tuned [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2),\n",
"* [BioMistral](https://huggingface.co/BioMistral/BioMistral-7B), tailored for the biomedical domain.\n",
"\n",
"As we use both models in the same proportions, we would expect the merge to inherit the alignment of the first model and the domain knowledge of the second model.\n",
"\n",
"You can find more examples of merging configurations in the [`mergekit`](https://github.com/arcee-ai/mergekit) documentation and at [docs.arcee.ai](https://docs.arcee.ai/merging/start-yaml-merge)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%writefile mistral_7b_merge.yaml\n",
"models:\n",
" - model: mistralai/Mistral-7B-Instruct-v0.2\n",
" parameters:\n",
" density: 0.5\n",
" weight: 0.5\n",
" - model: BioMistral/BioMistral-7B\n",
" parameters:\n",
" density: 0.5\n",
" weight: 0.5\n",
"merge_method: ties\n",
"base_model: mistralai/Mistral-7B-v0.1\n",
"parameters:\n",
" normalize: false\n",
" int8_mask: true\n",
"dtype: float16"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We're now ready to launch the merging operation. We'll use the `mergekit_yaml()` API and simply pass the names of the merging operation and of the configuration file."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"merging_name = \"Mistral 7B merge\"\n",
"merging_yaml = \"mistral_7b_merge.yaml\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"help(arcee.mergekit_yaml)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response = arcee.mergekit_yaml(merging_name, merging_yaml)\n",
"print(response)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's wait for the merging operation to complete. It should only takes a few minutes.\n",
"\n",
"The `merging_status` API lets us query the current state of the merging operation."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"help(arcee.merging_status)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from time import sleep\n",
"\n",
"while True:\n",
" response = arcee.merging_status(merging_name)\n",
" if response[\"processing_state\"] != \"completed\":\n",
" print(\"Merging is in progress. Waiting 30 seconds before checking again.\")\n",
" sleep(30)\n",
" else:\n",
" print(response)\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deploying our merged model\n",
"\n",
"Once merging is complete, we can deploy and test the merged model. As part of the Arcee Cloud free tier, this is free of charge and the endpoint will be automatically shut down after 2 hours.\n",
"\n",
"Deployment should take 5-7 minutes. Please see the model deployment sample notebook for details."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"deployment_name = \"Mistral 7B merge deployment\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response = arcee.start_deployment(deployment_name=deployment_name, merging=merging_name)\n",
"\n",
"while True:\n",
" response = arcee.deployment_status(deployment_name)\n",
" if response[\"deployment_processing_state\"] == \"pending\":\n",
" print(\"Deployment is in progress. Waiting 30 seconds before checking again.\")\n",
" sleep(30)\n",
" else:\n",
" print(response)\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once the model endpoint is up and running, we can prompt the model with a domain-specific question."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"query = \"As a biomedical expert, you provide the most accurate and detailed answer possible. \\\n",
" Explain the role of enzymes in the digestive system and give two examples of molecular reactions catalyzed by enzymes.\"\n",
"\n",
"response = arcee.generate(deployment_name=deployment_name, query=query)\n",
"print(response[\"text\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Stopping our deployment\n",
"\n",
"When we're done working with our model, we should stop the deployment to save resources and avoid unwanted charges.\n",
"\n",
"The `stop_deployment()` API only requires the deployment name."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"arcee.stop_deployment(deployment_name=deployment_name)\n",
"arcee.deployment_status(deployment_name)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This concludes the model merging demonstration. Thank you for your time!\n",
"\n",
"If you'd like to know more about using Arcee Cloud in your organization, please visit the [Arcee website](https://www.arcee.ai), or contact [[email protected]](mailto:[email protected]).\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.4"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

0 comments on commit f54fc09

Please sign in to comment.