Skip to content
This repository has been archived by the owner on Jan 14, 2025. It is now read-only.

Commit

Permalink
remove slow timing comparsion
Browse files Browse the repository at this point in the history
  • Loading branch information
dougbrn authored and wilsonbb committed Apr 4, 2024
1 parent ed50645 commit 07fc47d
Showing 1 changed file with 1 addition and 70 deletions.
71 changes: 1 addition & 70 deletions docs/tutorials/using_ray_with_the_ensemble.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"source": [
"# Using Dask on Ray with the Ensemble\n",
"\n",
"[Ray](https://docs.ray.io/en/latest/ray-overview/index.html) is an open-source unified framework for scaling AI and Python applications. Ray provides a scheduler for Dask ([dask_on_ray](https://docs.ray.io/en/latest/ray-more-libs/dask-on-ray.html)) which allows you to build data analyses using Dask’s collections and execute the underlying tasks on a Ray cluster. We have found with TAPE that the Ray scheduler is often more performant than Dasks scheduler. Ray can be used on TAPE using the setup shown in the following example."
"[Ray](https://docs.ray.io/en/latest/ray-overview/index.html) is an open-source unified framework for scaling AI and Python applications. Ray provides a scheduler for Dask ([dask_on_ray](https://docs.ray.io/en/latest/ray-more-libs/dask-on-ray.html)) which allows you to build data analyses using Dask’s collections and execute the underlying tasks on a Ray cluster. Ray can be used on TAPE using the setup shown in the following example."
]
},
{
Expand Down Expand Up @@ -86,75 +86,6 @@
" calc_sf2, use_map=False\n",
") # use_map is false as we repartition naively, splitting per-object sources across partitions"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c5692d75",
"metadata": {},
"source": [
"## Timing Comparison\n",
"\n",
"As mentioned above, we generally see that Ray is more performant than Dask. Below is a simple timing comparison."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "f128cdbf",
"metadata": {},
"source": [
"### Ray Timing"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dd960e10",
"metadata": {},
"outputs": [],
"source": [
"%%time\n",
"\n",
"ens = Ensemble(client=False) # Do not use a client\n",
"ens.from_dataset(\"s82_qso\", sorted=True)\n",
"ens.source = ens.source.repartition(npartitions=10)\n",
"ens.batch(calc_sf2, use_map=False)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "228e5114",
"metadata": {},
"source": [
"### Dask Timing"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "24a8f466",
"metadata": {},
"outputs": [],
"source": [
"disable_dask_on_ray() # unsets the dask_on_ray configuration settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1552c2b8",
"metadata": {},
"outputs": [],
"source": [
"%%time\n",
"\n",
"ens = Ensemble()\n",
"ens.from_dataset(\"s82_qso\", sorted=True)\n",
"ens.source = ens.source.repartition(npartitions=10)\n",
"ens.batch(calc_sf2, use_map=False).compute()"
]
}
],
"metadata": {
Expand Down

0 comments on commit 07fc47d

Please sign in to comment.