We introduce NoRTune, a resource-efficient and reliable configuration tuning framework for Spark that leverages subspace-based Bayesian optimization and a noise-robust acquisition function. NoRTune optimizes Spark configuration by effectively reducing high-dimensional parameters without requiring time-consuming and resource-intensive determination of the target dimensionality. Moreover, it is able to select an optimal configuration with reliable performance by reducing the impact of noise.
- The base of this code is here.
Bounce uses poetry for dependency management.
Bounce requires python>=3.10. To install poetry, see here.
To install the dependencies, run
poetry install
pip install configspace==0.6.1- python 3.10
- smac 2.0.2
- botorch 0.8.5
- gpytorch 1.10
python main.py --optimizer_method nsbo --workload ${WORKLOAD} --workload_size $WORKLOAR_SIZE --max_eval 50 --acquisition aei ---model_name NoRTune