DNF: Directional Neighbourhood Fitting

DNF is a nearest-neighbor approach for photometric redshift estimation developed at the CIEMAT (Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas). DNF computes the photo-z hyperplane that best fits the directional neighbourhood of a photometric galaxy in the training sample. A detailed description of DNF is available here.

If you have any questions or suggestions, please don't hesitate to contact us at [email protected] and/or [email protected].

Citing This Work

If you use this code in your research, we kindly ask you to cite the following article:

De Vicente, J., Sánchez, E., Sevilla-Noarbe, I., 2016, Directional Neighbourhood Fitting (DNF): photometric redshifts using nearest neighbours, MNRAS, 459, 3078.
DOI: 10.1093/mnras/stw824
arXiv:1511.07623

Here is the BibTeX entry:

@ARTICLE{2016MNRAS.459.3078D,
       author = {{De Vicente}, J. and {S{\'a}nchez}, E. and {Sevilla-Noarbe}, I.},
        title = "{DNF - Galaxy photometric redshift by Directional Neighbourhood Fitting}",
      journal = {\mnras},
     keywords = {methods: data analysis, surveys, galaxies: distances and redshifts, galaxies: statistics, large-scale structure of Universe, Astrophysics - Cosmology and Nongalactic Astrophysics},
         year = 2016,
        month = jul,
       volume = {459},
       number = {3},
        pages = {3078-3088},
          doi = {10.1093/mnras/stw857},
archivePrefix = {arXiv},
       eprint = {1511.07623},
 primaryClass = {astro-ph.CO},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2016MNRAS.459.3078D},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

DNF User Manual

How to Get Started with DNF

1. Download DNF and the Data Folder:

• Clone directly the repository:

If you are familiar with using GitHub, you can clone the repository where you will find everything you need. Type git clone and paste the URL:

git clone https://github.com/ltoribiosc/DNF_photoz.git

If you are not famliar with GitHub:

• Create a New Directory: You can name your directory as you prefer. For example, you can use the following command to create a directory called DNF_photoz:

mkdir DNF_photoz

• Download the DNF, Photoz and config codes:
  Visit the official DNF GitHub space to download the latest version:

  • dnf.py: This is the main code that calculates the photo-z. You don’t need to modify anything in this file.
  • photoz.py: This code handles loading the configuration and reading the training data (train.fits) as well as the validation data or the data for which you want to calculate the photo-z (e.g., valid.fits). You don’t need to modify anything in this file.
  • config.yaml and/or config.json: This files contain the selected configuration. You must to choose what type of file do you prefer use (config.yaml or config.json). Both files have the same function. Choose the one that is easier for you to modify or that you are more familiar with and delete the other one. DNF uses config.yaml by default. You will need to modify this file, but don’t worry, we’ll explain how to do it later.

• Download the Data Folder:
  Download the data folder required to run the example. Make sure this folder contains all the necessary files. In the downloaded data folder, you will find two important files:

  • train.fits: This file is used for training DNF.
  • valid.fits: Use this file to test the quality of photo-z results.

You are now ready to begin using DNF with the provided data!

2. Run DNF with Sample Data: To run our example, you need to execute the following command:

python3 photoz.py data/train.fits data/valid.fits output_results.fits

You have just run DNF. Congratulations on a successful run!

If everything has gone well, you should now have a new file named output_results.fits. This file contains all the galaxy information from the valid.fits file, with the photozs and other values provided by DNF.

If you encounter any errors or issues, please don't hesitate to contact us at [email protected] and/or [email protected].

In this command line:

• data/train.fits is the training data file, which contains the photometric information and spectroscopic redshift data for a sample of galaxies.
• data/valid.fits is the validation data file, which contains the galaxies for which you want to calculate the photo-zs.
• output_results.fits is the results data file, which contins the galaxies of data/valid.fits files with the photo-zs and other parameters.

Running DNF with Your Own Data

If you're interested in running DNF with your own data (that's probably why you're here.), follow these steps:

1. Prepare Your Data: DNF requires two input files:

• train.fits: This file contains photometric information and spectroscopic redshift data for a sample of galaxies.
• valid.fits: You will also need the file for which you want to calculate photoz. In this case, it should be a sample of galaxies with photometric information, exactly matching the filters used in the train file.

In the example, these files were named train.fits and valid.fits, but you can use whatever names you prefer. We recommend that you save your input files in the data folder, but this is not mandatory. You can create your own directory, just remember to include the correct path when running DNF.

2. Modify the config.yaml or config.json file if is necessary: The config.yaml and config.json files contain all the necessary information for DNF to run with the required customization for each type of data and analysis. In the example, the config file was configured for the data in train.fits, but now we will explain everything it contains so you can learn how to modify it.

• "filters": Contains the names of the filters used by DNF to calculate the photo-zs, which are available in the train.fits file. In the example, we have four filters named:
  • "MAG_G"
  • "MAG_R"
  • "MAG_I"
  • "MAG_Z"

You should modify these names to match those in your data. You can include as many filters as you want in your analysis. You only have to take care to put the correct name that appears in the train.fits file.

• "filters_err": These are the error values corresponding to your data. You should modify these names to match those in your data.

• "bins": Contains the information for calculating the PDFs, including the start, end, and number of bins. Unless you need to change this for a specific reason, you can leave the default configuration.

• "fit": Choose True or False depending on whether you want use the fit fuction. The fit function performs a better adjustment for the photo-z calculation, but it increases the execution time.

• "pdf": Select True or False depending on whether you want to calculate the PDFs.

• "metric": DNF has three types of metrics for determining the DNF. Choose the one that best fits your needs: ENF, ANF, or DNF. DNF provides three different metrics for calculating photoz:

  • ENF: Euclidean neighbourhood. it's a common distance metric used in kNN (k-Nearest Neighbors) for photometric redshift prediction.
  • ANF: uses normalized inner product for more accurate photo-z predictions.
  • DNF: combines Euclidean and angular metrics, improving accuracy, especially for larger neighborhoods, and maintaining proportionality in observable content.

For more than 4 filters, we recommend using ANF. You can learn more about these three metrics by referring to the following resource: here.

• "z_spec": Contains the name of the spectroscopic redshift column in the train.fits data.

3. Launching DNF with Your Data

Now, you can run DNF with your data by entering the following command in your console:

python3 photoz.py your_train_data.fits your_data.fits your_output_data_name.fits