1_write_metadata.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8825d5a1-14e1-44cb-8747-b55f579b5cc7",
   "metadata": {},
   "source": [
    "# 1. Writing a metadata file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "dc228dce",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:07.114161Z",
     "iopub.status.busy": "2023-09-15T12:06:07.113966Z",
     "iopub.status.idle": "2023-09-15T12:06:12.671777Z",
     "shell.execute_reply": "2023-09-15T12:06:12.670937Z",
     "shell.execute_reply.started": "2023-09-15T12:06:07.114134Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "                <script type=\"application/javascript\" id=\"jupyter_black\">\n",
       "                (function() {\n",
       "                    if (window.IPython === undefined) {\n",
       "                        return\n",
       "                    }\n",
       "                    var msg = \"WARNING: it looks like you might have loaded \" +\n",
       "                        \"jupyter_black in a non-lab notebook with \" +\n",
       "                        \"`is_lab=True`. Please double check, and if \" +\n",
       "                        \"loading with `%load_ext` please review the README!\"\n",
       "                    console.log(msg)\n",
       "                    alert(msg)\n",
       "                })()\n",
       "                </script>\n",
       "                "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import os\n",
    "import glob\n",
    "import pandas as pd\n",
    "import pypumatac as pum\n",
    "\n",
    "%load_ext jupyter_black"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aaa51933-7dff-40a7-a79e-961d65f52939",
   "metadata": {},
   "source": [
    "The metadata file is essentially a tab-delimited list of samples. For example, if you have 4 10x scATAC-seq runs named `ATAC_1` to `ATAC_4`, and each run has produced three FASTQ files (mate 1 and 2, and barcode read), your `metadata.tsv` would contain a header and four lines, with each line containing the paths to the three FASTQ files. An additional column (`technology`) refers to a set of instructions that the pipeline will use to extract and correct cell barcodes from the cell barcode FASTQ. There are 4 default technologies that come with the pipeline:\n",
    "- `standard`: the simplest case, where a barcode whitelist is provided, and each read in the `fastq_barcode` can be directly corrected using the provided whitelist. This strategy is the default strategy.\n",
    "- `ddseq`: for Bio-Rad SureCell ATAC ddSEQ samples. This workflow is a bit more complicated for two reasons: the barcode sequence itself contains adapters (constant regions that are shared between all barcodes), and the barcode read is followed by a gDNA insert read (i.e. barcode and mate 1 are read in the same sequencing read).\n",
    "- `hydrop_3x96` and  `hydrop_2x384`: for two variations of HyDrop. The HyDrop barcode read also contains constant regions, which are automatically removed by the pipeline.\n",
    "\n",
    "Later (in notebook `2_running_nextflow_pipeline.ipynb`, you will define a path to the whitelist that should be used for each `technology`.\n",
    "\n",
    "An example `metadata.tsv`, for many samples, looks like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3780dc1e-5497-426b-8066-dfcfef1fe6bc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:12.674210Z",
     "iopub.status.busy": "2023-09-15T12:06:12.673630Z",
     "iopub.status.idle": "2023-09-15T12:06:12.710341Z",
     "shell.execute_reply": "2023-09-15T12:06:12.709624Z",
     "shell.execute_reply.started": "2023-09-15T12:06:12.674185Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>technology</th>\n",
       "      <th>fastq_PE1_path</th>\n",
       "      <th>fastq_barcode_path</th>\n",
       "      <th>fastq_PE2_path</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sample_name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>BIO_ddseq_4</th>\n",
       "      <td>biorad</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EPF_hydrop_1</th>\n",
       "      <td>hydrop_2x384</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>OHS_s3atac_1</th>\n",
       "      <td>s3atac_1</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>VIB_10xv2_1</th>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/202...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                technology                                     fastq_PE1_path   \n",
       "sample_name                                                                     \n",
       "BIO_ddseq_4         biorad  /lustre1/project/stg_00002/lcb/fderop/data/202...  \\\n",
       "EPF_hydrop_1  hydrop_2x384  /lustre1/project/stg_00002/lcb/fderop/data/202...   \n",
       "OHS_s3atac_1      s3atac_1  /lustre1/project/stg_00002/lcb/fderop/data/202...   \n",
       "VIB_10xv2_1   atac_revcomp  /lustre1/project/stg_00002/lcb/fderop/data/202...   \n",
       "\n",
       "                                             fastq_barcode_path   \n",
       "sample_name                                                       \n",
       "BIO_ddseq_4   /lustre1/project/stg_00002/lcb/fderop/data/202...  \\\n",
       "EPF_hydrop_1  /lustre1/project/stg_00002/lcb/fderop/data/202...   \n",
       "OHS_s3atac_1  /lustre1/project/stg_00002/lcb/fderop/data/202...   \n",
       "VIB_10xv2_1   /lustre1/project/stg_00002/lcb/fderop/data/202...   \n",
       "\n",
       "                                                 fastq_PE2_path  \n",
       "sample_name                                                      \n",
       "BIO_ddseq_4   /lustre1/project/stg_00002/lcb/fderop/data/202...  \n",
       "EPF_hydrop_1  /lustre1/project/stg_00002/lcb/fderop/data/202...  \n",
       "OHS_s3atac_1  /lustre1/project/stg_00002/lcb/fderop/data/202...  \n",
       "VIB_10xv2_1   /lustre1/project/stg_00002/lcb/fderop/data/202...  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example = pd.read_csv(\"metadata.tsv\", sep=\"\\t\", index_col=0)\n",
    "example"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4aab064d-128d-4a7b-bd50-050fc69f5a3e",
   "metadata": {},
   "source": [
    "In this notebook, we use some simple Python string manipulation and list comprehensions to automatically write a `metadata.tsv` file for the pipeline to interpret."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62f6435b-e4b8-441e-a281-f9ca7092a596",
   "metadata": {},
   "source": [
    "# Case #1. Standard bcl2fastq format filenames\n",
    "Here, we work with standard bcl2fastq format filenames. 10x demultiplexed FASTQ files will also work seamlessly.\n",
    "I have put all sequencing files in the dir arbitrarily named `10x_fastq`. The code below will list and sort these files as we need.\n",
    "All of the runs were sequenced on NextSeq2000, which uses the reverse complement workflow (https://kb.10xgenomics.com/hc/en-us/articles/360056364852-Should-I-select-Workflow-A-or-Workflow-B-for-the-i5-index-sequence-). The barcode read is therefore read in reverse complement, which should be met with a reverse complemented whitelist (10x CellRanger detects and does this automatically)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3d9cbbc3-28ce-45e4-ae7e-869d50999d7c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:12.715788Z",
     "iopub.status.busy": "2023-09-15T12:06:12.715345Z",
     "iopub.status.idle": "2023-09-15T12:06:12.719976Z",
     "shell.execute_reply": "2023-09-15T12:06:12.719304Z",
     "shell.execute_reply.started": "2023-09-15T12:06:12.715766Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "fastq_dir = \"10x_fastq/\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3226ad8c-3631-4a57-8e0b-10b262e672fc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:12.725298Z",
     "iopub.status.busy": "2023-09-15T12:06:12.724866Z",
     "iopub.status.idle": "2023-09-15T12:06:12.736182Z",
     "shell.execute_reply": "2023-09-15T12:06:12.735548Z",
     "shell.execute_reply.started": "2023-09-15T12:06:12.725277Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['10x_fastq/ASA__0201f1__20220902_MO-016-b-ATAC_S5_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__0201f1__20220902_MO-016-b-ATAC_S5_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__0201f1__20220902_MO-016-b-ATAC_S6_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__0201f1__20220902_MO-016-b-ATAC_S6_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__0201f1__20220902_MO-016-b-ATAC_S7_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__0201f1__20220902_MO-016-b-ATAC_S7_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__0201f1__20220902_MO-016-b-ATAC_S8_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__0201f1__20220902_MO-016-b-ATAC_S8_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S10_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S10_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S11_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S11_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S12_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S12_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S1_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S1_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S2_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S2_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S3_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S3_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S4_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S4_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S9_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__089fab__20220902_MO-016-c-ATAC_S9_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__09f884__20230315_MO-018-b-ATAC_S5_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__09f884__20230315_MO-018-b-ATAC_S5_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__09f884__20230315_MO-018-b-ATAC_S6_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__09f884__20230315_MO-018-b-ATAC_S6_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__09f884__20230315_MO-018-b-ATAC_S7_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__09f884__20230315_MO-018-b-ATAC_S7_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__09f884__20230315_MO-018-b-ATAC_S8_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__09f884__20230315_MO-018-b-ATAC_S8_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__12bf4c__20230315_MO-018-h-ATAC_S29_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__12bf4c__20230315_MO-018-h-ATAC_S29_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__12bf4c__20230315_MO-018-h-ATAC_S30_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__12bf4c__20230315_MO-018-h-ATAC_S30_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__12bf4c__20230315_MO-018-h-ATAC_S31_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__12bf4c__20230315_MO-018-h-ATAC_S31_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__12bf4c__20230315_MO-018-h-ATAC_S32_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__12bf4c__20230315_MO-018-h-ATAC_S32_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1bde8b__20230315_MO-018-a-ATAC_S1_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1bde8b__20230315_MO-018-a-ATAC_S1_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1bde8b__20230315_MO-018-a-ATAC_S2_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1bde8b__20230315_MO-018-a-ATAC_S2_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1bde8b__20230315_MO-018-a-ATAC_S3_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1bde8b__20230315_MO-018-a-ATAC_S3_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1bde8b__20230315_MO-018-a-ATAC_S4_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1bde8b__20230315_MO-018-a-ATAC_S4_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1e598f__20230315_MO-018-d-ATAC_S13_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1e598f__20230315_MO-018-d-ATAC_S13_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1e598f__20230315_MO-018-d-ATAC_S14_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1e598f__20230315_MO-018-d-ATAC_S14_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1e598f__20230315_MO-018-d-ATAC_S15_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1e598f__20230315_MO-018-d-ATAC_S15_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1e598f__20230315_MO-018-d-ATAC_S16_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__1e598f__20230315_MO-018-d-ATAC_S16_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S10_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S10_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S11_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S11_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S12_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S12_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S21_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S21_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S22_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S22_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S23_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S23_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S24_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S24_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S9_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2b6050__20220927_MO-017-b-ATAC_S9_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2cb45a__20230315_MO-018-g-ATAC_S25_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2cb45a__20230315_MO-018-g-ATAC_S25_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2cb45a__20230315_MO-018-g-ATAC_S26_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2cb45a__20230315_MO-018-g-ATAC_S26_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2cb45a__20230315_MO-018-g-ATAC_S27_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2cb45a__20230315_MO-018-g-ATAC_S27_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2cb45a__20230315_MO-018-g-ATAC_S28_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__2cb45a__20230315_MO-018-g-ATAC_S28_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S17_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S17_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S18_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S18_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S19_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S19_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S20_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S20_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S5_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S5_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S6_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S6_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S7_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S7_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S8_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__4a5f45__20220927_MO-017-a-ATAC_S8_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__57620c__20230315_MO-018-c-ATAC_S10_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__57620c__20230315_MO-018-c-ATAC_S10_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__57620c__20230315_MO-018-c-ATAC_S11_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__57620c__20230315_MO-018-c-ATAC_S11_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__57620c__20230315_MO-018-c-ATAC_S12_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__57620c__20230315_MO-018-c-ATAC_S12_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__57620c__20230315_MO-018-c-ATAC_S9_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__57620c__20230315_MO-018-c-ATAC_S9_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S37_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S37_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S38_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S38_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S39_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S39_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S40_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S40_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S6_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S6_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S7_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S7_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S8_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S8_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S9_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__5d65d4__20220927_MO-017-f-ATAC_S9_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S13_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S13_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S14_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S14_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S15_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S15_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S16_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S16_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S29_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S29_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S30_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S30_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S31_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S31_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S32_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__620c11__20220927_MO-017-d-ATAC_S32_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S25_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S25_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S26_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S26_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S27_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S27_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S28_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S28_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S2_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S2_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S3_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S3_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S4_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S4_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S5_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__848fa1__20220927_MO-017-c-ATAC_S5_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__9e5bca__20230315_MO-018-f-ATAC_S21_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__9e5bca__20230315_MO-018-f-ATAC_S21_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__9e5bca__20230315_MO-018-f-ATAC_S22_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__9e5bca__20230315_MO-018-f-ATAC_S22_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__9e5bca__20230315_MO-018-f-ATAC_S23_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__9e5bca__20230315_MO-018-f-ATAC_S23_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__9e5bca__20230315_MO-018-f-ATAC_S24_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__9e5bca__20230315_MO-018-f-ATAC_S24_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ab17e7__20220902_MO-016-a-ATAC_S1_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ab17e7__20220902_MO-016-a-ATAC_S1_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ab17e7__20220902_MO-016-a-ATAC_S2_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ab17e7__20220902_MO-016-a-ATAC_S2_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ab17e7__20220902_MO-016-a-ATAC_S3_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ab17e7__20220902_MO-016-a-ATAC_S3_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ab17e7__20220902_MO-016-a-ATAC_S4_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ab17e7__20220902_MO-016-a-ATAC_S4_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S17_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S17_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S18_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S18_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S19_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S19_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S20_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S20_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S33_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S33_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S34_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S34_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S35_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S35_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S36_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__b6fa6d__20220927_MO-017-e-ATAC_S36_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__e27b63__20220902_MO-016-d-ATAC_S13_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__e27b63__20220902_MO-016-d-ATAC_S13_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__e27b63__20220902_MO-016-d-ATAC_S14_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__e27b63__20220902_MO-016-d-ATAC_S14_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__e27b63__20220902_MO-016-d-ATAC_S15_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__e27b63__20220902_MO-016-d-ATAC_S15_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__e27b63__20220902_MO-016-d-ATAC_S16_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__e27b63__20220902_MO-016-d-ATAC_S16_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ffd613__20230315_MO-018-e-ATAC_S17_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ffd613__20230315_MO-018-e-ATAC_S17_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ffd613__20230315_MO-018-e-ATAC_S18_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ffd613__20230315_MO-018-e-ATAC_S18_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ffd613__20230315_MO-018-e-ATAC_S19_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ffd613__20230315_MO-018-e-ATAC_S19_L002_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ffd613__20230315_MO-018-e-ATAC_S20_L001_R1_001.fastq.gz',\n",
       " '10x_fastq/ASA__ffd613__20230315_MO-018-e-ATAC_S20_L002_R1_001.fastq.gz']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filepaths = sorted(glob.glob(f\"{fastq_dir}/*R1*\"))\n",
    "filepaths"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "9b77f9c7-6788-4f5e-aa5a-12c1f8aeb0ca",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:12.739421Z",
     "iopub.status.busy": "2023-09-15T12:06:12.739091Z",
     "iopub.status.idle": "2023-09-15T12:06:12.744319Z",
     "shell.execute_reply": "2023-09-15T12:06:12.743667Z",
     "shell.execute_reply.started": "2023-09-15T12:06:12.739402Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(filepaths)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10fd9584-a607-4293-bf94-4899f46ff29d",
   "metadata": {},
   "source": [
    "We assume that each FASTQ file has a structured name indicating the sample and read. In this syntax, R1 is mate 1, R2 is the barcode read and R3 is mate 2. For ddSEq samples, R2 is mate 2. The file names are completely arbitrary, but can help to systematically generate the metadata file.  \n",
    "  \n",
    "In this case, the sample name can be extracted from the filename by taking everything that comes before `_S` (e.g.`S20_L002_R1_001.fastq.gz` is removed)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4a3c296a-cceb-4187-9870-a8832b9bee0b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:12.747595Z",
     "iopub.status.busy": "2023-09-15T12:06:12.747235Z",
     "iopub.status.idle": "2023-09-15T12:06:12.759519Z",
     "shell.execute_reply": "2023-09-15T12:06:12.758890Z",
     "shell.execute_reply.started": "2023-09-15T12:06:12.747574Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ASA__0201f1__20220902_MO-016-b-ATAC',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filenames = [x.split(\"/\")[-1] for x in filepaths]\n",
    "sample_names = sorted(list(set([x.split(\"_S\")[0] for x in filenames])))\n",
    "sample_names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d189638f-f4fe-4e3f-9862-a383a979952c",
   "metadata": {},
   "source": [
    "We can determine the sequencer used from the FASTQ:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "07bdfa32-ea1c-4a1c-8ee7-91333316d94c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:12.762689Z",
     "iopub.status.busy": "2023-09-15T12:06:12.762290Z",
     "iopub.status.idle": "2023-09-15T12:06:21.080136Z",
     "shell.execute_reply": "2023-09-15T12:06:21.079387Z",
     "shell.execute_reply.started": "2023-09-15T12:06:12.762670Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ASA__0201f1__20220902_MO-016-b-ATAC_S5_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S5_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S6_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S6_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S7_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S7_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S8_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S8_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S10_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S10_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S11_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S11_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S12_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S12_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S1_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S1_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S2_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S2_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S3_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S3_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S4_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S4_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S9_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S9_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S5_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S5_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S6_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S6_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S7_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S7_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S8_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S8_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S29_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S29_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S30_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S30_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S31_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S31_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S32_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S32_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S1_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S1_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S2_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S2_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S3_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S3_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S4_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S4_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S13_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S13_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S14_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S14_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S15_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S15_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S16_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S16_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S10_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S10_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S11_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S11_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S12_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S12_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S21_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S21_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S22_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S22_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S23_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S23_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S24_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S24_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S9_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S9_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S25_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S25_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S26_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S26_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S27_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S27_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S28_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S28_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S17_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S17_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S18_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S18_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S19_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S19_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S20_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S20_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S5_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S5_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S6_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S6_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S7_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S7_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S8_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S8_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S10_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S10_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S11_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S11_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S12_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S12_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S9_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S9_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S37_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S37_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S38_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S38_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S39_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S39_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S40_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S40_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S6_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S6_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S7_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S7_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S8_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S8_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S9_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S9_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S13_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S13_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S14_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S14_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S15_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S15_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S16_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S16_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S29_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S29_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S30_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S30_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S31_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S31_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S32_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S32_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S25_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S25_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S26_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S26_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S27_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S27_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S28_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S28_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S2_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S2_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S3_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S3_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S4_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S4_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S5_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S5_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S21_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S21_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S22_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S22_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S23_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S23_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S24_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S24_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S1_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S1_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S2_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S2_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S3_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S3_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S4_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S4_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S17_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S17_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S18_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S18_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S19_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S19_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S20_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S20_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S33_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S33_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S34_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S34_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S35_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S35_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S36_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S36_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S13_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S13_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S14_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S14_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S15_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S15_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S16_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S16_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S17_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S17_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S18_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S18_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S19_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S19_L002_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S20_L001_R1_001.fastq.gz': 'NextSeq 2000',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S20_L002_R1_001.fastq.gz': 'NextSeq 2000'}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sequencers_dict = {}\n",
    "for file in filepaths:\n",
    "    filename = file.split(\"/\")[-1]\n",
    "    message, sequencers = pum.sequencer_detection_message([file])\n",
    "\n",
    "    # print(f\"{filename}: {sequencers[0]}\")\n",
    "\n",
    "    sequencers_dict[filename] = sequencers[0]\n",
    "sequencers_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5361a79-c621-403a-ae43-419336638712",
   "metadata": {},
   "source": [
    "Determine whether this sequencer uses forward or reverse complement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "d9b937c5-148f-4b2a-82f8-e5b24cdf6b8f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.083668Z",
     "iopub.status.busy": "2023-09-15T12:06:21.083213Z",
     "iopub.status.idle": "2023-09-15T12:06:21.092353Z",
     "shell.execute_reply": "2023-09-15T12:06:21.091683Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.083645Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "worflow_dict = {\n",
    "    \"MiSeq\": \"forward\",\n",
    "    \"HiSeq 2500\": \"forward\",\n",
    "    \"HiSeq 3000\": \"revcomp\",\n",
    "    \"HiSeq X\": \"revcomp\",\n",
    "    \"NextSeq 500/550\": \"revcomp\",\n",
    "    \"NovaSeq 6000\": \"revcomp\",\n",
    "    \"NextSeq 2000\": \"revcomp\",\n",
    "    \"NovaSeq X\": \"revcomp\",\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "062a50cd-24ff-4183-9984-05d9d7f914b4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.093440Z",
     "iopub.status.busy": "2023-09-15T12:06:21.093223Z",
     "iopub.status.idle": "2023-09-15T12:06:21.106538Z",
     "shell.execute_reply": "2023-09-15T12:06:21.105903Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.093421Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ASA__0201f1__20220902_MO-016-b-ATAC_S5_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S5_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S6_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S6_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S7_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S7_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S8_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S8_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S10_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S10_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S11_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S11_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S12_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S12_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S1_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S1_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S2_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S2_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S3_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S3_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S4_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S4_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S9_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S9_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S5_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S5_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S6_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S6_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S7_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S7_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S8_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S8_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S29_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S29_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S30_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S30_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S31_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S31_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S32_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S32_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S1_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S1_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S2_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S2_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S3_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S3_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S4_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S4_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S13_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S13_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S14_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S14_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S15_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S15_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S16_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S16_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S10_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S10_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S11_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S11_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S12_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S12_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S21_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S21_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S22_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S22_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S23_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S23_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S24_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S24_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S9_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S9_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S25_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S25_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S26_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S26_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S27_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S27_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S28_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S28_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S17_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S17_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S18_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S18_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S19_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S19_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S20_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S20_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S5_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S5_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S6_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S6_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S7_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S7_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S8_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S8_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S10_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S10_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S11_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S11_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S12_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S12_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S9_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S9_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S37_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S37_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S38_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S38_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S39_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S39_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S40_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S40_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S6_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S6_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S7_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S7_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S8_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S8_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S9_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S9_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S13_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S13_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S14_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S14_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S15_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S15_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S16_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S16_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S29_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S29_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S30_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S30_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S31_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S31_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S32_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S32_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S25_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S25_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S26_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S26_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S27_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S27_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S28_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S28_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S2_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S2_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S3_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S3_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S4_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S4_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S5_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S5_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S21_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S21_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S22_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S22_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S23_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S23_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S24_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S24_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S1_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S1_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S2_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S2_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S3_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S3_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S4_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S4_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S17_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S17_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S18_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S18_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S19_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S19_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S20_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S20_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S33_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S33_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S34_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S34_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S35_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S35_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S36_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S36_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S13_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S13_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S14_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S14_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S15_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S15_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S16_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S16_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S17_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S17_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S18_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S18_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S19_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S19_L002_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S20_L001_R1_001.fastq.gz': 'revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S20_L002_R1_001.fastq.gz': 'revcomp'}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_workflow_dict = {x: worflow_dict[y] for x, y in sequencers_dict.items()}\n",
    "sample_workflow_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba5969de-d61b-4106-a54b-ccea042ca1d6",
   "metadata": {},
   "source": [
    "Then, define the type of debarcoding strategy you want to use. We have 10x libraries, so `standard` should do. I use the term `10x_atac`, which will default to `standard`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "4ae56833-e600-494b-92d0-ef20be7f398e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.107846Z",
     "iopub.status.busy": "2023-09-15T12:06:21.107385Z",
     "iopub.status.idle": "2023-09-15T12:06:21.119213Z",
     "shell.execute_reply": "2023-09-15T12:06:21.118569Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.107826Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ASA__0201f1__20220902_MO-016-b-ATAC_S5_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S5_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S6_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S6_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S7_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S7_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S8_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__0201f1__20220902_MO-016-b-ATAC_S8_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S10_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S10_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S11_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S11_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S12_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S12_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S1_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S1_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S2_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S2_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S3_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S3_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S4_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S4_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S9_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__089fab__20220902_MO-016-c-ATAC_S9_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S5_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S5_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S6_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S6_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S7_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S7_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S8_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__09f884__20230315_MO-018-b-ATAC_S8_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S29_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S29_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S30_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S30_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S31_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S31_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S32_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__12bf4c__20230315_MO-018-h-ATAC_S32_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S1_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S1_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S2_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S2_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S3_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S3_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S4_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1bde8b__20230315_MO-018-a-ATAC_S4_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S13_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S13_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S14_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S14_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S15_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S15_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S16_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__1e598f__20230315_MO-018-d-ATAC_S16_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S10_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S10_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S11_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S11_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S12_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S12_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S21_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S21_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S22_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S22_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S23_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S23_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S24_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S24_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S9_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2b6050__20220927_MO-017-b-ATAC_S9_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S25_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S25_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S26_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S26_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S27_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S27_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S28_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__2cb45a__20230315_MO-018-g-ATAC_S28_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S17_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S17_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S18_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S18_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S19_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S19_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S20_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S20_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S5_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S5_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S6_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S6_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S7_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S7_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S8_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__4a5f45__20220927_MO-017-a-ATAC_S8_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S10_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S10_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S11_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S11_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S12_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S12_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S9_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__57620c__20230315_MO-018-c-ATAC_S9_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S37_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S37_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S38_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S38_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S39_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S39_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S40_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S40_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S6_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S6_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S7_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S7_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S8_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S8_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S9_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__5d65d4__20220927_MO-017-f-ATAC_S9_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S13_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S13_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S14_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S14_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S15_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S15_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S16_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S16_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S29_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S29_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S30_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S30_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S31_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S31_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S32_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__620c11__20220927_MO-017-d-ATAC_S32_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S25_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S25_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S26_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S26_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S27_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S27_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S28_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S28_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S2_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S2_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S3_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S3_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S4_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S4_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S5_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__848fa1__20220927_MO-017-c-ATAC_S5_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S21_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S21_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S22_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S22_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S23_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S23_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S24_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__9e5bca__20230315_MO-018-f-ATAC_S24_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S1_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S1_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S2_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S2_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S3_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S3_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S4_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ab17e7__20220902_MO-016-a-ATAC_S4_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S17_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S17_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S18_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S18_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S19_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S19_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S20_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S20_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S33_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S33_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S34_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S34_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S35_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S35_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S36_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__b6fa6d__20220927_MO-017-e-ATAC_S36_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S13_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S13_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S14_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S14_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S15_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S15_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S16_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__e27b63__20220902_MO-016-d-ATAC_S16_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S17_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S17_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S18_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S18_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S19_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S19_L002_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S20_L001_R1_001.fastq.gz': 'atac_revcomp',\n",
       " 'ASA__ffd613__20230315_MO-018-e-ATAC_S20_L002_R1_001.fastq.gz': 'atac_revcomp'}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tech_dict = {x: f\"atac_{sample_workflow_dict[x]}\" for x in filenames}\n",
    "tech_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ec1f74a-5105-4eb3-bfc7-e9b011b6d363",
   "metadata": {},
   "source": [
    "We have multiple FASTQ sets belonging to the same sample though. We must therefore align to the reference genome and merge the resulting bam files before writing fragments from the merged bam file. We can achieve this by having one line for each set of FASTQs for each sample. For example, `sample_A` could have 8 lines, with each line denoting a set of FASTQ files that should be aligned, and of which the resulting `.bam` should be merged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d3a7b27b-f74b-4b64-ae94-762be8c8bf9f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.120487Z",
     "iopub.status.busy": "2023-09-15T12:06:21.120074Z",
     "iopub.status.idle": "2023-09-15T12:06:21.228637Z",
     "shell.execute_reply": "2023-09-15T12:06:21.227685Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.120467Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_metadata = pd.DataFrame()\n",
    "for sample in sample_names:\n",
    "    sample_R1_files = [\n",
    "        x for x in sorted(glob.glob(f\"{fastq_dir}/*{sample}*R1*\"))\n",
    "    ]\n",
    "    # print(len(sample_R1_files))\n",
    "    df_sub = pd.DataFrame([sample] * len(sample_R1_files), columns=[\"sample_name\"])\n",
    "    df_sub[\"technology\"] = [tech_dict[os.path.basename(x)] for x in sample_R1_files]\n",
    "    df_sub[\"fastq_PE1_path\"] = [os.path.realpath(x) for x in sample_R1_files]\n",
    "    df_sub[\"fastq_barcode_path\"] = [os.path.realpath(x.replace(\"_R1_\", \"_R2_\")) for x in sample_R1_files]\n",
    "    df_sub[\"fastq_PE2_path\"] = [os.path.realpath(x.replace(\"_R1_\", \"_R3_\")) for x in sample_R1_files]\n",
    "\n",
    "    df_metadata = pd.concat([df_metadata, df_sub])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f30b939b-db61-4ce2-977f-fdefc5476417",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.230073Z",
     "iopub.status.busy": "2023-09-15T12:06:21.229745Z",
     "iopub.status.idle": "2023-09-15T12:06:21.241405Z",
     "shell.execute_reply": "2023-09-15T12:06:21.240726Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.230045Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sample_name</th>\n",
       "      <th>technology</th>\n",
       "      <th>fastq_PE1_path</th>\n",
       "      <th>fastq_barcode_path</th>\n",
       "      <th>fastq_PE2_path</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ASA__0201f1__20220902_MO-016-b-ATAC</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ASA__0201f1__20220902_MO-016-b-ATAC</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ASA__0201f1__20220902_MO-016-b-ATAC</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ASA__0201f1__20220902_MO-016-b-ATAC</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ASA__0201f1__20220902_MO-016-b-ATAC</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ASA__ffd613__20230315_MO-018-e-ATAC</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ASA__ffd613__20230315_MO-018-e-ATAC</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>ASA__ffd613__20230315_MO-018-e-ATAC</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>ASA__ffd613__20230315_MO-018-e-ATAC</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>ASA__ffd613__20230315_MO-018-e-ATAC</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/ngs_runs/NextSe...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>200 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                            sample_name    technology   \n",
       "0   ASA__0201f1__20220902_MO-016-b-ATAC  atac_revcomp  \\\n",
       "1   ASA__0201f1__20220902_MO-016-b-ATAC  atac_revcomp   \n",
       "2   ASA__0201f1__20220902_MO-016-b-ATAC  atac_revcomp   \n",
       "3   ASA__0201f1__20220902_MO-016-b-ATAC  atac_revcomp   \n",
       "4   ASA__0201f1__20220902_MO-016-b-ATAC  atac_revcomp   \n",
       "..                                  ...           ...   \n",
       "3   ASA__ffd613__20230315_MO-018-e-ATAC  atac_revcomp   \n",
       "4   ASA__ffd613__20230315_MO-018-e-ATAC  atac_revcomp   \n",
       "5   ASA__ffd613__20230315_MO-018-e-ATAC  atac_revcomp   \n",
       "6   ASA__ffd613__20230315_MO-018-e-ATAC  atac_revcomp   \n",
       "7   ASA__ffd613__20230315_MO-018-e-ATAC  atac_revcomp   \n",
       "\n",
       "                                       fastq_PE1_path   \n",
       "0   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \\\n",
       "1   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "2   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "3   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "4   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "..                                                ...   \n",
       "3   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "4   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "5   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "6   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "7   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "\n",
       "                                   fastq_barcode_path   \n",
       "0   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \\\n",
       "1   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "2   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "3   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "4   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "..                                                ...   \n",
       "3   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "4   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "5   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "6   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "7   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...   \n",
       "\n",
       "                                       fastq_PE2_path  \n",
       "0   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \n",
       "1   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \n",
       "2   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \n",
       "3   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \n",
       "4   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \n",
       "..                                                ...  \n",
       "3   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \n",
       "4   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \n",
       "5   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \n",
       "6   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \n",
       "7   /lustre1/project/stg_00002/lcb/ngs_runs/NextSe...  \n",
       "\n",
       "[200 rows x 5 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_metadata"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad09677a-12e7-438f-9a89-d4a0e9ef6492",
   "metadata": {},
   "source": [
    "Then, write this to a csv file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "be2bf4de-2981-4d60-a1bc-66c006ab2ce9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.242761Z",
     "iopub.status.busy": "2023-09-15T12:06:21.242340Z",
     "iopub.status.idle": "2023-09-15T12:06:21.254892Z",
     "shell.execute_reply": "2023-09-15T12:06:21.254082Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.242741Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_metadata.to_csv(\"metadata.tsv\", sep=\"\\t\", index=False, header=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c760c217-365f-461c-aadf-adfbacaa9174",
   "metadata": {},
   "source": [
    "Be sure to include the 10x whitelist as an option for `atac_forward` or `atac_revcomp` in the `.config` file defined in notebook 2."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0559a2eb-4719-4375-b272-3f288224c26f",
   "metadata": {},
   "source": [
    "# Case #2. A directory containing FASTQ files with arbitrary names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "466b69fe-a4c5-4b65-8a70-c06eade57439",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-17T13:57:23.282436Z",
     "iopub.status.busy": "2023-04-17T13:57:23.281985Z",
     "iopub.status.idle": "2023-04-17T13:57:23.291497Z",
     "shell.execute_reply": "2023-04-17T13:57:23.290817Z",
     "shell.execute_reply.started": "2023-04-17T13:57:23.282363Z"
    }
   },
   "source": [
    "First, find all the FASTQ files in your `fastq_dir/`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0bdbe147-10d6-4ed4-977e-a1768a76be67",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.256288Z",
     "iopub.status.busy": "2023-09-15T12:06:21.255867Z",
     "iopub.status.idle": "2023-09-15T12:06:21.260276Z",
     "shell.execute_reply": "2023-09-15T12:06:21.259612Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.256267Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "fastq_dir = \"PUMATAC_example_fastq/\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "10a49f5d-4a3e-482e-adc8-e6a2adba189f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.264172Z",
     "iopub.status.busy": "2023-09-15T12:06:21.263849Z",
     "iopub.status.idle": "2023-09-15T12:06:21.270923Z",
     "shell.execute_reply": "2023-09-15T12:06:21.270296Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.264151Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['PUMATAC_example_fastq/BIO_ddseq_4__R1.LIBDS.fastq.gz',\n",
       " 'PUMATAC_example_fastq/EPF_hydrop_1__R1.LIBDS.fastq.gz',\n",
       " 'PUMATAC_example_fastq/OHS_s3atac_1__R1.LIBDS.fastq.gz',\n",
       " 'PUMATAC_example_fastq/VIB_10xv2_1__R1.LIBDS.fastq.gz',\n",
       " 'PUMATAC_example_fastq/test2_R1.fastq.gz',\n",
       " 'PUMATAC_example_fastq/test_R1.fastq.gz']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filepaths = sorted(glob.glob(f\"{fastq_dir}/*R1*\"))\n",
    "filepaths"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dab3d8be-47f0-46eb-bfa0-6ad7d5c24dd3",
   "metadata": {},
   "source": [
    "We assume that each FASTQ file has a structured name indicating the sample and read. In this syntax, R1 is mate 1, R2 is the barcode read and R3 is mate 2. For ddSEq samples, R2 is mate 2. The file names are completely arbitrary, but can help to systematically generate the metadata file. Of course, the scripts below assume the name structure described previously. If your FASTQ filenames have different structures, you should adapt these scripts or compile manual `metadata.tsv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f2656ac4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.272181Z",
     "iopub.status.busy": "2023-09-15T12:06:21.271769Z",
     "iopub.status.idle": "2023-09-15T12:06:21.283130Z",
     "shell.execute_reply": "2023-09-15T12:06:21.282495Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.272161Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['BIO_ddseq_4',\n",
       " 'EPF_hydrop_1',\n",
       " 'OHS_s3atac_1',\n",
       " 'VIB_10xv2_1',\n",
       " 'test2_R1.fastq.gz',\n",
       " 'test_R1.fastq.gz']"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filenames = [x.split(\"/\")[-1] for x in filepaths]\n",
    "sample_names = sorted(list(set([x.split(\"__\")[0] for x in filenames])))\n",
    "sample_names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93f4f629-c222-46ce-8b70-99174bcac575",
   "metadata": {},
   "source": [
    "We can determine the sequencer used from the FASTQ:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "7184a669-bccd-4db0-8088-497980c17fcb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.284351Z",
     "iopub.status.busy": "2023-09-15T12:06:21.283943Z",
     "iopub.status.idle": "2023-09-15T12:06:21.538828Z",
     "shell.execute_reply": "2023-09-15T12:06:21.538084Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.284331Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "BIO_ddseq_4__R1.LIBDS.fastq.gz: NextSeq 500/550\n",
      "EPF_hydrop_1__R1.LIBDS.fastq.gz: NextSeq 2000\n",
      "OHS_s3atac_1__R1.LIBDS.fastq.gz: UNKNOWN!\n",
      "VIB_10xv2_1__R1.LIBDS.fastq.gz: NextSeq 2000\n",
      "test2_R1.fastq.gz: NextSeq 2000\n",
      "test_R1.fastq.gz: NextSeq 2000\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'BIO_ddseq_4__R1.LIBDS.fastq.gz': 'NextSeq 500/550',\n",
       " 'EPF_hydrop_1__R1.LIBDS.fastq.gz': 'NextSeq 2000',\n",
       " 'OHS_s3atac_1__R1.LIBDS.fastq.gz': 'UNKNOWN!',\n",
       " 'VIB_10xv2_1__R1.LIBDS.fastq.gz': 'NextSeq 2000',\n",
       " 'test2_R1.fastq.gz': 'NextSeq 2000',\n",
       " 'test_R1.fastq.gz': 'NextSeq 2000'}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sequencers_dict = {}\n",
    "for file in filepaths:\n",
    "    filename = file.split(\"/\")[-1]\n",
    "    try:\n",
    "        message, sequencers = pum.sequencer_detection_message([file])\n",
    "        sequencer = sequencers[\n",
    "            0\n",
    "        ]  # multiple sequencers should only be detected when FASTQ files are the concatenation of FASTQ files sequenced on different insturments\n",
    "    except:\n",
    "        sequencer = \"UNKNOWN!\"\n",
    "\n",
    "    print(f\"{filename}: {sequencer}\")\n",
    "    sequencers_dict[filename] = sequencer\n",
    "sequencers_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35898d21-3f01-4cd1-9510-bfe5af738379",
   "metadata": {},
   "source": [
    "Determine whether this sequencer uses forward or reverse complement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "35485386-08ec-4597-9cf8-f8c8b7c2516a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.540157Z",
     "iopub.status.busy": "2023-09-15T12:06:21.539825Z",
     "iopub.status.idle": "2023-09-15T12:06:21.550208Z",
     "shell.execute_reply": "2023-09-15T12:06:21.549527Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.540133Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "worflow_dict = {\n",
    "    \"MiSeq\": \"forward\",\n",
    "    \"HiSeq 2500\": \"forward\",\n",
    "    \"HiSeq 3000\": \"revcomp\",\n",
    "    \"HiSeq X\": \"revcomp\",\n",
    "    \"NextSeq 500/550\": \"revcomp\",\n",
    "    \"NovaSeq 6000\": \"revcomp\",\n",
    "    \"NextSeq 2000\": \"revcomp\",\n",
    "    \"UNKNOWN!\": \"UNKNOWN!\",\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "f6d526e6-2012-41ab-a11b-b0be6c2bbdad",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.551466Z",
     "iopub.status.busy": "2023-09-15T12:06:21.551084Z",
     "iopub.status.idle": "2023-09-15T12:06:21.561710Z",
     "shell.execute_reply": "2023-09-15T12:06:21.561080Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.551446Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'BIO_ddseq_4__R1.LIBDS.fastq.gz': 'revcomp',\n",
       " 'EPF_hydrop_1__R1.LIBDS.fastq.gz': 'revcomp',\n",
       " 'OHS_s3atac_1__R1.LIBDS.fastq.gz': 'UNKNOWN!',\n",
       " 'VIB_10xv2_1__R1.LIBDS.fastq.gz': 'revcomp',\n",
       " 'test2_R1.fastq.gz': 'revcomp',\n",
       " 'test_R1.fastq.gz': 'revcomp'}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_workflow_dict = {x: worflow_dict[y] for x, y in sequencers_dict.items()}\n",
    "sample_workflow_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20789bf0-f62e-4907-80d6-f3f6ab6af252",
   "metadata": {},
   "source": [
    "Then, define the type of debarcoding strategy you want to use. We have 10x libraries, so `standard` should do. I use the term `10x_atac`, which will default to `standard`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "197a8f69-6158-4639-bb58-176391a0d419",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.562975Z",
     "iopub.status.busy": "2023-09-15T12:06:21.562568Z",
     "iopub.status.idle": "2023-09-15T12:06:21.570276Z",
     "shell.execute_reply": "2023-09-15T12:06:21.569641Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.562955Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'BIO_ddseq_4__R1.LIBDS.fastq.gz': 'atac_revcomp',\n",
       " 'EPF_hydrop_1__R1.LIBDS.fastq.gz': 'atac_revcomp',\n",
       " 'OHS_s3atac_1__R1.LIBDS.fastq.gz': 'atac_UNKNOWN!',\n",
       " 'VIB_10xv2_1__R1.LIBDS.fastq.gz': 'atac_revcomp',\n",
       " 'test2_R1.fastq.gz': 'atac_revcomp',\n",
       " 'test_R1.fastq.gz': 'atac_revcomp'}"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tech_dict = {x: f\"atac_{sample_workflow_dict[x]}\" for x in filenames}\n",
    "tech_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa1b903d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-02T18:21:01.866204Z",
     "iopub.status.busy": "2022-06-02T18:21:01.865839Z",
     "iopub.status.idle": "2022-06-02T18:21:01.871502Z",
     "shell.execute_reply": "2022-06-02T18:21:01.870527Z",
     "shell.execute_reply.started": "2022-06-02T18:21:01.866170Z"
    }
   },
   "source": [
    "Now, copy the dictionary and change `standard` to each sample's true method, for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "b3e41003",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.571530Z",
     "iopub.status.busy": "2023-09-15T12:06:21.571126Z",
     "iopub.status.idle": "2023-09-15T12:06:21.581042Z",
     "shell.execute_reply": "2023-09-15T12:06:21.580410Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.571511Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'BIO_ddseq_4__R1.LIBDS.fastq.gz': 'biorad',\n",
       " 'EPF_hydrop_1__R1.LIBDS.fastq.gz': 'hydrop_2x384',\n",
       " 'OHS_s3atac_1__R1.LIBDS.fastq.gz': 's3atac_1',\n",
       " 'VIB_10xv2_1__R1.LIBDS.fastq.gz': 'atac_revcomp'}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tech_dict = {\n",
    "    \"BIO_ddseq_4__R1.LIBDS.fastq.gz\": \"biorad\",  # this is a bio-rad sample, so we can use the `biorad` method.\n",
    "    \"EPF_hydrop_1__R1.LIBDS.fastq.gz\": \"hydrop_2x384\",  # this is a hydrop sample, so we can use `hydrop_2x384` method\n",
    "    \"OHS_s3atac_1__R1.LIBDS.fastq.gz\": \"s3atac_1\",  # this is an s3-atac sample, so we must supply a custom whitelist there in the .config file\n",
    "    \"VIB_10xv2_1__R1.LIBDS.fastq.gz\": \"atac_revcomp\",  # this is a niormal 10x v2 sample, so we can use `atac` workflow\n",
    "}\n",
    "tech_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63258da6-246c-4bc8-b5e0-5781bcbe0eb0",
   "metadata": {},
   "source": [
    "`biorad` is a standard method. `atac_revcomp` is not a standard method. As a result, `standard` will be used as a demultiplexing strategy for `atac_revcomp`, but using a whitelist that is specified for `atac_revcomp` in the `.config` file written in notebook 2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "8ec949bb-8abc-44d8-af9f-365a20f9b9a7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.582267Z",
     "iopub.status.busy": "2023-09-15T12:06:21.581857Z",
     "iopub.status.idle": "2023-09-15T12:06:21.620036Z",
     "shell.execute_reply": "2023-09-15T12:06:21.619181Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.582247Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_metadata = pd.DataFrame()\n",
    "for sample in sample_names:\n",
    "    sample_R1_files = [\n",
    "        os.path.realpath(x) for x in sorted(glob.glob(f\"{fastq_dir}/*{sample}*R1*\"))\n",
    "    ]\n",
    "    # print(len(sample_R1_files))\n",
    "    df_sub = pd.DataFrame([sample] * len(sample_R1_files), columns=[\"sample_name\"])\n",
    "    df_sub[\"technology\"] = [tech_dict[os.path.basename(x)] for x in sample_R1_files]\n",
    "    df_sub[\"fastq_PE1_path\"] = sample_R1_files\n",
    "    df_sub[\"fastq_barcode_path\"] = [x.replace(\"_R1_\", \"_R2_\") for x in sample_R1_files]\n",
    "    df_sub[\"fastq_PE2_path\"] = [x.replace(\"_R1_\", \"_R3_\") for x in sample_R1_files]\n",
    "\n",
    "    df_metadata = pd.concat([df_metadata, df_sub])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "828854ec-db81-43af-ba64-f5afbee172b2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.621358Z",
     "iopub.status.busy": "2023-09-15T12:06:21.621048Z",
     "iopub.status.idle": "2023-09-15T12:06:21.629980Z",
     "shell.execute_reply": "2023-09-15T12:06:21.629340Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.621338Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sample_name</th>\n",
       "      <th>technology</th>\n",
       "      <th>fastq_PE1_path</th>\n",
       "      <th>fastq_barcode_path</th>\n",
       "      <th>fastq_PE2_path</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BIO_ddseq_4</td>\n",
       "      <td>biorad</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>EPF_hydrop_1</td>\n",
       "      <td>hydrop_2x384</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>OHS_s3atac_1</td>\n",
       "      <td>s3atac_1</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>VIB_10xv2_1</td>\n",
       "      <td>atac_revcomp</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "      <td>/lustre1/project/stg_00002/lcb/fderop/data/000...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    sample_name    technology   \n",
       "0   BIO_ddseq_4        biorad  \\\n",
       "0  EPF_hydrop_1  hydrop_2x384   \n",
       "0  OHS_s3atac_1      s3atac_1   \n",
       "0   VIB_10xv2_1  atac_revcomp   \n",
       "\n",
       "                                      fastq_PE1_path   \n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...  \\\n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...   \n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...   \n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...   \n",
       "\n",
       "                                  fastq_barcode_path   \n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...  \\\n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...   \n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...   \n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...   \n",
       "\n",
       "                                      fastq_PE2_path  \n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...  \n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...  \n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...  \n",
       "0  /lustre1/project/stg_00002/lcb/fderop/data/000...  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_metadata"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e24a9ebe-ea30-4dfa-8cc0-bff336a0cdb9",
   "metadata": {},
   "source": [
    "Then, write this to a csv file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "f6be669a-fc00-455f-89aa-3e3b437eb6be",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:06:21.631264Z",
     "iopub.status.busy": "2023-09-15T12:06:21.630841Z",
     "iopub.status.idle": "2023-09-15T12:06:21.639882Z",
     "shell.execute_reply": "2023-09-15T12:06:21.639222Z",
     "shell.execute_reply.started": "2023-09-15T12:06:21.631243Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_metadata.to_csv(\"metadata.tsv\", sep=\"\\t\", index=False, header=True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}