update

EVALUATION.md

@@ -7,7 +7,7 @@ We assume that you have already done the installation steps in the [README.md](.
 When you run the container, you can download repositories of benchmarks and set the configuration:
 
 ```bash
-# Inside the image;
+# Inside the dokcer container;
 cd ~
 cd configuration
 python download_repo.py
@@ -22,6 +22,7 @@ If you can see `~/typebugs`, `~/bugsinpy`, and `~/excepy` directories, the downl
 It is necessary to preprocess the [homeassistant-core](https://github.com/home-assistant/core) project before running all tools:
 
 ```bash
+# Inside the dokcer container;
 cd ~
 python run/change_core_async.py
 ```
@@ -33,6 +34,7 @@ Then, you are ready to run Pyinder and other tools!
 You can run all tools with all projects by following the instructions:
 
 ```bash
+# Inside the dokcer container;
 cd ~
 cd run
 python pyinder_run.py
@@ -75,6 +77,7 @@ All tools generate other warnings than type errors, so you need to filter out th
 At first, you have to change result log file to json file of each tool:
 
 ```bash
+# Inside the dokcer container;
 cd ~
 cd run
 # Change the result log file to json file
@@ -89,6 +92,7 @@ It makes json files named `result_.json` in the `~/result/<each-tool>` directory
 Then, you can filter out the type errors from the results:
 
 ```bash
+# Inside the dokcer container;
 cd ~
 cd run
 python filter_error.py
@@ -99,6 +103,7 @@ You can see the filtered results in the `~/result/<each-tool>` directory (e.g.,
 Before checking the results, you have to run cloc to check the per kloc results:
 
 ```bash
+# Inside the dokcer container;
 cd ~
 cd eval
 python cloc.py
@@ -107,6 +112,7 @@ python cloc.py
 Then, you can see the results by following these steps:
 
 ```bash
+# Inside the dokcer container;
 cd ~
 cd eval
 python check_alarm.py # show the number of alarms by each tool

Pyinder/README.md

@@ -0,0 +1,22 @@
+### Structure
+
+```bash
+Pyinder
+├── client
+│   ├── pyre.py              # Command line interface for Pyinder
+│   ├── ...
+├── source
+│   ├── analysis             # Analysis module for Pyinder
+│   │   ├── ...                
+│   ├── ...
+│   ├── command
+│   │   ├── mineCommand.ml   # Start point of Pyinder
+│   │   ├── ...
+│   └── ...
+├── ...
+```
+
+We provide a brief description of the core parts of Pyinder.
+`pyre.py` is the command line interface for Pyinder.
+`mineCommand.ml` is the start point of Pyinder to run the analysis.
+`analysis` contains the analysis module for Pyinder.

README.md

@@ -59,7 +59,7 @@ We recommend setting the memory reservation to 32GB for the container to fully r
 You can build Pyinder by following the instructions:
 
 ```bash
-# Inside the image;
+# Inside the dokcer container;
 cd ~
 cd Pyinder/source
 make
@@ -80,7 +80,7 @@ You can skip instructions for other tools if you only want to run Pyinder.
 At first, clone the project and prepare the project for the analysis:
 
 ```bash
-# Inside the image;
+# Inside the dokcer container;
 cd ~
 cd configuration
 python download_repo.py --project luigi
@@ -90,6 +90,8 @@ python setting_config.py --project luigi
 Next, run the tools with the luigi project:
 
 ```bash
+# Inside the dokcer container;
+
 # It will take about 30 minutes 
 # to run all tools with the luigi project.
 cd ~
@@ -120,6 +122,7 @@ The result of each tool is stored in the `~/result/<each-tool>/<luigi-proejct>/r
 Next, post-process the results to collect the type errors and check the results:
 
 ```bash
+# Inside the dokcer container;
 cd ~
 cd run
 
@@ -163,6 +166,7 @@ The command `python cloc.py` makes the `~/cloc` directory that contains the resu
 Finally, you can see the results by following these steps:
 
 ```bash
+# Inside the dokcer container;
 cd ~
 cd eval
 python check_alarm.py -p luigi # show the number of alarms by each tool

configuration/README.md

@@ -0,0 +1,32 @@
+### Structure
+
+```bash
+configuration
+├── bugsinpy_repo.json                  # GitHub repositories and their commit id for bugsinpy
+├── config                              # Configuration files for all benchmark projects
+│   ├── bugsinpy                        # Configuration files for bugsinpy
+│   │   ├── ansible-1
+│   │   │   ├── .pyre_configuration     # Configuration file for Pyinder and pyre
+│   │   │   ├── mypy.ini                # Configuration file for mypy
+│   │   │   ├── pyrightconfig.json      # Configuration file for pyright
+│   │   │   └── pytype.cfg              # Configuration file for pytype
+│   │   ├── ansible-2
+│   │   │   ├── ...
+│   │   ├── ...
+|   ├── excepy                          # Configuration files for excepy
+│   │   ├── ...
+│   ├── pyinder                         # Configuration files for Pyinder
+│   │   ├── ...
+├── download_repo.py                    # The script to download repositories
+├── excepy_repo.json                    # GitHub repositories and their commit id for excepy
+├── setting_config.py                   # The script to set the configuration for evaluation
+└── typebugs_repo.json                  # GitHub repositories and their commit id for typebugs
+```
+
+### Options
+
+`download_repo.py` and `setting_config.py` have the option **`--project`**.
+The option **`--project`** denotes the project name to analyze.
+| Option | Description |
+|:------:|:------------|
+| `--project` | Set the project name to analyze. |

eval/README.md

@@ -0,0 +1,18 @@
+### Structure
+
+```bash
+eval
+├── check_alarm.py      # show the number of alarms by each tool
+├── check_correct.py    # show the number of detecting type errors by each tool
+├── check_time.py       # show the time taken by each tool
+├── cloc.py             # run cloc to check the per kloc results
+└── draw_venn.py        # draw the venn diagram of the results
+```
+
+### Options
+
+`check_alarm.py`, `check_correct.py`, and `check_time.py` have the option **`-p`** or **`--project`**.
+The option **`-p`** or **`--project`** denotes the project name to analyze.
+| Option | Description |
+|:------:|:------------|
+| `-p` or `--project` | Set the project name to analyze. |

run/README.md

@@ -0,0 +1,29 @@
+### Structure
+
+```bash
+run
+├── change_core_async.py      # change the async keyword for old version of core project  
+├── filter_error.py           # extract only the type errors from the results
+├── mypy_change_json.py       # change the result log file to json file for mypy
+├── mypy_run.py               # run mypy with the project
+├── pyinder_change_json.py    # change the result log file to json file for Pyinder
+├── pyinder_run.py            # run Pyinder with the project
+├── pyre_change_json.py       # change the result log file to json file for pyre
+├── pyre_run.py               # run pyre with the project
+├── pyright_run.py            # run pyright with the project
+├── pytype_change_json.py     # change the result log file to json file for pytype
+├── pytype_run.py             # run pytype with the project
+├── run.sh                    # script to run Pyinder (please use `pyinder_run.py` instead)
+├── run_mypy.sh               # script to run mypy (please use `mypy_run.py` instead)
+├── run_pyre.sh               # script to run pyre (please use `pyre_run.py` instead)
+├── run_pyright.sh            # script to run pyright (please use `pyright_run.py` instead)
+└── run_pytype.sh             # script to run pytype (please use `pytype_run.py` instead)
+```
+
+### Options
+
+`mypy_run.py`, `pyinder_run.py`, `pyre_run.py`, and `pytype_run.py` have the option **`-p`** or **`--project`**.
+The option **`-p`** or **`--project`** denotes the project name to analyze.
+| Option | Description |
+|:------:|:------------|
+| `-p` or `--project` | Set the project name to analyze. |