Added execution time to pipeline report

[norberttech]

Change Log

Added

• execution time to pipeline report

Fixed

Changed

Removed

Deprecated

Security

---

Description

$report = df()
    ->read(from_csv(...))
    ->run(analyze: true);

// $report->statistics()->executionTime->startedAt;
// $report->statistics()->executionTime->finishedAt;
// $report->statistics()->executionTime->inSeconds();

When the argument analyze in the DataFrame::run() method is set to true the return of that method is Report.

Now this report also provides total pipeline execution time (or just to be more precise, stared and finished at timestamps).

Resolves: #1379

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak        | mode             | rstdev          |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.746mb +0.25%  | 537.746ms -0.47% | ±0.48% -74.41%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.814mb +0.25%  | 1.073s +2.03%    | ±0.45% -55.12%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 86.468mb +0.01% | 921.445ms +3.12% | ±1.04% +175.21% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.479mb +0.27%  | 34.113ms +0.78%  | ±0.60% +47.61%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.456mb +0.27%  | 611.937ms +1.83% | ±2.80% +879.29% |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 108.474mb +0.01% | 58.548ms +1.19% | ±0.23% -89.80% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+-----------------+----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode            | rstdev         |
+--------------------+----------------+------+-----+------------------+-----------------+----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.022mb +0.02%  | 99.032ms +1.58% | ±1.12% -2.80%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 76.782mb +0.02%  | 92.151ms -0.20% | ±0.44% -52.70% |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 166.965mb +0.01% | 20.658s +1.03%  | ±0.40% +40.74% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.066mb +0.07%  | 30.594ms +0.13% | ±0.54% -47.79% |
+--------------------+----------------+------+-----+------------------+-----------------+----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev           |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 80.249mb +0.00%  | 3.222ms +0.16%   | ±2.53% +65.17%   |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 97.527mb +0.00%  | 189.802ms +0.15% | ±0.27% -42.65%   |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 80.247mb +0.00%  | 19.014ms +0.31%  | ±1.41% +573.85%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 81.124mb +0.00%  | 1.564ms +7.23%   | ±3.11% +128.66%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 81.124mb +0.00%  | 1.597ms +12.69%  | ±1.99% +224.43%  |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 79.285mb +0.00%  | 3.628ms +2.98%   | ±2.54% +44.08%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 79.814mb +0.00%  | 14.929ms +1.63%  | ±3.69% +172.14%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 79.814mb +0.00%  | 15.106ms +1.61%  | ±3.51% +1329.87% |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 78.505mb +0.00%  | 1.706μs +0.36%   | ±2.72% +0.00%    |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 78.505mb +0.00%  | 0.400μs +33.33%  | ±0.00% +0.00%    |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 86.842mb +0.00%  | 12.459ms -0.12%  | ±0.85% -47.07%   |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 114.190mb +0.00% | 58.606ms +0.02%  | ±0.91% -44.66%   |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 80.333mb +0.00%  | 1.468ms +18.18%  | ±2.80% -7.00%    |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 83.626mb +0.00%  | 62.308ms +2.57%  | ±0.93% -5.29%    |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 81.386mb +0.00%  | 3.716ms +4.83%   | ±0.36% -19.18%   |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 78.789mb +0.00%  | 43.111ms +3.72%  | ±0.12% -82.52%   |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 78.789mb +0.00%  | 43.343ms +4.78%  | ±1.00% -19.66%   |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 78.789mb +0.00%  | 43.511ms +3.87%  | ±0.72% +14.40%   |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 80.945mb +0.00%  | 8.108ms -0.98%   | ±0.51% +135.19%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 78.695mb +0.00%  | 29.281ms +0.11%  | ±1.74% +515.33%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 78.505mb +0.00%  | 13.982μs +2.19%  | ±1.02% -2.16%    |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 78.505mb +0.00%  | 15.321μs +1.38%  | ±2.93% +371.81%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 97.528mb +0.00%  | 192.304ms -2.13% | ±0.25% -72.54%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 98.640mb +0.00%  | 446.155ms +1.08% | ±0.47% -83.25%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 51.471mb +0.00%  | 221.681ms +0.14% | ±1.21% +230.12%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 13.643mb +0.01%  | 49.331ms +2.55%  | ±0.69% +8.16%    |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 43.774mb +0.00%  | 362.499ms +0.85% | ±0.51% -57.53%   |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 11.584mb +0.02%  | 73.463ms +1.25%  | ±0.25% +68.70%   |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+

[codecov]

Codecov Report

Attention: Patch coverage is 86.11111% with 5 lines in your changes missing coverage. Please review.

Project coverage is 82.51%. Comparing base (3c57230) to head (3dc9196).
Report is 3 commits behind head on 1.x.

Additional details and impacted files

@@            Coverage Diff             @@
##              1.x    #1380      +/-   ##
==========================================
- Coverage   82.51%   82.51%   -0.01%     
==========================================
  Files         645      648       +3     
  Lines       17361    17395      +34     
==========================================
+ Hits        14325    14353      +28     
- Misses       3036     3042       +6     

Components	Coverage Δ
etl	85.82% <86.11%> (-0.01%)	⬇️
cli	85.17% <ø> (ø)
lib-array-dot	94.53% <ø> (ø)
lib-azure-sdk	62.56% <ø> (ø)
lib-doctrine-dbal-bulk	97.36% <ø> (ø)
lib-filesystem	76.23% <ø> (ø)
lib-parquet	84.57% <ø> (ø)
lib-parquet-viewer	82.02% <ø> (ø)
lib-rdsl	87.09% <ø> (ø)
lib-snappy	90.69% <ø> (-0.47%)	⬇️
bridge-filesystem-async-aws	90.38% <ø> (ø)
bridge-filesystem-azure	89.92% <ø> (ø)
bridge-monolog-http	96.38% <ø> (ø)
symfony-http-foundation	77.10% <ø> (ø)
adapter-chartjs	86.45% <ø> (ø)
adapter-csv	89.49% <ø> (ø)
adapter-doctrine	90.14% <ø> (ø)
adapter-elasticsearch	97.19% <ø> (ø)
adapter-google-sheet	78.04% <ø> (ø)
adapter-http	59.15% <ø> (ø)
adapter-json	92.85% <ø> (ø)
adapter-logger	53.84% <ø> (ø)
adapter-meilisearch	97.75% <ø> (ø)
adapter-parquet	59.88% <ø> (ø)
adapter-text	84.44% <ø> (ø)
adapter-xml	83.15% <ø> (ø)