Skip to content

Commit

Permalink
Fix the visual appearance of lists in doc
Browse files Browse the repository at this point in the history
  • Loading branch information
piconti committed May 30, 2024
1 parent 5aec524 commit 5b2d3f9
Show file tree
Hide file tree
Showing 8 changed files with 27 additions and 22 deletions.
Binary file modified docs/_build/doctrees/environment.pickle
Binary file not shown.
Binary file modified docs/_build/doctrees/versioning.doctree
Binary file not shown.
2 changes: 1 addition & 1 deletion docs/_build/html/_sources/versioning.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The main goal of this approach is to version the data and track information at e
3. **Ensure transparency:** Citation of the various data stages and datasets should be straightforward; users should know when using the interface exactly what versions they are using, and should be able to consult the precise statistics related to them.


Data Statistics and NewspaperStatistics
Data Statistics and Newspaper Statistics
------------------------------------------

.. automodule:: impresso_commons.versioning.data_statistics
Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ <h1>Welcome to Impresso PyCommons’s documentation!<a class="headerlink" href="
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="versioning.html">Data Versioning</a><ul>
<li class="toctree-l2"><a class="reference internal" href="versioning.html#module-impresso_commons.versioning.data_statistics">Data Statistics and NewspaperStatistics</a></li>
<li class="toctree-l2"><a class="reference internal" href="versioning.html#module-impresso_commons.versioning.data_statistics">Data Statistics and Newspaper Statistics</a></li>
<li class="toctree-l2"><a class="reference internal" href="versioning.html#module-impresso_commons.versioning.data_manifest">Data Manifest</a></li>
<li class="toctree-l2"><a class="reference internal" href="versioning.html#module-impresso_commons.versioning.helpers">Versioning Helpers</a></li>
<li class="toctree-l2"><a class="reference internal" href="versioning.html#module-impresso_commons.versioning.compute_manifest">Manifest Computing Script</a></li>
Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/searchindex.js

Large diffs are not rendered by default.

24 changes: 14 additions & 10 deletions docs/_build/html/versioning.html
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
<li class="toctree-l1"><a class="reference internal" href="utils.html">Utilities</a></li>
<li class="toctree-l1"><a class="reference internal" href="images.html">Image handling</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Data Versioning</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#module-impresso_commons.versioning.data_statistics">Data Statistics and NewspaperStatistics</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#module-impresso_commons.versioning.data_statistics">Data Statistics and Newspaper Statistics</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#impresso_commons.versioning.data_statistics.DataStatistics"><code class="docutils literal notranslate"><span class="pre">DataStatistics</span></code></a><ul>
<li class="toctree-l4"><a class="reference internal" href="#impresso_commons.versioning.data_statistics.DataStatistics.stage"><code class="docutils literal notranslate"><span class="pre">DataStatistics.stage</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="#impresso_commons.versioning.data_statistics.DataStatistics.granularity"><code class="docutils literal notranslate"><span class="pre">DataStatistics.granularity</span></code></a></li>
Expand Down Expand Up @@ -200,7 +200,7 @@ <h1>Data Versioning<a class="headerlink" href="#data-versioning" title="Link to
<li><p><strong>Ensure transparency:</strong> Citation of the various data stages and datasets should be straightforward; users should know when using the interface exactly what versions they are using, and should be able to consult the precise statistics related to them.</p></li>
</ol>
<section id="module-impresso_commons.versioning.data_statistics">
<span id="data-statistics-and-newspaperstatistics"></span><h2>Data Statistics and NewspaperStatistics<a class="headerlink" href="#module-impresso_commons.versioning.data_statistics" title="Link to this heading"></a></h2>
<span id="data-statistics-and-newspaper-statistics"></span><h2>Data Statistics and Newspaper Statistics<a class="headerlink" href="#module-impresso_commons.versioning.data_statistics" title="Link to this heading"></a></h2>
<p>This module contains the definition of a data statistics class.</p>
<p>A DataStatstics object should be instantiated during each processing step of
the data preprocessing and augmentation of the Impresso project, and used to
Expand Down Expand Up @@ -592,9 +592,9 @@ <h1>Data Versioning<a class="headerlink" href="#data-versioning" title="Link to
<dl class="simple">
<dt>The steps of this computation are the following:</dt><dd><ul class="simple">
<li><p>Ensure <cite>_processing_stats</cite> is not empty so the manifest can be computed and
crystallize the time this function is called as the <cite>_generation_date</cite> .</p></li>
crystallize the time this function is called as the <cite>_generation_date</cite>.</p></li>
<li><p>Fetch the previous version of this manifest from S3, extract its media list.</p></li>
<li><p>Generate the new media list given the previous one and <cite>_processing_stats</cite> .</p></li>
<li><p>Generate the new media list given the previous one and <cite>_processing_stats</cite>.</p></li>
<li><p>Compute the new title and corpus level statistics using the new media list.</p></li>
<li><p>Compute the new version based on the performed updates.</p></li>
<li><p>Define the <cite>manifest_data</cite> attribute corresponding to the final manifest.</p></li>
Expand Down Expand Up @@ -894,14 +894,18 @@ <h1>Data Versioning<a class="headerlink" href="#data-versioning" title="Link to
<span class="sig-name descname"><span class="pre">update_media_stats</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">title</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">yearly_stats</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">dict</span><span class="p"><span class="pre">]</span></span></span></em>, <em class="sig-param"><span class="n"><span class="pre">old_media_list</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">dict</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">dict</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">list</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#impresso_commons.versioning.data_manifest.DataManifest.update_media_stats" title="Link to this definition"></a></dt>
<dd><p>Update a title’s media statistics given the its newly computed yearly stats.</p>
<p>Note that it’s actually the <cite>old_media_list</cite>’s contents which are updated when
necessary.
In addition, the value of <cite>self.only_counting</cite> will change the behavior:
- When False, the computation of the manifest should follow a processing, and
necessary.</p>
<dl class="simple">
<dt>In addition, the value of <cite>self.only_counting</cite> will change the behavior:</dt><dd><ul class="simple">
<li><p>When False, the computation of the manifest should follow a processing, and
all data within the <cite>_processing_stats</cite> (here <cite>yearly_stats</cite> for 1 title) will
be considered to have been modified (or re-generated).
- When True, the manifest is computed to verify the contents of the data, and
be considered to have been modified (or re-generated).</p></li>
<li><p>When True, the manifest is computed to verify the contents of the data, and
the media’s information will be update only if differences in statisitics are
found between the previous and current version.</p>
found between the previous and current version.</p></li>
</ul>
</dd>
</dl>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
Expand Down
2 changes: 1 addition & 1 deletion docs/versioning.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The main goal of this approach is to version the data and track information at e
3. **Ensure transparency:** Citation of the various data stages and datasets should be straightforward; users should know when using the interface exactly what versions they are using, and should be able to consult the precise statistics related to them.


Data Statistics and NewspaperStatistics
Data Statistics and Newspaper Statistics
------------------------------------------

.. automodule:: impresso_commons.versioning.data_statistics
Expand Down
17 changes: 9 additions & 8 deletions impresso_commons/versioning/data_manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -764,13 +764,14 @@ def update_media_stats(
Note that it's actually the `old_media_list`'s contents which are updated when
necessary.
In addition, the value of `self.only_counting` will change the behavior:
- When False, the computation of the manifest should follow a processing, and
all data within the `_processing_stats` (here `yearly_stats` for 1 title) will
be considered to have been modified (or re-generated).
- When True, the manifest is computed to verify the contents of the data, and
the media's information will be update only if differences in statisitics are
found between the previous and current version.
- When False, the computation of the manifest should follow a processing, and
all data within the `_processing_stats` (here `yearly_stats` for 1 title) will
be considered to have been modified (or re-generated).
- When True, the manifest is computed to verify the contents of the data, and
the media's information will be update only if differences in statisitics are
found between the previous and current version.
Args:
title (str): Media title for which to update the media list.
Expand Down Expand Up @@ -989,9 +990,9 @@ def compute(
The steps of this computation are the following:
- Ensure `_processing_stats` is not empty so the manifest can be computed and
crystallize the time this function is called as the `_generation_date` .
crystallize the time this function is called as the `_generation_date`.
- Fetch the previous version of this manifest from S3, extract its media list.
- Generate the new media list given the previous one and `_processing_stats` .
- Generate the new media list given the previous one and `_processing_stats`.
- Compute the new title and corpus level statistics using the new media list.
- Compute the new version based on the performed updates.
- Define the `manifest_data` attribute corresponding to the final manifest.
Expand Down

0 comments on commit 5b2d3f9

Please sign in to comment.