Merge commit '3d45b3897f577c10db3f89c2ddc8065ab6c8302d'

.hgtags

@@ -22,3 +22,4 @@ e27327f5da35c5feb660360336fdc94bd0afe719 1.8
 981e3bfef16a997bce6f46ce1b15631a0e234747 2.1
 be14a7e9755e54f0fd34911c72bdfa66981220bc 2.2
 3037c1448549ca920967831482c653e5892fa8ed 2.3
+e7a4dd48293b7956d4a20df257d23904cc78e376 2.4

doc/reST/api.rst

@@ -192,6 +192,12 @@ changes made to the parameters for auto-detection and other reasons::
 	 *      presets is not recommended without a more fine-grained breakdown of
 	 *      parameters to take this into account. */
 	int x265_encoder_reconfig(x265_encoder *, x265_param *);
+**x265_encoder_ctu_info**
+       /* x265_encoder_ctu_info:
+        *    Copy CTU information such as ctu address and ctu partition structure of all
+        *    CTUs in each frame. The function is invoked only if "--ctu-info" is enabled and
+        *    the encoder will wait for this copy to complete if enabled.
+        */
 
 Pictures
 ========
@@ -341,6 +347,14 @@ statistics from the encoder::
 Cleanup
 =======
 
+At the end of the encode, the application will want to trigger logging
+of the final encode statistics, if :option:`--csv` had been specified::
+
+ 	/* x265_encoder_log:
+	 *       write a line to the configured CSV file. If a CSV filename was not
+	 *       configured, or file open failed, this function will perform no write. */
+ 	void x265_encoder_log(x265_encoder *encoder, int argc, char **argv);
+ 	
 Finally, the encoder must be closed in order to free all of its
 resources. An encoder that has been flushed cannot be restarted and
 reused. Once **x265_encoder_close()** has been called, the encoder

doc/reST/cli.rst

@@ -52,8 +52,7 @@ Command line executable return codes::
 	2. unable to open encoder
 	3. unable to generate stream headers
 	4. encoder abort
-	5. unable to open csv file
-
+	
 Logging/Statistic Options
 =========================
 
@@ -83,9 +82,66 @@ Logging/Statistic Options
 	it adds one line per run. If :option:`--csv-log-level` is greater than
 	0, it writes one line per frame. Default none
 
-	Several frame performance statistics are available when 
-	:option:`--csv-log-level` is greater than or equal to 2:
-
+	The following statistics are available when :option:`--csv-log-level` is
+	greater than or	equal to 1:
+
+	**Encode Order** The frame order in which the encoder encodes.
+
+	**Type** Slice type of the frame.
+
+	**POC** Picture Order Count - The display order of the frames. 
+
+	**QP** Quantization Parameter decided for the frame. 
+
+	**Bits** Number of bits consumed by the frame.
+
+	**Scenecut** 1 if the frame is a scenecut, 0 otherwise. 
+
+	**RateFactor** Applicable only when CRF is enabled. The rate factor depends
+	on the CRF given by the user. This is used to determine the QP so as to 
+	target a certain quality.
+
+	**BufferFill** Bits available for the next frame. Includes bits carried
+	over from the current frame.
+
+	**Latency** Latency in terms of number of frames between when the frame 
+	was given in and when the frame is given out.
+
+	**PSNR** Peak signal to noise ratio for Y, U and V planes.
+
+	**SSIM** A quality metric that denotes the structural similarity between frames.
+
+	**Ref lists** POC of references in lists 0 and 1 for the frame.
+
+	Several statistics about the encoded bitstream and encoder performance are 
+	available when :option:`--csv-log-level` is greater than or equal to 2:
+
+	**I/P cost ratio:** The ratio between the cost when a frame is decided as an
+	I frame to that when it is decided as a P frame as computed from the 
+	quarter-resolution frame in look-ahead. This, in combination with other parameters
+	such as position of the frame in the GOP, is used to decide scene transitions.
+
+	**Analysis statistics:**
+
+	**CU Statistics** percentage of CU modes.
+
+	**Distortion** Average luma and chroma distortion. Calculated as
+	SSE is done on fenc and recon(after quantization).
+
+	**Psy Energy**  Average psy energy calculated as the sum of absolute
+	difference between source and recon energy. Energy is measured by sa8d
+	minus SAD.
+
+	**Residual Energy** Average residual energy. SSE is calculated on fenc 
+	and pred(before quantization).
+
+	**Luma/Chroma Values** minumum, maximum and average(averaged by area)
+	luma and chroma values of source for each frame.
+
+	**PU Statistics** percentage of PU modes at each depth.
+
+	**Performance statistics:**
+
 	**DecideWait ms** number of milliseconds the frame encoder had to
 	wait, since the previous frame was retrieved by the API thread,
 	before a new frame has been given to it. This is the latency
@@ -111,6 +167,8 @@ Logging/Statistic Options
 	**Stall Time ms** the number of milliseconds of the reported wall
 	time that were spent with zero worker threads, aka all compression
 	was completely stalled.
+
+	**Total frame time** Total time spent to encode the frame.
 
 	**Avg WPP** the average number of worker threads working on this
 	frame, at any given time. This value is sampled at the completion of
@@ -123,8 +181,6 @@ Logging/Statistic Options
 	is more of a problem for P frames where some blocks are much more
 	expensive than others.
 
-	**CLI ONLY**
-
 .. option:: --csv-log-level <integer>
 
     Controls the level of detail (and size) of --csv log files
@@ -133,8 +189,6 @@ Logging/Statistic Options
     1. frame level logging
     2. frame level logging with performance statistics
 
-    **CLI ONLY**
-
 .. option:: --ssim, --no-ssim
 
 	Calculate and report Structural Similarity values. It is
@@ -795,33 +849,31 @@ the prediction quad-tree.
 
 Analysis re-use options, to improve performance when encoding the same
 sequence multiple times (presumably at varying bitrates). The encoder
-will not reuse analysis if the resolution and slice type parameters do
-not match.
+will not reuse analysis if slice type parameters do not match.
 
-.. option:: --analysis-mode <string|int>
+.. option:: --analysis-reuse-mode <string|int>
 
-	Specify whether analysis information of each frame is output by encoder
-	or input for reuse. By reading the analysis data writen by an
-	earlier encode of the same sequence, substantial redundant work may
-	be avoided.
-
-	The following data may be stored and reused:
-	I frames   - split decisions and luma intra directions of all CUs.
-	P/B frames - motion vectors are dumped at each depth for all CUs.
+	This option allows reuse of analysis information from first pass to second pass.
+	:option:`--analysis-reuse-mode save` specifies that encoder outputs analysis information of each frame.
+	:option:`--analysis-reuse-mode load` specifies that encoder reuses analysis information from first pass.
+	There is no benefit using load mode without running encoder in save mode. Analysis data from save mode is
+	written to a file specified by :option:`--analysis-reuse-file`. The amount of analysis data stored/reused
+	is determined by :option:`--analysis-reuse-level`. By reading the analysis data writen by an earlier encode
+	of the same sequence, substantial redundant work may be avoided. Requires cutree, pmode to be off. Default 0.
 
 	**Values:** off(0), save(1): dump analysis data, load(2): read analysis data
 
-.. option:: --analysis-file <filename>
+.. option:: --analysis-reuse-file <filename>
 
-	Specify a filename for analysis data (see :option:`--analysis-mode`)
+	Specify a filename for analysis data (see :option:`--analysis-reuse-mode`)
 	If no filename is specified, x265_analysis.dat is used.
 
-.. option:: --refine-level <1..10>
+.. option:: --analysis-reuse-level <1..10>
 
-	Amount of information stored/reused in :option:`--analysis-mode` is distributed across levels.
+	Amount of information stored/reused in :option:`--analysis-reuse-mode` is distributed across levels.
 	Higher the value, higher the information stored/reused, faster the encode. Default 5.
 
-	Note that --refine-level must be paired with analysis-mode.
+	Note that --analysis-reuse-level must be paired with analysis-reuse-mode.
 
 	+--------+-----------------------------------------+
 	| Level  | Description                             |
@@ -835,6 +887,41 @@ not match.
 	| 10     | Level 5 + Full CU analysis-info         |
 	+--------+-----------------------------------------+
 
+.. option:: --scale-factor
+
+       Factor by which input video is scaled down for analysis save mode.
+       This option should be coupled with analysis-reuse-mode option, --analysis-reuse-level 10.
+       The ctu size of load should be double the size of save. Default 0.
+
+.. option:: --refine-intra <0|1|2>
+
+	Enables refinement of intra blocks in current encode. 
+
+	Level 0 - Forces both mode and depth from the previous encode.
+
+	Level 1 - Evaluates all intra modes for blocks of size one smaller than 
+	the min-cu-size of the incoming analysis data from the previous encode, 
+	forces modes for blocks of larger size.
+
+	Level 2 - Evaluates all intra modes for	blocks of size one smaller than 
+	the min-cu-size of the incoming analysis data from the previous encode. 
+	For larger blocks, force only depth when angular mode is chosen by the 
+	previous encode, force depth and mode when other intra modes are chosen.
+
+	Default 0.
+
+.. option:: --refine-inter-depth
+
+	Enables refinement of inter blocks in current encode. Evaluates all 
+	inter modes for blocks of size one smaller than the min-cu-size of the 
+	incoming analysis data from the previous encode. Default disabled.
+
+.. option:: --refine-mv
+
+	Enables refinement of motion vector for scaled video. Evaluates the best 
+	motion vector by searching the surrounding eight integer and subpel pixel
+    positions.
+
 Options which affect the transform unit quad-tree, sometimes referred to
 as the residual quad-tree (RQT).
 
@@ -1221,7 +1308,16 @@ Slice decision options
 	intra cost of a frame used in scenecut detection. For example, a value of 5 indicates,
 	if the inter cost of a frame is greater than or equal to 95 percent of the intra cost of the frame,
 	then detect this frame as scenecut. Values between 5 and 15 are recommended. Default 5.	
-
+
+.. option:: --ctu-info <0, 1, 2, 4, 6>
+
+   This value enables receiving CTU information asynchronously and determine reaction to the CTU information. Default 0.
+   1: force the partitions if CTU information is present.
+   2: functionality of (1) and reduce qp if CTU information has changed.
+   4: functionality of (1) and force Inter modes when CTU Information has changed, merge/skip otherwise.
+   This option should be enabled only when planning to invoke the API function x265_encoder_ctu_info to copy ctu-info asynchronously. 
+   If enabled without calling the API function, the encoder will wait indefinitely.
+
 .. option:: --intra-refresh
 
 	Enables Periodic Intra Refresh(PIR) instead of keyframe insertion.
@@ -1491,15 +1587,23 @@ Quality, rate control and rate distortion options
     and also redundant steps are skipped.
     In pass 1 analysis information like motion vector, depth, reference and prediction
     modes of the final best CTU partition is stored for each CTU.
-    Default disabled.
+    Multipass analysis refinement cannot be enabled when 'analysis-save/analysis-load' option
+    is enabled and both will be disabled when enabled together. This feature requires 'pmode/pme'
+    to be disabled and hence pmode/pme will be disabled when enabled at the same time.
+
+    Default: disabled.
 
 .. option:: --multi-pass-opt-distortion, --no-multi-pass-opt-distortion
 
     Enable/Disable multipass refinement of qp based on distortion data along with multipass
     ratecontrol. In pass 1 distortion of best CTU partition is stored. CTUs with high
     distortion get lower(negative)qp offsets and vice-versa for low distortion CTUs in pass 2.
     This helps to improve the subjective quality.
-    Default disabled.
+    Multipass refinement of qp cannot be enabled when 'analysis-save/analysis-load' option
+    is enabled and both will be disabled when enabled together. 'multi-pass-opt-distortion' 
+    requires 'pmode/pme' to be disabled and hence pmode/pme will be disabled when enabled along with it.
+
+    Default: disabled.
 
 .. option:: --strict-cbr, --no-strict-cbr
 
@@ -1573,6 +1677,11 @@ Quality, rate control and rate distortion options
    that this option is used through the tune grain feature where a combination 
    of param options are used to improve visual quality.
 
+.. option:: --const-vbv, --no-const-vbv
+
+   Enables VBV algorithm to be consistent across runs. Default disabled. 
+   Enabled when :option:'--tune' grain is applied.
+
 .. option:: --qblur <float>
 
 	Temporally blur quants. Default 0.5
@@ -1879,7 +1988,12 @@ VUI fields must be manually specified.
 
 .. option:: --dhdr10-info <filename>
 
-	Inserts tone mapping information as an SEI message.
+	Inserts tone mapping information as an SEI message. It takes as input, 
+	the path to the JSON file containing the Creative Intent Metadata 
+	to be encoded as Dynamic Tone Mapping into the bitstream. 
+
+	Click `here <https://www.sra.samsung.com/assets/User-data-registered-itu-t-t35-SEI-message-for-ST-2094-40-v1.1.pdf>`_
+	for the syntax of the metadata file. A sample JSON file is available in `the downloads page <https://bitbucket.org/multicoreware/x265/downloads/DCIP3_4K_to_400_dynamic.json>`_
 
 .. option:: --dhdr10-opt, --no-dhdr10-opt
 

doc/reST/releasenotes.rst

@@ -2,8 +2,33 @@
 Release Notes
 *************
 
-Release Notes
-*************
+Version 2.5
+===========
+
+Release date - 13th July, 2017.
+
+Encoder enhancements
+--------------------
+1. Improved grain handling with :option:`--tune` grain option by throttling VBV operations to limit QP jumps.
+2. Frame threads are now decided based on number of threads specified in the :option:`--pools`, as opposed to the number of hardware threads available. The mapping was also adjusted to improve quality of the encodes with minimal impact to performance.
+3. CSV logging feature (enabled by :option:`--csv`) is now part of the library; it was previously part of the x265 application. Applications that integrate libx265 can now extract frame level statistics for their encodes by exercising this option in the library.
+4.  Globals that track min and max CU sizes, number of slices, and other parameters have now been moved into instance-specific variables. Consequently, applications that invoke multiple instances of x265 library are no longer restricted to use the same settings for these parameter options across the multiple instances.
+5. x265 can now generate a seprate library that exports the HDR10+ parsing API. Other libraries that wish to use this API may do so by linking against this library. Enable ENABLE_HDR10_PLUS in CMake options and build to generate this library.
+6. SEA motion search receives a 10% performance boost from AVX2 optimization of its kernels.
+7. The CSV log is now more elaborate with additional fields such as PU statistics, average-min-max luma and chroma values, etc. Refer to documentation of :option:`--csv` for details of all fields.
+8. x86inc.asm cleaned-up for improved instruction handling.
+
+API changes
+-----------
+1. New API x265_encoder_ctu_info() introduced to specify suggested partition sizes for various CTUs in a frame. To be used in conjunction with :option:`--ctu-info` to react to the specified partitions appropriately.
+2. Rate-control statistics passed through the x265_picture object for an incoming frame are now used by the encoder.
+3. Options to scale, reuse, and refine analysis for incoming analysis shared through the x265_analysis_data field in x265_picture for runs that use :option:`--analysis-reuse-mode` load; use options :option:`--scale`, :option:`--refine-mv`, :option:`--refine-inter`, and :option:`--refine-intra` to explore. 
+4. VBV now has a deterministic mode. Use :option:`--const-vbv` to exercise.
+
+Bug fixes
+---------
+1. Several fixes for HDR10+ parsing code including incompatibility with user-specific SEI, removal of warnings, linking issues in linux, etc.
+2. SEI messages for HDR10 repeated every keyint when HDR options (:option:`--hdr-opt`, :option:`--master-display`) specified.
 
 Version 2.4
 ===========