3MF Spec references in test specification

[JimZuber]

This is a placeholder to revisit the topic of mapping some 3MF specification references to individual test cases once the beta test is complete. We hadn't really worked out a practical way to do this when we ran out of the current budget.

The specifications done for HP did have specific 3MF specification references for each test case, but this quickly became a big maintenance problem as the 3MF specification changed, and these references were not ported to the consolidated test specification put together for the consortium. Current plan is to review test cases and only document 3MF specification references where the requirement might not be obvious.

[martinweismann]

Good point.

Just some thoughts regarding the maintenance:
The testcases already specify the exact version of the specifications used. This can be permanently referenced via https://github.com/3MFConsortium/spec_core/blob/1.2.2/3MF%20Core%20Specification.md (which contains the version.)

I suggest to leave large numbers of references to the positions in the specification of this specific versions by referencing e.g. https://github.com/3MFConsortium/spec_core/blob/1.2.2/3MF%20Core%20Specification.md#611-jpeg-images

In an update of the testcases to a later version (say 1.3.3) one can easily update the references by updating the version string in the URL, and secondly verify, that the actual content of the spec has not changed.

[JimZuber]

Where practical, will add one URL reference to the appropriate 3MF specification section identifier for each test case defined in the test specification (i.e. P_???_902).

[JimZuber]

Referencing 3MF spec section in test cases as suggested by @martinweismann as follows...

https://github.com/3MFConsortium/spec_core/blob/1.2.2/3MF%20Core%20Specification.md#611-jpeg-images

...will only work if the latest versions of the 3MF specifications are released on GitHub with a tag created referencing the latest version. Only the core spec has been "released" on GitHub and has a tag (1.2.2), and it is already one version out of sync with the published version of the Core spec on the master branch (1.2.3). I also found it odd that the 3MF.io site link to the core specification took you to the master branch rather than the "released" 1.2.2 spec. I assume the master branch could point to an unpublished version of the spec, so I am not so sure it is a good idea for the links on the 3MF.io site to point to the master branch.

[martinweismann]

Hi Jim,

1. I agree that the 3MF.io site should probably link to the latest released version of each spec and all specification repos must have a release of their current version.
   Having said that, each new version on the master is "accepted" by the consortium via a pull request from a branch and could (should) essentially be released vie the github releases.
   I will bring this up in the next meeting.

2. I consider linking to a specific version tag (1.2.2 in https://github.com/3MFConsortium/spec_core/blob/1.2.2/3MF%20Core%20Specification.md#611-jpeg-images) in contrast to "the latest version on master" actually a feature of the test-suites repo, as the test suites are for a specific version of the specifications.
   If any of the specs is updated, the test suite needs to be adapted too:
   e.g. if we act on Rendering incorrect for P_XXM_0502_02 and P_XXM_0502_03 #21 and change the handling of P1, P2, P2 in the core spec in version 1.3.0, the test suite would still be correct for version 1.2.2, but not necessarily correct for v 1.3.0 (it would actually be incorrect for 1.3.0).

Do you agree with that?

[JimZuber]

Agreed! Once the test suites are updated to reflect any changes to a 3MF spec update, we will update the links in the spec with the new version number and validate that they still point to valid anchors in the 3MF specs.

Will need current master branch of public specs updated to released for me to move forward with the links.

[martinweismann]

We created github releases for all public specifications of 3MF extensions and the core spec.
Whenever changes to a specification are substantial enough, we intend to publish the updated spec via a github release.

This should enable us to link to specific versions and specific text passages of the specs from the test-suites.

[JimZuber]

I noticed that that for the Core specification the version on the master branch is 1.2.3 whereas the the released version is 1.2.2. Assume we would want the released version to be the same as master.

[jordig100]

Correct. The update was just clarifications.
But agreed not to include references in the first released version.

[JimZuber]

Note that competing links in the test specification was not funded as part of the 1.0 release. Work that still needs to be done includes...

1)When a PDF is available for version 1.2.1 of the material specification, links need to be added.

2)Links for the production specification need to be updated to the PDF for version 1.1.2

3)Links for the slice specification need to be updated to the PDF for version 1.0.2

4)There are a fair number of tests that need to have links added as this work was only partially completed.