LEP 0001: LJSON v3

Introduction

This enhancement proposal is a planning draft of version 3 of the LJSON file format.

LJSON v1 established a simple JSON-based format for storing landmark annotations. LJSON introduced the ability to attach semantic meaning to landmarks in two ways - by organising landmarks into labelled groups and by encoding connectivity relations between landmarks inside groups.

LJSON v2 moved the connectivity definitions to be for the while set of points instead of per-group - see the wiki page on the migration for more details.

Looking forward, LJSON v3 is an opportunity to revisit our file format with two goals in mind:

1. reduce usability pains with landmarker.io/menpo ecosystem
2. enable new features

Proposal: multiple landmark sets in one file

It's fairly common to require multiple annotations on a single asset (bounding box/points as an obvious example from images), but to date, each LJSON file only stores one set of landmarks. There are good reasons to keep to a one landmark set/one file system:

1. It's a simple model to understand for technical users. If a user wants to use two annotations together on an asset, they just need to get ahold of the two files, potentially from two different sources.
2. For the mass-annotation server case individual files are easier to administer. It's friendlier for version control (clearer in VCS checkins what exactly has been modified) and makes it easy to understand the data (e.g. to count the number of assets with lmtype: ls -l ./**/lmtype.lson | wc -l)
3. Merging multiple landmark files is trivial. In the case where one file has multiple landmarks, keys from two different files can collide.
4. Folders can be used to group one assets landmarks anyway (this is how our Dropbox service works).

However, as we look to make the landmarker more friendly for non-technical users, this design decision makes it harder to understand how to use landmarker. It's non-obvious how to position landmark files in Dropbox to have them appear in the landmarker (answer: for ./foo.jpg, ./landmarks/foo/some_name.ljson)

Current (v2) specification

• landmarks (object)
  • points (array) - the ordered locations of each points on the resource (2d or 3d as long as it consistent). Similar to the list of points in a pts file.
  • connectivity: (array) - list of tuples [a, b] where a and b are valid indices for 2 points which should appear connected.
• labels (array of objects) - the label field is the name of the group, while the mask field is a list of the indices for the points which belong to this group, they do not need to be contiguous.
• version (integer) - for validation.

Proposed (v3) specification

Two changes are made:

1. A new metadata key is added at the same level as landmarks and labels. This can contain any JSON data describing the landmarks. A special visible key is reserved - if this key is used under metadata it is assumed to be a array of numbers of length n_landmarks where the i'th number encodes the visability of the ith landmark. Values are 0 or 1.

2. All of the LSONv2 structure (except for version) is placed under a new top-level groups object. The groups object is a mapping from labels to landmark descriptions, each of which is declared exactly as v2 was, with the addition of the metadata key above.

An example file:

{
    "groups": {
        "bounding_tri": {
            "landmarks": {
                "points": [
                    [3, 4],
                    [6.6, 7.6],
                    [2.3, 6.4]
                ],
                "connectivity": [
                    [0, 1],
                    [1, 2],
                    [0, 1]
                ]
            },
            "labels": [
                {
                    "label": "first_edge",
                    "mask": [0, 1]
                },
                {
                    "label": "second_edge",
                    "mask": [1, 2]
                }
            ],
            "metadata": {
                "visibile": [1, 1, 0]
            }
        },
        "ibug68": {
            "landmarks": {
                "points": [
                    [4.5, 3.6],
                    [6.6, 7.1],
                    [15.2, 2.8],
                    [6.6, 8.6]
                ]
            },
            "labels": [
                {
                    "label": "all",
                    "mask": [0, 1, 2, 3]
                }
            ]
        }
    },
    "version": 3
}

This file contains two landmark sets "bounding_tri" and "ibug68". "bounding_tri" uses the full supported API - "ibug68" is a minimal landmark definition.