Use GPS time for videos if the difference with creation time is too big #659

malconsei · 2023-09-13T12:53:03Z

Some cameras reset their internal clock when the battery runs out, or otherwise get out of sync with the world clock.

Change PointWithFix to a more generic GpsPoint which contains GPS timestamps.
If available, save the GPS timestamp of the last point in the JSON description.
Upon upload, use it as timestamp of the CAMM track.
Server-side, use this timestamp if it's more than 86400s (1 day) apart from the video creation time.

ptpt · 2023-09-15T05:50:58Z

mapillary_tools/exiftool_read_video.py

@@ -173,7 +173,7 @@ def _aggregate_float_values_same_length(
        if timestamp is None or lon is None or lat is None:
            continue
        track.append(
-            geo.PointWithFix(
+            geo.GpsPoint(


Here I guess we can keep using geo.Point as unix_timestamp == time?

ptpt · 2023-09-15T06:22:49Z

mapillary_tools/geotag/blackvue_parser.py

@@ -44,12 +44,13 @@ def _parse_gps_box(gps_data: bytes) -> T.Generator[geo.Point, None, None]:
            if not nmea.is_valid:
                continue
            epoch_ms = int(match.group(1))
-            yield geo.Point(
+            yield geo.GpsPoint(
                time=epoch_ms,


It seems to be a bug here: Point.time should be always seconds (either seconds since epoch or offsets in seconds).

Could you change it to time=epoch_ms / 1000 and remove the / 1000 at L107?

ptpt · 2023-09-15T06:23:42Z

mapillary_tools/geotag/blackvue_parser.py

                time=epoch_ms,
                lat=nmea.latitude,
                lon=nmea.longitude,
                alt=nmea.altitude,
                angle=None,
+                unix_timestamp=int(epoch_ms / 1000),


Doesn't need as unix_timestamp == time

ptpt · 2023-09-15T06:29:27Z

mapillary_tools/geotag/camm_parser.py

            time=sample.time_offset,
            lat=box.data.latitude,
            lon=box.data.longitude,
            alt=box.data.altitude,
            angle=None,
+            unix_timestamp=int(_gps_timestamp_to_unix(box.data.time_gps_epoch)),
+            gps_fix=geo.GPSFix(box.data.gps_fix_type),


Adding gps_fix/gps_ground_speed/gps_horizontal_accuracy here will trigger gps filtering which affect GPS points and video slicing. Let's do it in a separate PR with proper testing to be safe.

ptpt · 2023-09-15T06:37:03Z

mapillary_tools/geotag/camm_parser.py

+    for some info on GPS timestamp. Here we arbitrarily use the offset as of 2023, since we
+    are not interested in precision to the second.
+    """
+    return ts + 315964800 - 18


why minus 18?

ptpt · 2023-09-15T06:37:11Z

mapillary_tools/geotag/camm_parser.py

+def _gps_timestamp_to_unix(ts: float) -> float:
+    """
+    Convert a GPS timestamp to UNIX timestamp.
+    See https://stackoverflow.com/questions/20521750/ticks-between-unix-epoch-and-gps-epoch


That's good to know!

ptpt · 2023-09-15T06:39:27Z

mapillary_tools/geotag/camm_parser.py

 def filter_points_by_elst(
-    points: T.Iterable[geo.Point], elst: T.Sequence[T.Tuple[float, float]]
-) -> T.Generator[geo.Point, None, None]:
+    points: T.Iterable[geo.GpsPoint], elst: T.Sequence[T.Tuple[float, float]]


Seems no need to be GpsPoint here: can cast GpsPoint to Point to use this function.

ptpt · 2023-09-15T06:42:16Z

mapillary_tools/geotag/geotag_images_from_gpx_file.py

                            lat=point.latitude,
                            lon=point.longitude,
                            alt=point.elevation,
                            angle=None,
+                            gps_ground_speed=point.speed,
+                            unix_timestamp=int(time),


no need unix_timestamp here as time == unix_timestamp

ptpt · 2023-09-15T06:42:42Z

mapillary_tools/geotag/geotag_images_from_nmea_file.py

+                    lon=lon,
+                    alt=alt,
+                    angle=None,
+                    unix_timestamp=int(time),


no need as time == unix_timestamp

ptpt · 2023-09-15T06:59:12Z

mapillary_tools/geo.py

+    gps_precision: T.Optional[float] = None
+    gps_ground_speed: T.Optional[float] = None
+    gps_horizontal_accuracy: T.Optional[float] = None
+    unix_timestamp: T.Optional[int] = None


Using float instead of int looks more consistent as time uses float (although it ended being rounded as video created time)

ptpt · 2023-10-02T06:53:27Z

LGTM.

The only concern is the ground speed added in mapillary_tools/geotag/camm_parser.py. It is used to determine if a GPS point is noisy or not (and noisy points and corresponding frames will be removed). If the velocity values were populated incorrectly, then normal GPS points would be mistakenly removed too. Need to test this change on real CAMM videos.

facebook-github-bot · 2024-04-01T02:05:28Z

Hi @malconsei!

Thank you for your pull request.

We require contributors to sign our Contributor License Agreement, and yours needs attention.

You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

eneerhut · 2024-05-21T22:27:10Z

Keeping an eye on this one. It would great to have GPS time set as the default to avoid capture time data issues when the camera resets its clock. This is a common issue on GoPros.

malconsei requested a review from ptpt September 13, 2023 12:53

facebook-github-bot added the cla signed label Sep 13, 2023

malconsei force-pushed the feat-gps-time-2 branch from e7ff738 to 4ca41e9 Compare September 13, 2023 13:08

ptpt reviewed Sep 15, 2023

View reviewed changes

malconsei added 3 commits September 25, 2023 15:32

Separate usage of Point and GpsPoint

cdcabc5

Set last gps timestamp in uploaded video. Fixes.

c98d2e9

Switch to seconds for creation ts, fixes

57e853d

malconsei force-pushed the feat-gps-time-2 branch from 4ca41e9 to 57e853d Compare September 25, 2023 13:33

ptpt approved these changes Oct 2, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use GPS time for videos if the difference with creation time is too big #659

Use GPS time for videos if the difference with creation time is too big #659

malconsei commented Sep 13, 2023

ptpt Sep 15, 2023

ptpt Sep 15, 2023

ptpt Sep 15, 2023

ptpt Sep 15, 2023

ptpt Sep 15, 2023

ptpt Sep 15, 2023

ptpt Sep 15, 2023

ptpt Sep 15, 2023

ptpt Sep 15, 2023

ptpt Sep 15, 2023

ptpt commented Oct 2, 2023

facebook-github-bot commented Apr 1, 2024

eneerhut commented May 21, 2024

Use GPS time for videos if the difference with creation time is too big #659

Are you sure you want to change the base?

Use GPS time for videos if the difference with creation time is too big #659

Conversation

malconsei commented Sep 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ptpt commented Oct 2, 2023

facebook-github-bot commented Apr 1, 2024

Process

eneerhut commented May 21, 2024