Skip to content

Sort images taken in the same second by their EXIF time - #960

Open
0z5a wants to merge 1 commit into
mapillary:mainfrom
0z5a:sort-by-exif-subsec
Open

0z5a wants to merge 1 commit into
mapillary:mainfrom
0z5a:sort-by-exif-subsec

Conversation

@0z5a

@0z5a 0z5a commented Nov 20, 2026

Copy link
Copy Markdown

Fixes #677.

Problem

ExifRead.extract_capture_time() prefers the GPS date/time, which usually has no subseconds. When two images are taken within the same second, they get the same capture time, and ImageMetadata.sort_key() breaks the tie by filename. OpenCamera names the second shot of a second IMG_..._170431 (1).jpg, which sorts before IMG_..._170431.jpg (space < .). Every such pair is put in reverse order. _interpolate_subsecs_for_sorting then spreads the times in that wrong order, so the uploaded sequence steps back and forth, as in the issue.

Change

As suggested in #677, images are now sorted by (capture time, EXIF time with subseconds, filename):

  • ImageMetadata gets an exif_time field: the EXIF DateTimeOriginal/SubSecTimeOriginal (or the DateTimeDigitized fallback) as Unix time. It is only used in sort_key(). It is not a MAP* field, so it isn't written to the description file or EXIF, and the description schema is unchanged.
  • ImageEXIFExtractor fills it in. Exiftool-based extraction goes through the same extractor. extract_exif_datetime() was already implemented by all three readers, and it is now declared on ExifReadABC.
  • Metadata without an EXIF time (for example, loaded back from a description file) sorts with math.inf in that slot, so the key stays comparable and the order falls back to the filename as before.

How I found it

I went through the open sequence/ordering issues and reproduced #677 on current main (0.15.1) with seven synthetic JPEGs written with piexif in the OpenCamera pattern. Each image has whole-second GPS times, SubSecTimeOriginal set, and latitude increasing in capture order. These are the mapillary_tools process results from mapillary_image_description.json:

file before: order / MAPCaptureTime after: order / MAPCaptureTime
IMG_20231115_170430.jpg 1 / …_04_30_000 1 / …_04_30_000
IMG_20231115_170431.jpg 3 / …_04_31_500 2 / …_04_31_000
IMG_20231115_170431 (1).jpg 2 / …_04_31_000 3 / …_04_31_500
IMG_20231115_170432.jpg 4 / …_04_32_000 4 / …_04_32_000

Before the change, the latitudes in output order went 52.50000, 52.50006, 52.50003, 52.50009, …. After it, they increase monotonically. The 170433 pair behaved the same way.

Where this can show up in practice (my estimate)

Phone camera apps that shoot faster than one frame per second with GPS on, such as OpenCamera with a short interval or burst mode. Other cameras whose GPS timestamps are whole seconds but whose filenames don't sort in capture order are affected too. Every same-second pair is reversed in the uploaded sequence.

Test plan

  • New test_sequences_sorted_by_exif_time_within_same_time in tests/unit/test_sequence_processing.py. It fails without the sort_key() change and passes with it.
  • pytest tests/unit: 1645 passed, 33 skipped (macOS, Python 3.11).
  • ruff check mapillary_tools, ruff format --check mapillary_tools tests and mypy mapillary_tools tests/cli are clean.

The capture time prefers the GPS time, which usually has no subseconds,
so images taken within the same second tie and were ordered by
filename. OpenCamera names the second one "IMG_... (1).jpg", which
sorts first, so each such pair was reversed in the sequence.

Keep the EXIF capture time (with subseconds) on ImageMetadata and use
it as the second sort key, before the filename.

Fixes mapillary#677
@meta-cla meta-cla Bot added the cla signed label Nov 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Broken image ordering for OpenCamera subsecond sequences

1 participant