ICLR 2027 Submission · Under Double-Blind Review

VDBench: A Benchmark for Visual Diachronic Analysis of Historical Depictions

Anonymous Author(s)
Affiliation withheld for review
Overview of the VDBench dataset construction pipeline, five stages: topic collection sourcing by Historian, Retrieval and Librarian roles; source-grounded topic collection of 148 topics; expert-accepted subset of 37 topics; controlled image generation with parameters, pre-verification, describe, render, and post-verification steps and a fail/retry loop; and the resulting dataset and benchmark. Human validation runs alongside via author audit, Stage 1 expert review, and Stage 2 image review.
Overview: (1) an LLM Historian proposes photographable diachronic topics and their periods while a Librarian retrieves and links supporting scholarly sources; (2)–(3) authors audit every proposed topic and domain experts review the survivors, retaining 148 of 266 source-grounded topics; (4) each accepted period is turned into a controlled when/where/what image-generation spec, grounded against real photographs, rendered, and CLIP-verified, retrying on failure; (5) the verified images and their full construction trace form the VDBench dataset and benchmark, with a further Stage-2 expert review of the rendered images running alongside.

Abstract

We introduce Visual Diachronic Analysis (VDA), a new computer vision challenge for studying how the visual context of a depicted subject or theme changes over time and how these changes relate to documented historical shifts in meaning. This raises the question to what extent such changes in meaning can be inferred computationally from historical depictions. To study such phenomena, we introduce VDBench, the first benchmark for VDA. In absence of available and sufficiently annotated historical image collections, we propose an unconventional approach: we construct a historically grounded synthetic benchmark dataset. We compile a source-grounded collection of human-centred diachronic themes and design a controlled text-to-image pipeline to generate plausible synthetic historical depictions. Domain experts confirm the suitability and plausibility of the data. We specify several computer vision tasks, propose a suitable evaluation methodology, and provide baseline methods and implementations as reference for future work. The dataset and an interactive explorer are available above. All code will be released upon acceptance.

Highlights

From Lexical Drift to Visual Drift

Diachronic analysis originates in linguistics: historical semantics studies how a word's meaning shifts while its form stays fixed — e.g. word-embedding neighbours of “gay” move from cheerful toward homosexual across the twentieth century. VDA asks the analogous question of images: a subject can remain visually recognisable across a century while its environment, activity, and co-occurring elements change, tracking a documented shift in social meaning.

Side-by-side timelines. Left, linguistic diachrony of the word 'gay': 1900s associated with daft, tasteful, flaunting, sweet, frolicsome; 1950s with pleasant, witty, bright, cheerful, jaunty; 1990s with gays, homosexual, lesbian, bisexual, queer. Right, visual diachrony of 'women in trousers': 1910s photograph with tasteless, inappropriate, extremist, scandalous, foolish, shocking; 1940s photograph with justified, practical, decent, appropriate, work garment, acceptable; 2010s photograph with ordinary, casual, daily attire, common, normal, typical, regular.
Linguistic vs. visual diachrony: the subject (a word, a garment) persists; what shifts is the context around it and the meaning read from that context. Linguistic example adapted from Hamilton et al.

Source-Grounded, Not Archive-Grounded

Historical archives are rarely catalogued by diachronic topic, so VDBench starts from the historical record and works forward to images. An LLM-based retrieval pipeline compiles a source-grounded collection of visual diachronic topics from scholarly literature; authors and humanities experts then filter and validate the resulting trajectories before any image is generated.

Two Benchmark Tasks

Task 1 reads the context out of a single image; Task 2 assembles many images — with either ground truth or Task-1 predictions — back into a topic's diachronic trajectory.

Task 1 · Visual context detection

In a single image I.
Out year, environment, activity, and a free-form caption.
  • When — year the depiction is set in
  • Where — environment, closed vocabulary
  • What — activity, closed vocabulary
  • Caption — open-vocabulary context description

Task 2 · Topic & trajectory discovery

In an image set for one (identity-hidden) topic, with per-image context.
Out topic title & description, plus an ordered period sequence.
  • Partition images into J periods
  • Title, year range & description per period
  • Chronological ordering of the trajectory
  • No Task-2 ground truth or human input provided
Illustration of the two benchmark tasks on the women-in-trousers example: six periods from Dress reform (1850-1899) through Full acceptance (1980-2025) laid out on a timeline. Task 1, per-image context, takes a single image and predicts when (year 1892), where (street), what (shopping), and a caption. Task 2, topic and trajectory discovery, takes the image collection plus Task 1 readings and predicts a topic title (women in trousers), overall description, and an ordered trajectory of periods, e.g. period P5 titled Liberation era, ranged 1960-1979, describing trousers gaining acceptance in fashion and workplaces.
Task illustration: Task 1 estimates when/where/what for one image; Task 2 partitions a topic's full image set into a titled, dated, described sequence of periods.

First Baselines, Neither Solving VDA

Two zero-shot, open-weight reference baselines — Open-MLLM (Qwen3-VL-32B-Instruct applied directly to both tasks) and TrajEx, a specialised pipeline combining a frozen EVA-CLIP year estimator, zero-shot environment/activity matching, and change-point detection over time — establish first reference points and expose how far current foundation models are from solving VDA.

Key Contributions

A new challenge: Visual Diachronic Analysis, with two concrete benchmark tasks — per-image context detection and diachronic trajectory discovery.
The VDBench benchmark: a historically grounded topic collection with plausible synthetic depictions, built via semi-automatic retrieval and generation pipelines, plus a full evaluation framework.
First baselines: two zero-shot, open-weight reference implementations for both tasks, exposing how far current foundation models are from solving VDA.

Dataset Statistics

20,100
Generated Images
41
Diachronic Topics
201
Periods
100
Images per Period
Topic collection: 148 of 266 proposed topics retained · 706 documented period descriptions · 174 verified publications/webpages
Visual vocabularies: 669 environment classes (Places365 + SUN) · 177 activity classes (ATUS, HETUS, MTUS) · 74 dated-trend labels (Wikipedia photo categories)

Generation pipeline: GPT-5.6 Sol writes the image prompt and caption for each grounded topic/period/environment/activity combination → Nano Banana Fast renders the image → CLIP verifies the result, regenerating on failure. Two release splits are provided for supervised methods — a uniform split (every topic in train/val/test) and a topic-disjoint split (held-out topics, for generalisation). Full field-level documentation, download instructions, and split files live on the dataset page linked above.

Expert Validation

Because the benchmark is synthetic by necessity, VDBench is validated at every stage: first by the authors, then by domain experts on the historical trajectories (Stage 1), then again by experts on the rendered images (Stage 2).

41 / 46
Stage 1 Topics Approved
4.02
Mean Period Rating (1–5)
84.2%
Stage 2 Images Judged Plausible
71.7%
Repeat-Rating Agreement

Six domain experts — in gender studies, sociology, social anthropology, media and communication studies, literary studies, digital humanities, film archiving and theory, and migration history — reviewed topics within their own expertise: judging historical validity and period boundaries in Stage 1 (270 period ratings, 501 image judgements across 29 topics in Stage 2), told the images were AI-generated.

Baselines & Results

Two zero-shot, open-weight reference baselines, neither using VDBench for training, fine-tuning, or prompt selection. Open-MLLM applies Qwen3-VL-32B-Instruct directly to both tasks. TrajEx (Trajectory Explorer) is a specialised pipeline: a frozen EVA-CLIP encoder estimates the year, EVA-CLIP zero-shot matching predicts environment/activity, and change-point detection over time — refined with DINOv3 features — recovers the period structure.

Task 1 · Per-image context detection
OutputMetricOpen-MLLMTrajEx
YearMAE (years) ↓4.724.65
EnvironmentAccuracy / macro-F1 ↑0.26 / 0.210.27 / 0.26
ActivityAccuracy / macro-F1 ↑0.28 / 0.240.31 / 0.32
CaptionBERTScore / MoverScore ↑0.84 / 0.120.86 / 0.14
Task 2 · Trajectory structure (ground-truth Task 1 inputs)
OutputMetricOpen-MLLMTrajEx
Image groupingARI / AMI ↑0.56 / 0.660.79 / 0.87
Period count|Ĵ−J| ↓0.631.17
Period rangeTemporal IoU ↑0.460.63
Matched orderKendall's τ ↑0.971.00
Confusion matrices comparing Open-MLLM and TrajEx image-to-period assignment on four topics: camera, bicycle, Buddha statue, and shopping mall. TrajEx reaches near-diagonal, near-100 percent assignment with high ARI (0.98, 0.36, 0.96, 0.37) on clean topics but confuses adjacent periods on harder ones; Open-MLLM shows more spread-out, blurrier assignment across periods (ARI 0.90, 0.88, 0.31, 0.52).
Per-topic image-to-period assignment: TrajEx is sharper on topics with clear visual eras (camera, bicycle) but both baselines blur adjacent periods on harder topics (Buddha statue, shopping mall).
Neither baseline solves VDA. Grouping quality drops sharply once each method relies on its own Task-1 predictions rather than ground truth — ARI falls by 0.34 for TrajEx and 0.15 for Open-MLLM. Swapping TrajEx's change-point detection for generic feature-space clustering collapses its ground-truth ARI from 0.79 to 0.13 — recovering a documented trajectory takes more than visual similarity alone.

Limitations & Future Work

Expert validation supports the historical relevance of the compiled trajectories and the plausibility of the generated images, and the first baselines — built on state-of-the-art foundation models — show the task is far from solved. Planned extensions broaden geographic and cultural coverage, gather multiple independent expert reviews per topic, and incorporate licensed archival images where reliable annotations exist. On the methods side, promising directions include hybrid evaluation across synthetic and archival data, uncertainty-aware trajectory modelling, and text evaluation supported by multiple references and human assessment.

Citation

Citation details will be added once the paper is de-anonymised after review.