Crossing Realities Has a Cost: Evaluating AR Second Screens for AI-Assisted Desktop Authoring
Abstract
AI-assisted authoring requires users to frequently exchange user-generated and AI-generated content between a primary workspace and a Large Language Model assistant. These repetitive actions can fragment attention and compete for limited display space. To examine whether augmented-reality second screens help or introduce new transition costs, we designed an AR-Second-Screen system with unified keyboard-and-mouse interaction and compared it with Single-Screen and Dual-Screen desktop setups. Dual-Screen remained more efficient, with lower task-completion time and switching cost. AR-Second-Screen produced higher task and LLM idle ratios, indicating re-engagement costs when crossing between physical and virtual workspaces. Participants nevertheless rated AR positively, preferred it over Single-Screen, and showed no significant preference difference between AR-Second-Screen and Dual-Screen.
Methodology
Following a formative study, we conducted a controlled within-subjects study with 18 participants. Participants completed AI-assisted LaTeX authoring tasks using Single-Screen, Dual-Screen, and AR-Second-Screen configurations. The refined AR system used unified keyboard-and-mouse input and simplified bidirectional content transfer. Measures included task time, switching cost, task and LLM idle ratios, workload, usability, user experience, and preference.
Text selection and interaction mechanism across physical desktop and AR display space.
Study 2 feedback interaction and content copy state in AR second screen interface.
Append and transfer workflow for integrating AI-generated text into desktop documents.
AR display region and visual feedback area during cross-reality authoring tasks.
Results
The physical Dual-Screen setup produced significantly lower task-completion times and switching costs than AR-Second-Screen. AR also led to higher task and LLM idle ratios, revealing re-engagement overhead at physical-virtual boundaries. Despite this performance cost, participants rated the AR experience positively and preferred it significantly over Single-Screen, with preference comparable to Dual-Screen.
Context switching cost between desktop and AR displays across task conditions.
LLM idle time ratio comparing single-screen vs. AR second-screen configurations.
Overall task idle ratio across experimental authoring conditions.
Task completion time across desktop authoring configurations.
NASA-TLX workload subscales and User Experience Questionnaire (UEQ) ratings.
Application & Implications
AR can extend an AI-assisted workspace when physical display space is unavailable, but adding virtual display area alone is insufficient. Cross-reality authoring tools should reduce the interaction and attention costs of moving context and generated content between the desktop workspace and the AR assistant.
Design Guidelines:
- Use unified input across physical and virtual workspaces to avoid repeated input-focus changes.
- Minimize manual content-transfer steps between the primary editor and the AI assistant.
- Preserve task context across transitions so users can resume productive work quickly.
- Use AR for persistent peripheral information when frequent cross-reality editing would create excessive switching costs.
BibTeX
@inproceedings{gemici2026crossing,
author = {Gemici, Mucahit and Bashar, Mohammad Raihanul and Stuerzlinger, Wolfgang and Phadnis, Vrushank and Batmaz, Anil Ufuk},
title = {Crossing Realities Has a Cost: Evaluating AR Second Screens for AI-Assisted Desktop Authoring},
booktitle = {Proceedings of the 2026 ACM Symposium on Spatial User Interaction},
series = {SUI '26},
address = {Bari, Italy},
numpages = {12},
year = {2026},
doi = {10.1145/3822518.3830046}
}