Research notes
79 notes · interactive index with search and filters
GUI memory · research atlas45 P0 papers · 10 branches · detailed notes in 中文↗- AndroidWorld: A Dynamic Benchmarking Environment for Autonomous AgentsRawles, Clinckemaillie, Chang et al. · ICLR 2025 · GUI agents
- GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile DevicesQuanfeng Lu, Wenqi Shao, Zitao Liu, et al. · ICCV 2025 · GUI agents
- OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer EnvironmentsXie, Zhang, Chen et al. · NeurIPS 2024 Datasets & Benchmarks · GUI agents
- Chain-of-Memory: Enhancing GUI Agents for Cross-Application NavigationSee original paper · Research paper · GUI agents
- Less is More: Empowering GUI Agent with Context-Aware Simplification (SimpAgent)See original paper · Research paper · GUI agents
- PAL-UI: Planning with Active Look-back for Vision-Based GUI AgentsSee original paper · Research paper · GUI agents
- MGA: Memory-Driven GUI Agent for Observation-Centric InteractionSee original paper · Research paper · GUI agents
- GUI-Rise: Structured Reasoning and History Summarization for GUI NavigationSee original paper · Research paper · GUI agents
- HiconAgent: History Context-aware Policy Optimization for GUI AgentsSee original paper · Research paper · GUI agents
- AgentProg: Empowering Long-Horizon GUI Agents with Program-Guided Context ManagementSee original paper · Research paper · GUI agents
- MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge EvolutionSee original paper · Research paper · GUI agents
- MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic EnvironmentsSee original paper · Research paper · GUI agents
- Spatio-Temporal Token Pruning for Efficient High-Resolution GUI AgentsSee original paper · Research paper · GUI agents
- M²: Dual-Memory Augmentation for Long-Horizon Web Agents via Trajectory Summarization and Insight RetrievalSee original paper · Research paper · GUI agents
- SecAgent: Efficient Mobile GUI Agent with Semantic ContextSee original paper · Research paper · GUI agents
- AndroTMem: From Interaction Trajectories to Anchored Memory in Long-Horizon GUI AgentsSee original paper · Research paper · GUI agents
- Where Did It Go Wrong? Process-Level Evaluation of Web Agents with Semantic State Tracking (WebStep)See original paper · Research paper · GUI agents
- UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy OptimizationSee original paper · Research paper · GUI agents
- Benchmarking and Improving GUI Agents in High-Dynamic Environments (DynamicGUIBench / DynamicUI)See original paper · Research paper · GUI agents
- Mem-W: Latent Memory-Native GUI AgentsSee original paper · Research paper · GUI agents
- ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy ReductionSee original paper · Research paper · GUI agents
- MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI AgentsSee original paper · Research paper · GUI agents
- STAMP: Training Explicit Memory for Mobile GUI Agents in Controllable and Scalable Virtual EnvironmentsSee original paper · Research paper · GUI agents
- A History-Aware Visually Grounded Critic for Computer Use Agents (HiViG)See original paper · Research paper · GUI agents
- Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web AgentsSee original paper · Research paper · GUI agents
- Naive Visual Memory is Not Enough: A Failure-Mode Study of GUI AgentsSee original paper · Research paper · GUI agents
- MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context ManagementSee original paper · Research paper · GUI agents
- Agent-Computer Observation Interfaces Enable Dynamic Computer Use (AOI)See original paper · Research paper · GUI agents
- OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World TasksSee original paper · Research paper · GUI agents
- What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving StatesSee original paper · Research paper · GUI agents
- HiSA: Hierarchical State Abstraction for Scalable GUI AgentsSee original paper · Research paper · GUI agents
- FocusMem: Factorizing Content, Readout, and Trust in Latent GUI MemorySee original paper · Research paper · GUI agents
- A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts (ReadAgent)See original paper · Research paper · GUI agents
- MemoRAG: Moving towards Next-Gen RAG Via Memory-Inspired Knowledge DiscoverySee original paper · Research paper · GUI agents
- LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced ColleaguesSee original paper · Research paper · GUI agents
- GUI-PRA: Process Reward Agent for GUI TasksSee original paper · Research paper · GUI agents
- M2A: Multimodal Memory Agent with Dual-Layer Hybrid Memory for Long-Term Personalized InteractionsSee original paper · Research paper · GUI agents
- Beyond Retrieval: Analytic Memory for Multimodal Agents (AdaMM)See original paper · Research paper · GUI agents
- A Task-State Representation for Long-Horizon Mobile GUI AgentsSee original paper · Research paper · GUI agents
- UI-Hawk: Unleashing the Screen Stream Understanding for Mobile GUI AgentsSee original paper · Research paper · GUI agents
- When History Is Multimodal: Rethinking Context Management for Long-Horizon Agents (VERA)See original paper · Research paper · GUI agents
- Personal Visual Context Learning in Large Multimodal ModelsSee original paper · Research paper · GUI agents
- MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent MemorySee original paper · Research paper · GUI agents
- DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual HistoriesSee original paper · Research paper · GUI agents
- DMV-Bench: Diagnosing Long-Horizon Multimodal Agents’ Visual Memory with Incidental Cue InjectionSee original paper · Research paper · GUI agents
- Beyond OAuth: Task-Scoped Authorization for AI Agents via Natural Language SlicesReshabh K Sharma, Linxi Jiang, Shuo Chen, Zhiqiang Lin · arXiv preprint · 2026 · Agent security
- Progent: Securing AI Agents with Privilege ControlTianneng Shi, Jingxuan He, Zhun Wang, Hongwei Li, Linyu Wu, Wenbo Guo, Dawn Song · arXiv preprint · 2025, revised 2026 · Agent security
- Delegated Authorization for Agents Constrained to Semantic Task-to-Scope MatchingMajed El Helou, Chiara Troiani, Benjamin Ryder, Jean Diaconu, Hervé Muyal, Marcelo Yannuzzi · arXiv preprint · 2025 · Agent security
- MiniScope: A Least Privilege Framework for Authorizing Tool Calling AgentsJinhao Zhu, Kevin Tseng, Gil Vernik, Xiao Huang, Shishir G. Patil, Vivian Fang, Raluca Ada Popa · arXiv preprint · 2025 · Agent security
- OAuth 2.0 Rich Authorization RequestsTorsten Lodderstedt, Justin Richer, Brian Campbell · IETF RFC 9396 · Proposed Standard · May 2023 · Agent security
- Critique of Agent ModelEric Xing, Mingkai Deng, Jinyu Hou · arXiv preprint · 2026 · Models & methods
- CogAgent: A Visual Language Model for GUI AgentsWenyi Hong, Weihan Wang, Qingsong Lv et al. · CVPR 2024 · GUI agents
- You Only Look at Screens: Multimodal Chain-of-Action AgentsZhuosheng Zhang and Aston Zhang · Findings of ACL 2024 · GUI agents
- SeeClick: Harnessing GUI Grounding for Advanced Visual GUI AgentsKanzhi Cheng, Qiushi Sun, Yougang Chu et al. · ACL 2024 · GUI agents
- OS-ATLAS: A Foundation Action Model for Generalist GUI AgentsZhiyong Wu, Zhenyu Wu, Fangzhi Xu et al. · ICLR 2025 · GUI agents
- Aguvis: Unified Pure Vision Agents for Autonomous GUI InteractionYiheng Xu, Zekun Wang, Junli Wang et al. · ICML 2025 · GUI agents
- UI-TARS: Pioneering Automated GUI Interaction with Native AgentsYujia Qin, Yining Ye, Junjie Fang et al. · arXiv preprint, 2025 · GUI agents
- AutoDroid: LLM-powered Task Automation in AndroidHao Wen, Yuanchun Li, Guohong Liu et al. · ACM MobiCom 2024 · GUI agents
- Do LLMs Need to See Everything? A Benchmark and Study of Failures in LLM-driven Smartphone Automation using Screentext vs. ScreenshotsShiquan Zhang, Tianyi Zhang, Le Fang et al. · arXiv preprint, 2026 · GUI agents
- OmniParser for Pure Vision Based GUI AgentYadong Lu, Jianwei Yang, Yelong Shen et al. · arXiv preprint, 2024 · GUI agents
- CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI AutomationXinbei Ma, Zhuosheng Zhang, Hai Zhao · Findings of ACL 2024 · GUI agents
- GUI-Actor: Coordinate-Free Visual Grounding for GUI AgentsQianhui Wu, Kanzhi Cheng, Rui Yang et al. · NeurIPS 2025 · GUI agents
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis et al. · ICLR 2022 · Models & methods
- FastVLM: Efficient Vision Encoding for Vision Language ModelsPavan Kumar Anasosalu Vasu, Fartash Faghri, Chun-Liang Li, et al. · CVPR 2025 · Models & methods
- TinyClick: Single-Turn Agent for Empowering GUI AutomationPawel Pawlowski, Krystian Zawistowski, Wojciech Lapacz, et al. · Interspeech 2025 · GUI agents
- ShowUI: One Vision-Language-Action Model for GUI Visual AgentKevin Qinghong Lin, Linjie Li, Difei Gao, et al. · CVPR 2025 · GUI agents
- Qwen2.5 Technical ReportQwen Team (An Yang et al.) · arXiv technical report, 2024; revised 2025 · Models & methods
- Android in the Zoo: Chain-of-Action-Thought for GUI AgentsJiwen Zhang, Jihao Wu, Yihua Teng, et al. · Findings of EMNLP 2024 · GUI agents
- Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI AgentsHanzhang Zhou, Panrong Tong, Xu Zhang, et al. · arXiv technical report, 2026 · GUI agents
- Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI AgentsGou, Wang, Zheng et al. · ICLR 2025 · GUI agents
- Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4VYang, Zhang, Li et al. · arXiv 2023 · GUI agents
- GPT-4V(ision) is a Generalist Web Agent, if GroundedZheng, Gou, Kil et al. · ICML 2024 · GUI agents
- Mind2Web: Towards a Generalist Agent for the WebDeng, Gu, Zheng et al. · NeurIPS 2023 Datasets & Benchmarks · GUI agents
- WebArena: A Realistic Web Environment for Building Autonomous AgentsZhou, Xu, Zhu et al. · ICLR 2024 · GUI agents
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM AgentsDebenedetti, Zhang, Balunović et al. · NeurIPS 2024 Datasets & Benchmarks · Agent security
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt InjectionGreshake, Abdelnabi, Mishra et al. · AISec 2023 (ACM CCS workshop) · Agent security
- Identifying the Risks of LM Agents with an LM-Emulated SandboxRuan, Dong, Wang et al. · ICLR 2024 · Agent security
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM AgentsAndriushchenko, Souly, Dziemian et al. · ICLR 2025 · Agent security
- Agent-SafetyBench: Evaluating the Safety of LLM AgentsZhang, Cui, Lu et al. · AAAI 2026 Workshop on Trust and Control in Agentic AI (TrustAgent) · Agent security