Research
Foundation models & publications
Multimodal foundation models for documents, images and interfaces, plus recent work on evaluating frontier language models.
Areas#
Document understanding
DocFormer and DocFormerv2: transformers that fuse text, layout and pixels to understand documents end to end.
Vision language and agents
LaTr for scene text visual question answering, MixGen for multimodal data augmentation and compressed history states for faster web agents.
LLM evaluations
Benchmarks and graded transcripts that measure when language models exploit flaws in their reward signal or test cases.
Mentees & Collaborators#
I am fortunate and proud to have worked with incredible researchers. I have learnt from them much more than what I could teach back.
- Yash Patel·learnt image compression·scientist at Meta
- Poojan Oza·research into multimodal OCR training·scientist at Amazon Go
- Chih-Hui (John) Ho·vision language research, ECCV·scientist at SAP
- Xiaoshuai Hao·vision language research·researcher at Xiaomi EV
- Ali Furkan Biten·vision language research, CVPR·at Computer Vision Center, Barcelona
- Yichu Zhou·document understanding with LLMs·scientist at Pinterest
- Chenge Li·vision language research·senior scientist at Amazon
- Yuan Gao·computer use vision language pretraining, CVPR·researcher at xAI
- Park Joonhyung·precise GUI grounding, ACL·postdoc at KAIST
- Varun Nagaraj Rao·retrieval augmented vision language learning·PhD at Princeton
- Tian Li·accelerated inference in transformer models·scientist at Amazon
- Ying Nian Zhao·multimodal transformer distillation·scientist at Amazon
- Xiyue Zhu·improving computer use automation with compressed history states·PhD at UIUC
- Shrinidhi Kumbhar·accelerating inference with diffusion LLMs, CVPR·PhD at Arizona State
- Sungnyun Kim·distillation on document understanding·postdoc at KAIST
- Simone Bombari·differential privacy research·PhD at ISTA
- Ofir Abramovich·multimodal research·PhD at Reichman University
- Tianyang Zhao·multimodal distillation research·scientist at Amazon Go
Selected Publications#
Full list on Google Scholar.