I am currently undergraduate student of Shanghai Jiao Tong University. I am interested in embodied AI and Foundation Models. My research interest lies in building generally capable embodied agents that continuously explore, plan, and develop new skills in open-ended worlds.

Publication

Building Cooperative Embodied Agents Modularly with Large Language Model

• Authors: Hongxin Zhang*, Weihua Du*, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B. Tenenbaum, Tianmin Shu, Chuang Gan.

• ICLR, 2024 Accept (poster)

• We propose CoELA, a decentralized multi-agent system that uses GPT-4’s language abilities for effective cooperation in complex tasks. CoELA combines reasoning and communication skills with perception and memory, outperforming traditional methods. It shows enhanced cooperation with humans and potential for future multi-agent research.

Constrained Human-AI: An Inclusive Embodied AI Assistance Challenge

• Authors: Andi Peng*, Weihua Du*, Jiaming Shan*, Tianmin Shu, Chuang Gan

• CVPR, 2024 (Conference Submission)

• We propose CHAIC, a 3D realistic challenge testing human-AI coordination where an AI helps a physically constrained human in household tasks. It focuses on AI‘s ability to understand tasks through observation, identify human constraints, and collaborate effectively. Using the TDW-Constrained simulator, it offers seven tasks for evaluating AI’s social interaction and intelligence.