ChatMolData: a Multimodal Agent for Automatic Molecular Data Processing

In recent years, the development of large language models (LLMs) has revolutionized various fields of natural science, yet their application in molecular data processing remains constrained due to the reliance on single-modality inputs and outputs. To bridge the gap between experimenters and computational tools, we introduce ChatMolData, a novel LLM-based multimodal agent designed to handle diverse molecular data forms, including molecular databases, images, structure-specific files and unstructured & structured documents. ChatMolData integrates the capabilities of LLMs (e.g., GPT-4 and GPT-3.5) with robust toolset that supports data retrieval, structuring, prediction, visualization, and search tasks. Our agent employs a systematic cycle of reasoning and action to efficiently process complex tasks in molecular science. The evaluation demonstrates that ChatMolData achieves over 90% accuracy for 128 diverse tasks, significantly lowering barriers for researchers without computer science background. Our findings highlight the agent's potential in advancing molecular research through improved data accessibility and usability.