Massively Scalable Workflows for Quantum Chemistry: BIGCHEM and CHEMCLOUD

26 September 2023, Version 2
This content is a preprint and has not undergone peer review at the time of posting.


Electronic structure theory, i.e. quantum chemistry, is the fundamental building block for many problems in computational chemistry. We present a new distributed computing framework (BIGCHEM) which allows efficient solution of many quantum chemistry problems in parallel. BIGCHEM is designed to be easily composable and leverages industry-standard middleware (e.g. Celery, RabbitMQ, and Redis) for distributed approaches to large scale problems. BIGCHEM can harness any collection of worker nodes, including ones on cloud providers (such as AWS or Azure), local clusters, or supercomputer centers (and any mixture of these). BIGCHEM builds upon MolSSI packages such as QCEngine to standardize the operation of numerous computational chemistry programs, demonstrated here with Psi4, xtb, geomeTRIC, and TeraChem. BIGCHEM delivers full utilization of compute resources at scale, offers a programable canvas for designing sophisticated quantum chemistry workflows, and is fault tolerant to node failures and network disruptions. We demonstrate linear scalability of BIGCHEM running computational chemistry workloads on up to 125 GPUs. Finally, we present CHEMCLOUD, a web API to BIGCHEM and successor to TeraChem Cloud. CHEMCLOUD delivers scalable and secure access to BIGCHEM over the internet.


distributed systems
horizontal scalability
distributed computing

Supplementary materials

Supplementary Information
Supplementary information to the main paper.

Supplementary weblinks


Comments are not moderated before they are posted, but they can be removed by the site moderators if they are found to be in contravention of our Commenting Policy [opens in a new tab] - please read this policy before you post. Comments should be used for scholarly discussion of the content in question. You can find more information about how to use the commenting feature here [opens in a new tab] .
This site is protected by reCAPTCHA and the Google Privacy Policy [opens in a new tab] and Terms of Service [opens in a new tab] apply.