The arrows depicting electron movement and the bond-making and breaking events are the maps that guide student instruction in organic chemistry curricula. For students, the pathways represented by electron pushing formalism (EPF) can be tough to navigate. For instructors, providing formative feedback to students to support their learning of the EPF arrow system is difficult to provide in a timely manner. The Mechanisms app (“Mechanisms”) was developed as a method for students to explore the electron movement of organic chemistry through a touch screen interface of a smart phone or tablet and do so within a game-like experience. In this paper the pedagogical content of the Mechanisms app (“Mechanisms”) is described along with studies of students’ use of the app to understand whether the open-ended experience to construct understanding of EPF is valuable as a formative assessment method. Presented in this paper are the results of Mechanisms use by analysis of a multi-institution anonymous student survey, with a usability study of organic chemistry students, and with three case studies detailing the use of the app in college classrooms.