The generalisation challenge: assessment of the efficacy of acoustic signals for state estimation of lithium-ion batteries via machine learning

Acoustic measurements of batteries are known to be correlated to their state-of-charge, creating opportunities for state estimation that do not rely on electrical signals. State estimators are typically parametric models fitted from data, often from the broad toolbox of machine learning. Such models can be easily designed to have millions of tuneable parameters, which endows them with tremendous but often misinterpreted fitting ability. The real performance metric, commonly omitted in the battery literature, is a model’s generalisation performance with respect to a population, which requires successful predictions to be made on data from one or more ‘held out’ cells. This study demonstrates that regression models based on neural networks can perform highly accurate state estimation on multiple cells; however, this is shown to be conditional on all cells being represented in the training dataset. Generalisation to the wider population is shown to be more challenging than other studies claim; a conclusion which follows from tests on multiple feature configurations and multiple model variants. It is hypothesized that success on multi-cell data in the absence of wider generalisation is due to the ability of models to learn cell-specific patterns implicitly, which is a type of 'overfitting'. This hypothesis is tested in two ways. First, classifiers performing a matching operation between acoustic waveforms and their respective cells are used to show that cell-specific characteristics are present in the waveforms. Next, unsupervised learning methods are used to perform a projection of all acoustic signals to two-dimensional latent space. In the latent space it is found that datapoints cluster according to the cell identity, indicating that the distinctiveness of cells dominates over any state-related commonalities in the acoustic dataset. The study highlights the need for caution in how the generalisation of machine learning models (of any kind) is evaluated in battery research.