Applying a voltage to metal electrodes in contact with aqueous electrolytes results in the electrolysis of water at low voltages and plasma formation in the electrolyte at high voltages referred to as contact glow discharge electrolysis (CGDE). While several studies explore parameters that lead to changes in the I-U characteristics in this voltage range, little is known about the evolution of the structural properties of the electrodes. Here we study this aspect on materials essential to electrocatalysis, namely Pt, Au, and Cu. The stationary I-U characteristics are almost identical for all electrodes. Detailed structural characterization by optical microscopy, scanning electron microscopy, and electrochemical approaches reveal that Pt is stable during electrolysis and CGDE, while Au and Cu exhibit a voltage-dependent oxide formation. More importantly, oxides are reduced when the Au and Cu electrodes are kept in the electrolysis solution. We suspect that H2O2 (formed during electrolysis) is responsible for the oxide reduction. The reduced oxides (which are also accessible via electrochemical reduction) form a porous film, representing a possible new class of materials in energy storage and conversion studies.