Our article is a systematic study toward understanding the mechanism of silver cluster formation. We calculated optical spectra of ultra-small silver clusters using time-dependent density functional theory (TDDFT) and compared our results with time-resolved UV-Vis spectra obtained from pulse radiolysis experiments during early stages of cluster formation. This comparative study indicates that the formation mechanism of silver clusters occurs through both monomer and ion addition growth pathways. Also, we calculated free energy of formation of small cationic and neutral clusters using density functional theory (DFT) which shows the thermodynamic stability of cationic clusters. In a conventional experimental system with the common reducing agents, the formation of cationic clusters is kinetically favored owing to the dominance of charged ions relative to neutral atoms in the system. While we show the stability of small cationic clusters relative to neutral clusters, collectively, we deduce that the monomer addition along with ion addition growth pathway is involved in silver cluster formation mechanism. We further show the effect of solvent and water ligands on structure, stability, and optical properties of small clusters.