While an FDA approved drug Ivermectin was reported to dramatically reduce the cell line of SARS-CoV-2 by ~5000 folds within 48 hours, the precise mechanism of action and the COVID-19 molecular target involved in interaction with this in-vitro effective drug are unknown yet. Among 12 different COVID-19 targets studied here, the RNA dependent RNA polymerase (RdRp) with RNA and Helicase NCB site show the strongest affinity to Ivermectin amounting -10.4 kcal/mol and -9.6 kcal/mol, respectively. Molecular dynamics of corresponding protein-drug complexes reveals that the drug bound state of RdRp with RNA has better structural stability than the Helicase NCB site, with MM/PBSA free energy of -135.2 kJ/mol, almost twice that of Helicase (-76.6 kJ/mol). The selectivity of Ivermectin to RdRp is triggered by a cooperative interaction of RNA-RdRp by ternary complex formation. Identification of the target and its interaction profile with Ivermectin can lead to more powerful drug designs for COVID-19 and experimental exploration.