Van der Waals heterojunction involving graphene (Gr) with transition metal dichalcogenides (TMDs) is regard as a promising structure for their outstanding performance in optical and optoelectronic response. The electron-hole thermalization has been deemed to be the main reason for the sub-bandgap-excitation charge transfer from Gr to TMDs. However, the role of the intricate interlayer interaction of the Gr and the TMDs still require intensive investigation. Here, we have investigated the photocarrier dynamics in 5-layer PtSe2/Gr heterojunction by using time-resolved optical pump and terahertz probe spectroscopy. Interestingly, after photoexcitation, electron transfer from PtSe2 to Gr in PtSe2/Gr/substrate heterojunction has been demonstrated successfully, by contrast, no observable charge transfer occurs in the Gr/PtSe2/substrate heterostructure. The prominent difference for the different stacking sequence between Gr and PtSe2 can be ascribed to the effective built-in field introduced by fused silica substrate. A physical picture accounting for built-in electric field introduced by substrate has been proposed to interpret the charge transfer process in the TMD/Gr heterostructure–the substrate built-in electric field plays a dominated role for controlling the charge transfer pathway in the TMDs/Gr heterojunction. This study not only shed the light to the substrate engineering but also provide a new insight into the dynamic in Gr/TMDs heterojunction, which provides a new method to optimize the performance of photodetection.