The orphan nuclear receptor TLX is expressed almost exclusively in neural stem cells. TLX acts as an essential factor for neural stem cell survival and is hence considered as a promising drug target in neurodegeneration. However, few studies have characterized the roles of TLX due to a lack of ligands and limited functional understanding. Here, we identify caffeine and istradefylline as TLX ligands that counteract the receptor’s intrinsic repressor activity in reporter gene assays and modulate TLX regulated SIRT1 and p21 expression. Mutagenesis of residues lining a cavity within the TLX ligand binding domain altered activity of these ligands suggesting direct interactions with helix 5. Using istradefylline as a tool compound, we observed ligand-sensitive recruitment of the co-repressor SMRT and heterodimerization of TLX with RXR. Both protein-protein complexes evolve as factors that modulate TLX function and suggest an unprecedented role of TLX in directly repressing other nuclear receptors.