LAUSR

Abstract The study of ligand interactions with nucleic acid structures such as G-quadruplexes versus double-stranded DNA is important because these interactions are fundamental for many intracellular processes. In the investigation of ligand-DNA binding process, achieving high fluorescent signal discrimination with strong binding affinity is challenging. To develop binding ligands with excellent recognition ability towards G4-DNA over duplex-DNA, the design of appropriate molecular scaffolds that are able to match with the G4-DNA binding pocket (G-quartet) is crucial. In the present study, the new fluorescent DNA binding ligands were designed and synthesized through the integration of a small and conjugated substituent group at the 2-position of the 1-methylquinolinium moiety of the nonspecific thiazole orange scaffold. The ligands were investigated in fluorescence binding assays and showed different interaction properties and significant fluorescent recognition selectivity towards G4-DNA over duplex-DNA in vitro. Molecular docking study of the ligands in complex with telo21 G4-DNA and ds26 duplex DNA also revealed different binding modes. In addition, the cytotoxicity of the fluorescent DNA binding ligands was evaluated in MTT assays against two selected cancer cell lines (human prostate cancer cell (PC3) and human hepatoma cell (hepG2)). The IC50 was found in the range of 6.3–12.5 μM, indicating a relatively high cytotoxicity of the ligands towards the cancer cells examined.