Introduction: Oral pre-exposure prophylaxis (PrEP) with emtricitabine/tenofovir (FTC/TDF) is highly effective in preventing HIV infection. This study aimed to identify factors associated with PrEP early loss to follow-up (ELFU) among gay, bisexual and other men who have sex with men (MSM), travestis and transgender women (TGW). Methodology: This was a prospective cohort study evaluating TGW and MSM who initiated PrEP at the Evandro Chagas National Institute of Infectious Diseases (INI-Fiocruz) from 2014 to 2020. ELFU was defined as not returning for a PrEP visit within 180 days after first dispensation. Exposure variables included age, gender, race, education, transactional sex, condomless anal intercourse [CAI] (both in the past six months), binge drinking and substance use (both in past three months) and syphilis diagnosis at baseline. Multilevel logistic regression models with random intercepts and fixed slopes were used to identify factors associated with ELFU accounting for clustering of participants according to their PrEP initiation study/context (PrEP Brasil, PrEParadas, ImPrEP and PrEP SUS). Results: Among 1,463 participants, the median age was 29 years (interquartile range 24-36), 83% self-identified as MSM, 17% as TGW, 24% were black, 37% mixed-black/pardo and 30% had < 12 years of education. Fifteen percent reported transactional sex, 59% reported CAI, 67% binge drinking, 33% substance use, and 15% had a syphilis diagnosis. Overall, 137 participants (9.7%) had ELFU. Younger age (18-24 years) (adjusted odds ratio [aOR] 1.9, 95%CI:1.2-3.2), TGW (aOR 2.8, 95%CI:1.6-4.8) and education < 12 years (aOR 1.9, 95%CI:1.2-2.9) were associated with greater odds of ELFU. Conclusion: TGW, young individuals and those with lower education were at higher risk of PrEP ELFU. Our results suggest that the development of specific strategies targeting these populations should be a priority, through policies that aim to reduce the incidence of HIV infection.