Objective: To investigate the association between family adversities in childhood and depression in three follow-up visits of a cohort of Brazilian adults. Methods: A total of 12,636 participants from the Longitudinal Study of Adult Health (ELSA-Brasil), who attended three interview/examination visits (2008-2010, 2012-2014, and 2017-2019), were included. Five family dysfunctions and the childhood family dysfunction score (0, 1, and 2+ dysfunctions) were used. The Clinical Interview Schedule-Revised-CIS-R assessed depression in 3 visits (no, 1, and 2 or 3) Multinomial logistic regression models estimated crude and adjusted Odds Ratios (OR) and 95% confidence intervals (95%CI). Results: Mean age was 59.6 years (SD=8.8), 7.4% presented depression in 1 visit and 2.2% in 2 or 3 visits. After adjustments, compared to the absence of family dysfunction the presence of mental disorder (OR: 3.91; 95%CI: 2.94-5.21), substance abuse (OR: 2.14; 95%CI: 1.65-2.77) and parental separation (OR: 1.55; 95%CI: 1.12-2.15) increased the odds of depression in 2 or 3 visits. Exposure to 2+ family dysfunctions increased the odds of depression in 1 and in 2 or 3 visits, with a dose-response gradient. Conclusion: Exposure to family dysfunction in childhood contributes to the occurrence and recurrence of depression in adults. Interventions to prevent dysfunctional family environments and their repercussions on children can reduce the burden of depression.