Background: Low back pain (LBP) is a leading cause of years lived with disability. However, the association of non-chronic LBP with levels of daily physical activity (PA) remains poorly explored. This study investigated the association between previous and current non-chronic LBP with daily PA and compliance with PA recommendations in middle and older-aged adults. Methods: This is a cross-sectional analysis including volunteers from the Baltimore Longitudinal Study of Aging who answered questions about LBP and wore an Actiheart accelerometer for 7days. Generalized linear models and logistic models were used and adjusted for potential confounders. Results: 662 volunteers were included (50.8% women, 68.1% white), aged 68.0 (± 11.4) years. Previous non-chronic LBP was reported by 240 (36.3%) participants with mean pain intensity of 3.5 ± 2.0 (ranging from 0 to 10). Current non-chronic LBP was reported by 5.7% (n = 38) with mean pain intensity of 4.1 ± 2.3. Participants reporting current non-chronic LBP had lower levels of total (β -0.18, 95% CI -0.34 to -0.02) and vigorous (β -0.29, 95% CI -0.56 to -0.007) daily PA and lower odds of meeting PA recommendations (OR 0.20, 95% CI 0.05 to 0.92) compared to those reporting no LBP. No association was observed for LBP intensity. Conversely, previous non-chronic LBP showed no statistically significant association with daily PA levels. Conclusion: The presence of current non-chronic LBP seems to be negatively associated with PA levels driven mainly by lower engagement in vigorous PA. No association was observed for pain intensity.