Accurate memory for discrete events is thought to rely on pattern separation to orthogonalize the representations of similar events. Previously, we reported that a behavioral index of pattern separation was correlated with activity in the hippocampus (dentate gyrus, CA3) and with integrity of the perforant path, which provides input to the hippocampus. If the hippocampus operates as part of a broader neural network, however, pattern separation would likely also relate to integrity of limbic tracts (fornix, cingulum bundle, and uncinate fasciculus) that connect the hippocampus to distributed brain regions. In this study, healthy adults (20–89 years) underwent diffusion tensor imaging and completed the Behavioral Pattern Separation Task-Object Version (BPS-O) and Rey Auditory Verbal Learning Test (RAVLT). After controlling for global effects of brain aging, exploratory skeleton-wise and targeted tractography analyses revealed that fornix integrity (fractional anisotropy, mean diffusivity, and radial diffusivity; but not mode) was significantly related to pattern separation (measured using BPS-O and RAVLT tasks), but not to recognition memory. These data suggest that hippocampal disconnection, via individual- and age-related differences in limbic tract integrity, contributes to pattern separation performance. Extending our earlier work, these results also support the notion that pattern separation relies on broad neural networks interconnecting the hippocampus.