Delays in vaccinating communities of color to COVID-19 have signaled a need to investigate structural barriers to vaccine uptake, with mass incarceration demanding greater characterization as a potential factor. In a nationally representative survey from February-March 2021 (N = 1,157), exposure to the criminal legal system, defined as having been incarcerated in prison or jail or having had a family member or close friend incarcerated, was associated with higher odds for COVID-19 vaccine deliberation. Individuals with criminal legal system exposure reported lower confidence in physician recommendation as a reason to get vaccinated. They were also more likely to decline vaccination out of fear it would cause COVID-19 infection, and that the vaccine might be promoted as a political tool. Our analysis suggests that populations impacted by the criminal legal system would benefit from targeted vaccine outreach by trusted community members who can address distrust during current and future pandemics.