Two parallel public health epidemics affecting the United States include the COVID-19 pandemic and a rise in substance use disorders (SUDs). Limited research has examined the potential relationship between these two epidemics. Our objective was therefore to perform an exploratory study in order to examine the association between public stigma toward people with a past history of opioid, methamphetamine, cocaine, and alcohol use disorder and COVID-19 vaccine hesitancy. A national sample of U.S. adults (N = 6515) completed a survey which assessed the degree of negative perceptions toward individuals with a past history of substance use disorders (referred to as substance use stigma) and COVID-19 vaccine hesitancy, along with variables such as racial prejudice, source of health news, and other demographics. We evaluated four multivariable logistic regression models to predict COVID-19 vaccine hesitancy, utilizing substance use stigma toward opioids, methamphetamine, cocaine, and alcohol use as independent variables. We found that COVID-19 vaccine hesitancy was positively associated with substance use stigma toward opioid (AOR = 1.34, p < 0.05), methamphetamine (AOR = 1.40, p < 0.01), and cocaine (AOR = 1.28, p < 0.05) use, but not alcohol use (AOR = 1.06, n.s.). Predictive models that incorporate substance use stigma may therefore improve our ability to identify individuals that may benefit from vaccine hesitancy interventions.