Endocannabinoids act as a major neuromodulatory system in a variety of physiological and behavioral functions. Three major lines of evidence suggest that the endocannabinoid system interacts with gonadal hormones. First, the endocannabinoid system is implicated in behaviors and physiological functions that are known to be regulated in part by gonadal hormones. Second, receptors and metabolic enzymes of the endocannabinoid system are localized extensively on structures in the hypothalamic-pituitary-gonadal axis. Third, changes in levels of gonadal hormones alter endocannabinoid signaling. Here we reviewed and summarized the current evidence regarding the interaction between the endocannabinoid system and androgens, estrogens, and progesterone. Overall, it appears that bidirectional interactions characterize the relationship between endocannabinoids and gonadal hormones, with endocannabinoids down-regulating hypothalamic-pituitary-gonadal activity and gonadal hormones modulating protein expression in the endocannabinoid system. An understanding of these interactions will have implications for elucidating the neuroendocrine mechanisms underlying a number of behavioral and physiological functions as well as potential pharmaceutical treatments for disorders of these functions.