Collaborative learning - an educational paradigm in which students work together to master content and complete projects - has long been a staple of classroom pedagogy. However, the rapid development of Large Language Model chatbots (LLMs), such as ChatGPT, Gemini, and Claude, has created the potential for a new frontier in collaborative learning in which students collaborate not with other students, but with LLMs. In this classroom-based study, we tested a novel intervention in which college students taking an introductory level social science elective (n = 154) were asked multiple times throughout a semester to write argumentative essays, have them critiqued by LLMs, and improve their essays by incorporating the LLMs’ critiques into their arguments or countering the LLMs’ arguments. We found that students enjoyed working with LLMs and that collaborating with LLMs improved students’ argumentative writing skills over the course of the semester - even when essay quality was assessed before students elicited the LLMs’ critiques. Students also improved their prompt engineering skills and increased their self-efficacy for working with LLMs. These findings suggest that collaborative learning with LLMs may be a highly scalable, low-cost intervention for improving students’ argumentative writing skills at the college level. Implications and limitations of the intervention are discussed.