Collaborative learning - an educational paradigm in which students work together to master content and complete projects - has long been a staple of classroom pedagogy. The rapid development of Large Language Model chatbots (LLMs), such as ChatGPT, Gemini, and Claude, has created the potential for a new frontier in collaborative learning in which students collaborate not with other students, but with conversational LLMs. In this exploratory, classroom-based study, we tested a novel intervention in which college students taking an introductory level social science elective (n = 154) were asked multiple times throughout a semester to write argumentative essays, have them critiqued by LLMs, and improve their essays by incorporating the LLMs’ critiques into their arguments or rebutting the LLMs’ arguments. We found that students engaged deeply with the LLMs, enjoyed working with LLMs, and that their argumentative writing skills improved over the course of the semester. This improvement was noted even in the essays that students wrote without the support of LLMs. Throughout the semester, students also improved in their prompt engineering skills and showed increased self-efficacy for working with LLMs, however future research is needed to isolate the causal effects of the intervention. Implications and limitations of the intervention are discussed.