Collaborative learning - an educational paradigm in which students work together to master content and complete projects - has long been a staple of classroom pedagogy. However, the rapid development of Large Language Models (LLMs), such as ChatGPT, has created the potential for a new frontier in collaborative learning in which students collaborate not with other students, but with LLMs.  In this classroom-based study, we tested a novel intervention in which college students were asked multiple times throughout a semester to write essays, have them critiqued by LLMs, and improve their essays using the critiques. We found that students enjoyed working with LLMs and that collaborating with LLMs improved students’ argumentative writing skills - even when assessed before eliciting the LLMs’ critiques. Students also improved their prompt engineering skills. These findings suggest that collaborative learning with LLMs may be a highly scalable, low-cost intervention for improving students’ argumentative writing skills.