Summary
Harvard researchers tested ChatGPT against graduate students in a molecular biology course, finding that students performed better by over two letter grades. The study highlighted ChatGPT's difficulties with remembering and applying knowledge, especially in analyzing scientific data, despite attempts to improve its performance with prompts.
Key Facts
- Harvard graduate students outperformed ChatGPT in a molecular biology course study.
- Students scored higher than ChatGPT by more than two letter grades.
- ChatGPT struggled with tasks involving remembering and applying information.
- Students scored 98% compared to ChatGPT's 82% on "remember" questions.
- ChatGPT did poorly on understanding, applying, and analyzing questions, scoring 66%.
- The AI's performance was tested using GPT-4o, released in May 2024.
- The test used pre-2022 assignments to prevent students from using AI assistance.
- Researchers noted ChatGPT's difficulty in interpreting scientific graphs and data.
