Can AI be funny? A new study by psychology researchers from the University of Southern California tested ChatGPT’s comedic ability with surprising results.
ChatGPT can write code, coherent content, and even poetry. But can it write content that captures the hard-to-quantify concept of humor?
It’s easy to evaluate if text is grammatically correct or if code works, but humor is far more subjective. The researchers ran two experiments to see if people found ChatGPT’s content funny.
For the first experiment, they hired 123 US-based laypeople (i.e. not professional comedians) on Amazon Mechanical Turk. The participants were asked to complete 3 humor production tasks:
- Generate one new, humorous phrase for each of the following three acronyms: “S.T.D.”, “C.L.A.P.”, and “C.O.W.”
- Fill in the blanks. Create one humorous answer for each of the three items: “A lesser talked about room in the White House: ___”, “A remarkable achievement you probably wouldn’t list on your resume: ____”, “Worst first date activity: ____.”
- Roast joke: Create a humorous, conversational response to fictional scenarios. E.g. a friend asks for feedback on their singing. “To be honest, listening to that was like ____”)
The researchers gave ChatGPT 3.5 the same tasks and instructions as the participants and generated 180 comedic responses.
They then hired 200 US-based CloudResearch approved MTurk workers to rate the humor of a random subset consisting of an equal amount of human and AI-generated responses.
The participants had to rate 54 responses based on a 7-point Likert scale (0 = Not Funny at All; 6 = Very Funny).
Funny results
After evaluating the responses, the researchers said, “When we compare the funniness of the jokes generated by each of our human participants in the humor production task with the funniness of the jokes produced by ChatGPT 3.5, ChatGPT outperformed the majority of our human humor producers on each task.”
ChatGPT was competing with laypeople in this test. How would it do when it came up against professionals?
The researchers compared ChatGPT 3.5’s ability to produce humorous, satirical news headlines to that of professional comedy writers at The Onion.
They recruited 217 students from the University of Southern California and asked each of them to rate the funniness of 10 satirical headlines (5 AI, 5 human).
The results showed that ChatGPT’s satirical headlines were rated on average as funny as those written by The Onion’s professional comedic writers.
Of the four top-rated headlines, two were generated by professional writers and two by ChatGPT.
ChatGPT wrote the highest-rated entry (“Local Man Discovers New Emotion, Still Can’t Describe It Properly”) and the fourth-highest-rated entry (“Man Achieves Personal Best in Avoiding Eye Contact With Neighbors During Awkward Elevator Ride”).
Those are pretty funny and raise some interesting questions and conclusions.
If you enjoy telling jokes but you’re not a pro, having ChatGPT write your material will get you more laughs than writing your own jokes.
If you’re a professional comedian, ChatGPT’s output is likely on par with what you currently write. Will an AI-generated comedian like George Carlin be coming for your job?
Probably not yet. Writing a joke is one thing. Getting the timing, delivery, and intonation right requires skills that few people have, never mind an AI.
AI doesn’t feel emotion so how does it know what’s funny? The researchers came to a disconcerting conclusion:
“Our studies suggest that the subjective experience of humor may not be required for the production of good humor–merely knowing the patterns that make up comedy may suffice.”
So ChatGPT can write funny jokes but still needs a human to deliver them. At least, for now.
