Michael Kosinski, a computational psychologist at Stanford University, is testing several iterations of the Chat GPT chatbot developed by OpenAI on its ability to pass the famous “Theory of Mind” test. In his research, posted on the arXiv preprint server, Kosinski says that when testing the latest version of Chat GPT, he found that it passed the level of an average 9-year-old child.
Chat GPT and other AI chatbots have advanced capabilities, such as writing perfect essays for high school and college students. As their capabilities improve, some users report that chatting with some software apps is almost no different from chatting with unknown and unseen people. Such findings have led some in the field of psychology to wonder about the impact of these applications on both individuals and society. Kosinski was curious to know if such chatbots are getting closer to passing the theory of mind testing in this new endeavor.
Kosinski tested a version of ChatGPT released before 2022 and found that it was unable to pass the Theory of Mind test. When he tested a new version shortly thereafter, he found that it was able to solve 70% of theory tests – about the same as a 7-year-old child. Then, last November when he tested the latest version, he found it was able to solve 93% of the tasks – about the same as a 9-year-old.
Microsoft, which has added ChatGPT capabilities to its Bing chatbot, is apparently aware of such results and has put a filter on related questions – when asked if the theory of mind was able to pass, Bing’s AI chatbot quickly replied, “I. I’m sorry, but I’d prefer to continue this conversation. I am still learning, so I appreciate your understanding and patience.”
More information:
Michal Kosinski, Theory of Mind May Have Spontaneously Emerged in Large Language Models, arXiv (2023)
A global media for the latest news, entertainment, music fashion, and more.
