2023-07-07 03:32:36
The University of Montana and its partners have conducted a groundbreaking study to evaluate the creativity of artificial intelligence ChatGPT, which uses the GPT-4 engine. They used the Torrance Tests of Creative Thinking (TTCT), a standard test that measures the ability to generate new, numerous and varied ideas. They compared ChatGPT responses to those of a control group of UM students and those of a national sample of students who took the test in 2016.
The answers were graded by Scholastic Testing Service, which was unaware that the AI was involved. The results revealed that ChatGPT ranked among the top 1% for fluency and originality, and among the top 3% for flexibility. The study suggests that AI can match or exceed human creativity and that creativity assessment tools need to be adapted to this new reality.
The study was led by Dr. Erik Guzik, assistant clinical professor in the College of Business at the University of Montana. He and his partners used the Torrance Tests of Creative Thinking, a well-known tool that has been used for decades to assess human creativity.
The researchers submitted eight responses generated by ChatGPT, the application powered by the GPT-4 artificial intelligence engine. They also submitted responses from a control group of 24 UM students taking Guzik’s courses in entrepreneurship and personal finance. These results were compared to those of 2,700 students nationwide who took the TTCT in 2016. All submissions were scored by Scholastic Testing Service, which was unaware that AI was involved.
ChatGPT, the AI application that surpasses students in creativity
The results put ChatGPT in the creative elite. The AI application was in the first percentile for fluency – the ability to generate a large number of ideas – and for originality – the ability to come up with new ideas. The application of AI fell slightly – up to the 97th percentile – for flexibility, i.e. the ability to generate different types and categories of ideas.
For ChatGPT and GPT-4, we showed for the first time that AI was in the top 1% for originality,” Guzik said. It was a novelty. He was pleased to see that some of his UM students also scored in the top 1%. However, ChatGTP performed better than the vast majority of students nationally.
For flexibility, the ability to generate a variety of ideas and switch between approaches, three tests scored in the top 1%, two in the top 2%, and the other three in the highest 7%.
Perhaps most importantly, all eight tests saw GPT-4 rank in the top 1% for originality, the ability to produce unusual or unique responses that defy established standards. “This is the first study showing that any AI tool is capable of generating original responses that match or exceed the current threshold of human creative abilities,” said Dr Guzik.
The research team predicts that every component of the entrepreneurial process, and of business in general, will be affected by AI over the next five years – including the identification of unique opportunities and important problem areas, the generation of ideas for new solutions, improving products, developing new marketing strategies, and imagining future scenarios and outcomes.
The impact on the launch of new start-ups and small businesses, which are important generators of economic activity and growth, will be significant. The exceptional creativity demonstrated by GPT-4 marks a new milestone in AI-powered brainstorming and solution building that might revolutionize the way startups and small businesses operate, innovate, and stay competitive. The creative capabilities of AI might significantly help startups and small businesses deal with typical resource limitations, which restrict their ability to employ a large team or invest in extensive market research. The inventiveness, adaptability and originality demonstrated by GPT-4 might serve as an indispensable innovation tool for humans,” said Dr Guzik.
Guzik tested the AI and its students during the spring semester. He was assisted in his work by Christian Gilde from UM Western and Christian Byrge from Vilnius University. The researchers presented their work in May at the Southern Oregon University Creativity Conference.
During the conference, we were careful not to over-interpret the data, Guzik said. We contented ourselves with presenting the results. But we have shared strong evidence that AI appears to develop a creative capacity equal to, or even greater than, that of humans. Guzik explains that he asked ChatGPT what it would mean if she scored well on the TTCT. The AI gave a solid answer, which it shared at the conference:
“ChatGPT told us that maybe we don’t fully understand human creativity, which is my correct opinion,” he said. He also suggested that we might need more sophisticated assessment tools that can tell the difference between human-generated ideas and those generated by artificial intelligence.
The introduction of ChatGPT creates new opportunities in almost every field and industry, but especially in those that depend on creativity, or the ability to generate new and original ideas that have value,” said Dr Gilde. . This development is also forcing society to reconsider some of the most fundamental questions regarding creativity, including the nature of human creativity and whether creativity is an ability that uniquely separates humans from machines. Overall, this might prompt us to reassess how we define, assess and develop creativity in education, organizations and society,” he added.
A professor from UM Western passionate regarding the subject
Guzik has long been interested in creativity. As a seventh grader in the small town of Palmer, Massachusetts, he attended a program for gifted and talented students. This experience introduced him to the future problem solving process pioneered by Ellis Paul Torrance, the pioneering psychologist who also created the TTCT. Guzik says he fell in love with brainstorming at that time and the way it harnesses the human imagination, and he remains active with the Future Problem Solving organization – he even met his wife during one of his conferences.
Guzik and his team decided to test the creativity of ChatGPT following playing with it over the past year. We’ve all explored ChatGPT and noticed that it does some interesting things that we didn’t expect,” he explains. Some answers were new and surprising. That’s when we decided to put it to the test to see how creative it is.
In conclusion, this study shows that the ChatGPT artificial intelligence, powered by the GPT-4 engine, can compete with the top 1% of human thinkers in a standard test of creativity, the Torrance Tests of Creative Thinking. This exceptional performance is a testament to the potential of AI to generate new ideas, many and varied, which can have applications in many fields, including entrepreneurship and business.
However, this study also raises important questions regarding the definition, measurement and development of creativity, for both humans and machines. How to assess the relevance, quality and value of ideas produced by AI? How to foster collaboration and complementarity between human creativity and artificial creativity? How to preserve ethics, responsibility and humanity in creative processes involving AI? These are all challenges that call for continued research and dialogue on the creativity of AI.
Source : University of Montana Western
And you ?
Are the results of his research relevant?
What are the pros and cons of the TTCT test for assessing AI creativity?
Do you have experience with ChatGPT AI? Is it of any use in your work?
What do you see as the risks and opportunities of using ChatGPT AI to generate ideas?
What are the limits and perspectives of the study conducted by the research team from the University of Montana?
See as well :
OpenAI on the economic impact of GPT-4: 80% of workers will have at least 10% of their tasks affected, the company says its products will automate a vast range of professional tasks
GPT-4 would produce erroneous information, much more than GPT-3.5, according to a study by NewsGuard, yet OpenAI had declared the opposite
1688710926
#match #top #human #thinkers #standard #test #creativity #University #Montana #researchers
