Identification of thesaurus relationships with the support of ChatGPT and Gemini

The study aims to verify if Large Language Models (LLMs) - ChatGPT and Gemini - can support experts during some critical stages of thesaurus construction: categorization and recognition of semantic relationships between terms/concepts. To this end, domain-independent prompts were defined and tested through a pilot experiment in the sub-domain of archaeological sites. The experimentation required the development of a benchmark consisting of terms and concepts linked by thesaurus’ semantic relationships. The results show that LLMs, combined with the experts’ skills, can be useful in accomplishing these tasks more efficiently. The interaction between experts and AI services has been mutually beneficial: experts helped AI services fully understand the requests – providing context and improving prompts – while AI services provided reasoning for categorization and semantic network construction, which could provide experts with insights for improving tasks. The study highlights opportunities for improving AI services performances in disambiguating categories and recognizing semantic relationships.