Portal de Periódicos da CAPES

Chat of the Town: Gathering User Perception About ChatGPT

2023; RELX Group (Netherlands); Linguagem: Inglês

10.2139/ssrn.4502004

1556-5068

Arundhati Dixit, Ridhi Jain,

Privacy, Security, and Data Protection

ChatGPT is a chat bot that generates responses to textual questions. We studied human interaction with this application, by focusing on (1) nature and frequency of questions to better understand what users expect from such an application, and (2) how users perceive the responses provided by ChatGPT, what utility and satisfaction they derive in their chosen domain of use. The study consisted of usage logs from 10 participants over a period of two weeks. We saw that most queries, 74.05%, pertained to seeking information or for professional, academic or factual requirements. 90.54% of the answers provided were satisfactory, and the rest were either inaccurate or incomplete or misinterpreted, thus requiring paraphrasal. Users indicated that maintaining ask history and the ability to respond to follow ups aligned well with the conversational nature of the application, and that it works well for specific requirements like coding or structured writing, but is not the go-to app for everything, everywhere, all at once. Users also indicated that while posing questions, they were mindful of contextual enrichment of queries, and the application's limitations and unsatisfactory answers in the past, if any, thus helping them use the app more efficiently. More research is required to better understand the degree of usefulness, reliability, and replaceability of the information sought from ChatGPT.