top of page
Ravit banner.jpg

Post page

How (in)accurate is generative AI?



A new study, Magesh et al. (2024), finds that Generative AI is mostly wrong.


➤ Highlights



👉 A study from May tested four platforms, focusing on legal tech. 


(Excuse the typo in the pdf, the study is from May, not March)



👉 The researchers sent prompts with legal questions and measured the accuracy of the results, sorting them into false, incomplete, or accurate answers.



👉 They found that the answers are mostly wrong, false or incomplete – see the image attached to this post 







➤ Reflections



💡 Generative AI is mostly wrong



💡Platforms vary on how accurate they are 



💡Make sure you inquire about accuracy when you choose platforms, whether for personal use or when procuring platforms for your business.



💡 Don’t believe vendors who tell you that their chatbots are error-free! 




➤ Comments welcome! What do people think of this finding?


Comments


FOR UPDATES

Join my newsletter for tech ethics resources.

I will never use your email for anything else.

bottom of page