Citation: Sejin Paik, Sarah Bonna, Ekaterina Novozhilova, Ge Gao, Jongin Kim, Derry Wijaya, and Margrit Betke. The Affective Nature of AI-Generated News Images: Impact on Visual Journalism. The International Conference on Affective Computing and Intelligent Interaction, September 10-13, MIT Media Lab, Cambridge, MA. 10 pages.
Abstract: This study explores the affective responses and newsworthiness perceptions of generative AI for visual journalism. While generative AI offers advantages for newsrooms in terms of producing unique images and cutting costs, the potential misuse of AI-generated news images is a cause for concern. For our study, we designed a 3-part news image codebook for affect-labeling news images based on journalism ethics and photography guidelines. We collected 200 news headlines and images retrieved from a variety of U.S. news sources on the topics of gun violence and climate change, generated corresponding news images from DALL-E 2 and asked study participants to annotate their emotional responses to the human-selected and AIgenerated news images following the codebook. We also examined the impact of modality on emotions by measuring the effects of visual and textual modalities on emotional responses. The findings of this study provide insights into the quality and emotional impact of generative news images produced by humans and AI. Further, results of this work can be useful in developing technical guidelines as well as policy measures for the ethical use of generative AI systems in journalistic production. The codebook, images and annotations are made publicly available to facilitate future research in affective computing, specifically tailored to civic and public-interest journalism.
Image annotations used in this work
The images are found here: Gun Violence (Human) Gun Violence (AI) Climate Change (Human) Climate Change (AI)Related Papers by BU's AIEM Team:
Lei Guo, Chao Su, Sejin Paik, Vibhu Bhatia, Vidya Prasad Akavoor, Ge Gao, Margrit Betke and Derry Wijaya. Proposing an open-sourced tool for computational framing analysis of multilingual data. Digital Journalism, 11(2): 276-297. 2023. pdf. Open Framing Code
Ge Gao, Sejin Paik, Carley Reardon, Yanling Zhao, Lei Guo, Prakash Ishwar, Margrit Betke, and Derry Wijaya. Prediction of People's Emotional Response towards Multi-modal News. The 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (AACL-IJCNLP), November 2022. 10 pages.
Carley Reardon, Sejin Paik, Ge Gao, Meet Parekh, Yanling Zhao, Lei Guo, Margrit Betke, and Derry Wijaya. NEmo: An Affective Dataset of Gun Violence News. Proceedings of the 13th Conference on Language Resources and Evaluation ( LREC 2022) Palais du Pharo, Marseille, France, June 20-25, 2022, pages 2507-2516, pdf.
Sha Lai, Yanru Jiang, Lei Guo, Margrit Betke, Prakash Ishwar, and Derry Tanti Wijaya. An Unsupervised Approach to Discover Media Frames. Proceedings of The LREC 2022 Workshop on Natural Language Processing for Political Science, Marseille, France, June 20-25, 2022, ISBN: 979-10-95546-88-7. Pages 22-31 pdf.
Isidora Tourni, Lei Guo, Taufiq Husad Daryanto, Fabian Zhafransyah, Edward Edberg Halim, Mona Jalal, Boqi Chen, Sha Lai, Hengchang Hu, Margrit Betke, Prakash Ishwar, and Derry Tanti Wijaya. Detecting Frames in News Headlines and Lead Images in U.S. Gun Violence Coverage. In Findings of the Assocation for Computational Linguistics: 2021 Conference on Empirical Methods in Natural Language Processing. November 2021, pages 4037-4050, Punta Cana, Dominican Republic. DOI 10.18653/v1/2021.findings-emnlp.339
Lei Guo, Kate Mays, Yiyan Zhang, Derry Wijaya, and Margrit Betke. What makes gun violence a (less) prominent issue? A computational analysis of compelling arguments and selective agenda setting. Mass Communication & Society:24(5):651-675, April 2021. https://doi.org/10.1080/15205436.2021.1898644
Vibhu Bhatia, Vidya Prasad Akavoor, Sejin Paik, Lei Guo, Mona Jalal, Alyssa Smith, David Assefa Tofu, Edward Edberg Halim, Yimeng Sun, Margrit Betke, Prakash Ishwar and Derry Tanti Wijaya. OpenFraming: Open-sourced Tool for Computational Framing Analysis of Multilingual Data. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 242-250. Online and Punta Cana, Dominican Republic. November 2021. DOI 10.18653/v1/2021.emnlp-demo.28
Afra Feyza Akyürek, Lei Guo, Randa Elanwar, Margrit Betke, Prakash Ishwar, and Derry T. Wijaya. Multi-label and Multilingual News Framing Analysis Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), July 2020, pages 8614-8624.
L. Guo, K. Mays, S. Lai, M. Jalal, P. Ishwar, and M. Betke. Accurate, Fast, But Not Always Cheap: Evaluating "Crowdcoding" as an Alternative Approach to Analyze Social Media Data. Journalism & Mass Communication Quarterly, 97(3): 811-834. September 2020. https://doi.org/10.1177/1077699019891437
Siyi Liu, Lei Guo, Kate Mays, Margrit Betke, and Derry Wijaya. Detecting Frames in News Headlines and Its Application to Analyzing News Framing Trends Surrounding U.S. Gun Violence. Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), Hong Kong, November 2019, 504-514. Gun Violence Frame Corpus (GVFC) dataset