Unleashing the Power of Generative AI in Literature Review

Table of Contents

1. Introduction
2. The Use of Generative AI in Literature Review
   • Overview of the study
   • Human coding process
   • Use of generative AI in the analysis
3. The Research Questions
   • Main research question
   • Different types of syllabi
4. The AI Tool: AI Lies
   • Uploading the transcript file
   • Analyzing the transcript
   • Evaluating the consistency of AI tool's responses
5. Reflections on the AI Tool
   • The reliability of AI Tools
   • Differentiating between AI responses
   • AI's ability to create a code book
6. Comparison with Human Coding
   • Comparison of AI tool's codes with human code book
   • Evaluation of AI's contributions
7. Pros and Cons of AI Tool for Literature Review
   • Pros of using AI in literature review
   • Cons and limitations of AI in literature review
8. Future Developments and Recommendations
   • Improvements in AI tools for literature review
   • Recommendations for researchers using AI in literature review
9. Conclusion

The Use of Generative AI in Literature Review

In recent years, there has been an increasing interest in the use of artificial intelligence (AI) in various research domains. One area where AI has shown promise is in conducting literature reviews. Traditionally, literature reviews involve a labor-intensive process of manually reviewing and coding numerous articles. However, the advent of generative AI tools has opened up new possibilities for researchers to streamline and automate this process.

Overview of the study

The study discussed in this article focuses on the use of generative AI in reviewing literature. The researchers conducted a literature review study on a specific topic and employed human coding to analyze the articles. The study involved screening 1,300 articles and reviewing the full text of 170 articles. The human coding process was completed before they turned to AI to explore its capabilities in conducting the same analysis.

Human coding process

The human coding process involved evaluating the selected articles based on specific criteria and coding the Relevant information. This process was time-consuming and required considerable effort from the researchers. However, it provided a comprehensive understanding of the literature in the field.

Use of generative AI in the analysis

Once the human coding process was completed, the researchers turned to a generative AI tool called AI Lies to analyze the transcripts of interviews conducted as part of the study. This tool allowed them to upload the transcripts and analyze them according to their research questions.

Main research question

The main research question of the analysis was to determine what information from the syllabus the students used to complete the Course evaluation. The syllabi used in the study were of two types: core-centered and community-centered. The researchers aimed to identify the differences between the two types of syllabi and examine their impact on the students' course evaluation.

The AI Tool: AI Lies

The researchers used AI Lies, a generative AI tool specifically designed for analyzing transcripts. They uploaded the transcript file to the tool's web page and utilized various options to analyze the data. The AI Lies tool provided insights into the transcripts and generated summaries and codes based on the uploaded data.

Evaluation of AI tool's responses

The researchers evaluated the consistency of the AI tool's responses by running the analysis multiple times. They found that in some cases, the responses were inconsistent and even contradictory. This raised questions about the reliability of the AI tool and its ability to provide accurate and consistent results.

Reflections on the AI Tool

The inconsistencies observed in the AI tool's responses led the researchers to reflect on the nature of AI-generated analysis. They suggested that different responses from the AI tool might indicate different perspectives rather than being classified as right or wrong. The researchers also explored the option of using AI to create a code book for further analysis.

Comparison with Human Coding

To assess the effectiveness of the AI tool, the researchers compared its results with the human-coded analysis. They found that the AI tool addressed a significant portion of the codes identified by the research team. However, some of the additional codes generated by the AI tool were found to be irrelevant to the research questions.

Pros and Cons of AI Tool for Literature Review

There are several advantages and limitations of using AI tools like AI Lies in literature review. One of the pros is the ability to automatically analyze large volumes of data, saving time and effort for researchers. However, limitations include the potential for incorrect responses, the need for continuous updates and improvements in AI tools, and the challenge of balancing reliance on AI with researcher expertise.

Future Developments and Recommendations

In the future, researchers using AI for literature review should stay updated with the latest advancements and improvements in AI tools. It is crucial to critically evaluate the responses generated by AI tools and consider them as an additional resource rather than a definitive answer. Researchers should also provide clear instructions and context to AI tools to generate more accurate and contextually relevant results.

Conclusion

The use of generative AI in literature review has both pros and cons. While AI tools like AI Lies can offer efficiency and automation, they also come with limitations and challenges. Researchers should approach AI tools as complementary resources and continue to exercise their own expertise and judgment throughout the literature review process.

Note: AI Lies is a fictional AI tool created for the purpose of this article and does not represent an actual existing AI tool.

Highlights:

• The use of generative AI in literature review provides automation and efficiency.
• Comparison between AI-generated analysis and human coding reveals potential benefits and limitations.
• AI tools should be used as complementary resources and not as a replacement for human expertise.
• Continuous learning, updates, and critical evaluation are essential when using AI tools for literature review.

Resources:

• AI Lies (Fictional AI tool)