Advanced Techniques in Text Summarization: Leveraging Generative AI and Prompt Engineering

Introduction:

In today’s information-rich world, extracting key information from vast amounts of textual data is a time-consuming task. Generative AI and Language Models like OpenAI’s GPT-3.5 have emerged as powerful tools for text summarization. In this blog, we will explore advanced techniques in text summarization, specifically focusing on prompt engineering. We will set up our working environment and delve into the process of summarizing texts at different levels.

Prerequisite: Setting Up the Working Environment

Before we embark on our journey, let’s ensure we have the necessary tools to leverage Generative AI for text summarization.

1. Create a Virtual Environment: We start by isolating our project dependencies in a virtual environment using the command:

python -m venv venv

2. Activate the Virtual Environment: Depending on the operating system, use the appropriate command to activate the virtual environment.

For Windows: .\venv\Scripts\activate
For macOS/Linux: source venv/bin/activate

3. Install Required Packages: With the virtual environment active, install the necessary packages using the commands:

pip install openai langchain tiktoken.

With these steps completed, our working environment is now set up and ready to explore the power of Generative AI in text summarization.

Level 1: Basic Summarization.

In this section, we will explore basic summarization by selecting a grim and controversial paragraph discussing the widely debated ethical topic known as “The Trolley Dilemma.”

We will summarize the key points of this paragraph to provide a concise overview of the topic..

Code Snippet — Setting Up Environment :

from langchain import OpenAI

OPENAI_API_KEY = '<ADD YOUR OPENAI API KEY HERE😛>'
llm = OpenAI(temperature=0, openai_api_key=OPENAI_API_KEY)

Code Snippet — Paragraph summarization:

prompt = """
 Please provide a summary of the following text

 TEXT:

 The trolley dilemma, an intricate moral thought experiment, presents a perplexing predicament
 whereby an uncontrolled trolley hurtles down a track, inevitably leading to the demise of five
 individuals. As a bystander, you find yourself faced with a choice: should you intervene by 
 pulling a lever, redirecting the trolley onto an alternative track where a lone person stands,
 thereby sacrificing their life to save the others? This quandary delves into the depths of 
 ethical inquiry, eliciting profound questions surrounding the nature of human agency and the
 conflict between consequentialist and deontological frameworks. It provokes contemplation on
 the intricate interplay between maximizing overall welfare and adhering
 steadfastly to moral principles, unfazed by the consequences. The trolley dilemma, in its 
 multifarious incarnations, serves as a captivating instrument for philosophers and ethicists to
 dissect and navigate the intricacies of decision-making processes, offering insights into the 
 complexities of our moral landscape.
 """
 num_tokens = llm.get_num_tokens(prompt)
 print (f"Our prompt has {num_tokens} tokens")
 output = llm(prompt)
 print (output)

Output

Our prompt has 222 tokens

The Trolley Dilemma is a moral thought experiment that presents a difficult decision. A trolley is hurtling down a track, leading to the death of five individuals. As a bystander, one must decide whether to intervene by pulling a lever, redirecting the trolley onto an alternative track where a lone person stands, sacrificing their life to save the others. This dilemma raises questions about human agency, the conflict between consequentialist and deontological frameworks, and the interplay between maximizing overall welfare and adhering to moral principles. It serves as a tool for philosophers and ethicists to explore the complexities of decision-making processes and gain insight into our moral landscape.

1.2 Levelling Up with Prompt Engineering

Did you notice how the English sentences became simpler after summarization? Well, let’s take it to the next level! It’s time to dive into the world of prompt engineering.

Now, you might be wondering, what on earth is prompt engineering and how do we use it?

It’s actually a fancy term for a cool trick.We can spice up our prompt by adding something like, “Hey, please explain it to me as if I’m a 5-year-old 😛”. Let’s see how it works!

“Do we really need to explain the Trolley Dilemma to a 5-year-old? It’s quite a complex topic!”

Code Snippet Paragraph summarization using prompt engineering:

prompt = """
 Please provide a summary of the following text.
 Please provide your output in a manner that a 5 year old would understandTEXT:
 
 The trolley dilemma, an intricate moral thought experiment, presents a perplexing predicament
 whereby an uncontrolled trolley hurtles down a track, inevitably leading to the demise of five
 individuals. As a bystander, you find yourself faced with a choice: should you intervene by
 pulling a lever, redirecting the trolley onto an alternative track where
 a lone person stands, thereby sacrificing their life to save the others? This quandary delves
 into the depths of ethical inquiry, eliciting profound questions surrounding the nature of
 human agency and the conflict between consequentialist and deontological frameworks. It
 provokes contemplation on the intricate interplay between maximizing overall welfare and
 adhering
 steadfastly to moral principles, unfazed by the consequences. The trolley dilemma, in its
 multifarious incarnations, serves as a captivating instrument for philosophers and ethicists 
 to dissect and navigate the intricacies of decision-making processes, offering insights into
 the complexities of our moral landscape.
 """

 num_tokens = llm.get_num_tokens(prompt)
 print (f"Our prompt has {num_tokens} tokens")
 output = llm(prompt)
 print (output)

Output

Our prompt has 238 tokens

There is a trolley going down a track that will hurt five people. Someone has to make a decision to save the five people by pulling a lever and sending the trolley onto a different track where one person will be hurt instead. This is a hard decision because it makes you think about if it is okay to hurt one person to save five people. It also makes you think about if it is okay to break a rule to save people. This is a problem that people think about to try and figure out what is the right thing to do.

Level 2: Map Reduce — Summarize a couple of pages.

When it comes to summarizing multiple pages of text, you might encounter a token limit constraint. While not always an issue, it’s essential to know how to handle it if it arises. This is where the “Map Reduce” chain type comes into play. It’s a method that can help overcome token limit challenges.

2.1 Understanding Map Reduce

• Token Limit Challenge: Discuss the issue of token limit constraints when summarizing multiple pages of text.
• Map Reduce Approach: Explain how the Map Reduce method breaks down the text into smaller chunks and generates summaries for each chunk.
• Obtaining a Concise Summary: Highlight the process of summarizing the individual chunk summaries to obtain a comprehensive summary of the entire document.

Note

1. We use “RecursiveCharacterTextSplitter” method provided by langchain to split the large data to smaller chunks.

2. We use the “load_summarize_chain” method provided by langchain to summarize content across multiple chunks.

2.2 Applying Map Reduce to Text Summarization

To explain the map-reduce, we will use the contents of our previous blog named “Enhancing Chatbot Capabilities with Generative AI”.

To read this blog:

Enhancing Chatbot Capabilities with Generative AI

To download the txt file:

https://chatbot-widget-test.s3.ap-south-1.amazonaws.com/Enhancing+Chatbot+Capabilities+with+Generative+AI.txt

Code Snippet — Importing Libraries:

 from langchain import OpenAI
 from langchain import PromptTemplate
 from langchain.chains.summarize import load_summarize_chain
 from langchain.text_splitter import RecursiveCharacterTextSplitter

Code Snippet — Setting Up Environment and Loading Data:

 OPENAI_API_KEY = '<:ADD YOUR OPENAI API KEY HERE😛>'
 llm = OpenAI(temperature=0, openai_api_key=OPENAI_API_KEY)

 chatbot_ai_txt = "Enhancing Chatbot Capabilities with Generative AI.txt"
 with open(chatbot_ai_txt, 'r') as file:
     essay = file.read()
 num_tokens = llm.get_num_tokens(essay)
 print(num_tokens)

Number of tokens — 1528

That’s too many, let’s split our text up into chunks so they fit into the prompt limit. I’m going for a chunk size of 5,000 characters. (You can go up to 10,000 characters)

For English text, 1 token is approximately 4 characters or 0.75 words.

This means the number of tokens we should expect is 3,000 / 4 = 750 token chunks. But this will vary, each body of text/code will be different

Code Snippet — Splitting Text into Chunks:

chatbot_ai_txt = "Enhancing Chatbot Capabilities with Generative AI.txt"
 
 with open(chatbot_ai_txt, 'r') as file:
     essay = file.read()
 text_splitter = RecursiveCharacterTextSplitter(separators=["\n\n", "\n"], 
 chunk_size=3000, chunk_overlap=100)
 
 docs = text_splitter.create_documents([essay])
 num_docs = len(docs)
 num_tokens_first_doc = llm.get_num_tokens(docs[0].page_content)
 
 print (f"Now we have {num_docs} documents and the first one has {num_tokens_first_doc} tokens")

Code Snippet — Running Map Reduce Summarization:

 summary_chain = load_summarize_chain(llm=llm, chain_type='map_reduce')
 output = summary_chain.run(docs)
 print(output)

OUTPUT -

Now we have 3 documents and the first one has 503 tokens

Generative AI can be used to enhance the capabilities of chatbots, such as OpenAI’s natural language processing technology, handling generic questions, language translation, internationalization, voice to text, contextual awareness, personalization, and continuous learning. These features can help the chatbot to quickly and accurately respond to user queries in multiple languages, making it more accessible and user-friendly for a global audience. OpenAI’s advanced features enable chatbots to understand the context and intent behind user queries, personalize their interactions with users, and continuously learn from user interactions, thus improving user experience.

2.3 How to add prompts to instruct the model on how to generate the response

We will explore how to add prompts to instruct the model on generating the desired response. We will utilize map_prompt and combine_prompt along with langchain’s PromptTemplate to create custom templates for instructing the model.

Code Snippet — Importing Libraries:

from langchain import OpenAI
 from langchain import PromptTemplate
 from langchain.chains.summarize import load_summarize_chain
 from langchain.text_splitter import RecursiveCharacterTextSplitter

Code Snippet — Loading Text:

chatbot_ai_txt = "Enhancing Chatbot Capabilities with Generative AI.txt"
 with open(chatbot_ai_txt, 'r') as file:
   essay = file.read()

Code Snippet — Text Splitting:

 text_splitter = RecursiveCharacterTextSplitter(separators=["\n\n", "\n"], 
 chunk_size=3000, chunk_overlap=100)
 docs = text_splitter.create_documents([essay])
 num_docs = len(docs)

Code Snippet — Creating the Map Prompt Template:

 map_prompt = """
 Write a concise summary of the following:
 "{text}"
 CONCISE SUMMARY:
 """
 map_prompt_template = PromptTemplate(template=map_prompt, input_variables=["text"])

Code Snippet — Creating the Combine Prompt Template

combine_prompt = """
 Write a concise summary of the following text delimited by triple backquotes.
 Return your response in bullet points which covers the key points of the text.
 ```{text}```
 BULLET POINT SUMMARY:
 """
 combine_prompt_template = PromptTemplate(template=combine_prompt, input_variables=["text"])

Code Snippet — Running Map Reduce Summarization with custom map and combine template

summary_chain = load_summarize_chain(llm=llm,
 chain_type='map_reduce',
 map_prompt=map_prompt_template,
 combine_prompt=combine_prompt_template,
 )

 output = summary_chain.run(docs)
 print (output)

OUTPUT -

- Generative AI can be used to enhance the capabilities of chatbots, such as natural language processing, automation, and more.
- 10 features that can be incorporated into a chatbot using Generative AI, such as OpenAI’s Natural Language Processing Technology, handling generic questions, language translation, internationalization, voice to text, contextual awareness, personalization, and continuous learning.
- OpenAI models can be integrated into a chatbot to enhance its capabilities, such as semantic search, language translation, internationalization, voice to text, and fine-tuning to learn about a company.
- OpenAI’s advanced features enable chatbots to understand the context and intent behind user queries, personalize their interactions, and continuously learn from user interactions.

Level 3: Agents — Summarize an unknown amount of text.

In this level, we will learn how to summarize an unknown amount of text using Wikipedia as a source. We will leverage the capabilities of the GPT-3.5-turbo model and a Wikipedia API wrapper.

Installing Dependencies:

pip install wikipedia

And we’ll use the model gpt-3.5-turbo to reduce cost.

Code Snippet — Importing Libraries:

 from langchain import OpenAI
 from langchain.chat_models import ChatOpenAI
 from langchain.agents import initialize_agent, Tool
 from langchain.utilities import WikipediaAPIWrapper
 import sys

Code Snippet — Initializing the OpenAI Model:

OPENAI_API_KEY = 'ADD YOUR OPENAI API KEY HERE😛'
 llm = ChatOpenAI(temperature=0, model_name='gpt-3.5-turbo', openai_api_key=OPENAI_API_KEY)

Code Snippet — Initializing the Wikipedia API Wrapper:

wikipedia = WikipediaAPIWrapper()

Code Snippet — Initializing the Agent Tools:

tools = [
 Tool(
     name="Wikipedia",
     func=wikipedia.run,
     description="Useful for when you need to get information from wikipedia
 about a single topic"
 ),
 ]

Code Snippet — Initializing the Agent Executor:

agent_executor = initialize_agent(tools, llm, 
 agent='zero-shot-react-description', verbose=True)

Code Snippet — Running the Agent Executor:

output = agent_executor.run("Can you please search the anime named Black Clover? 
 ")
print(output)

*Note that we have added verbose=True to see how the model is using wikipedia.

Logs:
Entering the new AgentExecutor chain…
I should use Wikipedia to search for information about Black Clover.
Action: Wikipedia
Action Input: “Black Clover anime”
Observation:I have found all the information about Black Clover that was requested.

Output -
Black Clover is a Japanese manga and anime series about a young boy named Asta who is born without magic power in a world where magic is common. He joins the Black Bulls to become the next Wizard King and has many adventures along the way. The anime series aired from October 2017 to March 2021 and has 170 episodes. An anime film titled Black Clover: Sword of the Wizard King is set to premiere in June 2023.

Summarizing an unknown amount of text, such as information from Wikipedia, is made possible by utilizing the GPT-3.5-turbo model and appropriate tools. By leveraging the power of AI, we can extract key information efficiently and enhance our understanding of complex texts. In the upcoming blog, we will explore more advanced techniques like embedding for summarizing entire books.

Useful Information

References

Blog: https://www.bluetickconsultants.com/advanced-techniques-in-text-summarization.html

Our Open Source Projects

Open Source Projects: https://www.bluetickconsultants.com/open-source.html

GitHub Open Source repos: https://github.com/orgs/bluetickconsultants/repositories

Author:

Bluetick Consultants LLP — https://www.bluetickconsultants.com