Gen AI –Part 3: Building a Chatbot with LLaMA and Streamlit: A Beginner’s Guide

9 min readApr 12, 2024

If you’re new to the world of generative AI and looking to dive in, you’ve come to the right place. In this series, I’ll take you through the journey of setting up your AI-powered chatbot step by step.

Creating a chatbot can seem daunting, especially if you’re new to the world of artificial intelligence and machine learning. However, with the right tools and a bit of guidance, it’s a process that can be both educational and enjoyable.

No prior experience? No problem!

You can read the Part 1 and Part 2 if you want to understand the basics of Gen AI and LLMs

Part 1 —

Gen AI–Part 1: Understanding the Basics

Welcome to the first entry in a Blog series designed to decode the world of Generative AI (Gen AI), a term that’s…

ai.plainenglish.io

Part 2 —

Unravelling Generative AI –Part 2: Bridging the Gap Between LLMs and Applications

In our first blog post, “Unraveling Generative AI — Part 1: Understanding the Basics,” we dipped our toes into the…

ai.plainenglish.io

In this blog, I’ll take you through the steps to build a simple chatbot using the LLAMA2 model and the Streamlit framework, starting with the basics.

Since Llama is an Opensource model and we are installing it locally, we would not need any API keys such as Open AI API keys.

What is LLAMA?

LLAMA, short for “Language Model Adaptation,” is an open-source language model developed by Meta AI. It is a large language model trained on a vast amount of text data, allowing it to understand and generate human-like text.

LLAMA is notable for its impressive performance on various NLP tasks, such as text generation, question answering, and language understanding. One of the key advantages of LLAMA is its open-source nature, which allows developers and researchers to access and modify the model for their specific use cases.

In the context of this project, we will be using LLAMA as the underlying language model to power our chatbot. The LangChain library provides a convenient interface for integrating LLAMA with our application.

What is Streamlit?

Streamlit is an open-source Python library that allows developers to create interactive web applications with minimal effort. It provides a simple and intuitive way to build data visualization tools, machine learning applications, and interactive dashboards.

One of the key features of Streamlit is its ability to create reactive user interfaces, which update in real-time as the user interacts with the application. This makes it an ideal choice for building chatbots and other conversational interfaces.

In our project, we will be using Streamlit to create a user-friendly interface for our chatbot. Streamlit will handle the rendering of the user interface, including the text input field and the display of the chatbot’s responses.

Why LLaMA and Streamlit?

LLaMA is a powerful language model that can understand and generate human-like text. Streamlit, on the other hand, is a framework for turning Python scripts into beautiful web applications with ease. Together, they make building chatbots both accessible and fun!

By combining the power of LLAMA, LangChain, and Streamlit, we can create a robust and interactive chatbot that can understand and respond to user queries in a natural and engaging manner.

Prerequisites

Before we get started, make sure you have the following installed on your machine:

Python 3.x — If you do not have Python already, you can download and install it from the official Python website
Pip (Python’s package manager)
A code editor or IDE (like Visual Studio Code or PyCharm)

Installing Ollama

Go to the official website of Ollama

Download the version according to your OS and follow the steps to run and install llama on your local machine.

Setting up your environment

Open your IDE — Vscode/pycharm, any IDE of your choice
Create a project folder
Add requirements.txt in the folder

streamlit
langchain

Run the following command to install all the libraries.

pip install -r requirements.txt

Creating the Chatbot

Create a new file and name it chatbot.py

First, we need to import the necessary modules:

# Import necessary modules
from langchain_core.prompts import ChatPromptTemplate
from langchain_community.llms import Ollama
import streamlit as st

Here’s what each import does:

ChatPromptTemplate: This is a tool from langchain-core that helps us create structured prompts for our chatbot.
Ollama: This is the module that gives us access to the LLaMA model.
streamlit as st: This imports the Streamlit library and allows us to use it with the shorthand st.

Initializing the Streamlit app

With Streamlit, we can quickly create a web interface for our chatbot. Here’s how:

# Set up the Streamlit framework
st.title('Langchain Chatbot With LLAMA2 model')  # Set the title of the Streamlit app
input_text=st.text_input("Ask your question!")  # Create a text input field in the Streamlit app

The st.title function sets the title of our web app, and st.text_input creates a field where users can type in their questions.

Initializing the LLaMA Model

Before you run this, make sure you have downloaded the llama model on your local machine and its running.

Open the terminal and run the following command -

ollama run llama2

This will ensure llama2 model is running successfully.

Open the chatbot.py file again and type the below code —

# Initialize the Ollama model
llm=Ollama(model="llama2")

Prompt Engineering

Prompt engineering is about giving clear and specific instructions to a virtual assistant or AI system to ensure it understands exactly what you want it to do. Just like you would provide detailed instructions to a human assistant, you need to give precise prompts to an AI model to get the desired results.

In simpler words, we set up our chatbot’s brain with a prompt template.

For example, if you want a language model to generate a short story about a magical adventure, you would need to craft a prompt that provides enough context and guidance for the model to understand the genre, setting, characters, and plot elements you have in mind. A well-designed prompt might look something like this:

“Write a short story about a young wizard who discovers a secret portal to a magical realm filled with fantastical creatures and unexpected challenges. The story should be around 500 words long and include elements of adventure, mystery, and friendship.”

By providing this specific prompt, you are “engineering” the input to the language model, increasing the chances of it generating a story that aligns with your expectations.

We’ll set up a prompt template for our chatbot. This defines how the chatbot will interact with the user:

# Define a prompt template for the chatbot
prompt=ChatPromptTemplate.from_messages(
    [
        ("system","You are a helpful assistant. Please respond to the questions"),
        ("user","Question:{question}")
    ]
)

Creating the Chain

Imagine you have a set of instructions or a recipe that you want to follow to accomplish a task. Each step in the instructions or recipe can be thought of as a separate component or a “link” in a chain.

In the context of LangChain, a chain is a way to connect different components or “links” together to perform a specific task. These components can be things like prompts, language models, or other processing steps.

In our code, the two main components or “links” are:

The prompt template
The LLAMA language model

# Create a chain that combines the prompt and the Ollama model
chain=prompt|llm

Essentially, the chain links the components together to enable the conversational flow between the user and the language model.

Invoke the chain

By invoking the chain with the user’s input and displaying the output, it brings the entire chatbot application together, allowing users to engage in a conversational interface with the language model.

# Invoke the chain with the input text and display the output
if input_text:
    st.write(chain.invoke({"question":input_text}))

The if input_text: condition checks if the user has entered any text in the input field. If the input_text variable is not empty (i.e., the user has entered some text), the code inside the if block will execute.

The chain.invoke({"question":input_text}) line is where the magic happens. It invokes (or calls) the chain that we created earlier by combining the prompt template and the LLAMA language model. The invoke method takes a dictionary as an argument, where the key "question" corresponds to the placeholder {question} in the prompt template we defined earlier.

So, when the user enters some text in the input field, that text is passed as the value for the "question" key in the dictionary. The chain then takes this input, passes it through the prompt template, and feeds it to the LLAMA language model. The language model processes the input and generates a response based on its training.

Putting it all together —

Your chatbot.py will look like this

# chatbot.py
# Import necessary modules
from langchain_core.prompts import ChatPromptTemplate
from langchain_community.llms import Ollama
import streamlit as st

# Define a prompt template for the chatbot
prompt=ChatPromptTemplate.from_messages(
    [
        ("system","You are a helpful assistant. Please response to the questions"),
        ("user","Question:{question}")
    ]
)

# Set up the Streamlit framework
st.title('Langchain Chatbot With LLAMA2 model')  # Set the title of the Streamlit app
input_text=st.text_input("Ask your question!")  # Create a text input field in the Streamlit app

# Initialize the Ollama model
llm=Ollama(model="llama2")

# Create a chain that combines the prompt and the Ollama model
chain=prompt|llm

# Invoke the chain with the input text and display the output
if input_text:
    st.write(chain.invoke({"question":input_text}))

Requirements.txt will have these two libs-

# requirements.txt
# Streamlit is an open-source Python library that makes it easy to create and share beautiful, custom web apps for machine learning and data science.
streamlit

# Langchain is a Python library for language detection and translation using Google's Translation API.
langchain