💻
RAG and LLM Bootcamp
  • Welcome to the Bootcamp
    • Course Structure
    • Course Syllabus and Timelines
    • Know your Educators
    • Action Items and Prerequisites
    • Kick-Off Session for the Bootcamp
  • Basics of LLMs
    • What is Generative AI?
    • What is a Large Language Model?
    • Advantages and Applications of LLMs
    • Bonus Resource: Multimodal LLMs and Google Gemini
  • Word Vectors, Simplified
    • What is a Word Vector?
    • Word Vector Relationships
    • Role of Context in LLMs
    • Transforming Vectors into LLM Responses
    • Bonus: Overview of the Transformer Architecture
      • Attention Mechanism
      • Multi-Head Attention and Transformer Architecture
      • Vision Transformers (ViTs)
    • Bonus: Future of LLMs? | By Transformer Co-inventor
    • Graded Quiz 1
  • Prompt Engineering and Token Limits
    • What is Prompt Engineering
    • Prompt Engineering and In-context Learning
    • For Starters: Best Practices
    • Navigating Token Limits
    • Hallucinations in LLMs
    • Prompt Engineering Excercise (Ungraded)
      • Story for the Excercise: The eSports Enigma
      • Your Task fror the Module
  • RAG and LLM Architecture
    • What is Retrieval Augmented Generation (RAG)?
    • Primer to RAG: Pre-trained and Fine-Tuned LLMs
    • In-context Learning
    • High-level LLM Architecture Components for In-context Learning
    • Diving Deeper: LLM Architecture Components
    • Basic RAG Architecture with Key Components
    • RAG versus Fine-Tuning and Prompt Engineering
    • Versatility and Efficiency in RAG
    • Key Benefits of using RAG in an Enterprise/Production Setup
    • Hands-on Demo: Performing Similarity Search in Vectors (Bonus Module)
    • Using kNN and LSH to Enhance Similarity Search (Bonus Module)
    • Bonus Video: Implementing End-to-End RAG | 1-Hour Session
    • Graded Quiz 2
  • Hands-on Development
    • Prerequisites (Must)
    • Docker Basics
    • Your Hands-on RAG Journey
    • 1 – First RAG Pipeline
      • Building with Open AI
      • How it Works
      • Using Open AI Alternatives
      • RAG with Open Source and Running "Examples"
    • 2 – Amazon Discounts App
      • How the Project Works
      • Building the App
    • 3 – Private RAG with Mistral, Ollama and Pathway
      • Building a Private RAG project
      • (Bonus) Adaptive RAG Overview
    • 4 – Realtime RAG with LlamaIndex/Langchain and Pathway
      • Understand the Basics
      • Implementation with LlamaIndex and Langchain
  • Final Project + Giveaways
    • Prizes and Giveaways
    • Suggested Tracks for Ideation
    • Sample Projects and Additional Resources
    • Submit Project for Review
Powered by GitBook
On this page
  • Step 1: Verify Docker Installation
  • Step 2: Clone the LLM App Templates Repository
  • Step 3: Navigate to the relevant project directory
  • Step 4: Create a .env File and put your Open API key
  • Step 5: Build the Docker Image
  • Step 6: Run the Docker Container
  • Step 7: Check the List of Files
  • Step 8: Last Step – Run the RAG Service
  1. Hands-on Development
  2. 1 – First RAG Pipeline

Building with Open AI

Previous1 – First RAG PipelineNextHow it Works

Last updated 10 months ago

Step 1: Verify Docker Installation

First, let’s verify that Docker is properly installed and open in your system. Open your terminal (Command Prompt on Windows) and run:

docker --version

You should see the Docker version information if it's installed correctly. If not, you might want to revist the and the section.

Step 2: Clone the Templates Repository

Next, clone the llm-app repository from GitHub. This repository contains all the files you’ll need.

git clone https://github.com/pathwaycom/llm-app.git

If you get an error because you have previously cloned an older version of the repository, ensure you're in the correct repository directory and update it using:

git pull

This will update your local repository with the latest changes from the remote repository.

Step 3: Navigate to the relevant project directory

Change to the directory where the example is located.

cd examples/pipelines/demo-question-answering

Step 4: Create a .env File and put your Open API key

Create a file named .env and add your Open AI API key using the Bash command below. 

echo "OPENAI_API_KEY=<your_openai_api_key>" > .

Alternatively, you can open your preferred text editor (e.g., Notepad), create a new file and add just the following line:

OPENAI_API_KEY=<your_openai_api_key>

Then, save the file as environment (.env) in the demo-question-answering folder. It will do the same thing.

Step 5: Build the Docker Image

Now, let’s build the Docker image. This step might take a few minutes depending on your machine. Ensure you have enough space (approximately 8 GB).

docker build -t rag .

The -t rag part is tagging the Docker image with the name ‘rag’. Whereas the . at the end specifies the build context directory, which is the current directory. This tells Docker to look for the Dockerfile in the current directory and include any files/subdirectories in the build context.

Step 6: Run the Docker Container

Run the Docker container, mounting (described below) the data folder, and exposing port 8000.

For Windows:

docker run -v "%cd%/data:/app/data" -p 8000:8000 rag

For Linux/Mac:

docker run -v "$(pwd)/data:/app/data" -p 8000:8000 --env-file .env rag

Note: You will see the logs for parsing & embedding documents in the Docker image logs. Give it a few minutes to finish up on embeddings. You will see 0 entries (x minibatch(es)) have been... message. If there are no more updates, this means the app is ready for use!

Handling Port Conflicts: If port 8000 is already in use and you see an error related to it, you can specify a different port. For example, if you want to use port 8080 instead, modify the command as follows:

For Windows:

docker run -v "${PWD}/data:/app/data" -p 8000:8000 rag

For Linux/Mac:

docker run -v "$(pwd)/data:/app/data" -p 8080:8000 --env-file .env rag

This will map port 8080 on your local machine to port 8000 in the Docker container. Just remember to update the port in the next step as well.

Open up another terminal window and follow the next steps.

Step 7: Check the List of Files

curl -X 'POST'   'http://localhost:8000/v1/pw_list_documents'   -H 'accept: */*'   -H 'Content-Type: application/json'

[{"created_at": null, "modified_at": 1718810417, "owner": "root", "path":"data/IdeanomicsInc_20160330_10-K_EX-10.26_9512211_EX-10.26_Content License Agreement.pdf", "seen_at": 1718902304}]

Step 8: Last Step – Run the RAG Service

You can now run the RAG service. Start by asking a simple question. For example:

curl -X 'POST' \
  'http://0.0.0.0:8000/v1/pw_ai_answer' \
  -H 'accept: */*' \
  -H 'Content-Type: application/json' \
  -d '{
  "prompt": "What is the start date of the contract?"
}'

It should return the following answer:

December 21, 2015

But how do you tweak this for your use-case? Let's see that by understanding the contents of the repo which just used.

You will see the logs for parsing & embedding documents in the Docker image logs. Give it a few minutes to finish up on embeddings, you will see 0 entries (x minibatch(es)) have been… message. If there are no more updates, this means the app is ready for use! Now let’s see the files from which we’ll retrieve information for our LLMs.To test it, let's query to get the list of available inputs and associated metadata using :

This will return the list of files e.g. if you start with the provided in the demo, the answer will be as follows:

prerequisites
Docker Basics
LLM App
llm-app
the curl command
data folder