PDF Knowledge Base

The PDFKnowledgeBase reads local PDF files, converts them into vector embeddings and loads them to a vector databse.

Usage

We are using a local PgVector database for this example. Make sure it’s running

pip install pypdf

knowledge_base.py

from bitca.knowledge.pdf import PDFKnowledgeBase, PDFReader
from bitca.vectordb.pgvector import PgVector

pdf_knowledge_base = PDFKnowledgeBase(
    path="data/pdfs",
    # Table name: ai.pdf_documents
    vector_db=PgVector(
        table_name="pdf_documents",
        db_url="postgresql+psycopg://ai:ai@localhost:5532/ai",
    ),
    reader=PDFReader(chunk=True),
)

Then use the knowledge_base with an Agent:

agent.py

from bitca.agent import Agent
from knowledge_base import knowledge_base

agent = Agent(
    knowledge=knowledge_base,
    search_knowledge=True,
)
agent.knowledge.load(recreate=False)

agent.print_response("Ask me about something from the knowledge base")

Params

Parameter
Type
Default
Description

path

Union[str, Path]

-

Path to PDF files. Can point to a single PDF file or a directory of PDF files.

vector_db

VectorDb

-

Vector Database for the Knowledge Base. Example: PgVector

reader

Union[PDFReader, PDFImageReader]

PDFReader()

A PDFReader that converts the PDFs into Documents for the vector database.

num_documents

int

5

Number of documents to return on search.

optimize_on

int

-

Number of documents to optimize the vector db on. For Example: Create an index for PgVector.

chunking_strategy

ChunkingStrategy

FixedSizeChunking

The chunking strategy to use.

Last updated