PDF Knowledge Base
The PDFKnowledgeBase reads local PDF files, converts them into vector embeddings and loads them to a vector databse.
Usage
We are using a local PgVector database for this example. Make sure it’s running
pip install pypdf
knowledge_base.py
from bitca.knowledge.pdf import PDFKnowledgeBase, PDFReader
from bitca.vectordb.pgvector import PgVector
pdf_knowledge_base = PDFKnowledgeBase(
path="data/pdfs",
# Table name: ai.pdf_documents
vector_db=PgVector(
table_name="pdf_documents",
db_url="postgresql+psycopg://ai:ai@localhost:5532/ai",
),
reader=PDFReader(chunk=True),
)
Then use the knowledge_base
with an Agent:
agent.py
from bitca.agent import Agent
from knowledge_base import knowledge_base
agent = Agent(
knowledge=knowledge_base,
search_knowledge=True,
)
agent.knowledge.load(recreate=False)
agent.print_response("Ask me about something from the knowledge base")
Params
path
Union[str, Path]
-
Path to PDF
files. Can point to a single PDF file or a directory of PDF files.
vector_db
VectorDb
-
Vector Database for the Knowledge Base. Example: PgVector
reader
Union[PDFReader, PDFImageReader]
PDFReader()
A PDFReader
that converts the PDFs
into Documents
for the vector database.
num_documents
int
5
Number of documents to return on search.
optimize_on
int
-
Number of documents to optimize the vector db on. For Example: Create an index for PgVector
.
chunking_strategy
ChunkingStrategy
FixedSizeChunking
The chunking strategy to use.
Last updated