Show HN: I compressed 10k PDFs into a 1.4GB video for LLM memory https://ift.tt/qJpQbOU

Show HN: I compressed 10k PDFs into a 1.4GB video for LLM memory While building a Retrieval-Augmented Generation (RAG) system, I was frustrated by my vector database consuming 8GB RAM just to search my own PDFs. After incurring $150 in cloud costs, I had an unconventional idea: what if I encoded my documents into video frames? The concept sounded absurd—storing text in video? But modern video codecs have been optimized for compression over decades. So, I converted text into QR codes, then encoded those as video frames, letting H.264/H.265 handle the compression. The results were surprising. 10,000 PDFs compressed down to a 1.4GB video file. Search latency was around 900ms compared to Pinecone’s 820ms—about 10% slower. However, RAM usage dropped from over 8GB to just 200MB, and it operates entirely offline without API keys or monthly fees. Technically, each document chunk is encoded into QR codes, which become video frames. Video compression handles redundancy between similar documents effectively. Search works by decoding relevant frame ranges based on a lightweight index. You get a vector database that’s just a video file you can copy anywhere. GitHub: https://ift.tt/tAzoXyI https://ift.tt/tAzoXyI May 29, 2025 at 02:54AM