Self-Hosted LLM RAG with Occult Books

[Swampdweller900]

I recently spun up a local LLM on a home server just for funzies. It's fine on its own, and the goal is to dump a bunch of documents into a RAG db and get some cross-referencing, some copyediting, etc. Apparently any OCR'd PDF or text would work, which immediately made me think of the library here and how it might be worth chunking in a dozen or so favorite occult books and see what I get.

I'm curious if anyone has either used a local LLM for something like this, or something similar.

 

[Frater M.M.]

Nope, but this is a fascinating idea. I usually dump a single file into ChatGPT if I want more concrete analysis, but the idea of putting loads of files in a db for indexing and then analysis with a local LLM sounds great. Who knows what it will abstract out of them?

Maybe when I have more time/money, I might look into something like this myself.

 

[Swampdweller900]

Maybe when I have more time/money

Plenty of free models out there. I'm using an old laptop from 2019 as the home server, and haven't paid a cent to anyone. Docker is free, Ollama is free, Mistral 7B is free. Even entirely unrestricted models are free on Hugging Face. Sure, it's slooooow sometimes, but I'm running it on 12GB of RAM. It's something I set up and let run and check back on later. I'm hoping to get it set up to process large documents overnight for copyediting.

FWIW, the Mistral training data includes at least a cursory level of occult knowledge. No idea what books that's originally from, it cites the source of a test I did just now to give me an example of a RHP spell as "chatbot schema" lol.

 

[Kepler]

This mapping of the geometry of meaning found in a single subject, like occult books, could be very revealing of patterns that are helpful to realize how those books found meaning within their purview and develop a way to reason similarly without getting locked into the same conclusions.

 

[Frater M.M.]

I'm hoping to get it set up to process large documents overnight for copyediting.

You may need more than one night for Michael Ford's books

FWIW, the Mistral training data includes at least a cursory level of occult knowledge. No idea what books that's originally from, it cites the source of a test I did just now to give me an example of a RHP spell as "chatbot schema" lol.

That's both interesting and hilarious.

Yeah, from some investigation with ChatGPT it's possible I could dig out an old mini-pc I have to investigate this for free without blowing £15k on a 512GB M5 Ultra later on this year.

It's also suggesting to me I could use a hybrid model of working:

100 books on your machine
→ local RAG searches/indexes them
→ retrieves a set of potentially relevant passages
→ reranks them and selects the most relevant chunks
→ exports those chunks with title/author/page/chapter metadata
→ you give that extracted material to me
→ I analyse/compare/synthesise it

And to actually send it retrieved chunks and get it to answer questions based only on them:

SOURCE: Michael Aquino, The Temple of Set
PAGE: 143
CHAPTER: Xeper
TEXT:
[retrieved passage]

SOURCE: Don Webb, Uncle Setnakt's Essential Guide...
PAGE: 87
TEXT:
[retrieved passage]

Also, I think if you dumped in a few entries from places like the

Please, Log in or Register to view URLs content!

, it could make analysis of texts more interesting by using more rigorous philosophical concepts/frameworks.