My Own Private (or public) Google

Post date: 2020-12-04 04:37:13

I have a hard drive with ~3TB of assorted files (html, video, etc.) scraped from a large (public) website at a finite point in time. The files aren't arranged in a particularly human-readable way, but (I think?) in folders by file type. How do I make them my own private google -- or open to the public is fine?

My goal is to have this archive in a format where a relatively small number of people could pull up a browser, enter text (or filetype) in a search field, and have relevant results pop up -- really, exactly what Google does. It could be a system where they need to set up an account (ideally free for them), or something open to the public (not sensitive, if not popular either).

Difficulty: I understand computers, and 10 years ago might have clawed my way into setting up my own CMS, and maybe an SQL install or something, but I'd rather just have an off-the-shelf product that works quickly and that I can set up with less terminal and more mouse. I'm willing to pay ~$20/month, or maybe more (since this is potentially time limited).

One idea I had was just to set up a google drive account, create a shared drive, and upload everything there (though I think uploads are limited to 750GB or something/day). I can try to trim it to under 2GB (the jump from 2GB for $9.99 to 10TB for $49.99 is massive)... Or should I just try to get the data into the cloud somewhere, hope Google indexes it, and create a one-page web interface that routes searches to site:xyz? (Does "hope Google indexes it" work here?)

Other ideas are welcome! Thank you!

Please click Here to read the full story.