Sample Page

Bookologia

The right words find you.

Bookologia is a specialized search engine, to find any book within seconds. It is open source, and can be self-hosted easily on Docker.

Now, why were we interested in creating this project?

We believe there was a misunderstanding of the book market online, and a proof of that, none of the book platforms in the last 20 years were successful. They were shaped around the idea that people would read books and talk about them with others. Yet, every time a new project was launched, it was met with an insufficient market for growth.

That is because they targeted the wrong audience. People who read books for entertainment, are not the ones who search for them. The people who search for books are called: Knowledge workers. Engineers, doctors, lawyers, marketers... they use bits of information from here and there to find solutions to problems they have in hand. Most of the books they deal with are technical and specialized. They do not read the books, they browse them.

In such a market, you cannot simply sell books, yet these people are the total majority of those consuming them. Hence why profitability in this market is not possible, and only a public library model would work.

How does Bookologia work?

Bookologia is a specialized search engine. Which means, it queries Google and other search engines on your behalf, retrieves the links related to the book in question, analyze them, and present them to you. Our value proposition is the organization of books in such a way that you know a lot about a book before even opening it. The editions, other works by the author, and recommended similar ones picked carefully.

A scraper engine is provided as well, to add as much meta-data to your library as you wish, or missing books that you need to catalogue. All self-hosted locally.
See documentation for more details.

Who created this project?

We are Blank Research. An industrial research company. Our job is to conduct applied research to develop new products, processes and technologies for commercial use. We help startups build their products, and estblished companies to innovate new markets.

Main founding engineer is : Youssef (yousbot).
Find more details about us visiting : https://blankresearch.com

Bookologia is divided into 3 main parts, Elastic Search, Flack (backend) and Scraper (Scrapy).
Elastic Search is the database, Flask is the backend server, and Scraper, is a local agent that scrapes book metadata on demand.

As the goal of this project is to be self-hosted, we chose to use simple technologies.
Javascript, CSS Tailwind, HTML and Flask. No frameworks were used as no scalability was intended. The entire setup can take up to 1 minute (excluding download time).

The easiest way to get Bookologia is through Docker. More details below.

Docker installation

It is assumed you have Docker installed. If not, install it here then follow the next steps.


# wget https://raw.githubusercontent.com/blankresearch/Bookologia/refs/heads/main/docker-compose.yml 
# docker compose up

This will deploy both bookologia-es (Elastic) and bookologia-app in a dedicated network (bookologia-net), then you can access the app directly at : http://localhost:5001

Manual Deployment

To deploy Bookologia manually from the source. You must first host Elastic Search database on Docker following the next steps.


# docker run -d --name bookologia-es -p 9200:9200 \ 
 -e discovery.type=single-node \ 
 -e xpack.security.enabled=false \ 
 yousb0t/bookologia-elastic

If you're interested in the dump directly. Find it here.
Then, download and run flask backend as follow :


# git clone https://github.com/blankresearch/Bookologia.git
# cd Bookologia 
# pyenv install 3.10.7  
# pyenv local 3.10.7  
# python -m venv venv && source venv/bin/activate && pip install -r requirements.txt 
# cd Backend 
# python app.py

then you can access the app directly at : http://localhost:5001

Scraper Engine

The database comes preloaded with almost 2 million books. To add more books to this library, you need to launch the scraping engine, by providing it a range of IDs from GoodReads it doesnt have locally.

Books are checked for deduplication before insertion. You can run the scraper as follow :


# cd Bookologia/Scraper/billionBook
# scrapy crawl goodreads -a start_id=start_id -a end_id=end_id  

# Example: scrapy crawl goodreads -a start_id=120000 -a end_id=120100