From c690c1d4217992e7557c261092385b7236af3b78 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vir=C3=A1g=20G=C3=A1bor?= Date: Wed, 6 Nov 2024 11:03:00 +0100 Subject: [PATCH] upload --- README.md | 733 +----------------------------------------------------- 1 file changed, 3 insertions(+), 730 deletions(-) diff --git a/README.md b/README.md index a774508..c64b164 100644 --- a/README.md +++ b/README.md @@ -1,731 +1,4 @@ -![Whoogle Search](docs/banner.png) +![RP's Search](docs/banner.png) -[![Latest Release](https://img.shields.io/github/v/release/benbusby/whoogle-search)](https://github.com/benbusby/shoogle/releases) -[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) -[![tests](https://github.com/benbusby/whoogle-search/actions/workflows/tests.yml/badge.svg)](https://github.com/benbusby/whoogle-search/actions/workflows/tests.yml) -[![buildx](https://github.com/benbusby/whoogle-search/actions/workflows/buildx.yml/badge.svg)](https://github.com/benbusby/whoogle-search/actions/workflows/buildx.yml) -[![codebeat badge](https://codebeat.co/badges/e96cada2-fb6f-4528-8285-7d72abd74e8d)](https://codebeat.co/projects/github-com-benbusby-shoogle-master) -[![Docker Pulls](https://img.shields.io/docker/pulls/benbusby/whoogle-search)](https://hub.docker.com/r/benbusby/whoogle-search) - - - - - - -
SourceHutGitHub
- -Get Google search results, but without any ads, JavaScript, AMP links, cookies, or IP address tracking. Easily deployable in one click as a Docker app, and customizable with a single config file. Quick and simple to implement as a primary search engine replacement on both desktop and mobile. - -Contents -1. [Features](#features) -3. [Install/Deploy Options](#install) - 1. [Heroku Quick Deploy](#heroku-quick-deploy) - 1. [Render.com](#render) - 1. [Repl.it](#replit) - 1. [Fly.io](#flyio) - 1. [Koyeb](#koyeb) - 1. [pipx](#pipx) - 1. [pip](#pip) - 1. [Manual](#manual) - 1. [Docker](#manual-docker) - 1. [Arch/AUR](#arch-linux--arch-based-distributions) - 1. [Helm/Kubernetes](#helm-chart-for-kubernetes) -4. [Environment Variables and Configuration](#environment-variables) -5. [Usage](#usage) -6. [Extra Steps](#extra-steps) - 1. [Set Primary Search Engine](#set-whoogle-as-your-primary-search-engine) - 2. [Custom Redirecting](#custom-redirecting) - 2. [Custom Bangs](#custom-bangs) - 3. [Prevent Downtime (Heroku Only)](#prevent-downtime-heroku-only) - 4. [Manual HTTPS Enforcement](#https-enforcement) - 5. [Using with Firefox Containers](#using-with-firefox-containers) - 6. [Reverse Proxying](#reverse-proxying) - 1. [Nginx](#nginx) -7. [Contributing](#contributing) -8. [FAQ](#faq) -9. [Public Instances](#public-instances) -10. [Screenshots](#screenshots) - -## Features -- No ads or sponsored content -- No JavaScript\* -- No cookies\*\* -- No tracking/linking of your personal IP address\*\*\* -- No AMP links -- No URL tracking tags (i.e. utm=%s) -- No referrer header -- Tor and HTTP/SOCKS proxy support -- Autocomplete/search suggestions -- POST request search and suggestion queries (when possible) -- View images at full res without site redirect (currently mobile only) -- Light/Dark/System theme modes (with support for [custom CSS theming](https://github.com/benbusby/whoogle-search/wiki/User-Contributed-CSS-Themes)) -- Randomly generated User Agent -- Easy to install/deploy -- DDG-style bang (i.e. `! `) searches -- User-defined [custom bangs](#custom-bangs) -- Optional location-based searching (i.e. results near \) -- Optional NoJS mode to view search results in a separate window with JavaScript blocked - -*No third party JavaScript. Whoogle can be used with JavaScript disabled, but if enabled, uses JavaScript for things like presenting search suggestions. - -**No third party cookies. Whoogle uses server side cookies (sessions) to store non-sensitive configuration settings such as theme, language, etc. Just like with JavaScript, cookies can be disabled and not affect Whoogle's search functionality. - -***If deployed to a remote server, or configured to send requests through a VPN, Tor, proxy, etc. - -## Install -There are a few different ways to begin using the app, depending on your preferences: - -___ - -### [Heroku Quick Deploy](https://heroku.com/about) -[![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy?template=https://github.com/benbusby/whoogle-search/tree/main) - -Provides: -- Easy Deployment of App -- A HTTPS url (https://\.herokuapp.com) - -Notes: -- Requires a **PAID** Heroku Account. -- Sometimes has issues with auto-redirecting to `https`. Make sure to navigate to the `https` version of your app before adding as a default search engine. - -___ - -### [Render](https://render.com) - -Create an account on [render.com](https://render.com) and import the Whoogle repo with the following settings: - -- Runtime: `Python 3` -- Build Command: `pip install -r requirements.txt` -- Run Command: `./run` - -___ - -### [Repl.it](https://repl.it) -[![Run on Repl.it](https://repl.it/badge/github/benbusby/whoogle-search)](https://repl.it/github/benbusby/whoogle-search) - -*Note: Requires a (free) Replit account* - -Provides: -- Free deployment of app -- Free HTTPS url (https://\.\\.repl\.co) - - Supports custom domains -- Downtime after periods of inactivity ([solution](https://repl.it/talk/learn/How-to-use-and-setup-UptimeRobot/9003)\) - -___ - -### [Fly.io](https://fly.io) - -You will need a [Fly.io](https://fly.io) account to deploy Whoogle. - -#### Install the CLI: https://fly.io/docs/hands-on/installing/ - -#### Deploy the app - -```bash -flyctl auth login -flyctl launch --image benbusby/whoogle-search:latest -``` - -The first deploy won't succeed because the default `internal_port` is wrong. -To fix this, open the generated `fly.toml` file, set `services.internal_port` to `5000` and run `flyctl launch` again. - -Your app is now available at `https://.fly.dev`. - -Notes: -- Requires a [**PAID**](https://fly.io/docs/about/pricing/#free-allowances) Fly.io Account. - -___ - -### [Koyeb](https://www.koyeb.com) - -Use one of the following guides to install Whoogle on Koyeb: - -1. Using GitHub: https://www.koyeb.com/docs/quickstart/deploy-with-git -2. Using Docker: https://www.koyeb.com/docs/quickstart/deploy-a-docker-application - -___ - -### [pipx](https://github.com/pipxproject/pipx#install-pipx) -Persistent install: - -`pipx install https://github.com/benbusby/whoogle-search/archive/refs/heads/main.zip` - -Sandboxed temporary instance: - -`pipx run --spec git+https://github.com/benbusby/whoogle-search.git whoogle-search` - -___ - -### pip -`pip install whoogle-search` - -```bash -$ whoogle-search --help -usage: whoogle-search [-h] [--port ] [--host ] [--debug] [--https-only] [--userpass ] - [--proxyauth ] [--proxytype ] [--proxyloc ] - -Whoogle Search console runner - -optional arguments: - -h, --help Show this help message and exit - --port Specifies a port to run on (default 5000) - --host Specifies the host address to use (default 127.0.0.1) - --debug Activates debug mode for the server (default False) - --https-only Enforces HTTPS redirects for all requests - --userpass - Sets a username/password basic auth combo (default None) - --proxyauth - Sets a username/password for a HTTP/SOCKS proxy (default None) - --proxytype - Sets a proxy type for all connections (default None) - --proxyloc - Sets a proxy location for all connections (default None) -``` -See the [available environment variables](#environment-variables) for additional configuration. - -___ - -### Manual - -*Note: `Content-Security-Policy` headers can be sent by Whoogle if you set `WHOOGLE_CSP`.* - -#### Dependencies -- [Python3](https://www.python.org/downloads/) -- `libcurl4-openssl-dev` and `libssl-dev` - - macOS: `brew install openssl curl-openssl` - - Ubuntu: `sudo apt-get install -y libcurl4-openssl-dev libssl-dev` - - Arch: `pacman -S curl openssl` - -#### Install - -Clone the repo and run the following commands to start the app in a local-only environment: - -```bash -git clone https://github.com/benbusby/whoogle-search.git -cd whoogle-search -python3 -m venv venv -source venv/bin/activate -pip install -r requirements.txt -./run -``` -See the [available environment variables](#environment-variables) for additional configuration. - -#### systemd Configuration -After building the virtual environment, you can add something like the following to `/lib/systemd/system/whoogle.service` to set up a Whoogle Search systemd service: - -```ini -[Unit] -Description=Whoogle - -[Service] -# Basic auth configuration, uncomment to enable -#Environment=WHOOGLE_USER= -#Environment=WHOOGLE_PASS= -# Proxy configuration, uncomment to enable -#Environment=WHOOGLE_PROXY_USER= -#Environment=WHOOGLE_PROXY_PASS= -#Environment=WHOOGLE_PROXY_TYPE= -# Site alternative configurations, uncomment to enable -# Note: If not set, the feature will still be available -# with default values. -#Environment=WHOOGLE_ALT_TW=farside.link/nitter -#Environment=WHOOGLE_ALT_YT=farside.link/invidious -#Environment=WHOOGLE_ALT_RD=farside.link/libreddit -#Environment=WHOOGLE_ALT_MD=farside.link/scribe -#Environment=WHOOGLE_ALT_TL=farside.link/lingva -#Environment=WHOOGLE_ALT_IMG=farside.link/rimgo -#Environment=WHOOGLE_ALT_WIKI=farside.link/wikiless -#Environment=WHOOGLE_ALT_IMDB=farside.link/libremdb -#Environment=WHOOGLE_ALT_QUORA=farside.link/quetre -#Environment=WHOOGLE_ALT_SO=farside.link/anonymousoverflow -# Load values from dotenv only -#Environment=WHOOGLE_DOTENV=1 -# specify dotenv location if not in default location -#Environment=WHOOGLE_DOTENV_PATH=/whoogle.env -Type=simple -User= -# If installed as a package, add: -ExecStart=/python3 /whoogle-search --host 127.0.0.1 --port 5000 -# For example: -# ExecStart=/usr/bin/python3 /home/my_username/.local/bin/whoogle-search --host 127.0.0.1 --port 5000 -# Otherwise if running the app from source, add: -ExecStart=/run -# For example: -# ExecStart=/var/www/whoogle-search/run -WorkingDirectory= -ExecReload=/bin/kill -HUP $MAINPID -Restart=always -RestartSec=3 -SyslogIdentifier=whoogle - -[Install] -WantedBy=multi-user.target -``` -Then, -``` -sudo systemctl daemon-reload -sudo systemctl enable whoogle -sudo systemctl start whoogle -``` - -#### Tor Configuration *optional* -If routing your request through Tor you will need to make the following adjustments. -Due to the nature of interacting with Google through Tor we will need to be able to send signals to Tor and therefore authenticate with it. - -There are two authentication methods, password and cookie. You will need to make changes to your torrc: - * Cookie - 1. Uncomment or add the following lines in your torrc: - - `ControlPort 9051` - - `CookieAuthentication 1` - - `DataDirectoryGroupReadable 1` - - `CookieAuthFileGroupReadable 1` - - 2. Make the tor auth cookie readable: - - This is assuming that you are using a dedicated user to run whoogle. If you are using a different user replace `whoogle` with that user. - - 1. `chmod tor:whoogle /var/lib/tor` - 2. `chmod tor:whoogle /var/lib/tor/control_auth_cookie` - - 3. Restart the tor service: - - `systemctl restart tor` - - 4. Set the Tor environment variable to 1, `WHOOGLE_CONFIG_TOR`. Refer to the [Environment Variables](#environment-variables) section for more details. - - This may be added in the systemd unit file or env file `WHOOGLE_CONFIG_TOR=1` - - * Password - 1. Run this command: - - `tor --hash-password {Your Password Here}`; put your password in place of `{Your Password Here}`. - - Keep the output of this command, you will be placing it in your torrc. - - Keep the password input of this command, you will be using it later. - - 2. Uncomment or add the following lines in your torrc: - - `ControlPort 9051` - - `HashedControlPassword {Place output here}`; put the output of the previous command in place of `{Place output here}`. - - 3. Now take the password from the first step and place it in the control.conf file within the whoogle working directory, ie. [misc/tor/control.conf](misc/tor/control.conf) - - If you want to place your password file in a different location set this location with the `WHOOGLE_TOR_CONF` environment variable. Refer to the [Environment Variables](#environment-variables) section for more details. - - 4. Heavily restrict access to control.conf to only be readable by the user running whoogle: - - `chmod 400 control.conf` - - 5. Finally set the Tor environment variable and use password variable to 1, `WHOOGLE_CONFIG_TOR` and `WHOOGLE_TOR_USE_PASS`. Refer to the [Environment Variables](#environment-variables) section for more details. - - These may be added to the systemd unit file or env file: - - `WHOOGLE_CONFIG_TOR=1` - - `WHOOGLE_TOR_USE_PASS=1` - -___ - -### Manual (Docker) -1. Ensure the Docker daemon is running, and is accessible by your user account - - To add user permissions, you can execute `sudo usermod -aG docker yourusername` - - Running `docker ps` should return something besides an error. If you encounter an error saying the daemon isn't running, try `sudo systemctl start docker` (Linux) or ensure the docker tool is running (Windows/macOS). -2. Clone and deploy the docker app using a method below: - -#### Docker CLI - -Through Docker Hub: -```bash -docker pull benbusby/whoogle-search -docker run --publish 5000:5000 --detach --name whoogle-search benbusby/whoogle-search:latest -``` - -or with docker-compose: - -```bash -git clone https://github.com/benbusby/whoogle-search.git -cd whoogle-search -docker-compose up -``` - -or by building yourself: - -```bash -git clone https://github.com/benbusby/whoogle-search.git -cd whoogle-search -docker build --tag whoogle-search:1.0 . -docker run --publish 5000:5000 --detach --name whoogle-search whoogle-search:1.0 -``` - -Optionally, you can also enable some of the following environment variables to further customize your instance: - -```bash -docker run --publish 5000:5000 --detach --name whoogle-search \ - -e WHOOGLE_USER=username \ - -e WHOOGLE_PASS=password \ - -e WHOOGLE_PROXY_USER=username \ - -e WHOOGLE_PROXY_PASS=password \ - -e WHOOGLE_PROXY_TYPE=socks5 \ - -e WHOOGLE_PROXY_LOC=ip \ - whoogle-search:1.0 -``` - -And kill with: `docker rm --force whoogle-search` - -#### Using [Heroku CLI](https://devcenter.heroku.com/articles/heroku-cli) -```bash -heroku login -heroku container:login -git clone https://github.com/benbusby/whoogle-search.git -cd whoogle-search -heroku create -heroku container:push web -heroku container:release web -heroku open -``` - -This series of commands can take a while, but once you run it once, you shouldn't have to run it again. The final command, `heroku open` will launch a tab in your web browser, where you can test out Whoogle and even [set it as your primary search engine](https://github.com/benbusby/whoogle#set-whoogle-as-your-primary-search-engine). -You may also edit environment variables from your appโ€™s Settings tab in the Heroku Dashboard. - -___ - -### Arch Linux & Arch-based Distributions -There is an [AUR package available](https://aur.archlinux.org/packages/whoogle-git/), as well as a pre-built and daily updated package available at [Chaotic-AUR](https://chaotic.cx). - -___ - -### Helm chart for Kubernetes -To use the Kubernetes Helm Chart: -1. Ensure you have [Helm](https://helm.sh/docs/intro/install/) `>=3.0.0` installed -2. Clone this repository -3. Update [charts/whoogle/values.yaml](./charts/whoogle/values.yaml) as desired -4. Run `helm install whoogle ./charts/whoogle` - -___ - -#### Using your own server, or alternative container deployment -There are other methods for deploying docker containers that are well outlined in [this article](https://rollout.io/blog/the-shortlist-of-docker-hosting/), but there are too many to describe set up for each here. Generally it should be about the same amount of effort as the Heroku deployment. - -Depending on your preferences, you can also deploy the app yourself on your own infrastructure. This route would require a few extra steps: - - A server (I personally recommend [Digital Ocean](https://www.digitalocean.com/pricing/) or [Linode](https://www.linode.com/pricing/), their cheapest tiers will work fine) - - Your own URL (I suppose this is optional, but recommended) - - SSL certificates (free through [Let's Encrypt](https://letsencrypt.org/getting-started/)) - - A bit more experience or willingness to work through issues - -## Environment Variables -There are a few optional environment variables available for customizing a Whoogle instance. These can be set manually, or copied into `whoogle.env` and enabled for your preferred deployment method: - -- Local runs: Set `WHOOGLE_DOTENV=1` before running -- With `docker-compose`: Uncomment the `env_file` option -- With `docker build/run`: Add `--env-file ./whoogle.env` to your command - -| Variable | Description | -| -------------------- | ----------------------------------------------------------------------------------------- | -| WHOOGLE_URL_PREFIX | The URL prefix to use for the whoogle instance (i.e. "/whoogle") | -| WHOOGLE_DOTENV | Load environment variables in `whoogle.env` | -| WHOOGLE_DOTENV_PATH | The path to `whoogle.env` if not in default location | -| WHOOGLE_USER | The username for basic auth. WHOOGLE_PASS must also be set if used. | -| WHOOGLE_PASS | The password for basic auth. WHOOGLE_USER must also be set if used. | -| WHOOGLE_PROXY_USER | The username of the proxy server. | -| WHOOGLE_PROXY_PASS | The password of the proxy server. | -| WHOOGLE_PROXY_TYPE | The type of the proxy server. Can be "socks5", "socks4", or "http". | -| WHOOGLE_PROXY_LOC | The location of the proxy server (host or ip). | -| WHOOGLE_USER_AGENT | The desktop user agent to use. Defaults to a randomly generated one. | -| WHOOGLE_USER_AGENT_MOBILE | The mobile user agent to use. Defaults to a randomly generated one. | -| WHOOGLE_USE_CLIENT_USER_AGENT | Enable to use your own user agent for all requests. Defaults to false. | -| WHOOGLE_REDIRECTS | Specify sites that should be redirected elsewhere. See [custom redirecting](#custom-redirecting). | -| EXPOSE_PORT | The port where Whoogle will be exposed. | -| HTTPS_ONLY | Enforce HTTPS. (See [here](https://github.com/benbusby/whoogle-search#https-enforcement)) | -| WHOOGLE_ALT_TW | The twitter.com alternative to use when site alternatives are enabled in the config. Set to "" to disable. | -| WHOOGLE_ALT_YT | The youtube.com alternative to use when site alternatives are enabled in the config. Set to "" to disable. | -| WHOOGLE_ALT_RD | The reddit.com alternative to use when site alternatives are enabled in the config. Set to "" to disable. | -| WHOOGLE_ALT_TL | The Google Translate alternative to use. This is used for all "translate ____" searches. Set to "" to disable. | -| WHOOGLE_ALT_MD | The medium.com alternative to use when site alternatives are enabled in the config. Set to "" to disable. | -| WHOOGLE_ALT_IMG | The imgur.com alternative to use when site alternatives are enabled in the config. Set to "" to disable. | -| WHOOGLE_ALT_WIKI | The wikipedia.org alternative to use when site alternatives are enabled in the config. Set to "" to disable. | -| WHOOGLE_ALT_IMDB | The imdb.com alternative to use when site alternatives are enabled in the config. Set to "" to disable. | -| WHOOGLE_ALT_QUORA | The quora.com alternative to use when site alternatives are enabled in the config. Set to "" to disable. | -| WHOOGLE_ALT_SO | The stackoverflow.com alternative to use when site alternatives are enabled in the config. Set to "" to disable. | -| WHOOGLE_AUTOCOMPLETE | Controls visibility of autocomplete/search suggestions. Default on -- use '0' to disable. | -| WHOOGLE_MINIMAL | Remove everything except basic result cards from all search queries. | -| WHOOGLE_CSP | Sets a default set of 'Content-Security-Policy' headers | -| WHOOGLE_RESULTS_PER_PAGE | Set the number of results per page | -| WHOOGLE_TOR_SERVICE | Enable/disable the Tor service on startup. Default on -- use '0' to disable. | -| WHOOGLE_TOR_USE_PASS | Use password authentication for tor control port. | -| WHOOGLE_TOR_CONF | The absolute path to the config file containing the password for the tor control port. Default: ./misc/tor/control.conf WHOOGLE_TOR_PASS must be 1 for this to work.| -| WHOOGLE_SHOW_FAVICONS | Show/hide favicons next to search result URLs. Default on. | -| WHOOGLE_UPDATE_CHECK | Enable/disable the automatic daily check for new versions of Whoogle. Default on. | -| WHOOGLE_FALLBACK_ENGINE_URL | Set a fallback Search Engine URL when there is internal server error or instance is rate-limited. Search query is appended to the end of the URL (eg. https://duckduckgo.com/?k1=-1&q=). | - -### Config Environment Variables -These environment variables allow setting default config values, but can be overwritten manually by using the home page config menu. These allow a shortcut for destroying/rebuilding an instance to the same config state every time. - -| Variable | Description | -| ------------------------------------ | --------------------------------------------------------------- | -| WHOOGLE_CONFIG_DISABLE | Hide config from UI and disallow changes to config by client | -| WHOOGLE_CONFIG_COUNTRY | Filter results by hosting country | -| WHOOGLE_CONFIG_LANGUAGE | Set interface language | -| WHOOGLE_CONFIG_SEARCH_LANGUAGE | Set search result language | -| WHOOGLE_CONFIG_BLOCK | Block websites from search results (use comma-separated list) | -| WHOOGLE_CONFIG_BLOCK_TITLE | Block search result with a REGEX filter on title | -| WHOOGLE_CONFIG_BLOCK_URL | Block search result with a REGEX filter on URL | -| WHOOGLE_CONFIG_THEME | Set theme mode (light, dark, or system) | -| WHOOGLE_CONFIG_SAFE | Enable safe searches | -| WHOOGLE_CONFIG_ALTS | Use social media site alternatives (nitter, invidious, etc) | -| WHOOGLE_CONFIG_NEAR | Restrict results to only those near a particular city | -| WHOOGLE_CONFIG_TOR | Use Tor routing (if available) | -| WHOOGLE_CONFIG_NEW_TAB | Always open results in new tab | -| WHOOGLE_CONFIG_VIEW_IMAGE | Enable View Image option | -| WHOOGLE_CONFIG_GET_ONLY | Search using GET requests only | -| WHOOGLE_CONFIG_URL | The root url of the instance (`https:///`) | -| WHOOGLE_CONFIG_STYLE | The custom CSS to use for styling (should be single line) | -| WHOOGLE_CONFIG_PREFERENCES_ENCRYPTED | Encrypt preferences token, requires preferences key | -| WHOOGLE_CONFIG_PREFERENCES_KEY | Key to encrypt preferences in URL (REQUIRED to show url) | -| WHOOGLE_CONFIG_ANON_VIEW | Include the "anonymous view" option for each search result | - -## Usage -Same as most search engines, with the exception of filtering by time range. - -To filter by a range of time, append ":past