OpenDistro Multi-node ElasticSearch Cluster

Why OpenDistro?

ElasticSearch started as a fully open source project, but over time (and popularity) started putting features behind a proprietary paywall. Hence OpenDistro. You can read more about this here:

Multi-Node Cluster Configuration Walkthrough

We’re going with a multi-node setup because by distributing the documents in an index across multiple shards and distributing those shards across multiple nodes, Elasticsearch can ensure redundancy, which both protects against hardware failures and increases query capacity as nodes are added to a cluster. As the cluster grows (or shrinks), Elasticsearch automatically migrates shards to rebalance the cluster

  1. Clone repo: https://github.com/kave/opendistro-elasticsearch
cd certs
./ssl_creation.sh
macOS Keychain
  • Open the macOS Keychain app
  • Go to File > Import Items..
  • Select your root certificate file (i.e. root.pem)
Trust Root CA
  • Double click on your root certificate in the list
  • Expand the Trust section
  • Change the When using this certificate: select box to Always Trust
docker-compose --compatibility up
Goldblum, Goldbluming
curl 'https://admin:admin@local.es:9200/_cluster/health?pretty' --cacert $PWD/certs/root-ca.pem

Tips for Production

  • Set Xmx and Xms of your Java heap space to no more than 50% of your physical RAM.
ES_JAVA_OPTS=-Xms1g -Xmx1g
deploy:
resources:
limits:
memory: 2G
  • Ensure ES can create the necessary amount of threads. This can be done by setting ulimit -n 4096 as root before starting Elasticsearch, or by setting nproc -n 4096 in /etc/security/limits.conf.
<app_user>         soft       nproc          4096<app_user>         hard       nproc          4096
  • Increate VM File Descriptors. set ulimit -n 65535 as root before starting Elasticsearch, or set nofile to 65535 in /etc/security/limits.conf.
<app_user>         soft       nofile          1024<app_user>         hard       nofile          65535
  • If you are bind-mounting a local directory or file, it must be readable by the elasticsearch user. Also, this user must have write access to the data and log dirs. A good strategy is to grant group access to gid 0 for the local directory. For example, to prepare a local directory for storing data through a bind-mount:
mkdir esdatadir
chmod g+rwx esdatadir
chgrp 0 esdatadir
  • If possible you want to have your nodes on different machines, but on the same network to improve redundancy and reduce the blast radius of a disaster.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Emmanuel Apau

Emmanuel Apau

Site Reliability Engineer with 9 years of experience developing innovative and valuable solutions for clients with Python/Go