docker|September 06, 2019|1 min read

How to run ElasticSearch cluster on Docker with Kibana on Linux

TL;DR

Use docker-compose to spin up a multi-node Elasticsearch cluster with Kibana. Configure discovery settings, memory limits, and network bindings for a local development cluster.

How to run ElasticSearch cluster on Docker with Kibana on Linux

Introduction

This post is about hosting ElasticSearch cluster on dockerised environment. We are also running Kibana configured on running ElasticSearch cluster.

Steps to create ElasticSearch cluster using Docker

Create docker compose file: docker-compose.yml

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.3.1
    container_name: es01
    environment:
      - node.name=es01
      - discovery.seed_hosts=es02
      - cluster.initial_master_nodes=es01,es02
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - esnet
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.3.1
    container_name: es02
    environment:
      - node.name=es02
      - discovery.seed_hosts=es01
      - cluster.initial_master_nodes=es01,es02
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata02:/usr/share/elasticsearch/data
    networks:
      - esnet
  kibana:
    image: docker.elastic.co/kibana/kibana:7.3.1
    container_name: kibana
    environment:
      SERVER_NAME: kibana.local
      ELASTICSEARCH_HOSTS: http://es01:9200
    ports:
      - 5601:5601
    networks:
      - esnet

volumes:
  esdata01:
    driver: local
  esdata02:
    driver: local

networks:
  esnet:

Run docker-compose

docker-compose up -d

Access Kibana

<host>:5601

For first few seconds, you might see message: Kibana is not ready yet Try refreshing after some time. It will open up Kibana for you.

Special Notes

You might need to do following configuration, if your container died after few seconds (see the logs of your container):

In file: /etc/sysctl.conf

Append in end:
    vm.max_map_count=262144

Check health of ElasticSearch and Kibana

For ElasticSearch:

curl http://127.0.0.1:9200/_cat/health

Output will be look like:
1567757045 08:04:05 docker-cluster green 2 2 10 5 0 0 0 0 - 100.0%

For Kibana:

Either open in browser:
<hostname>:5601/status

Or, via api:
curl http://127.0.0.1:5601/api/status 

Related Posts

How to Copy Local Docker Image to Another Host Without Repository and Load

How to Copy Local Docker Image to Another Host Without Repository and Load

Introduction Consider a scenario where you are building a docker image on your…

How to connect to a running mysql service on host from a docker container on same host

How to connect to a running mysql service on host from a docker container on same host

Introduction I have a host running mysql (not on a container). I have to run an…

Docker: unauthorized: incorrect username or password.

Docker: unauthorized: incorrect username or password.

While running docker commands with some images, I started getting error: The…

Docker image for Drupal 7, and Php extension MongoDB installed.

Docker image for Drupal 7, and Php extension MongoDB installed.

You have drupal 7 image from docker hub, and want to connect tomongo db via php…

Docker Push&#58; How to push your docker image to your organization in hub.docker.com

Docker Push&#58; How to push your docker image to your organization in hub.docker.com

Tag the image, by seeing its image id, from docker images command docker tag 04d…

How to connect Php docker container with Mongo DB docker container

How to connect Php docker container with Mongo DB docker container

Goto your command terminal. Type: This will expose port: 27017 by default. You…

Latest Posts

Staff Engineer Study Plan for MAANG Interviews — The Complete 12-Week Roadmap

Staff Engineer Study Plan for MAANG Interviews — The Complete 12-Week Roadmap

If you’re a Senior Engineer (L5) preparing for Staff (L6+) roles at MAANG…

XSS and CSRF Explained — The Complete Guide with Real Attack Examples and Defenses

XSS and CSRF Explained — The Complete Guide with Real Attack Examples and Defenses

XSS and CSRF have been in the OWASP Top 10 for over a decade. They’re among the…

OWASP Top 10 (2021) — Every Vulnerability Explained with Code

OWASP Top 10 (2021) — Every Vulnerability Explained with Code

The OWASP Top 10 is the industry standard for web application security risks. If…

HTTP Cookies Security — Everything Developers Get Wrong

HTTP Cookies Security — Everything Developers Get Wrong

Cookies are the single most important mechanism for web authentication. Every…

Format String Vulnerabilities — The Read-Write Primitive Hiding in printf()

Format String Vulnerabilities — The Read-Write Primitive Hiding in printf()

Format string vulnerabilities are unique in the exploit world. Most memory…

Buffer Overflow Attacks — How Memory Corruption Actually Works

Buffer Overflow Attacks — How Memory Corruption Actually Works

Buffer overflows are the oldest and most consequential vulnerability class in…