Es集群部署

目录

组件全家套

版本说明

主机准备

1.解压安装

2.运行环境配置

2.1修改每个节点linux系统限制

2.2 修改每个节点  linux 系统配置 

2.3 调整vm.max_map_count的大小

2.4 重启验证配置

3. 配置ES

3.1 每个节点创建ES用户，ES不能使用root启动

3.2 每个节点创建数据目录，日志目录

3.3 配置es文件目录，跨域请求,修改每个节点特殊的内容

3.4修改JVM启动参数

3.5 配置开机启动，启动服务

---

组件全家套

• LogStash数据同步组件相关
  Lostash同步Mysql数据到Elasticsearch（一）服务介绍及环境配置
  Lostash同步Mysql数据到ElasticSearch（二）logstash脚本配置和常见坑点
  Lostash同步Mysql数据到Elasticsearch（三）Elasticsearch模板与索引设置
  Lostash同步Mysql数据到Elasticsearch（四）通过kibana辅助查看同步情况
• kibana开发工具相关
  Kibana开发工具安装

版本说明

主机准备

1.解压安装

注意：Elasticsearch需要安装JDK环境，配置前请先行安装JDK环境

tar -zxvf elasticsearch-7.7.1-linux-x86_64.tar.gz -C /usr/local/

2.运行环境配置

2.1修改每个节点linux系统限制

 vim /etc/security/limits.conf 

* hard nproc 4096
* soft nproc 4096
* hard nofile 655300
* soft nofile 655300

* soft memlock unlimited
* hard memlokc unlimited

2.2 修改每个节点  linux 系统配置 

编辑文件，查找如下参数进行配置
vim /etc/systemd/system.conf

DefaultLimitNOFILE=65536
DefaultLimitNPROC=32000
DefaultLimitMEMLOCK=infinity

ulimit -a  #查看重启前系统配置

2.3 调整vm.max_map_count的大小

sysctl -a|grep vm.max_map_count  #查看当前系统配置大小，可以设置为262144 建议设置为262144,最大655360
vim /etc/sysctl.conf 

vm.max_map_count=655360

sysctl -p  #生效配置

2.4 重启验证配置

ulimit -a  #重启后查看配置是否生效
sysctl -a|grep vm.max_map_count #重启后查看配置是否生效

3. 配置ES

3.1 每个节点创建ES用户，ES不能使用root启动

3.2 每个节点创建数据目录，日志目录

3.3 配置es文件目录，跨域请求,修改每个节点特殊的内容

vi /usr/local/elasticsearch-7.7.1/config/elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application

cluster.name: es_cluster


#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1

node.name: es_znyg104
node.master: true
node.data: true

#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#

path.data: /mnt/elk/elasticsearch/data

#
# Path to log files:
#

path.logs: /mnt/elk/elasticsearch/logs

#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true

bootstrap.memory_lock: true

#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1

network.host: 0.0.0.0


#
# Set a custom port for HTTP:
#
#http.port: 9200

http.port: 9200

http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-credentials: true

#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]

discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 3s
discovery.seed_hosts: ["172.168.9.104", "172.168.9.105", "172.168.9.203", "172.168.9.204", "172.168.9.205"]


#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]

cluster.initial_master_nodes: ["es_znyg104", "es_znyg204"]

#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

transport.tcp.port: 9300

3.4修改JVM启动参数

vi /usr/local/elasticsearch-7.7.1/config/jvm.options

主要修改的是运行内存，日志文件路径等，按照自己定义的进行处理

3.5 配置开机启动，启动服务

cd /etc/init.d
vi elasticsearch

#!/bin/bash
#chkconfig: 345 63 37
#description: elasticsearch
#processname: elasticsearch-7.7.1

export ES_HOME=/usr/local/elasticsearch-7.7.1     # elasticsearch所在目录

case $1 in
        start)
                su elasticsearch< 
保存退出，赋予执行权限
 
chmod +x elasticsearch 
 
添加到开机启动任务
 
chkconfig --add elasticsearch
 
启动，重启，停止
 
service elasticsearch start/stop/restart
 
遇到could not find java in bundled jdk at /usr/local/elasticsearch-7.7.1/jdk/bin/java，查看自己的es安装目录是否是root权限，修改授权为ES
 
验证服务启动
 
ps aux|grep elasticsearch
curl 127.0.0.1:9200