完整安装datax-web教程

1.安装mysql5.7

a.创建目录下载安装rpm包

mkdir -p /opt/software & cd /opt/software/  & wget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm & du -sh mysql57-community-release-el7-10.noarch.rpm & sudo yum -y install mysql57-community-release-el7-10.noarch.rpm

b.安装mysql

sudo yum -y install mysql-community-server （提示输入 y即可）

可能会报错：Public key for mysql-community-server-5.7.37-1.el7.x86_64.rpm is not installed

是Mysql的GPG升级了，重新安装即可

rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022

最后启动mysql并查看状态

systemctl start mysqld &  systemctl enable mysqld &systemctl status mysqld

查看状态的时候可能会报错mysqld.service holdoff time over, scheduling restart.之类的错误

直接忽略他，重启你的服务器，reboot。然后重新执行启动命令，就可以搞定。

重要：mysql设置免密登录，要不然后面的datax-web admin默认配置启动不了

vim /etc/my.cnf  

 在[mysqld]后面任意一行添加“skip-grant-tables”用来跳过密码验证的过程。

 
[mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
skip-grant-tables
 

service mysqld restart 

c.进入mysql修改密码

获取初始密码

cat /var/log/mysqld.log | grep password| grep "root@localhost"

进入mysql：mysql -u root -pxxxx

修改密码：ALTER USER 'root'@'localhost' IDENTIFIED BY '密码为数字、小写字母、大写字母 、特殊字符、长度至少8位';

d.修改远程登陆权限

use mysql & update user set host = '%' where user = 'root';

刷新生效 flush privileges;

mysql搞定！！！

2.安装jdk1.8（自己确保原先jdk卸载干净，最好是没装过jdk）

yum install java-1.8.0-openjdk* -y &  java -version

3.安装maven3.6.3

a.下包：

 wget https://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz

b.解压并添加环境变量

 tar -zxvf apache-maven-3.6.3-bin.tar.gz -C /usr/local/

vi  /etc/profile

#文件尾部追加 Maven
MAVEN_HOME=/usr/local/apache-maven-3.6.3
export PATH=${MAVEN_HOME}/bin:${PATH}

source /etc/profile

mvn -v  (验证是否成功)

c.配置maven国内镜像、指定jdk编译版本。进入maven根目录/conf，编辑文件settings.xml,替换为以下内容

[root@iZbp1ax83amkj727yqg272Z conf]# cat settings.xml 
"1.0" encoding="UTF-8"?>
 
 
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  
 
  
 
  
 
  
  <pluginGroups>
    
  pluginGroups>
 
  
  <proxies>
    
  proxies>
 
  
  <servers>
    
 
    
  servers>
 
  
  <mirrors>
    
  mirrors>
 
  
  <profiles>
    
 
    
  profiles>
 
  
settings>

4.安装python2.7

centos7.9阿里云自带的,应该是 yum install python2

5.安装datax

wget http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz & tar -zxvf  datax.tar.gz -C /opt/datax & rm -rf /opt/datax/plugin/*/._* 

检查是否可用

 python2 /opt/datax/bin/datax.py /opt/datax/job/job.json
 

6.安装datax web

a.下载地址：https://pan.baidu.com/s/13yoqhGpD00I82K4lOYtQhg 提取码：cpsk

b.tar -xvf datax-web-2.1.2.tar  -C  /opt & mv datax-web-2.1.2 datax-web

c.sh  /opt/datax-web/bin/install.sh --force

  按提示输入mysql连接基本信息

d.vim  /opt/datax-web/modules/datax-executor/conf/application.yml

   修改正确ip和pypath（为datax的python脚本执行路径，非python2.7路径）

e.修改文件/opt/datax-web/modules/datax-executor/bin/datax-executor.sh

将-Dpython.path的值，修改为datax python执行路径，/opt/datax/bin/datax.py

f.启动服务  sh /opt/datax-web/bin/start-all.sh

e.打开 https://ip:9999

注意：datax-web启动有点消耗内存，会启动执行器失败，报错here is insufficient memory for the Java Runtime Environment to continue.；这时候就需要掏钱扩容了。

囊中羞涩办法：调整jvm默认运行内存。

/opt/datax-web/modules/datax-executor/bin/datax-executor.sh文件的104行

JAVA_OPTS=" -Xms2g -Xmx2g -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8"

修改成：（勉强能用，内存大小按自己实际情况调整）

JAVA_OPTS=" -Xms600m -Xmx600m -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8"

打开：http://11.17.23.18:9527/index.html

admin  123456