hive数据初始化

mysql版本：8.0.31

hive版本：   3.1.3 

hive连接配置

    javax.jdo.option.ConnectionURL
    jdbc:mysql://node88:3306/hive?createDatabaseIfNotExist=true    
 
 
    javax.jdo.option.ConnectionDriverName
    com.mysql.cj.jdbc.Driver
 
 
    javax.jdo.option.ConnectionUserName
    root
 
 
    javax.jdo.option.ConnectionPassword
    123456
 

执行导入初始化

./schematool -dbType mysql -initSchema --verbose

User root is not allowed to perform this API call 

报错信息

启动

./hive --service metastore 
 

./hive --service hiveserver2

或者

nohup hivemetastore2 &
nohup hiveserver2 &
 

插入数据到hive

beeline> !connect jdbc:hive2://10.10.10.99:10000
Connecting to jdbc:hive2://10.10.10.99:10000
Enter username for jdbc:hive2://10.10.10.99:10000: root
Enter password for jdbc:hive2://10.10.10.99:10000: ******
Connected to: Apache Hive (version 3.1.3)
Driver: Hive JDBC (version 3.1.3)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://10.10.10.99:10000> use demo;
INFO  : Compiling command(queryId=root_20231123010751_48eab3cd-3c01-418e-be8a-9d4d500e4df1): use demo
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=root_20231123010751_48eab3cd-3c01-418e-be8a-9d4d500e4df1); Time taken: 0.084 seconds
INFO  : Executing command(queryId=root_20231123010751_48eab3cd-3c01-418e-be8a-9d4d500e4df1): use demo
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=root_20231123010751_48eab3cd-3c01-418e-be8a-9d4d500e4df1); Time taken: 0.015 seconds
INFO  : OK
No rows affected (0.174 seconds)
0: jdbc:hive2://10.10.10.99:10000> insert into wy values(1,'hive-hadoop-test');
INFO  : Compiling command(queryId=root_20231123010829_6e713c6f-918d-4d40-a1a2-a750d94b1d20): insert into wy values(1,'hive-hadoop-test')
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col1, type:int, comment:null), FieldSchema(name:col2, type:string, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=root_20231123010829_6e713c6f-918d-4d40-a1a2-a750d94b1d20); Time taken: 0.741 seconds
INFO  : Executing command(queryId=root_20231123010829_6e713c6f-918d-4d40-a1a2-a750d94b1d20): insert into wy values(1,'hive-hadoop-test')
WARN  : Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
INFO  : Query ID = root_20231123010829_6e713c6f-918d-4d40-a1a2-a750d94b1d20
INFO  : Total jobs = 3
INFO  : Launching Job 1 out of 3
INFO  : Starting task [Stage-1:MAPRED] in serial mode
INFO  : Number of reduce tasks determined at compile time: 1
INFO  : In order to change the average load for a reducer (in bytes):
INFO  :   set hive.exec.reducers.bytes.per.reducer=
INFO  : In order to limit the maximum number of reducers:
INFO  :   set hive.exec.reducers.max=
INFO  : In order to set a constant number of reducers:
INFO  :   set mapreduce.job.reduces=
INFO  : number of splits:1
INFO  : Submitting tokens for job: job_1700671555135_0005
INFO  : Executing with tokens: []
INFO  : The url to track the job: http://dss-service:18088/proxy/application_1700671555135_0005/
INFO  : Starting Job = job_1700671555135_0005, Tracking URL = http://dss-service:18088/proxy/application_1700671555135_0005/
INFO  : Kill Command = /cluster/hadoop3/bin/mapred job  -kill job_1700671555135_0005
INFO  : Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
INFO  : 2023-11-23 01:09:44,742 Stage-1 map = 0%,  reduce = 0%
INFO  : 2023-11-23 01:09:57,002 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 8.16 sec
INFO  : 2023-11-23 01:10:07,647 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 14.46 sec
INFO  : MapReduce Total cumulative CPU time: 14 seconds 460 msec
INFO  : Ended Job = job_1700671555135_0005
INFO  : Starting task [Stage-7:CONDITIONAL] in serial mode
INFO  : Stage-4 is selected by condition resolver.
INFO  : Stage-3 is filtered out by condition resolver.
INFO  : Stage-5 is filtered out by condition resolver.
INFO  : Starting task [Stage-4:MOVE] in serial mode
INFO  : Moving data to directory hdfs://10.10.10.99:9000/cluster/hive/warehouse/demo.db/wy/.hive-staging_hive_2023-11-23_01-08-29_068_2124975396479935317-7/-ext-10000 from hdfs://10.10.10.99:9000/cluster/hive/warehouse/demo.db/wy/.hive-staging_hive_2023-11-23_01-08-29_068_2124975396479935317-7/-ext-10002
INFO  : Starting task [Stage-0:MOVE] in serial mode
INFO  : Loading data to table demo.wy from hdfs://10.10.10.99:9000/cluster/hive/warehouse/demo.db/wy/.hive-staging_hive_2023-11-23_01-08-29_068_2124975396479935317-7/-ext-10000
INFO  : Starting task [Stage-2:STATS] in serial mode
ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.StatsTask
INFO  : MapReduce Jobs Launched: 
INFO  : Stage-Stage-1: Map: 1  Reduce: 1   Cumulative CPU: 14.46 sec   HDFS Read: 15131 HDFS Write: 247 SUCCESS
INFO  : Total MapReduce CPU Time Spent: 14 seconds 460 msec
INFO  : Completed executing command(queryId=root_20231123010829_6e713c6f-918d-4d40-a1a2-a750d94b1d20); Time taken: 101.733 seconds
 

yarn截图

问题1

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.StatsTask (state=08S01,code=1)

hive-site.xml

    hive.stats.column.autogather
    false
 
 
 
    hive.optimize.sort.dynamic.partition
    true
 

yarn-site.xml

        yarn.scheduler.minimum-allocation-mb
        4096
   
    
        yarn.nodemanager.vmem-pmem-ratio
        3.0
   

Attempt to do update or delete on table demo.wy that is not transactional (state=42000,code=10297)