安装CDH

cdh-6.3.2

服务器环境 centos7.9

name ip
master.hadoop

192.168.10.187
node1.hadoop 192.168.10.188
node2.hadoop 192.168.10.189
node3.hadoop 192.168.10.190

第一步

注意把dns解析做好

或者更改etc文件把所有的服务器的/etc/hosts文件增加手工解析

第二步

关闭selinux 和防火墙,每台机器上都要做

systemctl stop firewalld
systemctl disable firewalld
sed -i '/SELINUX/s/enforcing/disabled/g' /etc/selinux/config
cat /etc/selinux/config

第三步

做免密
在master机器上生成密钥，通过ssh-copy-id拷贝到四台机器

[root@master ~]#ssh_keygen 

一直回车

[root@master ~]# ssh-copy-id node{1-3}.hadoop
第四步 安装时间同步

yum -y install ntpdate
ntpdate cn.pool.ntp.org

第五步 安装jdk

yum -y install java java-devel

第六步，使用swap

sysctl -w vm.swappiness=0

echo ‘vm.swappiness=0’ >> /etc/sysctl.conf

sysctl -p

第7步 ，在master机器上安装mysql5.7

[root@master ~]# wget https://dev.mysql.com/get/mysql80-community-release-el7-3.noarch.rpm
--2022-12-31 20:40:10--  https://dev.mysql.com/get/mysql80-community-release-el7-3.noarch.rpm
Resolving dev.mysql.com (dev.mysql.com)... 23.63.37.189, 2600:140b:2:99d::2e31, 2600:140b:2:99c::2e31
Connecting to dev.mysql.com (dev.mysql.com)|23.63.37.189|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://repo.mysql.com//mysql80-community-release-el7-3.noarch.rpm [following]
--2022-12-31 20:40:11--  https://repo.mysql.com//mysql80-community-release-el7-3.noarch.rpm
Resolving repo.mysql.com (repo.mysql.com)... 23.39.0.221
Connecting to repo.mysql.com (repo.mysql.com)|23.39.0.221|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 26024 (25K) [application/x-redhat-package-manager]
Saving to: ‘mysql80-community-release-el7-3.noarch.rpm.1’

100%[===============================================================================================>] 26,024       162KB/s   in 0.2s

[root@master ~]# rpm -ivh mysql80-community-release-el7-3.noarch.rpm
warning: mysql80-community-release-el7-3.noarch.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:mysql80-community-release-el7-3  ################################# [100%]
[root@master ~]#

让5.7 mysql生效

vi /etc/yum.repos.d/mysql-community.repo

# Enable to use MySQL 5.7
[mysql57-community]
name=MySQL 5.7 Community Server
baseurl=http://repo.mysql.com/yum/mysql-5.7-community/el/7/$basearch/
enabled=1 # 改成1 
gpgcheck=0 # 改成0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql

[mysql80-community]
name=MySQL 8.0 Community Server
baseurl=http://repo.mysql.com/yum/mysql-8.0-community/el/7/$basearch/

enabled=0 改成0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql

然后安装mysql

[root@master ~]#yum -y install mysql-community-server
....

[root@master ~]#systemctl start mysqld
[root@master ~]#systemctl enable mysqld


[root@master ~]# cat /var/log/mysqld.log |grep password
2022-12-31T12:48:20.149714Z 1 [Note] A temporary password is generated for root@localhost: 2p#GJBWtg1g4

正常安装完毕，初始化cdh所需要的账户名和密码.

mysql5.7/8.0 需要修改初始密码才能执行创建用户

mysql> alter user user() identified by 'Root@123';
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql>
执行一下脚本，把下面脚本保存成jiaoben.sql

create database scm DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database rman DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database metastore DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database sentry DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database nav DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database navms DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

grant all on scm.* to 'scm'@'%' identified by 'Root@123';
grant all on hive.* to 'hive'@'%' identified by 'Root@123';

grant all on amon.* to 'amon'@'%' identified by 'Root@123';

grant all on rman.* to 'rman'@'%' identified by 'Root@123';

grant all on hue.* to 'hue'@'%' identified by 'Root@123';

grant all on metastore.* to 'hive'@'%' identified by 'Root@123';

grant all on sentry.* to 'sentry'@'%' identified by 'Root@123';

grant all on nav.* to 'nav'@'%' identified by 'Root@123';

grant all on navms .* to navms @'%' identified by 'Root@123';

grant all on oozie.* to 'oozie'@'%' identified by 'Root@123';

flush privileges;

修改密码并执行jiaoben.sql

[root@master ~]# mysql -uroot -p"2p#GJBWtg1g4"
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 6
Server version: 5.7.40

Copyright (c) 2000, 2022, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> alter user user() identified by 'Root@123';
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye
[root@master ~]# mysql -uroot -p"Root@123"   <
jiaoben.sql

第八步 建立仓库

建立yum源方便安装cdh。

建立仓库的主要代码是创建nginx一个虚拟网站，然后创建仓库。

mkdir -p /www/wwwroot/yum.in.io/cm6
将下载的cm包文件移到此目录下: /www/wwwroot/yum.in.io/cm6

    包含这些文件：

cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
oracle-j2sdk1.8-1.8.0update181-1.x86_64.rpm

    cd /www/wwwroot/yum.in.io/cm6

创建仓库： createrepo .

如无此命令： yum install createrepo 后再执行 createrepo .

成功后会出现：repodata文件夹


在master 上创建仓库文件

tee < /etc/yum.repos.d/cloudera-manager.repo
[cloudera-manager]
name=cloudera-manager
baseurl=http://yum.in.io/cm6/
enabled=1
gpgcheck=0
EOF

在master上执行 

yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

把以下文件 移动到 /opt/cloudera/parcel-repo

CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256

初始化mysql脚本

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm Root@123

发现没有mysql驱动，下载mysql驱动,再执行代码完成。

[root@master parcel-repo]# cp -i  ~/mysql-connector-java-5.1.49-bin.jar /opt/cloudera/cm/lib/
[root@master parcel-repo]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm Root@123                                        JAVA_HOME=/usr/lib/jvm/java-openjdk
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/lib/jvm/java-openjdk/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Sat Dec 31 21:22:32 CST 2022 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!
[root@master parcel-repo]#

注意，这个驱动还要复制到
/usr/share/java 下面，每台机器都需要
而且必须把名字改成 mysql-connector-java.jar,因为我发现log日志中文件名称是写死的。

至此进入模块安装就完成了

hive问题

如何hive起不来，需要在服务器上执行
sudo -u hdfs hdfs dfs -mkdir /tmp
如果/tmp已经存在
sudo -u hdfs hdfs dfs -chmod -R 777 /tmp
因为 hdfs用户有权限对数据仓库操作。