CDH 05.cdh本地源搭建配置

CDH 05.cdh本地源搭建配置

1、解决安装的时候报一下警告(禁用透明页)：

echo never > /sys/kernel/mm/transparent_hugepage/defrag;
echo never > /sys/kernel/mm/transparent_hugepage/enabled;

vi /etc/rc.local # 添加如下两行，并同步该文件到其他服务器
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

 

cd ~/scripts/;
sh ./ssh_to_all_node.sh "echo never > /sys/kernel/mm/transparent_hugepage/defrag;"
sh ./ssh_to_all_node.sh "echo never > /sys/kernel/mm/transparent_hugepage/enabled;"
sh ./sync_to_all_node.sh /etc/rc.local /etc

2、配置本地Parcel存储库

mkdir -p /opt/cloudera/parcel-repo

将下载好的parcel和manifest.json文件放置到该目录

cd /opt/cloudera/parcel-repo

sha1sum CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel | awk '{ print $1 }' > CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha

3、创建新用户和修改文件权限

useradd cloudera-scm;
chown -R cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/*

4、安装rpm包

yum -y install edhat-lsb portmap mod_ssl openssl-devel python-psycopg2 MySQL-python chkconfig bind-utils libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs

cd ~/scripts/;
sh ./ssh_to_all_node.sh "yum -y install edhat-lsb portmap mod_ssl openssl-devel python-psycopg2 MySQL-python chkconfig bind-utils libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs"

5、安装cloudera-manager (在 cdh-cm-v01 服务器上执行)

yum -y install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

6、创建cdh需要数据库

mysql -uroot -prootROOT@1234

drop database if exists scm;
drop database if exists hive;
drop database if exists hue;

drop database if exists amon;
drop database if exists rman;
drop database if exists metastore;
drop database if exists sentry;
drop database if exists nav;
drop database if exists oozie;

生产：%严格控制到IP段或者具体IP地址，密码区别开
create database scm default character set utf8 default collate utf8_general_ci;
grant all on scm.* to 'scm'@'%' identified by 'rootROOT@1234';
create database hive default character set utf8 default collate utf8_general_ci;
grant all on hive.* to 'hive'@'%' identified by 'rootROOT@1234';
create database hue default character set utf8 default collate utf8_general_ci;
grant all on hue.* to 'hue'@'%' identified by 'rootROOT@1234';
flush privileges;

create database amon default character set utf8 default collate utf8_general_ci;
grant all on amon.* to 'amon'@'%' identified by 'rootROOT@1234';
create database rman default character set utf8 default collate utf8_general_ci;
grant all on rman.* to 'rman'@'%' identified by 'rootROOT@1234';
create database metastore default character set utf8 default collate utf8_general_ci;
grant all on metastore.* to 'hive'@'%' identified by 'rootROOT@1234';
create database sentry default character set utf8 default collate utf8_general_ci;
grant all on sentry.* to 'sentry'@'%' identified by 'rootROOT@1234';
create database nav default character set utf8 default collate utf8_general_ci;
grant all on nav.* to 'nav'@'%' identified by 'rootROOT@1234';
create database oozie default character set utf8 default collate utf8_general_ci;
grant all on oozie.* to 'oozie'@'%' identified by 'rootROOT@1234';
flush privileges;

7、设置cdh数据库

7.1、MySQL数据库与CDH Server在同一台主机上

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm

 

7.2、MySQL数据库与CDH Server不在同一台主机上

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h --scm-host scm scm

8、启动Cloudera Manager

systemctl enable cloudera-scm-server;
systemctl start cloudera-scm-server;
systemctl status cloudera-scm-server;

 

systemctl enable cloudera-scm-agent;
systemctl start cloudera-scm-agent;
systemctl status cloudera-scm-agent;

# 停止Cloudera Manager
systemctl stop cloudera-scm-server;
systemctl stop cloudera-scm-agent;

systemctl start cloudera-scm-server;
systemctl start cloudera-scm-agent;

systemctl status cloudera-scm-server;
systemctl status cloudera-scm-agent;

9、查看端口被监听

netstat -anlp |grep -E "7182|7180"|grep LISTEN;

10、查看 Cloudera Manager 日志(主要是看是否有其它错误)：

tail -n 400 /var/log/cloudera-scm-server/cloudera-scm-server.log
tail -n 400 /var/log/cloudera-scm-agent/cloudera-scm-agent.log

 

11、浏览器查看：默认账号密码是admin:admin

http://cdh-cm-v01.yunes.com:7180