测试集群网络拓扑
| 实例名 | IP | 域名 | 软件 |
|---|---|---|---|
| CA | 10.60.22.160 | ca.testpp.com | puppet server [nginx] |
| master1 | 10.60.22.161 | puppet-master1.testpp.com | puppet server [nginx] |
| master2 | 10.60.22.162 | puppet-master2.testpp.com | puppet server [nginx] |
| puppet-server(LB) | 10.60.22.161(测试用) | puppet-server.testpp.com | nginx |
| puppet-agent | 10.60.22.162(测试用) | puppet-agent.testpp.com | puppet |
首先排雷：puppet最大的坑，就是客户端必须配置hosts，且必须和主机名保持一致，否则证书无法正常生成
公共部分(CA、Master节点)设置yum：vi /etc/yum.repos.d/puppet.repo
需要配合公网yum，采用阿里的yum源
[puppetlabs-products]
name=Puppet Labs Products El 7 - $basearch
baseurl=http://yum.puppetlabs.com/el/7/products/$basearch
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs
enabled=1
gpgcheck=0
[puppetlabs-deps]
name=Puppet Labs Dependencies El 7 - $basearch
baseurl=http://yum.puppetlabs.com/el/7/dependencies/$basearch
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs
enabled=1
gpgcheck=0
[puppetlabs-devel]
name=Puppet Labs Devel El 7 - $basearch
baseurl=http://yum.puppetlabs.com/el/7/devel/$basearch
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs
enabled=0
gpgcheck=1
[puppetlabs-products-source]
name=Puppet Labs Products El 7 - $basearch - Source
baseurl=http://yum.puppetlabs.com/el/7/products/SRPMS
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs
failovermethod=priority
enabled=0
gpgcheck=1
[puppetlabs-deps-source]
name=Puppet Labs Source Dependencies El 7 - $basearch - Source
baseurl=http://yum.puppetlabs.com/el/7/dependencies/SRPMS
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs
enabled=0
gpgcheck=1
[puppetlabs-devel-source]
name=Puppet Labs Devel El 7 - $basearch - Source
baseurl=http://yum.puppetlabs.com/el/7/devel/SRPMS
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs
enabled=0
gpgcheck=1
yum初始化
yum clean all && yum makecache
安装软件包
yum install -y ruby-irb ruby-devel ruby ruby-shadow rubygems facter hiera libselinux-ruby ruby-augeas ruby-rgen gcc gcc-c++ curl-devel openssl-devel zlib-devel ruby-devel pcre pcre-devel make rubygem-net-ldap git puppet puppet-server -y
gem安装Passenger/Rake/Rack
gem sources --add https://gems.ruby-china.com/ --remove https://rubygems.org/
gem install rake --version=0.9.6
gem install rack --version=1.6.4
gem install passenger --version=5.3.1
wget http://rubygems.org/downloads/rake-0.9.6.gem
wget http://rubygems.org/downloads/rack-1.6.4.gem
wget http://rubygems.org/downloads/passenger-5.3.1.gem
#生成ssl证书，CA节点、master节点以及puppet-server(LB)节点
puppet cert --generate --allow-dns-alt-names testppmaster.pplive.com
现网只申请了lb节点的ssl证书，内网+公网，内网机器不建立ssl连接，所有节点共用一份证书
生成测试证书testppmaster.pplive.com
拷贝ca服务器/var/lib/puppet/ssl文件夹至master端的/var/lib/puppet/ssl文件夹下
注意：可只申请一张lb的证书，一同绑定在ca与master端的certname中，验证通过
修改/etc/puppet/puppet.conf
[main]
logdir = /var/log/puppet
rundir = /var/run/puppet
ssldir = $vardir/ssl
[master]
confdir = /etc/puppet
certname = testppmaster.pplive.com
ca = true
编辑/etc/puppet/auth.conf文件，ca节点需要同步配置
path /certificate_revocation_list/ca
auth any
method find
allow *
path /certificate_status/ca
auth any
path /certificate_status/testppmaster.pplive.com
auth any
path /certificate_status
auth any
allow *
service puppetmaster restart && chkconfig puppetmaster on
至此ca节点启动完毕，可通过netstat -natp查看8140端口开放情况
修改/etc/puppet/puppet.conf
[main]
logdir = /var/log/puppet
rundir = /var/run/puppet
ssldir = $vardir/ssl
pluginsync = true
environmentpath = $confdir/environments
default_manifest = ./manifests
basemodulepath = $confdir/modules
[master]
classfile = $vardir/classes.txt
localconfig = $vardir/localconfig
fileserverconfig = /etc/puppet/fileserver.conf
reportdir = /home/logs/puppet/reports
masterhttplog = /home/logs/puppet/masterhttp.log
reports = log
ca = false
autosign = true
autosign = /etc/puppet/autosign.conf
certname = testppmaster.pplive.com
ssl_client_verify_header = HTTP_X_CLIENT_VERIFY
ssl_client_header = HTTP_X_CLIENT_DN
启动master节点
service puppetmaster restart
使用nginx+passenger代替ruby内置服务器，扩大负载量
关闭master服务
service puppetmaster stop
安装内置nginx, 设置prefix directory为 /usr/local/nginx，如无法下载找到脚本的下载地址将https修改成http，可执行以下命令
sed -i "s#https://nginx.org/download/#http://nginx.org/download/#g" /usr/local/share/gems/gems/passenger-5.3.1/bin/passenger-install-nginx-module
安装nginx，该方式为编译安装，附加了passenger模块
/usr/local/share/gems/gems/passenger-5.3.1/bin/passenger-install-nginx-module
一路回车，需注意指定nginx安装目录为 /usr/local/nginx
设置rack，nginx工作目录权限
mkdir -p /etc/puppet/rack/public
cp /usr/share/puppet/ext/rack/config.ru /etc/puppet/rack
chown -R puppet:puppet /etc/puppet/rack
修改nginx主配置文件 /usr/local/nginx/conf/nginx.conf
user root;
worker_processes 32;
worker_rlimit_nofile 65535;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#error_log /home/logs/nginx/error.log info;
pid var/run/nginx.pid;
events {
use epoll;
worker_connections 65535;
}
http {
passenger_root /usr/local/share/gems/gems/passenger-5.3.1;
passenger_ruby /usr/bin/ruby;
passenger_max_pool_size 120;
passenger_max_requests 4000;
passenger_pool_idle_time 100;
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr $host $scheme [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" $upstream_response_time $upstream_addr $upstream_status $request_time';
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
gzip on;
gzip_min_length 1k;
gzip_comp_level 6;
gzip_buffers 4 16k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/x-javascript text/xml application/xml+rss text/javascript;
gzip_vary on;
gzip_proxied any;
client_max_body_size 100m;
client_body_buffer_size 1024k;
proxy_buffer_size 100m;
proxy_buffers 8 100m;
proxy_busy_buffers_size 100m;
proxy_temp_file_write_size 100m;
proxy_read_timeout 500;
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
include vhost/*.conf;
}
添加vhost，/usr/local/nginx/conf/vhost/puppet.conf
server{
listen 8140;
server_name puppet-master1.testpp.com 192.168.43.140;
root /etc/puppet/rack/public;
passenger_enabled on;
access_log /home/logs/nginx/puppet.access.log main;
error_log /home/logs/nginx/puppet.error.log debug;
}
编辑/etc/puppet/auth.conf文件，ca节点需要同步配置
path /certificate_revocation_list/ca
auth any
method find
allow *
path /certificate_status/ca
auth any
path /certificate_status/ppmaster.pplive.cn
auth any
path /certificate_status/puppetmaster.idc.pplive.cn
auth any
path /certificate_status
auth any
allow 10.206.10.118
配置nginx service文件, 创建/usr/lib/systemd/system/nginx.service
[Unit]
Description=A high performance web server and a reverse proxy server
After=network.target
[Service]
Type=forking
PIDFile=/usr/local/nginx/var/run/nginx.pid
PrivateDevices=yes
SyslogLevel=err
ExecStart=/usr/local/nginx/sbin/nginx -g 'pid /usr/local/nginx/var/run/nginx.pid;'
ExecReload=/usr/bin/kill -HUP $MAINPID
KillSignal=SIGQUIT
KillMode=mixed
[Install]
WantedBy=multi-user.target
启动nginx
systemctl start nginx
至此ca节点启动完毕，亦通过netstat -natp查看8140端口开放情况，需注意nginx反向代理的puppet服务并未使用ssl协议，故无法通过puppet -t测试，具体需配合后续lb的搭建部分一起食用
需单独安装nginx，然后配置反向代理服务
yum install -y nginx
拷贝CA服务器生成的用于LB的证书 testppmaster.pplive.com
wget wget http://10.60.22.160:8000/ssl.tar.gz
nginx主配置文件 /usr/local/nginx/conf/nginx.conf
user root;
worker_processes 32;
worker_rlimit_nofile 65535;
error_log /home/logs/nginx/error.log info;
events {
use epoll;
worker_connections 65535;
}
http {
#passenger_root /usr/local/share/gems/gems/passenger-5.3.1;
#passenger_ruby /usr/bin/ruby;
#passenger_max_pool_size 120;
#passenger_max_requests 4000;
#passenger_pool_idle_time 100;
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr $host $scheme [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" $upstream_response_time $upstream_addr $upstream_status $request_time';
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
puppet配置文件，新建/usr/local/nginx/conf/vhost/puppet.conf，需注意四个证书的存放路径
upstream osp_ca {
server 10.60.22.160:8140 weight=1;
}
upstream osp_master {
server 10.60.22.161:8140 weight=1;
server 10.60.22.162:8140 weight=1;
}
server {
server_name testppmaster.pplive.com;
listen 443 default;
ssl on ;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:512m;
ssl_session_timeout 20m;
ssl_certificate /var/lib/puppet/ssl/certs/puppet-server.testpp.com.pem;
ssl_certificate_key /var/lib/puppet/ssl/private_keys/puppet-server.testpp.com.pem;
ssl_crl /var/lib/puppet/ssl/ca/ca_crl.pem;
ssl_client_certificate /var/lib/puppet/ssl/certs/ca.pem;
ssl_verify_client optional;
ssl_verify_depth 1;
proxy_set_header X-SSL-Subject $ssl_client_s_dn;
proxy_set_header X-Client-DN $ssl_client_s_dn;
proxy_set_header X-Client-Verify $ssl_client_verify;
proxy_buffers 256 8k;
proxy_connect_timeout 2;
proxy_read_timeout 5;
proxy_send_timeout 15;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_headers_hash_max_size 2048;
proxy_headers_hash_bucket_size 128;
proxy_set_header Host $host;
#proxy_set_header X-Real-IP $remote_addr;
#proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 20m;
location ~* /.+/(certificate|certificate_request|certificate_revocation_list)/ {
proxy_pass http://osp_ca;
}
location / {
proxy_pass http://osp_master;
}
修改/etc/puppet/puppet.conf
[main]
logdir = /var/log/puppet
rundir = /var/run/puppet
ssldir = $vardir/ssl
listen = true
pluginsync = true
[agent]
classfile = $vardir/classes.txt
localconfig = $vardir/localconfig
runinterval = 10
ignorecache = true
user = root
group = root
listen = false
puppetdlog = /var/log/puppet/puppetd.log
report = false
config = /etc/puppet/puppet.conf
http_compression = true
server = testppmaster.pplive.com
masterport = 443
客户端开始同步内容
puppet agent --test
测试节点添加hosts，以10.60.22.16为例
10.60.22.161 testppmaster.pplive.com
在master2节点新建vhost ，testppmaster.conf
server {
server_name testppmaster.pplive.com;
listen 443 default;
ssl on ;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:512m;
ssl_session_timeout 20m;
ssl_certificate /var/lib/puppet/ssl/certs/testppmaster.pplive.com.pem;
ssl_certificate_key /var/lib/puppet/ssl/private_keys/testppmaster.pplive.com.pem;
ssl_crl /var/lib/puppet/ssl/ca/ca_crl.pem;
ssl_client_certificate /var/lib/puppet/ssl/certs/ca.pem;
ssl_verify_client optional;
ssl_verify_depth 1;
proxy_set_header X-SSL-Subject $ssl_client_s_dn;
proxy_set_header X-Client-DN $ssl_client_s_dn;
proxy_set_header X-Client-Verify $ssl_client_verify;
proxy_buffers 256 8k;
proxy_connect_timeout 2;
proxy_read_timeout 5;
proxy_send_timeout 15;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_headers_hash_max_size 2048;
proxy_headers_hash_bucket_size 128;
proxy_set_header Host $host;
client_max_body_size 20m;
location ~* /.+/(certificate|certificate_request|certificate_revocation_list)/ {
proxy_pass http://10.60.22.160:8140;
}
location / {
proxy_pass http://10.60.22.161:8140;
}
}
master端新建测试配置
- 创建module
module的目录结构是固定的，目录的结构一般如下所示：
├── files
├── manifests
└── templates
- files: 属于模块的文件
- manifests: 脚本文件
- templates：模板文件
mkdir -p /etc/puppet/modules/helloworld/{files,templates,manifests}
新建模块的init.pp文件
vi /etc/puppet/modules/helloworld/manifests/init.pp
class helloworld{
file { '/tmp/hello.txt':
owner => 'root',
group => 'root',
mode => '0440',
source => 'puppet:///modules/helloworld/hello_old.txt'
}
}
- 配置file
在 /etc/puppet/modules/helloworld/files 预先新建hello_old文件
echo 'helloworld' > /etc/puppet/modules/helloworld/files/hello_old.txt
- 配置入口
编辑入口文件 vi /etc/puppet/manifests/site.pp ，无则新建
node 'Hostname(agent端的主机名)' {
include helloworld
}
客户端进行puppet agent -t测试
欢迎分享，转载请注明来源：内存溢出
微信扫一扫
支付宝扫一扫
评论列表（0条）