My Blog
Rook Ceph Deployment
2019-02-23 10:35:23    1    0    0

Rook is an open source cloud-native storage orchestrator, providing the platform, framework, and support for a diverse set of storage solutions to natively integrate with cloud-native environments.

Rook turns storage software into self-managing, self-scaling, and self-healing storage services. It does this by automating deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management. Rook uses the facilities provided by the underlying cloud-native container management, scheduling and orchestration platform to perform its duties.

This guide will walk you through the basic setup of a Ceph cluster and enable you to consume block, object, and file storage from other pods running in your cluster.

Kubernetes v1.8 or higher is supported by Rook.

TL;DR -> (Too Long, Donot Read)

Deployment rook ceph

fetch source

  1. git clone https://github.com/rook/rook
kubernetes with stacked etcd 部署
2019-02-23 06:32:44    3    0    0

preapre

disable firewall

  1. systemctl stop firewalld
  2. systemctl disable firewalld

disable swap

  1. swapoff -a

config hostname ensure local hostname reachable

  1. vi /etc/hosts
  3. 139.162.127.39  node1
  4. 139.162.121.213   node2
  5. 139.162.97.24  node3
  6. 139.162.127.39  apiserver.example.com

Install docker

  1. # Install Docker from CentOS/RHEL repository:
  2. yum install -y docker
  4. ----------------------------
  5. # or install Docker CE 18.06 from Docker's CentOS repositories:
  7. ## Install prerequisites.
  8. yum install yum-utils device-mapper-persistent-data lvm2
  10. ## Add docker repository.
  11. yum-config-manager \
  12.     --add-repo \
  13.     https://download.docker.com/linux/centos/docker-ce.repo
  15. ## Install docker.
  16. yum update && yum install docker-ce-18.06.1.ce
  18. ## Create /etc/docker directory.
  19. mkdir /etc/docker
  20. mkdir /docker
  22. # Setup daemon.
  23. cat > /etc/docker/daemon.json <<EOF
  24. {
  25.   "log-driver": "json-file",
  26.   "log-opts": {
  27.     "max-size": "100m"
  28.   },
  29.   "storage-opts": [
  30.     "overlay2.override_kernel_
Kubernetes部署
2018-12-06 22:27:14    1    0    0

preapre

  1. swapoff -a

Install docker

  1. # Install Docker from CentOS/RHEL repository:
  2. yum install -y docker
  4. ----------------------------
  5. # or install Docker CE 18.06 from Docker's CentOS repositories:
  7. ## Install prerequisites.
  8. yum install yum-utils device-mapper-persistent-data lvm2
  10. ## Add docker repository.
  11. yum-config-manager \
  12.     --add-repo \
  13.     https://download.docker.com/linux/centos/docker-ce.repo
  15. ## Install docker.
  16. yum update && yum install docker-ce-18.06.1.ce
  18. ## Create /etc/docker directory.
  19. mkdir /etc/docker
  21. # Setup daemon.
  22. cat > /etc/docker/daemon.json <<EOF
  23. {
  24.   "log-driver": "json-file",
  25.   "log-opts": {
  26.     "max-size": "100m"
  27.   },
  28.   "storage-opts": [
  29.     "overlay2.override_kernel_check=true"
  30.   ],
  31.   "bip": "172.17.0.1/24"
  32. }
  33. EOF
  35. // "bip": "192.168.35.1/24"
  36. // ip link del docker0 
  37. mkdir -p /etc/systemd/system/docker.service.d
  39. # Restart docker.
  40. systemctl daemon-reload
  41. systemctl restart docker

Install kubeadm, kubelet and kubectl

Some users on RHE

Flink安装文档
2018-10-16 10:01:57    6    0    0

环境准备

配置ssh互信

在flink各个服务器上执行如下命令，配置ssh免密码登录

  1. ssh-keygen -t rsa
  2. ssh-copy-id -i .ssh/id_rsa.pub flink-01
  3. ssh-copy-id -i .ssh/id_rsa.pub flink-02
  4. ssh-copy-id -i .ssh/id_rsa.pub flink-03

配置java环境

下载并安装jdk8

  1. wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" https://download.oracle.com/otn-pub/java/jdk/8u191-b12/2787e4a523244c269598db4e85c51e0c/jdk-8u191-linux-i586.tar.gz
  2. tar zxvf jdk-8u191-linux-i586.tar.gz 
  3. mv jdk1.8.0_191 /opt

配置java环境变量

  1. echo 'export JAVA_HOME=/opt/jdk1.8.0_181' >> ~/.bashrc
  2. echo 'export PATH=$PATH:$JAVA_HOME/bin' >> ~/.bashrc
  3. source ~/.bashrc

zookeeper集群搭建

flink的jobmanager借助zookeeper实现HA，因此需要先搭建一套zookeeper集群。

下载安装zookeeper

  1. wget http://apache.website-solution.net/zookeeper/zookeeper-3.4.10/zookeeper-3.4.10.tar.gz
  2. tar zxvf zookeeper-3.4.10.tar.gz

修改zookeeper配置文件

  1. vi config/zoo.cfg
  3. dataDir=/var/zookeeper/
  4. clientPort=2181
  5. server.1=zoo-01:2888:3888
  6. server.2=zoo-0
Flink kubernates部署
2018-12-06 23:15:26    3    0    0

Build The Flink Docker Image

Dockerfile

  1. FROM java:8-jre-alpine
  3. # Install requirements
  4. RUN apk add --no-cache bash snappy
  6. # Flink environment variables
  7. ENV FLINK_INSTALL_PATH=/opt
  8. ENV FLINK_HOME $FLINK_INSTALL_PATH/flink
  9.     ENV FLINK_LIB_DIR $FLINK_HOME/lib
  10. ENV PATH $PATH:$FLINK_HOME/bin
  12. # flink-dist can point to a directory or a tarball on the local system
  13. ARG flink_dist=NOT_SET
  15. # Install build dependencies and flink
  16. ADD $flink_dist $FLINK_INSTALL_PATH
  18. RUN set -x && \
  19.   ln -s $FLINK_INSTALL_PATH/flink-* $FLINK_HOME && \
  20.   ln -s $FLINK_INSTALL_PATH/job.jar $FLINK_LIB_DIR && \
  21.   addgroup -S flink && adduser -D -S -H -G flink -h $FLINK_HOME flink && \
  22.       chown -R flink:flink $FLINK_INSTALL_PATH/flink-* && \
  23.   chown -h flink:flink $FLINK_HOME
  25. COPY docker-entrypoint.sh /
  27. USER flink
  28. EXPOSE 8081 6123
  29. ENTRYPOINT ["/docker-entrypoint.sh"]
  30. CMD ["help"]
  1. FROM centos:centos7.5.1804
  3. # Install requirements
  4. RUN yum install -y snappy wget telnet unzip cu
Nifi+OpenLDAP安装配置
2018-11-13 10:43:49    7    0    0

Build openLDAP

You can obtain a copy of the software by following the instructions on the OpenLDAP Software download page (http://www.openldap.org/software/download/). It is recommended that new users start with the latest release. 

  1. gunzip -c openldap-VERSION.tgz | tar xvfB -
  3. ./configure --prefix=/home/mingjue/openldap2446 
  4. make depend 
  5. make
  6. make install
  7. make test
  9. mkdir /home/mingjue/openldap2446/openldap-data

Edit the configuration file.

Use your favorite editor to edit the provided slapd.ldif example (usually installed as /home/mingjue/openldap2446/etc/openldap/slapd.ldif) to contain a MDB database definition of the form:

  1. dn: olcDatabase=mdb,cn=config 
  2. objectClass: olcDatabaseConfig 
  3. objectClass: olcMdbConfig 
  4. olcDatabase: mdb 
  5. OlcDbMaxSize: 1073741824 
  6. olcSuffix: dc=mingjue,dc=com
  7. olcRootDN: cn=Manager,dc=mingjue,dc=com 
  8. olcRootPW: secret 
  9. olcDbDirectory: /home/mingjue/openldap2446/openldap-data 
  10. olcDbIndex: objectClass eq

Impor

kubernetes部署
2018-09-24 10:27:59    9    0    0

主机规划

主机节点

172.105.197.104 k-master
103.29.68.48 k-node01
172.104.116.143 k-node02

设置hostname

hostnamectl set-hostname k-master

配置hosts文件

vi /etc/hosts

master节点安装

关闭防火墙

  1. systemctl stop firewalld
  2. systemctl disable firewalld

安装kubernetes和etcd

  1. yum install etcd kubernetes -y
  2. yum install *rhsm*

安装flannel

  1. yum install flannel -y

配置master节点

  1. vi /etc/etcd/etcd.conf
  2. :1,$s/localhost/0.0.0.0/g
  4. vi /etc/sysconfig/docker
  5. OPTIONS='--selinux-enabled=false --insecure-registry gcr.io --log-driver=journald --signature-verification=false'
  7. vi /etc/kubernetes/apiserver
  8. #去掉ServiceAccount
  9. KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
  10. KUBE_ETCD_SERVERS="--etcd-servers=http://k-master:2379"
  11. KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
  12. KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"
  13. KUBE_API_ARGS="--insecure-port=8080 -
Nginx安装配置
环境搭建    2017-05-23 03:31:43    64    0    0

1. 下载解压nginx，pcre，zlib

    wget http://nginx.org/download/nginx-1.11.2.tar.gz
    tar zxvf nginx-1.11.2.tar.gz 
    wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.37.tar.bz2
    tar jxvf pcre-8.37.tar.bz2 
    wget http://zlib.net/zlib-1.2.8.tar.gz
    tar zxvf zlib-1.2.8.tar.gz ​

2. 添加nginx水印支持

    Nginx-image-filter-watermark

    wget https://codeload.github.com/intaro/nginx-image-filter-watermark/zip/master
    unzip master 
    cd nginx-image-filter-watermark-master/
    cp ngx_http_image_filter_module.c ../nginx-1.11.2/src/http/modules/

3. 编译安装

cd nginx-1.11.2
sudo yum install openssl openssl-devel gd gd-devel
./configure --prefix=/root/programs/nginx --with-http_ssl_module --with-pcre=../pcre-8.37 --with-zlib=../zlib-1.2.8 --with-http_image_filter_module
make
make install​

4.配置文件

user root;
worker_processes 20;

http {
 include mime.types;
 default_type application/octet-stream;
 
 client_max_body_size 10m;
 
 server {
        listen 10000;
        server_name www.xx
PostgreSQL 10 - pgpool II
2017-10-15 23:05:37    736    0    0

Install PostgreSQL

Install from repository

Install the repository RPM

  1. yum install https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-centos10-10-1.noarch.rpm

Install the client packages:

  1. yum install postgresql10

Optionally install the server packages:

  1. yum install postgresql10-server

Optionally initialize the database and enable automatic start:

  1. /usr/pgsql-10/bin/postgresql-10-setup initdb
  2. systemctl enable postgresql-10
  3. systemctl start postgresql-10
  5. su - postgres
  6. psql
  7. \password postgres
  8. xxxpgxxx

Install from source

  1. wget https://ftp.postgresql.org/pub/source/v10.0/postgresql-10.0.tar.gz
  2. tar zxvf postgresql-10.0.tar.gz
  3. cd postgresql-10.0
  5. export PYTHON=/usr/bin/python3
  6. ./configure --prefix=/usr/local/postgres10 --with-python --with-openssl  --with-libxml --with-ldap --with-libxslt --enable-thread-safety
  7. make
  8. make install
  10. mkdir /home/pgdata/pg10/pgdata
  11. ./initdb -D /home/pgdata/pg10/pgdata
  13. psql
  14. \pass
Ignite安装
2018-07-22 05:33:49    10    0    0

Install ignite

To get started with Apache Ignite:

  • Download Apache Ignite as a ZIP archive from https://ignite.apache.org/
  • Unzip the ZIP archive into the installation folder in your system
  • (Optional) Set IGNITE_HOME environment variable to point to the installation folder and make sure there is no trailing / in the path

To start a cluster node with the default configuration, open the command shell and, assuming you are in IGNITE_HOME (Ignite installation folder), just type this:

  1. $ bin/ignite.sh

By default ignite.sh starts a node with the default configuration which is config/default-config.xml.

If you want to use a custom configuration file, then pass it as a parameter to ignite.sh/bat as follows:

  1. bin/ignite.sh examples/config/example-ignite.xml

The path to the configuration file can be absolute, or relative to either IGNITE_HOME (Ignite installation folder) or META-INF folder in your classpath.

Install Ignite Web

1/7
Proudly powered by Leanote