Mesos Setup

# Downloading Mesos

wget http://mirror.dsrg.utoronto.ca/apache/mesos/1.3.0/mesos-1.3.0.tar.gz
    tar -zxf mesos-1.3.0.tar.gz

# System Requirements

Make sure your hostname is resolvable via DNS or via /etc/hosts to allow full support of Docker’s host-networking capabilities, needed for some of the Mesos tests. When in doubt, please validate that /etc/hosts contains your hostname.

## CentOS 7.1

# Install a few utility tools
    sudo yum install -y tar wget git
    
    #Fetch the Apache Maven repo file.
    sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
    
    # Install the EPEL repo so that we can pull in 'libserf-1' as part of our
    # subversion install below.
    sudo yum install -y epel-release
    
    # 'Mesos > 0.21.0' requires 'subversion > 1.8' devel package,
    # which is not available in the default repositories.
    # Create a WANdisco SVN repo file to install the correct version:
    sudo bash -c 'cat > /etc/yum.repos.d/wandisco-svn.repo <<EOF
    [WANdiscoSVN]
    name=WANdisco SVN Repo 1.9
    enabled=1
    baseurl=http://opensource.wandisco.com/centos/7/svn-1.9/RPMS/\$basearch/
    gpgcheck=1
    gpgkey=http://opensource.wandisco.com/RPM-GPG-KEY-WANdisco
    EOF'
    
    # Parts of Mesos require systemd in order to operate. However, Mesos
    # only supports versions of systemd that contain the 'Delegate' flag.
    # This flag was first introduced in 'systemd version 218', which is
    # lower than the default version installed by centos. Luckily, centos
    # 7.1 has a patched 'systemd < 218' that contains the 'Delegate' flag.
    # Explicity update systemd to this patched version.
    sudo yum update systemd
    
    # Install essential development tools.
    sudo yum groupinstall -y "Development Tools"
    
    # Install other Mesos dependencies.
    sudo yum install -y apache-maven python-devel python-six python-virtualenv java-1.8.0-openjdk-devel zlib-devel libcurl-devel openssl-devel cyrus-sasl-devel cyrus-sasl-md5 apr-devel subversion-devel apr-util-devel

# Building Mesos (Posix)

# Change working directory.
    cd mesos-1.3.0
     
    # Configure and build.
    mkdir build
    cd build
    ../configure
    make
    
    # Run test suite.
    make check
    
    #Install (Optional).
    make install
    
In order to speed up the build and reduce verbosity of the logs, you can append -j <number of cores> V=0 to make.

# Configure Ssh

[Configure SSh](https://note.louyj.com/blog/post/louyj/SSH%E5%85%8D%E5%AF%86%E7%A0%81%E7%99%BB%E9%99%86)

# Configure Mesos

he Mesos master and agent can take a variety of configuration options through command-line arguments or environment variables. A list of the available options can be seen by running mesos-master --help or mesos-agent --help. Each option can be set in two ways:

- By passing it to the binary using --option_name=value, either specifying the value directly, or specifying a file in which the value resides (--option_name=file://path/to/file). The path can be absolute or relative to the current working directory.
 - By setting the environment variable MESOS_OPTION_NAME (the option name with a MESOS_ prefix added to it).

Configuration values are searched for first in the environment, then on the command-line.

see http://mesos.apache.org/documentation/latest/configuration/

cd /root/mesos/etc/mesos
    vi masters
    node1
    node2
    
    vi slavers
    node1
    node2
    node3
    
    cd etc/mesos
    
    #create credentials text file, only when Authentication
    vi master_credentials.txt
    {
       "credentials" : [
         {
           "principal": "principal1",
           "secret": "secret1"
         },
         {
           "principal": "principal2",
           "secret": "secret2"
         }
       ]
    }
    
    vi agent_credentials.txt
    {
       "principal": "principal1",
       "secret": "secret1"
    }
    
    vi http_credentials.txt
    {
       "principal": "principal1",
       "secret": "secret1"
    }
     
    mv mesos-master-env.sh.template mesos-master-env.sh
    vi mesos-master-env.sh
    #add
    export MESOS_log_dir=/var/log/mesos
    export MESOS_work_dir=/var/lib/mesos
    #Authentication
    export MESOS_authenticate=true
    export MESOS_authenticate_http_readonly=true
    export MESOS_authenticate_http_readwrite=true
    export MESOS_authenticate_agents=true
    export MESOS_authenticators=crammd5
    export MESOS_http_authenticators=basic
    export MESOS_credentials=/root/mesos/etc/mesos/master_credentials.txt
    #High-Availability Mode
    export MESOS_zk=zk://louyj.top:2181/mesos
    export MESOS_quorum=1
    
    mv mesos-agent-env.sh.template mesos-agent-env.sh
    vi mesos-agent-env.sh
    #add
    export MESOS_master=node1:5050
    export MESOS_log_dir=/var/log/mesos
    export MESOS_work_dir=/var/run/mesos
    export MESOS_isolation=cgroups
    # Agent Recovery
    export MESOS_strict=true
    export MESOS_recover=reconnect
    export MESOS_recovery_timeout=15mins
    #Authentication
    export MESOS_authenticatee=crammd5
    export MESOS_credential=/root/mesos/etc/mesos/agent_credentials.txt
    export MESOS_authenticate_http_readonly=true
    export MESOS_authenticate_http_readwrite=true
    export MESOS_authenticate_http_executors=true
    export MESOS_http_authenticators=basic
    export MESOS_http_credentials=/root/mesos/etc/mesos/http_credentials.txt
    #High-Availability Mode
    export MESOS_master=zk://louyj.top:2181/mesos
     
    mv mesos-deploy-env.sh.template mesos-deploy-env.sh
    vi mesos-deploy-env.sh
    #add options

# Start Mesos

These examples assume you are running Mesos on your local machine. Following them will not allow you to access the Mesos web page in a production environment (e.g. on AWS). For that you will need to specify the actual IP of your host when launching the Mesos master and ensure your firewall settings allow access to port 5050 from the outside world.
    
    scp -r mesos node2:~
    scp -r mesos node3:~
    
    # Start Mesos master (ensure work directory exists and has proper permissions).
    ./sbin/mesos-start-masters.sh
    
    # Start Mesos agent (ensure work directory exists and has proper permissions).
    ./sbin/mesos-start-agents.sh
    
    # Start Mesos master and agents
    ./sbin/mesos-start-cluster.sh 
    
    # Visit the Mesos web page.
    http://node1:5050

# Test Mesos

Mesos comes bundled with example frameworks written in C++, Java and Python. The framework binaries will only be available after running make check, as described in the Building Mesos section above.

# Run C++ framework (exits after successfully running some tasks).
    $ ./src/test-framework --master=127.0.0.1:5050
    
    # Run Java framework (exits after successfully running some tasks).
    $ ./src/examples/java/test-framework 127.0.0.1:5050
    
    # Run Python framework (exits after successfully running some tasks).
    $ ./src/examples/python/test-framework 127.0.0.1:5050

Navigation

Recent Posts

Friend Links