Kubeless Deployment

#Installation

##Install kubeless

Installation is made of three steps:

- Download the kubeless CLI from the release page.
- Create a kubeless namespace (used by default)
- Then use one of the YAML manifests found in the release page to deploy kubeless. It will create a functions Custom Resource Definition and launch a controller.

#Download the kubeless CLI
    wget https://github.com/kubeless/kubeless/releases/download/v1.0.4/kubeless_linux-amd64.zip
    cd bundles/kubeless_linux-amd64
    cp kubeless /usr/local/bin
    
    #Create a kubeless namespace
    kubectl create ns kubeless
    
    
    export RELEASE=$(curl -s https://api.github.com/repos/kubeless/kubeless/releases/latest | grep tag_name | cut -d '"' -f 4)
    #or
    export RELEASE=v1.0.4
    kubectl create -f https://github.com/kubeless/kubeless/releases/download/$RELEASE/kubeless-$RELEASE.yaml
    
    #verify
    $ kubectl get pods -n kubeless
    NAME                                           READY     STATUS    RESTARTS   AGE
    kubeless-controller-manager-567dcb6c48-ssx8x   1/1       Running   0          1h
    
    $ kubectl get deployment -n kubeless
    NAME                          DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
    kubeless-controller-manager   1         1         1            1           1h
    
    $ kubectl get customresourcedefinition
    NAME                          AGE
    cronjobtriggers.kubeless.io   1h
    functions.kubeless.io         1h
    httptriggers.kubeless.io      1h

For installing kubeless CLI using execute:

```
export OS=$(uname -s| tr '[:upper:]' '[:lower:]')
curl -OL https://github.com/kubeless/kubeless/releases/download/$RELEASE/kubeless_$OS-amd64.zip && \
  unzip kubeless_$OS-amd64.zip && \
  sudo mv bundles/kubeless_$OS-amd64/kubeless /usr/local/bin/
```

##Install kubeless-ui

To run the UI inside your Kubernetes cluster as a Deployment and Service you can run the following:

kubectl create -f https://raw.githubusercontent.com/kubeless/kubeless-ui/master/k8s.yaml

ui address

http://localhost:3000
    http://localhost:8001/api/v1/namespaces/kubeless/services/ui:ui-port/proxy/#/

##Sample function

You can use the CLI to create a function. Here is a toy:

```
def hello(event, context):
  print event
  return event['data']
```

You create it with:

```
kubeless function deploy hello -n serverless --runtime python2.7 \
                                --from-file test.py \
                                --handler test.hello
```

You will see the function custom resource created:

kubectl get functions -n serverless
    
or

kubeless function ls -n serverless
    
You can then call the function with:

kubeless function call hello -n serverless --data 'Hello world!'
    
Or you can curl directly with kubectl proxyusing an apiserver proxy URL.

```
kubectl proxy -p 8080 &
curl -L --data '{"Another": "Echo"}' \
  --header "Content-Type:application/json" \
  localhost:8080/api/v1/namespaces/serverless/services/hello:http-function-port/proxy/
```

You can delete the function

kubeless function delete -n serverless hello

##Install Triggers

Kubeless architecture is built on core concepts of Functions, Triggers and Runtime. A Trigger in Kubeless represents association between an event source and functions that need to be invoked on an event in the event source. Kubeless fully leverages the Kubernetes concepts of custom resource definition(CRD) and custom controllers. Each trigger is expected to be modelled as Kubernetes CRD. A trigger specific custom resource controller is expected to be written that realizes how deployed functions are invoked when event occurs. Following sections document how one can add a new event source as Trigger into Kubeless.

###kafka

####Use an existing Kafka cluster with Kubeless

In Kubeless release page, we provide along with Kubeless manifests a collection of Kafka and Zookeeper statefulsets which helps user to quickly deploying PubSub function. These statefulsets are deployed in kubeless namespace. However, if you have a Kafka cluster already running in the same Kubernetes cluster, this doc will walk you through how to deploy Kubeless PubSub function with it.

Now we need to deploy the Kafka consumer and the Kafka Trigger CRD. We can do that extracting the Deployment, CRD and ClusterRoles from the generic Kafka manifest. The key part is adding the environment variable KAFKA_BROKERS pointing to the right URL:

```
echo '
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  labels:
    kubeless: kafka-trigger-controller
  name: kafka-trigger-controller
  namespace: kubeless
spec:
  selector:
    matchLabels:
      kubeless: kafka-trigger-controller
  template:
    metadata:
      labels:
        kubeless: kafka-trigger-controller
    spec:
      containers:
      - image: bitnami/kafka-trigger-controller:latest
        imagePullPolicy: IfNotPresent
        name: kafka-trigger-controller
        env:
        - name: KAFKA_BROKERS
          value: kafka.pubsub:9092 # CHANGE THIS!
      serviceAccountName: controller-acct
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: kafkatriggers.kubeless.io
spec:
  group: kubeless.io
  names:
    kind: KafkaTrigger
    plural: kafkatriggers
    singular: kafkatrigger
  scope: Namespaced
  version: v1beta1
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kafka-controller-deployer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kafka-controller-deployer
subjects:
- kind: ServiceAccount
  name: controller-acct
  namespace: kubeless
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: kafka-controller-deployer
rules:
- apiGroups:
  - ""
  resources:
  - services
  - configmaps
  verbs:
  - get
  - list
- apiGroups:
  - kubeless.io
  resources:
  - functions
  - kafkatriggers
  verbs:
  - get
  - list
  - watch
  - update
  - delete
' | kubectl create -f -
```

A function can be as simple as:

```
def foobar(event, context):
  print event['data']
  return event['data']
```

Now you can deploy a pubsub function.

```
kubeless function deploy test --runtime python2.7 \
                                --handler test.foobar \
                                --from-file test.py
```

You need to create a Kafka trigger that lets you associate a function with a topic specified by --trigger-topic as below:

```
kubeless trigger kafka create test --function-selector created-by=kubeless,function=test --trigger-topic test-topic
```

After that you can invoke the function by publishing messages in that topic.

To allow you to easily manage topics kubeless provides a convenience function kubeless topic. You can create/delete and publish to a topic easily.

###http-trigger(Nginx Ingress)

Kubeless leverages Kubernetes ingress to provide routing for functions. By default, a deployed function will be matched to a Kubernetes service using ClusterIP as the service. That means that the function is not exposed publicly. Because of that, we provide the kubeless trigger http command that can make a function publicly available. This guide provides a quick sample on how to do it.

see https://github.com/kubernetes/ingress-nginx/blob/master/docs/deploy/index.md

!!! attention The default configuration watches Ingress object from all the namespaces. To change this behavior use the flag --watch-namespace to limit the scope to a particular namespace.

!!! warning If multiple Ingresses define different paths for the same host, the ingress controller will merge the definitions.

The following Mandatory Command is required for all deployments.

**If you are using a Kubernetes version previous to 1.14, you need to change kubernetes.io/os to beta.kubernetes.io/os at line 217 of mandatory.yaml, see Labels details.**

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
    
    
Verify installation

kubectl get pods --all-namespaces -l app.kubernetes.io/name=ingress-nginx --watch
    
Detect installed version

To detect which version of the ingress controller is running, exec into the pod and run nginx-ingress-controller version command.

```
POD_NAMESPACE=ingress-nginx
POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app.kubernetes.io/name=ingress-nginx -o jsonpath='{.items[0].metadata.name}')

kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version
```

Expose a function

In order to expose a function, it is necessary to create a HTTP Trigger object. The Kubeless CLI provides the commands required to do so:

kubeless trigger http create hello -n serverless --function-name hello

This command will create an ingress object. We can see it with kubectl

kubectl get ing -n serverless
    
Kubeless creates a default hostname in form of ..nip.io. Alternatively, you can provide a real hostname with --hostname flag or use a different --path like this:

But you have to make sure your hostname is configured properly.

kubeless trigger http create hello -n serverless --function-name hello --path echo --hostname example.com

You can test the created HTTP trigger with the following command:

```
#kubectl get ing -n serverless
curl --data '{"Another": "Echo"}' \
  --header "Host: hello.k8s.mjcndc01.com.nip.io" \
  --header "Content-Type:application/json" \
  10.109.105.154/echo
{"Another": "Echo"}
```

###CronJob Trigger

It is possible to deploy functions that should be triggered following a certain schedule. For specifying the execution frequency we use the Cron format. Every time a scheduled function is executed, a Job is started. This Job will do a HTTP GET request to the function service and will be successful as far as the function returns 200 OK.

kubeless trigger cronjob create hello -n serverless --function hello  --schedule '*/1 * * * *'
    kubeless trigger cronjob list -n serverless
    
If for some reason you want to modify one of the default values for a certain function you can execute kubectl edit cronjob trigger-<func_name> (where func_name is the name of your function) and modify the fields required. Once it is saved the CronJob will be updated.

##Autoscaling

Kubernetes introduces HorizontalPodAutoscaler for pod autoscaling. In kubeless, each function is deployed into a separate Kubernetes deployment, so naturally we leverage HPA to automatically scale function based on defined workload metrics.

If you're on Kubeless CLI, this below command gives you an idea how to setup autoscaling for deployed function:

kubeless autoscale create hello -n serverless --min 1 --max 5 --value 50 --metric cpu

#Implementing a New Runtime

see https://github.com/kubeless/runtimes/blob/master/DEVELOPER_GUIDE.md

###Create the Runtime Docker Images

The first step is to create the different docker images required to run, compile or install dependencies for our runtime. The Kubeless build process for functions contain 4 different steps:

- Preparation. This step is managed by the Kubeless core and it's not customizable. In this step, Kubeless retrieve the function source code (and dependencies) from the Function CRD and places it in the volume mounted at /kubeless. If the function is compressed in a ZIP file it will extract it so the next phases have the code ready to be used.
- Installation. This step is optional depending on the runtime. The goal of this phase is to install any required dependency that the user specifies in its dependency file. All the dependencies should be installed in the volume mounted to be able to use it in other phases. In this phase the following environment variables are available:
  - KUBELESS_INSTALL_VOLUME: Path to the volume that should contain the dependencies (/kubeless by default).
  - KUBELESS_DEPS_FILE: Path to the dependencies file.
- Compilation. This step is optional as well depending on the runtime. At this point, the mounted volume will contain the result of executing the previous phases. The goal of this phase is to run any required compilation required by the runtime. In case that the compilation took place in the previous phase or if the runtime doesn't require compilation at all, this step may be skipped. As in the installation phase, the container has the following environment variables available:
  - KUBELESS_INSTALL_VOLUME: Path to the volume that should contain the dependencies (/kubeless by default).
  - KUBELESS_FUNC_NAME: Name of the function. Useful in case it's necessary to substitute a place holder in the source file with the function name to execute.
- Runtime. This is the phase that contains the information related to the execution time of the function. It's important to set here the environment variables required to load dependencies added in the installation. For example, for NodeJS functions, it's necessary to point the NODE_PATH environment variable to the installation volume.

All the information related to the different phases is specified in the Jsonnet manifest (explained in the next section). Jsonnet is a templating language that will be used to merge all the different manifest (one per runtime) in the root one.

##bash runtime

`kubeless-shell.py`

```
#!/usr/bin/env python

from subprocess import call
import bottle
import os
import prometheus_client as prom

app = application = bottle.app()

cmdline = '/kubeless/' + os.getenv('MOD_NAME') + '.sh'
cmdlineprep = 'chmod -R 755 /kubeless'

def func(osparam):
    global cmdline
    if osparam:
        cmdline = cmdline + ' ' + osparam
    call(cmdlineprep, shell=True)
    call(cmdline, shell=True)

func_calls = prom.Counter('function_calls_total',
                          'Number of calls to user function',
                          ['method'])
func_errors = prom.Counter('function_failures_total',
                           'Number of exceptions in user function',
                           ['method'])
func_hist = prom.Histogram('function_duration_seconds',
                           'Duration of user function in seconds',
                           ['method'])

@app.route('/', method=['GET', 'POST'])
def handler():
    req = bottle.request
    method = req.method
    func_calls.labels(method).inc()
    print str(bottle.request)
    with func_errors.labels(method).count_exceptions():
        with func_hist.labels(method).time():
            if method == 'GET':
                return func(None)
            else:
                return func(bottle.request.body.read())

@app.get('/healthz')
def healthz():
    return 'OK'

@app.get('/metrics')
def metrics():
    bottle.response.content_type = prom.CONTENT_TYPE_LATEST
    return prom.generate_latest(prom.REGISTRY)

if __name__ == '__main__':
    import logging
    import sys
    import requestlogger
    loggedapp = requestlogger.WSGILogger(
        app,
        [logging.StreamHandler(stream=sys.stdout)],
        requestlogger.ApacheFormatter())
    bottle.run(loggedapp, server='cherrypy', host='0.0.0.0', port=8080)
```

`Dockerfile`

```
FROM centos:7.5.1804

ADD  ./jdk-8u201-linux-x64.tar.gz /opt/

RUN yum groupinstall "Development tools" -y
RUN  yum install unzip libffi-devel net-tools lsof openssl-devel zlib-devel bzip2-devel \
     expat-devel gdbm-devel readline-devel ncurses-devel sqlite-devel gcc automake autoconf \
     libtool make install kde-l10n-Chinese glibc-common postgresql-devel  fontconfig mkfontscale python-devel \
     bind-utils git less -y
RUN  rm /etc/localtime && \
     ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
ADD  ./bin/* /usr/local/bin/
ADD  ./locale.conf /etc/locale.conf
ADD  ./pgsql.tar.gz /opt/
RUN  localedef -c -f UTF-8 -i zh_CN zh_CN.utf8  && \
     chmod a+x /usr/local/bin/*
RUN pip install bottle==0.12.13 cherrypy==8.9.1 wsgi-request-logger prometheus_client
ENV  LC_ALL zh_CN.UTF-8
ENV  LANG="zh_CN.UTF-8"
ENV  PGHOME=/opt/pgsql
ENV  PATH=$PGHOME/bin:$PATH
ENV  LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PGHOME/lib

ENV  JAVA_HOME=/opt/jdk1.8.0_201
ENV  JRE_HOME=${JAVA_HOME}/jre
ENV  CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
ENV  PATH=${JAVA_HOME}/bin:$PATH

ADD kubeless-shell.py /

EXPOSE 8080
CMD ["python", "/kubeless-shell.py"]
```

`build.sh`

```
#!/bin/bash
docker build -t registry.dataenlighten.com:5000/kubeless-shell:v1.0 .
docker push registry.dataenlighten.com:5000/kubeless-shell:v1.0
```

`shell.jsonnet`

```
{
  ID: "shell",
  versions: [
    {
      name: "shell",
      version: "1.0",
      images: [{
        phase: "installation",
        image: "registry.dataenlighten.com:5000/kubeless-shell",
        command: "sh $KUBELESS_DEPS_FILE"
      }, {
        phase: "runtime",
        image: "registry.dataenlighten.com:5000/kubeless-shell",
        env: {
          PATH: "$PATH:$(KUBELESS_INSTALL_VOLUME)",
        },
      }],
    },
  ],
  depName: "requirements.sh",
  fileNameSuffix: ".sh",
}
```

Update the kubeless-config ConfigMap

The manifest of the previous point is part of the Kubeless ConfigMap that should be deployed in the kubeless namespace. In order to test your changes you can copy the result of executing jsonnet <runtime>.jsonnet and paste it in the Kubeless ConfigMap. For example:

```
cd stable/nodejs
$ jsonnet nodejs.jsonnet
# Copy the result and
# Paste the result within the "runtime-images" section
$ kubectl edit cm -n kubeless kubeless-config
# Restart the controller pod
$ kubectl delete pod -n kubeless -l kubeless=controller
```

function example

```
echo param $1
```

You create it with:

```
kubeless function deploy shell -n serverless --runtime shell1.0 \
                                             --from-file test.sh \
                                             --handler test.any
```

You can then call the function with:

```
kubeless function -n serverless call  shell  --data="ddd"
```

#Configure Deployment

##mount volume

```
apiVersion: kubeless.io/v1beta1
kind: Function
metadata:
  name: hello
  namespace: serverless
  label:
    created-by: kubeless
    function: hello
spec:
  runtime: python2.7
  timeout: "180"
  handler: test.hello
  deps: ""
  function-content-type: text
  function: |
    def hello(event, context):
      print event
      return event['data']
  deployment:
    spec:
      metadata:
        namespace: serverless
      template:
        spec:
          containers:
          - resources:
              limits:
                cpu: 100m
                memory: 50Mi
              requests:
                cpu: 100m
                memory: 10Mi
            volumeMounts:
            - mountPath: /mnt/cephfs/kubeless
              name: ceph-kubeless
          volumes:
          - name: ceph-kubeless
            hostPath:
              path: /mnt/cephfs/kubeless
```

url

```
apiVersion: kubeless.io/v1beta1
kind: Function
metadata:
  name: hello
  namespace: serverless
  label:
    created-by: kubeless
    function: hello
spec:
  runtime: python2.7
  timeout: "180"
  handler: test.hello
  deps: ""
  function-content-type: url
  function: file:///mnt/cephfs/kubeless/test.py
  deployment:
    spec:
      metadata:
        namespace: serverless
      template:
        spec:
          containers:
          - resources:
              limits:
                cpu: 100m
                memory: 50Mi
              requests:
                cpu: 100m
                memory: 10Mi
            volumeMounts:
            - mountPath: /mnt/cephfs/kubeless
              name: ceph-kubeless
          volumes:
          - name: ceph-kubeless
            hostPath:
              path: /mnt/cephfs/kubeless
```

Navigation

Recent Posts

Friend Links