Router Manager

KubeEdge supports routing management with the help of Kubernetes CRDs and a router module. Users can deliver their custom messages between cloud and edge by mqtt broker.

Usage scenarios

  • used for user’s control data delivery;
  • not suitable for large data delivery;
  • the data size of delivery at one time is limited to 12MB.

Use Cases

  • user can deliver custom message from cloud to edge by calling api in cloudcore. And it will deliver the message to mqttbroker on edge finally.

    1. user creates a rule which defines the message’s source and target info.
    2. user’s app in cloud calls an api of cloudcore to deliver custom message; the message is delivered to edge, and published to mqttbroker with a topic.
    3. user’s app in edge subscribes the message with the same topic in step 2.

  • user can deliver custom message from edge to cloud by publishing message to mqttbroker. And it will deliver the message to rest api in cloud finally.

    1. user creates a rule which defines the message’s source and target info.
    2. user’s app in edge published message to mqttbroker with a topic.
    3. user’s app in cloud got the message in step 2.

  • user can deliver custom message from cloud to edge by calling api in cloudcore. And it will call rest api on edgenode finally. And get the result.

    1. user creates a rule which defines the message’s source and target info.
    2. user’s app in cloud calls an api of cloudcore to deliver custom message; cloudcore delivered the message to edgecore. Edgecore will call edge’s rest api.
    3. user’s app in cloud got the result of api call in step 2.

Rule And ruleendpoint Model

RuleEndpoint and Rule definition

  • A ruleEndpoint defines where messages come from, or where messages go to. It contains 3 types: rest, eventbus, servicebus.

    1. rest: a rest endpoint in the cloud. It is a source ruleEndpoint to send rest requests to the edge. Or a target ruleEndpoint to receive message from edge.

    2. eventbus: It is a source ruleEndpoint to send data to the cloud, or a target ruleEndpoint to receive messages from the cloud.

    3. servicebus: a rest api on edge node. It is a target ruleEndpoint to receive messages delivered from cloud.

  • Rule defines how messages were delivered from source ruleendpoint to target ruleendpoint. It contains 3 rule types.

    1. rest->eventbus: users’ app calls rest api in cloud to send messages, finally messages were sent to mqttbroker in edge.
    2. eventbus->rest: users publish messages to mqttbroker in edge, finally messages were sent to rest api in cloud.
    3. rest->servicebus: users’ app calls rest api in cloud to send messages, finally messages were sent to user’s app in edge.

RuleEndpoint and Rule CRD design

  • RuleEndpoint
  1. apiVersion: apiextensions.k8s.io/v1
  2. kind: CustomResourceDefinition
  3. metadata:
  4.   name: ruleendpoints.rules.kubeedge.io
  5. spec:
  6.   group: rules.kubeedge.io
  7.   versions:
  8.     - name: v1
  9.       served: true
  10.       storage: true
  11.       schema:
  12.         openAPIV3Schema:
  13.           type: object
  14.           properties:
  15.             spec:
  16.               type: object
  17.               properties:
  18.                 ruleEndpointType:
  19.                   description: |
  20.                     ruleEndpointType is a string value representing rule-endpoint type. its value is
  21.                     one of rest/eventbus/servicebus.
  22.                   type: string
  23.                 properties:
  24.                   description: |
  25.                     properties is not required except for servicebus rule-endpoint type. It is a map
  26.                     value representing rule-endpoint properties.when ruleEndpointType is servicebus,
  27.                     its value is {"service_port":"8080"}.
  28.                   type: object
  29.                   additionalProperties:
  30.                     type: string
  31.               required:
  32.                 - ruleEndpointType
  33.   scope: Namespaced
  34.   names:
  35.     plural: ruleendpoints
  36.     singular: ruleendpoint
  37.     kind: RuleEndpoint
  38.     shortNames:
  39.       - re
  • Rule
  1. apiVersion: apiextensions.k8s.io/v1
  2. kind: CustomResourceDefinition
  3. metadata:
  4.   name: rules.rules.kubeedge.io
  5. spec:
  6.   group: rules.kubeedge.io
  7.   versions:
  8.     - name: v1
  9.       served: true
  10.       storage: true
  11.       schema:
  12.         openAPIV3Schema:
  13.           type: object
  14.           properties:
  15.             spec:
  16.               type: object
  17.               properties:
  18.                 source:
  19.                   description: |
  20.                     source is a string value representing where the messages come from. Its
  21.                     value is the same with rule-endpoint name. For example, my-rest or my-eventbus.
  22.                   type: string
  23.                 sourceResource:
  24.                   description: |
  25.                     sourceResource is a map representing the resource info of source. For rest
  26.                     rule-endpoint type its value is {"path":"/a/b"}. For eventbus rule-endpoint type its
  27.                     value is {"topic":"<user define string>","node_name":"edge-node"}
  28.                   type: object
  29.                   additionalProperties:
  30.                     type: string
  31.                 target:
  32.                   description: |
  33.                     target is a string value representing where the messages go to. its value is
  34.                     the same with rule-endpoint name. For example, my-eventbus or my-rest or my-servicebus.
  35.                   type: string
  36.                 targetResource:
  37.                   description: |
  38.                     targetResource is a map representing the resource info of target. For rest
  39.                     rule-endpoint type its value is {"resource":"http://127.0.0.1:8081/hello"}. For eventbus rule-endpoint
  40.                     type its value is {"topic":"/test"}. For servicebus rule-endpoint type its value is
  41.                     {"path":"/request_path"}.
  42.                   type: object
  43.                   additionalProperties:
  44.                     type: string
  45.               required:
  46.                 - source
  47.                 - sourceResource
  48.                 - target
  49.                 - targetResource
  50.             status:
  51.               type: object
  52.               properties:
  53.                 successMessages:
  54.                   type: integer
  55.                 failMessages:
  56.                   type: integer
  57.                 errors:
  58.                   items:
  59.                     type: string
  60.                   type: array
  61.   scope: Namespaced
  62.   names:
  63.     plural: rules
  64.     singular: rule
  65.     kind: Rule

how to deliver custom messages

  • first, you should enable router module in cloudcore by modifying the cloudcore.yaml: add “enable: true” in router module, and restart cloudcore.
  1. cloud to edge : rest->eventbus 1.1 create rest and eventbus type ruleEndpoint if they don’t exist. Exec command:

kubectl create -f create-ruleEndpoint-rest.yaml

kubectl create -f create-ruleEndpoint-eventbus.yaml

create-ruleEndpoint-rest.yaml’s contents are as follows:

  1. apiVersion: rules.kubeedge.io/v1
  2. kind: RuleEndpoint
  3. metadata:
  4.   name: my-rest
  5.   labels:
  6.     description: test
  7. spec:
  8.   ruleEndpointType: "rest"
  9.   properties: {}

create-ruleEndpoint-eventbus.yaml’s contents are as follows:

  1. apiVersion: rules.kubeedge.io/v1
  2. kind: RuleEndpoint
  3. metadata:
  4.   name: my-eventbus
  5.   labels:
  6.     description: test
  7. spec:
  8.   ruleEndpointType: "eventbus"
  9.   properties: {}

1.2 create rule. Exec command:

kubectl create -f create-rule-rest-eventbus.yaml

create-rule-rest-eventbus.yaml’s contents are as follows:

  1. apiVersion: rules.kubeedge.io/v1
  2. kind: Rule
  3. metadata:
  4.   name: my-rule
  5.   labels:
  6.     description: test
  7. spec:
  8.   source: "my-rest"
  9.   sourceResource: {"path":"/test"}
  10.   target: "my-eventbus"
  11.   targetResource: {"topic":"test"}

1.3 call rest api in the cloud to send messages to edge.

The rest api in the cloud can be called to send messages to eventbus on an edge node based on the node name and sourceResource.

  • Method: POST
  • URL: http://{rest\_endpoint}/{node\_name}/{namespace}/{path}, {rest_endpoint} is {cloudcore_ip}:9443, {node_name} is name of edgenode, {namespace} is the namespace of rule, {path}’s prefix is source ruleEndpoint’s sourceResource.
  • Body: {user_message}, {user_message} is user’s message

For example: - Method: POST - URL: http://{cloudcore\_ip}:9443/{node\_name}/default/test - Body： {“message”:“123”}

1.4 User’s app subscribes custom topics from mqtt-broker in edge to receive messages from the cloud.

  • Topic: {topic}, {topic} is target ruleEndpoint’s targetResource.
  • Message: {user_message}

For example: - subscribe Topic: ‘test’, exec command with mosquitto: mosquitto_sub -t 'test' -d - Get Message: {“message”:“123”}

  1. edge to cloud: eventbus->rest:

2.1 create rest type and eventbus type ruleEndpoint if they don’t exist. Exec command:

kubectl create -f create-ruleEndpoint-rest.yaml

kubectl create -f create-ruleEndpoint-eventbus.yaml

create-ruleEndpoint-rest.yaml’s contents are as follows:

  1. apiVersion: rules.kubeedge.io/v1
  2. kind: RuleEndpoint
  3. metadata:
  4.   name: my-rest
  5.   labels:
  6.     description: test
  7. spec:
  8.   ruleEndpointType: "rest"
  9.   properties: {}

create-ruleEndpoint-eventbus.yaml’s contents are as follows:

  1. apiVersion: rules.kubeedge.io/v1
  2. kind: RuleEndpoint
  3. metadata:
  4.   name: my-eventbus
  5.   labels:
  6.     description: test
  7. spec:
  8.   ruleEndpointType: "eventbus"
  9.   properties: {}

2.2 create rule. Exec command:

kubectl create -f create-rule-eventbus-rest.yaml

create-rule-eventbus-rest.yaml’s contents are as follows:

  1. apiVersion: rules.kubeedge.io/v1
  2. kind: Rule
  3. metadata:
  4.   name: my-rule-eventbus-rest
  5.   labels:
  6.     description: test
  7. spec:
  8.   source: "my-eventbus"
  9.   sourceResource: {"topic": "test","node_name": "edge-node"}
  10.   target: "my-rest"
  11.   targetResource: {"resource":"http://127.0.0.1:8080/hello"}

2.3 User’s app in edge publishes messages with custom topic to MQTT broker on edge node. (target ruleEndpoint’s targetResource http://127.0.0.1:8080/hello should be available before this step) - Topic: {namespace}/{topic} - Message: {user_api_body}

for example: - publish data with mosquitto, exec command:

mosquitto_pub -t 'default/test' -d -m '{"edgemsg":"msgtocloud"}'

2.4 Kubeedge delivers messages to user api address in cloud.

For example: user’s app in cloud gets the data {“edgemsg”:“msgtocloud”}

  1. cloud to edge : rest->servicebus

3.1 create rest and servicebus type ruleEndpoint if they don’t exist. Exec command:

kubectl create -f create-ruleEndpoint-rest.yaml

kubectl create -f create-ruleEndpoint-servicebus.yaml

create-ruleEndpoint-rest.yaml’s contents are as follows:

  1. apiVersion: rules.kubeedge.io/v1
  2. kind: RuleEndpoint
  3. metadata:
  4.   name: my-rest
  5.   labels:
  6.     description: test
  7. spec:
  8.   ruleEndpointType: "rest"
  9.   properties: {}

create-ruleEndpoint-servicebus.yaml’s contents are as follows:

  1. apiVersion: rules.kubeedge.io/v1
  2. kind: RuleEndpoint
  3. metadata:
  4.   name: my-servicebus
  5.   labels:
  6.     description: test
  7. spec:
  8.   ruleEndpointType: "servicebus"
  9.   properties: {"service_port":"6666"}

3.2 create rule. Exec command:

kubectl create -f create-rule-rest-servicebus.yaml

create-rule-rest-servicebus.yaml’s contents are as follows:

  1. apiVersion: rules.kubeedge.io/v1
  2. kind: Rule
  3. metadata:
  4.   name: my-rule-rest-servicebus
  5.   labels:
  6.     description: test
  7. spec:
  8.   source: "my-rest"
  9.   sourceResource: {"path":"/source"}
  10.   target: "my-servicebus"
  11.   targetResource: {"path":"/target"}

3.3 user’s app calls rest api in the cloud to send messages to edge.

The rest api in the cloud can be called to send messages to servicebus on an edge node based on the node name and sourceResource.

  • Method: POST/GET/DELETE/PUT
  • URL: http://{rest\_endpoint}/{node\_name}/{namespace}/{path}, {rest_endpoint} is {cloudcore_ip}:9443, {node_name} is name of edgenode, {namespace} is namespace of rule. {path} is source ruleEndpoint’s sourceResource.
  • Body: {user_message}, {user_message} is user’s message

finally, kubeedge’s servicebus will call api on edgen node.

  • Method: POST/GET/DELETE/PUT

  • URL: http://127.0.0.1:{port}/{path}, {port} is target ruleEndpoint’s properties, {path} is target ruleEndpoint’s targetResource.

  • Body: {user_message}, {user_message} is user’s message

For example:

finnaly, kubeedge’s servicebus calls api on edge node. For example:

user’s app gets the result of the api on edge node.

Exception scenarios/Corner cases handling

CloudCore restart

  • When cloudcore restarts or starts normally, it will get all rules from edge-controller, and store in memory.

Implementation plan

  • Alpha: v1.6 rest->eventbus, eventbus->rest
  • Beta: v1.7 rest->servicebus
  • GA: TBD

Last updated on Mar 15, 2021