Deployment

  1. cd inlong-agent

Configuration

Online operation needs to pull the configuration from inlong-manager, the configuration conf/agent.properties is as follows:

  1. # whether enable http service
  2. agent.http.enable=true
  3. # http default port
  4. agent.http.port=Available ports
  5. agent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (the class name for fetch tasks, default ManagerFetcher）
  6. agent.local.ip=Write local ip
  7. agent.manager.vip.http.host=manager web host
  8. agent.manager.vip.http.port=manager web port

Start

After decompression, run the following command

  1. sh agent.sh start

Add job configuration in real time

  1.     curl --location --request POST 'http://localhost:8008/config/job' \
  2.     --header 'Content-Type: application/json' \
  3.     --data '{
  4.     "job": {
  5.     "dir": {
  6.     "path": "",
  7.     "pattern": "/data/inlong-agent/test.log"
  8.     },
  9.     "trigger": "org.apache.inlong.agent.plugin.trigger.DirectoryTrigger",
  10.     "id": 1,
  11.     "thread": {
  12.     "running": {
  13.     "core": "4"
  14.     },
  15.     "onejob": true
  16.     },
  17.     "name": "fileAgentTest",
  18.     "source": "org.apache.inlong.agent.plugin.sources.TextFileSource",
  19.     "sink": "org.apache.inlong.agent.plugin.sinks.ProxySink",
  20.     "channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel"
  21.     },
  22.     "proxy": {
  23.   "inlongGroupId": "groupId10",
  24.   "inlongStreamId": "groupId10"
  25.     },
  26.     "op": "add"
  27.     }'

The meaning of each parameter is ：

  • job.dir.pattern: Configure the read file path, which can include regular expressions
  • job.trigger: Trigger name, the default is DirectoryTrigger, the function is to monitor the files under the folder to generate events
  • job.source: The type of data source used, the default is TextFileSource, which reads text files
  • job.sink：The type of writer used, the default is ProxySink, which sends messages to the proxy
  • proxy.groupId: The groupId type used when writing proxy, groupId is group id showed on data access in inlong-manager, not the topic name.
  • proxy.streamId: The streamId type used when writing proxy, streamId is the data flow id showed on data flow window in inlong-manager