Description

Sink to local or HDFS files in CSV format.

Parameters

Name Description Type Required？ Default Value
filePath File path String ✓
fieldDelimiter Field delimiter String “,”
rowDelimiter Row delimiter String “\n”
quoteChar quote char Character “\””
overwriteSink Whether to overwrite existing data. Boolean false
numFiles Number of files Integer 1

Script Example

batch sink

  1. filePath = 'http://alink-dataset.cn-hangzhou.oss.aliyun-inc.com/csv/iris.csv'
  2. schema = 'sepal_length double, sepal_width double, petal_length double, petal_width double, category string'
  3. csvSource = CsvSourceBatchOp()\
  4.     .setFilePath(filePath)\
  5.     .setSchemaStr(schema)\
  6.     .setFieldDelimiter(",")
  7. csvSink = CsvSinkBatchOp()\
  8.     .setFilePath('~/csv_test.txt')
  10. csvSource.link(csvSink)
  12. BatchOperator.execute()

stream sink

  1. filePath = 'http://alink-dataset.cn-hangzhou.oss.aliyun-inc.com/csv/iris.csv'
  2. schema = 'sepal_length double, sepal_width double, petal_length double, petal_width double, category string'
  3. csvSource = CsvSourceStreamOp()\
  4.     .setFilePath(filePath)\
  5.     .setSchemaStr(schema)\
  6.     .setFieldDelimiter(",")
  7. csvSink = CsvSinkStreamOp()\
  8.     .setFilePath('~/csv_test_s.txt')
  10. csvSource.link(csvSink)
  12. StreamOperator.execute()