Description

Sink to local or HDFS files in CSV format.

Parameters

Name Description Type Required? Default Value
filePath File path String
fieldDelimiter Field delimiter String “,”
rowDelimiter Row delimiter String “\n”
quoteChar quote char Character “\””
overwriteSink Whether to overwrite existing data. Boolean false
numFiles Number of files Integer 1

Script Example

batch sink

  1. filePath = 'https://alink-release.oss-cn-beijing.aliyuncs.com/data-files/iris.csv'
  2. schema = 'sepal_length double, sepal_width double, petal_length double, petal_width double, category string'
  3. csvSource = CsvSourceBatchOp()\
  4. .setFilePath(filePath)\
  5. .setSchemaStr(schema)\
  6. .setFieldDelimiter(",")
  7. csvSink = CsvSinkBatchOp()\
  8. .setFilePath('~/csv_test.txt')
  9. csvSource.link(csvSink)
  10. BatchOperator.execute()

stream sink

  1. filePath = 'https://alink-release.oss-cn-beijing.aliyuncs.com/data-files/iris.csv'
  2. schema = 'sepal_length double, sepal_width double, petal_length double, petal_width double, category string'
  3. csvSource = CsvSourceStreamOp()\
  4. .setFilePath(filePath)\
  5. .setSchemaStr(schema)\
  6. .setFieldDelimiter(",")
  7. csvSink = CsvSinkStreamOp()\
  8. .setFilePath('~/csv_test_s.txt')
  9. csvSource.link(csvSink)
  10. StreamOperator.execute()