Description

outputColNames CAN NOT have the same colName with keepOriginColName except the selectedColName.

Parameters

Name Description Type Required？ Default Value
selectedCols Names of the columns used for processing String[] ✓
outputCol Name of the output column String ✓
reservedCols Names of the columns to be retained in the output table String[] null

Script Example

Code

  1. source = CsvSourceStreamOp()\
  2.     .setSchemaStr("sepal_length double, sepal_width double, petal_length double, petal_width double, category string")\
  3.     .setFilePath("http://alink-dataset.cn-hangzhou.oss.aliyun-inc.com/csv/iris.csv")
  5. udfOp = UDFStreamOp() \
  6.     .setFunc(lambda x: x + 1) \
  7.     .setResultType("DOUBLE") \
  8.     .setSelectedCols(['sepal_length']) \
  9.     .setOutputCol('sepal_length_t') \
  10.     .setReservedCols(['sepal_width'])
  11. res = udfOp.linkFrom(source)
  13. res.print()
  14. StreamOperator.execute()

Results

  1.      sepal_length_t    sepal_width
  2. 1               6.9         3.2
  3. 2               6.4         3.7
  4. 3               7.9         3.1
  5. 4               6.5         2.5
  6. 5               6.4         3.4
  7. ...             ...         ...
  8. 96              8.9         3.8
  9. 97              6.2         2.7
  10. 98              7.4         2.7
  11. 99              7.8         3.0
  12. 100                6.7         2.5