Description

Ftrl predictor receive two stream : model stream and data stream. It using updated model by model stream real-time, and using the newest model predict data stream.

Parameters

Name Description Type Required？ Default Value
vectorCol Name of a vector column String null
reservedCols Names of the columns to be retained in the output table String[] null
predictionCol Column name of prediction. String ✓
predictionDetailCol Column name of prediction result, it will include detailed info. String

Script Example

Script

  1.  data = np.array([
  2.     [2, 1, 1],
  3.     [3, 2, 1],
  4.     [4, 3, 2],
  5.     [2, 4, 1],
  6.     [2, 2, 1],
  7.     [4, 3, 2],
  8.     [1, 2, 1],
  9.     [5, 3, 2]])
  10. df = pd.DataFrame({"f0": data[:, 0], 
  11.                    "f1": data[:, 1],
  12.                    "label": data[:, 2]})
  14. batchData = dataframeToOperator(df, schemaStr='f0 int, f1 int, label int', op_type='batch')
  15. streamData = dataframeToOperator(df, schemaStr='f0 int, f1 int, label int', op_type='stream')
  17. model = LogisticRegressionTrainBatchOp() \
  18.             .setFeatureCols(["f0", "f1"]) \
  19.             .setLabelCol("label") \
  20.             .setMaxIter(5).linkFrom(batchData);
  23. models = FtrlTrainStreamOp(model) \
  24.             .setFeatureCols(["f0", "f1"]) \
  25.             .setLabelCol("label") \
  26.             .setTimeInterval(1) \
  27.             .setAlpha(0.1) \
  28.             .setBeta(0.1) \
  29.             .setL1(0.1) \
  30.             .setL2(0.1).setVectorSize(2).setWithIntercept(True) \
  31.             .linkFrom(streamData);
  33. FtrlPredictStreamOp(model) \
  34.         .setPredictionCol("pred") \
  35.         .setReservedCols(["label"]) \
  36.         .setPredictionDetailCol("details") \
  37.         .linkFrom(models, streamData).print()
  38. StreamOperator.execute()

Result

  1. label    pred    details
  2. 1    1    {"1":"0.9999917437501057","2":"8.2562498943117...
  3. 1    1    {"1":"0.965917838185468","2":"0.03408216181453...
  4. 2    2    {"1":"0.00658782416074899","2":"0.993412175839...
  5. 1    1    {"1":"0.9810760570397847","2":"0.0189239429602...
  6. 1    1    {"1":"0.9998904582473768","2":"1.0954175262323...
  7. 2    2    {"1":"0.00658782416074899","2":"0.993412175839...
  8. 1    1    {"1":"0.9999996598523875","2":"3.4014761252088...
  9. 2    2    {"1":"2.0589409516880153E-5","2":"0.9999794105...

```