Description

pca predict for stream data, it need a pca model which is train from PcaTrainBatchOp

Parameters

Name Description Type Required？ Default Value
transformType ‘SIMPLE’ or ‘SUBMEAN’, SIMPLE is data model, SUBMEAN is (data - mean) model String “SIMPLE”
reservedCols Names of the columns to be retained in the output table String[] null
predictionCol Column name of prediction. String ✓
vectorCol Name of a vector column String null

Script Example

Script

  1. data = np.array([
  2.         [0.0,0.0,0.0],
  3.         [0.1,0.2,0.1],
  4.         [0.2,0.2,0.8],
  5.         [9.0,9.5,9.7],
  6.         [9.1,9.1,9.6],
  7.         [9.2,9.3,9.9]
  8. ])
  10. df = pd.DataFrame({"x1": data[:, 0], "x2": data[:, 1], "x3": data[:, 2]})
  12. # batch source 
  13. inOp = dataframeToOperator(df, schemaStr='x1 double, x2 double, x3 double', op_type='batch')
  15. trainOp = PcaTrainBatchOp()\
  16.        .setK(2)\
  17.        .setSelectedCols(["x1","x2","x3"])
  19. predictOp = PcaPredictBatchOp()\
  20.         .setPredictionCol("pred")
  22. # batch train
  23. inOp.link(trainOp)
  25. # batch predict
  26. predictOp.linkFrom(trainOp,inOp)
  28. predictOp.print()
  30. # stream predict
  31. inStreamOp = dataframeToOperator(df, schemaStr='x1 double, x2 double, x3 double', op_type='stream')
  33. predictStreamOp = PcaPredictStreamOp(trainOp)\
  34.         .setPredictionCol("pred")
  36. predictStreamOp.linkFrom(inStreamOp)
  38. predictStreamOp.print()
  40. StreamOperator.execute()

Result

x1 x2 x3 pred
9.0 9.5 9.7 3.2280384305400736,1.1516225426477789E-4
0.2 0.2 0.8 0.13565076707329407,0.09003329494282108
9.2 9.3 9.9 3.250783163664603,0.0456526246528135
9.1 9.1 9.6 3.182618319978973,0.027469531992220464
0.1 0.2 0.1 0.045855205015063565,-0.012182917696915518
0.0 0.0 0.0 0.0,0.0