Description

The stream operator that predict the data using the quantile discretizer model.

Parameters

Name Description Type Required? Default Value
selectedCols Names of the columns used for processing String[]
reservedCols Names of the columns to be retained in the output table String[] null
outputCols Names of the output columns String[] null
handleInvalid Strategy to handle unseen token when doing prediction, one of “keep”, “skip” or “error” String “keep”
encode Encode method,”INDEX”, “VECTOR”, “ASSEMBLED_VECTOR” String INDEX
dropLast drop last Boolean true

Script Example

Code

  1. import numpy as np
  2. import pandas as pd
  3. from pyalink.alink import *
  4. def exampleData():
  5. return np.array([
  6. ["a", 1, 1, 2.0, True],
  7. ["c", 1, 2, -3.0, True],
  8. ["a", 2, 2, 2.0, False],
  9. ["c", 0, 0, 0.0, False]
  10. ])
  11. def sourceFrame():
  12. data = exampleData()
  13. return pd.DataFrame({
  14. "f_string": data[:, 0],
  15. "f_long": data[:, 1],
  16. "f_int": data[:, 2],
  17. "f_double": data[:, 3],
  18. "f_boolean": data[:, 4]
  19. })
  20. def batchSource():
  21. return dataframeToOperator(
  22. sourceFrame(),
  23. schemaStr='''
  24. f_string string,
  25. f_long long,
  26. f_int int,
  27. f_double double,
  28. f_boolean boolean
  29. ''',
  30. op_type='batch'
  31. )
  32. def streamSource():
  33. return dataframeToOperator(
  34. sourceFrame(),
  35. schemaStr='''
  36. f_string string,
  37. f_long long,
  38. f_int int,
  39. f_double double,
  40. f_boolean boolean
  41. ''',
  42. op_type='stream'
  43. )
  44. trainOp = (
  45. QuantileDiscretizerTrainBatchOp()
  46. .setSelectedCols(['f_double'])
  47. .setNumBuckets(8)
  48. )
  49. predictBatchOp = (
  50. QuantileDiscretizerPredictBatchOp()
  51. .setSelectedCols(['f_double'])
  52. )
  53. (
  54. predictBatchOp
  55. .linkFrom(
  56. batchSource().link(trainOp),
  57. batchSource()
  58. )
  59. .print()
  60. )
  61. predictStreamOp = (
  62. QuantileDiscretizerPredictStreamOp(
  63. batchSource().link(trainOp)
  64. )
  65. .setSelectedCols(['f_double'])
  66. )
  67. (
  68. predictStreamOp
  69. .linkFrom(
  70. streamSource()
  71. )
  72. .print()
  73. )
  74. StreamOperator.execute()

Result

  1. f_string f_long f_int f_double f_boolean
  2. 0 a 1 1 2 True
  3. 1 a 2 2 2 False
  4. 2 c 1 2 0 True
  5. 3 c 0 0 1 False