MLeap Troubleshooting

Must provide a sample dataset for the X transformer

This error occurs because you are trying to serialize a Sparktransformer that normally relies on metadata available in the SparkDataFrame. In order to serialize properly, MLeap needs access to themetadata so we can store all of the necessary values in the MLeapBundle. The solution is to provide a sample DataFrame that has beentransformed by your Spark ML Pipeline.

Fixed Code

// Use your Spark ML Pipeline to transform the Spark DataFrame
val transformedDataset = sparkTransformer.transform(sparkDataset)
// Create a custom SparkBundleContext and provide the transformed DataFrame
implicit val sbc = SparkBundleContext().withDataset(transformedDataset)
// Serialize the pipeline as you would normally
(for(bf <- managed(BundleFile(file))) yield {
  sparkTransformer.writeBundle.save(bf).get
}).tried.get