Boosting Transform Speed
MLeap’s performance with transforming leap frames is good out of thebox, but sometimes there are situations where even more speed is needed.Luckily, we have a row-based transformer that eliminates a lot ofoverhead in transforming leap frames. The overhead that is beingeliminated is transforming the schema of the leap frame on every call totransform. How is this done?
- Calculate the input/output schema of a leap frame before any calls totransform
- When we transform a row, preallocate enough space to store allrequired results
- Fill the preallocated space with results from our transformers andreturn the transformed row
A lot of time is spent in transforming the schema relative to actuallyprocessing your data, so this can be a huge speed increase in certaincases.
当前内容版权归 combust.ml 或其关联方所有,如需对内容或内容相关联开源项目进行关注与资助,请访问 combust.ml .