mindinsight.lineagemgr

Lineagemgr Module Introduction.

This module provides Python APIs to collect and query the lineage of models.Users can add the TrainLineage/EvalLineage callback to the MindSpore train/eval callback list tocollect the key parameters and results, such as, the name of the network and optimizer, theevaluation metric and results.The APIs can be used to get the lineage information of the models. For example,what hyperparameter is used in the model training, which model has the highestaccuracy among all the versions, etc.

  • class mindinsight.lineagemgr.TrainLineage(summary_record, raise_exception=False)[source]

  • Collect lineage of a training job.

    • Parameters

      • summary_record (SummaryRecord) – SummaryRecord is used to recordthe summary value, and summary_record is an instance of SummaryRecord,see mindspore.train.summary.SummaryRecord.

      • raise_exception (bool) – Whether to raise exception when error occurs inTrainLineage. If True, raise exception. If False, catch exceptionand continue. Default: False.

    • Raises

      • MindInsightException – If validating parameter fails.

      • LineageLogError – If recording lineage information fails.

Examples

  1. Copy>>> from mindinsight.lineagemgr import TrainLineage
  2. >>> from mindspore.train.callback import ModelCheckpoint, SummaryStep
  3. >>> from mindspore.train.summary import SummaryRecord
  4. >>> model = Model(train_network)
  5. >>> model_ckpt = ModelCheckpoint(directory='/dir/to/save/model/')
  6. >>> summary_writer = SummaryRecord(log_dir='./')
  7. >>> summary_callback = SummaryStep(summary_writer, flush_step=2)
  8. >>> lineagemgr = TrainLineage(summary_record=summary_writer)
  9. >>> model.train(epoch_num, dataset, callbacks=[model_ckpt, summary_callback, lineagemgr])
  • begin(run_context)[source]

  • Initialize the training progress when the training job begins.

    • Parameters

    • run_context (RunContext) – It contains all lineage information,see mindspore.train.callback.RunContext.

    • Raises

    • MindInsightException – If validating parameter fails.

  • end(run_context)[source]

  • Collect lineage information when the training job ends.

    • Parameters

    • run_context (RunContext) – It contains all lineage information,see mindspore.train.callback.RunContext.

    • Raises

    • LineageLogError – If recording lineage information fails.
  • class mindinsight.lineagemgr.EvalLineage(summary_record, raise_exception=False)[source]

  • Collect lineage of an evaluation job.

    • Parameters

      • summary_record (SummaryRecord) – SummaryRecord is used to recordthe summary value, and summary_record is an instance of SummaryRecord,see mindspore.train.summary.SummaryRecord.

      • raise_exception (bool) – Whether to raise exception when error occurs inEvalLineage. If True, raise exception. If False, catch exceptionand continue. Default: False.

    • Raises

      • MindInsightException – If validating parameter fails.

      • LineageLogError – If recording lineage information fails.

Examples

  1. Copy>>> from mindinsight.lineagemgr import EvalLineage
  2. >>> from mindspore.train.callback import ModelCheckpoint, SummaryStep
  3. >>> from mindspore.train.summary import SummaryRecord
  4. >>> model = Model(train_network)
  5. >>> model_ckpt = ModelCheckpoint(directory='/dir/to/save/model/')
  6. >>> summary_writer = SummaryRecord(log_dir='./')
  7. >>> summary_callback = SummaryStep(summary_writer, flush_step=2)
  8. >>> lineagemgr = EvalLineage(summary_record=summary_writer)
  9. >>> model.eval(epoch_num, dataset, callbacks=[model_ckpt, summary_callback, lineagemgr])
  • end(run_context)[source]

  • Collect lineage information when the training job ends.

    • Parameters

    • run_context (RunContext) – It contains all lineage information,see mindspore.train.callback.RunContext.

    • Raises

      • MindInsightException – If validating parameter fails.

      • LineageLogError – If recording lineage information fails.

  • mindinsight.lineagemgr.getsummary_lineage(_summary_dir, keys=None)[source]
  • Get the lineage information according to summary directory and keys.

The function queries lineage information of single train processcorresponding to the given summary directory. Users can query theinformation according to keys.

  • Parameters

    • summary_dir (str) – The summary directory. It contains summary logs forone training.

    • keys (list[str]) – The filter keys of lineage information. The acceptablekeys are metric, hyper_parameters, algorithm, train_dataset,model, valid_dataset and dataset_graph. If it is None, allinformation will be returned. Default: None.

  • Returns

  • dict, the lineage information for one training.

  • Raises

    • LineageParamSummaryPathError – If summary path is invalid.

    • LineageQuerySummaryDataError – If querying summary data fails.

    • LineageFileNotFoundError – If the summary log file is not found.

Examples

  1. Copy>>> summary_dir = "/path/to/summary"
  2. >>> summary_lineage_info = get_summary_lineage(summary_dir)
  3. >>> hyper_parameters = get_summary_lineage(summary_dir, keys=["hyper_parameters"])
  • mindinsight.lineagemgr.filtersummary_lineage(_summary_base_dir, search_condition=None)[source]
  • Filter the lineage information under summary base directory according to search condition.

Users can filter and sort all lineage information according to the searchcondition. The supported filter fields include summary_dir, network,etc. The filter conditions include eq, lt, gt, le, ge and in.At the same time, the combined use of these fields and conditions issupported. If you want to sort based on filter fields, the field ofsorted_name and sorted_type should be specified.

Users can use lineage_type to decide what kind of lineage information toquery. If the lineage_type is dataset, the query result is only thelineage information related to data augmentation. If the lineage_type ismodel or None, the query result is all lineage information.

Users can paginate query result based on offset and limit. The offset_refers to page number. The _limit refers to the number in one page.

  • Parameters

    • summary_base_dir (str) – The summary base directory. It contains summarydirectories generated by training.

    • search_condition (dict) –

The search condition. When filtering andsorting, in addition to the following supported fields, fieldsprefixed with metric are also supported. The fields prefixed with_metric are related to the _metrics parameter in the trainingscript. For example, if the key of metrics parameter isaccuracy, the field should be metric_accuracy. Default: None.

  1.   -

summary_dir (dict): The filter condition of summary directory.

  1.   -

loss_function (dict): The filter condition of loss function.

  1.   -

train_dataset_path (dict): The filter condition of train dataset path.

  1.   -

train_dataset_count (dict): The filter condition of train dataset count.

  1.   -

test_dataset_path (dict): The filter condition of test dataset path.

  1.   -

test_dataset_count (dict): The filter condition of test dataset count.

  1.   -

network (dict): The filter condition of network.

  1.   -

optimizer (dict): The filter condition of optimizer.

  1.   -

learning_rate (dict): The filter condition of learning rate.

  1.   -

epoch (dict): The filter condition of epoch.

  1.   -

batch_size (dict): The filter condition of batch size.

  1.   -

loss (dict): The filter condition of loss.

  1.   -

model_size (dict): The filter condition of model size.

  1.   -

dataset_mark (dict): The filter condition of dataset mark.

  1.   -

offset (int): Page number, the value range is [0, 100000].

  1.   -

limit (int): The number in one page, the value range is [1, 100].

  1.   -

sorted_name (str): Specify which field to sort by.

  1.   -

sortedtype (str): Specify sort order. It can be _ascending ordescending.

  1.   -

lineagetype (str): It decides what kind of lineage information toquery. It can be _dataset or model. If it is dataset,the query result is only the lineage information related to dataaugmentation. If it is model or None, the query result is alllineage information.

  • Returns

  • dict, all lineage information under summary base directory according tosearch condition.

  • Raises

    • LineageSearchConditionParamError – If search_condition param is invalid.

    • LineageParamSummaryPathError – If summary path is invalid.

    • LineageFileNotFoundError – If the summary log file is not found.

    • LineageQuerySummaryDataError – If querying summary log file data fails.

Examples

  1. Copy>>> summary_base_dir = "/path/to/summary_base"
  2. >>> search_condition = {
  3. >>>     'summary_dir': {
  4. >>>         'in': [
  5. >>>             os.path.join(summary_base_dir, 'summary_1'),
  6. >>>             os.path.join(summary_base_dir, 'summary_2'),
  7. >>>             os.path.join(summary_base_dir, 'summary_3')
  8. >>>         ]
  9. >>>     },
  10. >>>     'loss': {
  11. >>>         'gt': 2.0
  12. >>>     },
  13. >>>     'batch_size': {
  14. >>>         'ge': 128,
  15. >>>         'le': 256
  16. >>>     },
  17. >>>     'metric_accuracy': {
  18. >>>         'lt': 0.1
  19. >>>     },
  20. >>>     'sorted_name': 'summary_dir',
  21. >>>     'sorted_type': 'descending',
  22. >>>     'limit': 3,
  23. >>>     'offset': 0,
  24. >>>     'lineage_type': 'model'
  25. >>> }
  26. >>> summary_lineage = filter_summary_lineage(summary_base_dir)
  27. >>> summary_lineage_filter = filter_summary_lineage(summary_base_dir, search_condition)