mindspore.parallel

This interface is ONLY used in Auto-parallel procedure.

• mindspore.parallel.setfusion_strategy_by_idx(_idxList, group='hccl_world_group')[source]
• A function set gradient segment strategy according to the index list.

Note

In the back propagation,the fusion of the allreduce operators with a fusion attribute equals 1,will be performed according to the idxList,to achieve the effect of parallel between calculation and communication.

• Parameters

• • idxList (list) – The index list of the gradient.

  • group (str) – The hccl communication group.

• Raises

• • TypeError – If group is not a python str.

  • TypeError – If IdxList is not a python list.

  • TypeError – If type of idxList item is not int.

  • ValueError – If group name length is out of range.

  • ValueError – If idxList length is 0.

  • ValueError – If idxList item is less than 0.

  • RuntimeError – If allreduce split failed.

• mindspore.parallel.setfusion_strategy_by_size(_dataSizeList, group='hccl_world_group')[source]
• A function set gradient segment strategy according to the data size percentage list.

Note

In the back propagation,the fusion of the allreduce operators with a fusion attribute equals 1,will be performed according to dataSizeList,to achieve the effect of parallel between calculation and communication.

• Parameters

• • dataSizeList (list) – The data size percentage list of the gradient.

  • group (str) – The hccl communication group.

• Raises

• • TypeError – If group is not a python str.

  • TypeError – If dataSizeList is not a python list.

  • TypeError – If type of dataSizeList item is not int or float.

  • ValueError – If group name length is out of range.

  • ValueError – If dataSizeList length is 0.

  • ValueError – If dataSizeList item is less than 0.

  • RuntimeError – If allreduce split failed.

• mindspore.parallel.getalgo_parameters(_attr_key)[source]
• Get algo parameter config attributes.

Note

Return value according to the attribute value.

• Parameters

• attr_key (str) – The key of the attribute.

• Raises

• ValueError – If context keyword is not recognized.

• mindspore.parallel.reset_algo_parameters()[source]
• Reset algo parameter attributes.

• mindspore.parallel.setalgo_parameters(**kwargs_)[source]
• Set algo parameter config.

Note

Attribute name is needed.

• Parameters

• • simplify_cal (bool) – Whether simplifying calculations in strategy-searching algorithm. Default: True

  • tensor_slice_align_enable (bool) – Whether checking tensor slice shape. Default: False

  • tensor_slice_align_size (int) – The minimum tensor slice shape, the value must be in [1, 1024]. Default: 16

  • not_fully_use_devices (bool) – Whether generating strategies that not fully use devices. Default: False

  • elementwise_op_strategy_follow (bool) – Whether the elementwise operator have the same strategies as itssubsequent operators. Default: False

• Raises

• ValueError – If context keyword is not recognized.