mindspore.nn

Neural Networks Cells.

Pre-defined building blocks or computing units to construct Neural Networks.

• class mindspore.nn.Accuracy(eval_type='classification')[source]
• Calculates the accuracy for classification and multilabel data.

The accuracy class creates two local variables, correct number and total number that are used to compute thefrequency with which predictions matches labels. This frequency is ultimately returned as the accuracy: anidempotent operation that simply divides correct number by total number.

• Parameters
• eval_type (str) – Metric to calculate the accuracy over a dataset, forclassification (single-label), and multilabel (multilabel classification).Default: ‘classification’.

Examples

Copy>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]]), mindspore.float32)
>>> y = Tensor(np.array([1, 0, 1]), mindspore.float32)
>>> metric = nn.Accuracy('classification')
>>> metric.clear()
>>> metric.update(x, y)
>>> accuracy = metric.eval()

• clear()[source]

• Clears the internal evaluation result.

• eval()[source]

• Computes the accuracy.

  • Returns

  • Float, the computed result.

  • Raises

  • RuntimeError – If the sample size is 0.

• update(*inputs)[source]

• Updates the internal evaluation result

and

.

- Parameters
- 

inputs – Input y_pred and y. y_pred and y are a Tensor, a list or an array.y_pred is in most cases (not strictly) a list of floating numbers in range

and the shape is

, where

is the number of cases and

is the number of categories. For ‘multilabel’ evaluation type, y_pred can only be one-hotencoding with values 0 or 1. Indices with 1 indicate positive category. y contains valuesof integers. The shape is

if one-hot encoding is used. One-hot encodingshould be used when ‘eval_type’ is ‘multilabel’. Shape can also be

if categoryindex is used in ‘classification’ evaluation type.

- Raises
- 

ValueError – If the number of the input is not 2.

• class mindspore.nn.Adam(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-08, use_locking=False, use_nesterov=False, weight_decay=0.0, loss_scale=1.0)[source]
• Updates gradients by Adaptive Moment Estimation (Adam) algorithm.

The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.

The updating formulas are as follows,

represents the 1st moment vector moment1,

represents the 2nd moment vector moment2,

represents gradients,

represents scaling factor lr,

representbeta1 and beta2,

represents updating step while

and

representbeta1_power and beta2_power,

represents learning_rate,

represents params,

represents eps.

• Parameters

• • params (list[Parameter]) – A list of parameter, which will be updated. The element in _params_should be class mindspore.Parameter.

  • learning_rate (float) – The Learning rate.

  • beta1 (float) – The exponential decay rate for the 1st moment estimates. Should be in range (0.0, 1.0).

  • beta2 (float) – The exponential decay rate for the 2nd moment estimates. Should be in range (0.0, 1.0).

  • eps (float) – Term added to the denominator to improve numerical stability. Should be greater than 0.

  • use_locking (bool) – Whether to enable a lock to protect updating variable tensors.If True, updating of the var, m, and v tensors will be protected by a lock.If False, the result is unpredictable. Default: False.

  • use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients.If True, updates the gradients using NAG.If False, updates the gradients without using NAG. Default: False.

  • weight_decay (float) – Weight decay (L2 penalty). Default: 0.0.

  • loss_scale (float) – A floating point value for the loss scale. Default: 1.0.Should be equal to or greater than 1.

• Inputs:

• • gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.

• Outputs:

• Tensor[bool], the value is True.

Examples

Copy>>> net = Net()
>>> loss = nn.SoftmaxCrossEntropyWithLogits()
>>> optim = nn.Adam(params=net.trainable_params())
>>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None)

• class mindspore.nn.AdamWeightDecay(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-06, weight_decay=0.0)[source]

• Implements Adam algorithm weight decay fix.

  • Parameters

  • • params (list[Parameter]) – A list of parameter, which will be updated. The element in _params_should be class mindspore.Parameter.

    • learning_rate (float) – A floating point value for the learning rate. Default: 1e-3.

    • beta1 (float) – The exponential decay rate for the 1st moment estimates. Default: 0.9.Should be in range (0.0, 1.0).

    • beta2 (float) – The exponential decay rate for the 2nd moment estimates. Default: 0.999.Should be in range (0.0, 1.0).

    • eps (float) – Term added to the denominator to improve numerical stability. Default: 1e-6.Should be greater than 0.

    • weight_decay (float) – Weight decay (L2 penalty). Default: 0.0.

  • Inputs:

  • • gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.

  • Outputs:

  • tuple[Parameter], the updated velocity value, the shape is the same as params.

Examples

Copy>>> net = Net()
>>> loss = nn.SoftmaxCrossEntropyWithLogits()
>>> optim = nn.AdamWeightDecay(params=net.trainable_params())
>>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None)

• class mindspore.nn.AdamWeightDecayDynamicLR(params, decay_steps, learning_rate=0.001, end_learning_rate=0.0001, power=10.0, beta1=0.9, beta2=0.999, eps=1e-06, weight_decay=0.0)[source]

• Adam Weight Decay Dynamic Learning Rate (LR).

  • Parameters

  • • params (list[Parameter]) – A list of parameter, which will be updated. The element in _params_should be class mindspore.Parameter.

    • decay_steps (int) – The steps of the decay.

    • learning_rate (float) – A floating point value for the learning rate. Default: 0.001.

    • end_learning_rate (float) – A floating point value for the end learning rate. Default: 0.0001.

    • power (float) – Power. Default: 10.0.

    • beta1 (float) – The exponential decay rate for the 1st moment estimates. Default: 0.9.Should be in range (0.0, 1.0).

    • beta2 (float) – The exponential decay rate for the 2nd moment estimates. Default: 0.999.Should be in range (0.0, 1.0).

    • eps (float) – Term added to the denominator to improve numerical stability. Default: 1e-6.Should be greater than 0.

    • weight_decay (float) – Weight decay (L2 penalty). Default: 0.0.

  • Inputs:

  • • gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.

  • Outputs:

  • tuple[Parameter], the updated velocity value, the shape is the same as params.

Examples

Copy>>> net = Net()
>>> loss = nn.SoftmaxCrossEntropyWithLogits()
>>> optim = nn.AdamWeightDecayDynamicLR(params=net.trainable_params(), decay_steps=10)
>>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None)

• class mindspore.nn.AvgPool2d(kernel_size=1, stride=1, pad_mode='VALID', padding=0)[source]
• Average pooling for temporal data.

Applies a 2D average pooling over an input Tensor which can be regarded as a composition of 2D input planes.

Typically the input is of shape

, AvgPool2d outputsregional average in the

-dimension. Given kernel size

and stride

, the operation is as follows.

Note

pad_mode for training only supports “same” and “valid”.

• Parameters

• • kernel_size (int) – Size of the window to take a max over.

  • stride (int) – Stride size of the window. Default: None.

  • pad_mode (str) –

Select the mode of the pad. The optional values are“same”, “valid”. Default: “valid”.

  - 

same: Adopts the way of completion. Output height and width will be the same asthe input. Total number of padding will be calculated for horizontal and verticaldirection and evenly distributed to top and bottom, left and right if possible. Otherwise, thelast extra padding will be done from the bottom and the right side.

  - 

valid: Adopts the way of discarding. The possibly largest height and width of output will be returnwithout padding. Extra pixels will be discarded.

- 

padding (int) – Now is not supported, implicit zero padding to be added on both sides. Default: 0.

• Inputs:
• • input (Tensor) - Tensor of shape

.

• Outputs:
• Tensor of shape

.

Examples

Copy>>> pool = AvgPool2d(kernel_size=3, stride=1)
>>> x = Tensor(np.random.randint(0, 10, [1, 2, 4, 4]), mindspore.float32)
[[[[5. 5. 9. 9.]
    [8. 4. 3. 0.]
    [2. 7. 1. 2.]
    [1. 8. 3. 3.]]
   [[6. 8. 2. 4.]
    [3. 0. 2. 1.]
    [0. 8. 9. 7.]
    [2. 1. 4. 9.]]]]
>>> output = pool(x)
>>> output.shape()
(1, 2, 2, 2)
>>> output
[[[[4.888889  4.4444447]
   [4.111111  3.4444444]]
  [[4.2222223 4.5555553]
   [3.2222223 4.5555553]]]]

• class mindspore.nn.BatchNorm1d(num_features, eps=1e-05, momentum=0.9, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=True)[source]
• Batch normalization layer over a 2D input.

Batch Normalization is widely used in convolutional networks. This layerapplies Batch Normalization over a 2D input (a mini-batch of 1D inputs) toreduce internal covariate shift as described in the paperBatch Normalization: Accelerating Deep Network Training byReducing Internal Covariate Shift. Itrescales and recenters the feature using a mini-batch of data andthe learned parameters which can be described in the following formula.

• Parameters

• • num_features (int) – C from an expected input of size (N, C).

  • eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.

  • momentum (float) – A floating hyperparameter of the momentum for therunning_mean and running_var computation. Default: 0.9.

  • gamma_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight.The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’,‘he_uniform’, etc. Default: ‘ones’.

  • beta_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight.The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’,‘he_uniform’, etc. Default: ‘zeros’.

  • moving_mean_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean.The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’,‘he_uniform’, etc. Default: ‘zeros’.

  • moving_var_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance.The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’,‘he_uniform’, etc. Default: ‘ones’.

  • use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data, else usethe mean value and variance value of specified value. Default: True.

• Inputs:

• • input (Tensor) - Tensor of shape

.

• Outputs:
• Tensor, the normalized, scaled, offset tensor, of shape

.

Examples

Copy>>> net = nn.BatchNorm1d(num_features=16)
>>> input = Tensor(np.random.randint(0, 255, [3, 16]), mindspore.float32)
>>> net(input)

• class mindspore.nn.BatchNorm2d(num_features, eps=1e-05, momentum=0.9, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=True)[source]
• Batch normalization layer over a 4D input.

Batch Normalization is widely used in convolutional networks. This layerapplies Batch Normalization over a 4D input (a mini-batch of 2D inputs withadditional channel dimension) to avoid internal covariate shift as describedin the paper Batch Normalization: Accelerating Deep Network Training byReducing Internal Covariate Shift. Itrescales and recenters the feature using a mini-batch of data andthe learned parameters which can be described in the following formula.

• Parameters

• • num_features (int) – C from an expected input of size (N, C, H, W).

  • eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.

  • momentum (float) – A floating hyperparameter of the momentum for therunning_mean and running_var computation. Default: 0.9.

  • gamma_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight.The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’,‘he_uniform’, etc. Default: ‘ones’.

  • beta_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight.The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’,‘he_uniform’, etc. Default: ‘zeros’.

  • moving_mean_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean.The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’,‘he_uniform’, etc. Default: ‘zeros’.

  • moving_var_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance.The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’,‘he_uniform’, etc. Default: ‘ones’.

  • use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data, else usethe mean value and variance value of specified value. Default: True.

• Inputs:

• • input (Tensor) - Tensor of shape

.

• Outputs:
• Tensor, the normalized, scaled, offset tensor, of shape

.

Examples

Copy>>> net = nn.BatchNorm2d(num_features=3)
>>> input = Tensor(np.random.randint(0, 255, [1, 3, 224, 224]), mindspore.float32)
>>> net(input)

• class mindspore.nn.Cell(auto_prefix=True)[source]
• Base class for all neural network.

A ‘Cell’ could be a single neural network cell, such as conv2d, relu, batch_norm, etc. or a composition ofcells to constructing a network.

Note

In general, the autograd algorithm will automatically generate the implementation of the gradient function,but if bprop method is implemented, the gradient functionwill be replaced by the bprop. The bprop implementation will receive a Tensor dout containing the gradientof the loss w.r.t. the output, and a Tensor out containing the forward result. The bprop need to compute thegradient of the loss w.r.t. the inputs, gradient of the loss w.r.t. Parameter variables is not supportedcurrently.

• Parameters
• auto_prefix (bool) – Recursively generate namespaces. Default: True.

Examples

Copy>>> class MyCell(Cell):
>>>    def __init__(self):
>>>        super(MyCell, self).__init__()
>>>        self.relu = P.ReLU()
>>>
>>>    def construct(self, x):
>>>        return self.relu(x)

• cells()[source]

• Returns an iterator over immediate cells.

• cellsand_names(_cells=None, name_prefix='')[source]

• Returns an iterator over all cells in the network.

Includes the cell’s name and itself.

- Parameters
- 
  - 

cells (str) – Cells to iterate over. Default: None.

  - 

name_prefix (str) – Namespace. Default: ‘’.

Examples

Copy>>> n = Net()
>>> names = []
>>> for m in n.cells_and_names():
>>>     if m[0]:
>>>         names.append(m[0])

• compileand_run(*inputs_)[source]

• Compiles and runs cell.

  • Parameters

  • inputs (tuple) – Input parameters.

  • Returns

  • Object, the result of executing.

• construct(*inputs)[source]

• Defines the computation to be performed.

This method should be overridden by all subclasses.

Note

The inputs of the top cell only allow Tensor.Other types (tuple, list, int etc.) are forbidden.

- Returns
- 

Tensor, returns the computed result.

• exec_checkpoint_graph()[source]

• Executes saving checkpoint graph operation.

• extend_repr()[source]

• Sets the extended representation of the Cell.

To print customized extended information, re-implement this method in your own cells.

• generate_scope()[source]

• Generate the scope for every cell object in the network.

• get_func_graph_proto()[source]

• Return graph binary proto.

• getparameters(_expand=True)[source]

• Returns an iterator over cell parameters.

Yields parameters of this cell. If expand is True, yield parameters of this cell and all subcells.

- Parameters
- 

expand (bool) – If True, yields parameters of this cell and all subcells. Otherwise, yields only parametersthat are direct members of this cell. Default: True.

Examples

Copy>>> net = Net()
>>> for item in net.get_parameters():
>>>     print(item)

• get_scope()[source]

• Returns the scope of a cell object in one network.

• insertchild_to_cell(_child_name, child)[source]

• Adds a child cell to the current cell.

Inserts a subcell with given name to current cell.

- Parameters
- 
  - 

child_name (str) – Name of the child cell.

  - 

child (Cell) – The child cell to be inserted.

- Raises
- 
  - 

KeyError – Child Cell’s name is incorrect or duplicated with the other child name.

  - 

TypeError – Child Cell’s type is incorrect.

• insertparam_to_cell(_param_name, param, check_name=True)[source]
• Adds a parameter to the current cell.

Inserts a parameter with given name to the cell. Please refer to the usage insource code of mindspore.nn.Cell.setattr.

- Parameters
- 
  - 

param_name (str) – Name of the parameter.

  - 

param (Parameter) – Parameter to be inserted to the cell.

  - 

check_name (bool) – Determines whether the name input is compatible. Default: True.

- Raises
- 
  - 

KeyError – If the name of parameter is null or contains dot.

  - 

AttributeError – If user did not call init() first.

  - 

TypeError – If the type of parameter is not Parameter.

• loadparameter_slice(_params)[source]
• Replace parameters with sliced tensors by parallel strategies.

Please refer to the usage in source code of mindspore.common._Executor.compile.

- Parameters
- 

params (dict) – The parameters dictionary used for init data graph.

• name_cells()[source]
• Returns an iterator over all cells in the network.

Include name of the cell and cell itself.

• parametersand_names(_name_prefix='', expand=True)[source]
• Returns an iterator over cell parameters.

Includes the parameter’s name and itself.

- Parameters
- 
  - 

name_prefix (str) – Namespace. Default: ‘’.

  - 

expand (bool) – If True, yields parameters of this cell and all subcells. Otherwise, yields only parametersthat are direct members of this cell. Default: True.

Examples

Copy>>> n = Net()
>>> names = []
>>> for m in n.parameters_and_names():
>>>     if m[0]:
>>>         names.append(m[0])

• parametersdict(_recurse=True)[source]
• Gets parameters dictionary.

Gets the parameters dictionary of this cell.

- Parameters
- 

recurse (bool) – Whether contains the parameters of subcells. Default: True.

- Returns
- 

OrderedDict, return parameters dictionary.

• setbroadcast_flag(_mode=True)[source]
• Set the cell to data_parallel mode.

The cell can be accessed as an attribute using the given name.

- Parameters
- 

mode (bool) – Specifies whether the model is data_parallel. Default: True.

• settrain(_mode=True)[source]
• Sets the cell to training mode.

The cell itself and all children cells will be set to training mode.

- Parameters
- 

mode (bool) – Specifies whether the model is training. Default: True.

• tofloat(_dst_type)[source]
• Add cast on all inputs of cell and child cells to run with certain float type.

If dst_type is mindspore.dtype.float16, all the inputs of Cell including input, Parameter, Tensoras const will be cast to float16. Please refer to the usage in source code ofmindspore.train.amp.build_train_network.

Note

Call multiple times will overwrite the previous.

- Parameters
- 

dst_type (mindspore.dtype) – Transfer Cell to Run with dsttype.dst_type can be _mindspore.dtype.float16 or mindspore.dtype.float32.

- Raises
- 

ValueError – If dst_type is not float32 or float16.

• trainableparams(_recurse=True)[source]
• Returns all trainable parameters.

Returns a list of all trainable parmeters.

- Parameters
- 

recurse (bool) – Whether contains the trainable parameters of subcells. Default: True.

- Returns
- 

List, the list of trainable parameters.

• untrainableparams(_recurse=True)[source]
• Returns all untrainable parameters.

Returns a list of all untrainable parmeters.

- Parameters
- 

recurse (bool) – Whether contains the untrainable parameters of subcells. Default: True.

- Returns
- 

List, the list of untrainable parameters.

• updateparameters_name(_prefix='', recurse=True)[source]
• Updates the names of parameters with given prefix string.

Adds the given prefix to the names of parameters.

- Parameters
- 
  - 

prefix (str) – The prefix string.

  - 

recurse (bool) – Whether contains the parameters of subcells. Default: True.

• class mindspore.nn.CellList(*args)[source]
• Holds Cells in a list.

CellList can be indexed like a regular Python list, but cells itcontains are properly registered, and will be visible by all Cell methods.

• Parameters
• args (list, __optional) – List of subclass of Cell.

Examples

Copy>>> conv = nn.Conv2d(100, 20, 3)
>>> bn = nn.BatchNorm2d(20)
>>> relu = nn.ReLU()
>>> cell_ls = nn.CellList([bn])
>>> cell_ls.insert(0, conv)
>>> cell_ls.append(relu)
>>> x = Tensor(np.random.random((1, 3, 4, 4)), dtype=mindspore.float32)
>>> # not same as nn.SequentialCell, `cell_ls(x)` is not correct
>>> cell_ls
CellList< (0): Conv2d<input_channels=100, ..., bias_init=None>
          (1): BatchNorm2d<num_features=20, ..., moving_variance=Parameter (name=variance)>
          (2): ReLU<> >

• append(cell)[source]

• Appends a given cell to the end of the list.

• extend(cells)[source]

• Appends cells from a Python iterable to the end of the list.

  • Raises
  • TypeError – If the cells is not a list of subcells.

• insert(index, cell)[source]

• Inserts a given cell before a given index in the list.

• class mindspore.nn.ClipByNorm[source]
• Clips tensor values to a maximum

-norm.

The output of this layer remains the same if the

-norm of the input tensoris not greater than the argument clip_norm. Otherwise the tensor will be normalized as:

where

is the

-norm of

.

• Inputs:

• • input (Tensor) - Tensor of shape N-D.

  • clip_norm (Tensor) - A scalar Tensor of shape

or

and ofthe same type as the input Tensor.

• Outputs:
• Tensor, clipped tensor with the same shape as the input.

Examples

Copy>>> net = nn.ClipByNorm()
>>> input = Tensor(np.random.randint(0, 10, [4, 16]), mindspore.float32)
>>> clip_norm = Tensor(np.array([100]).astype(np.float32))
>>> net(input, clip_norm)

• class mindspore.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')[source]
• 2D convolution layer.

Applies a 2D convolution over an input tensor which is typically of shape

,where

is batch size and

is channel number. For each batch of shape

, the formula is defined as:

where

is cross correlation operator,

is the input channel number,

rangesfrom

to

,

corresponds to

-th channel of the

-thfilter and

corresponds to the

-th channel of the output.

is a sliceof kernel and it has shape

, where

and

are height and width of the convolution kernel. The full kernel has shape

, where group is the group numberto split the input in the channel dimension.

If the ‘pad_mode’ is set to be “valid”, the output height and width will be

and

respectively.

The first introduction can be found in paper Gradient Based Learning Applied to Document Recognition.

• Parameters
• • in_channels (int) – The number of input channel

.

- 

out_channels (int) – The number of output channel

.

- 

kernel_size (Union__[int, tuple]) – The data type is int or tuple with 2 integers. Specifies the heightand width of the 2D convolution window. Single int means the value if for both height and width ofthe kernel. A tuple of 2 ints means the first value is for the height and the other is for thewidth of the kernel.

- 

stride (int) – Specifies stride for all spatial dimensions with the same value. Value of stride should begreater or equal to 1 but bounded by the height and width of the input. Default: 1.

- 

pad_mode (str) –

Specifies padding mode. The optional values are“same”, “valid”, “pad”. Default: “same”.

  - 

same: Adopts the way of completion. Output height and width will be the same as the input.Total number of padding will be calculated for horizontal and verticaldirection and evenly distributed to top and bottom, left and right if possible. Otherwise, thelast extra padding will be done from the bottom and the right side. If this mode is set, _padding_must be 0.

  - 

valid: Adopts the way of discarding. The possibly largest height and width of output will be returnwithout padding. Extra pixels will be discarded. If this mode is set, _padding_must be 0.

  - 

pad: Implicit paddings on both sides of the input. The number of padding will be padded to the inputTensor borders. padding should be greater than or equal to 0.

- 

padding (int) – Implicit paddings on both sides of the input. Default: 0.

- 

dilation (int) – Specifying the dilation rate to use for dilated convolution. If set to be

,there will be

pixels skipped for each sampling location. Its value should be greateror equal to 1 and bounded by the height and width of the input. Default: 1.

- 

group (int) – Split filter into groups, in channels and _out_channels should bedivisible by the number of groups. Default: 1.

- 

has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.

- 

weight_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel.It can be a Tensor, a string, an Initializer or a numbers.Number. When a string is specified,values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as wellas constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values ofInitializer for more details. Default: ‘normal’.

- 

bias_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. PossibleInitializer and string are the same as ‘weight_init’. Refer to the values ofInitializer for more details. Default: ‘zeros’.

• Inputs:
• • input (Tensor) - Tensor of shape

.

• Outputs:
• Tensor of shape

.

Examples

Copy>>> net = nn.Conv2d(120, 240, 4, has_bias=False, weight_init='normal')
>>> input = Tensor(np.ones([1, 120, 1024, 640]), mindspore.float32)
>>> net(input).shape()
(1, 240, 1024, 637)

• class mindspore.nn.Conv2dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')[source]
• 2D transposed convolution layer.

Compute a 2D transposed convolution, which is also know as a deconvolution(although it is not actual deconvolution).

Input is typically of shape

, where

is batch size and

is channel number.

• Parameters

• • in_channels (int) – The number of channels in the input space.

  • out_channels (int) – The number of channels in the output space.

  • kernel_size (Union__[int, tuple]) – int or tuple with 2 integers, which specifies the heightand width of the 2D convolution window. Single int means the value is for both height and width ofthe kernel. A tuple of 2 ints means the first value is for the height and the other is for thewidth of the kernel.

  • stride (int) – Specifies the same value for all spatial dimensions. Default: 1.

  • pad_mode (str) –

Select the mode of the pad. The optional values are“pad”, “same”, “valid”. Default: “same”.

  - 

pad: Implicit paddings on both sides of the input.

  - 

same: Adopted the way of completion.

  - 

valid: Adopted the way of discarding.

- 

padding (int) – Implicit paddings on both sides of the input. Default: 0.

- 

dilation (int) – Specifies the dilation rate to use for dilatedconvolution. Default: 1.

- 

group (int) – Split filter into groups, in_channels and out_channels should bedivisible by the number of groups. Default: 1.

- 

has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.

- 

weight_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel.It can be a Tensor, a string, an Initializer or a numbers.Number. When a string is specified,values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as wellas constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values ofInitializer for more details. Default: ‘normal’.

- 

bias_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. PossibleInitializer and string are the same as ‘weight_init’. Refer to the values ofInitializer for more details. Default: ‘zeros’.

• Inputs:
• • input (Tensor) - Tensor of shape

.

• Outputs:
• Tensor of shape

.

Examples

Copy>>> net = nn.Conv2dTranspose(3, 64, 4, has_bias=False, weight_init='normal')
>>> input = Tensor(np.ones([1, 3, 16, 50]), mindspore.float32)
>>> net(input)

• class mindspore.nn.DataWrapper(network, dataset_types, dataset_shapes, queue_name)[source]
• Network training package class for dataset.

DataWrapper wraps the input network with a dataset which automatically fetches data with ‘GetNext’function from the dataset channel ‘queue_name’ and does forward computation in the construct function.

• Parameters

• • network (Cell) – The training network for dataset.

  • dataset_types (list) – The type of dataset. The list contains describes the types of the inputs.

  • dataset_shapes (list) – The shapes of dataset. The list contains multiple sublists that describesthe shape of the inputs.

  • queue_name (str) – The identification of dataset channel which specifies the dataset channel to supplydata for the network.

• Outputs:

• Tensor, network output whose shape depends on the network.

Examples

Copy>>> # call create_dataset function to create a regular dataset, refer to mindspore.dataset
>>> train_dataset = create_dataset()
>>> dataset_helper = mindspore.DatasetHelper(train_dataset)
>>> net = Net()
>>> net = DataWrapper(net, *(dataset_helper.types_shapes()), train_dataset.queue_name)

• class mindspore.nn.Dense(in_channels, out_channels, weight_init='normal', bias_init='zeros', has_bias=True, activation=None)[source]
• The fully connected layer.

Applies dense-connected layer for the input. This layer implements the operation as:

where

is the activation function passed as the activationargument (if passed in),

is a weight matrix with the samedata type as the inputs created by the layer, and

is a bias vectorwith the same data type as the inputs created by the layer (only if has_bias is True).

• Parameters

• • in_channels (int) – The number of channels in the input space.

  • out_channels (int) – The number of channels in the output space.

  • weight_init (Union__[Tensor, str, Initializer, numbers.Number]) – The trainable weightinit parameter. The dtypeis same as input x. The values of str refer to the function _initializer. Default: ‘normal’.

  • bias_init (Union__[Tensor, str, Initializer, numbers.Number]) – The trainable biasinit parameter. The dtype issame as input x. The values of str refer to the function _initializer. Default: ‘zeros’.

  • has_bias (bool) – Specifies whether the layer uses a bias vector. Default: True.

  • activation (str) – Regularizer function applied to the output of the layer, eg. ‘relu’. Default: None.

• Raises

• ValueError – If weight_init or bias_init shape is incorrect.

• Inputs:

• • input (Tensor) - Tensor of shape

.

• Outputs:
• Tensor of shape

.

Examples

Copy>>> net = nn.Dense(3, 4)
>>> input = Tensor(np.random.randint(0, 255, [2, 3]), mindspore.float32)
>>> net(input)
[[ 2.5246444   2.2738023   0.5711005  -3.9399147 ]
 [ 1.0739875   4.0155234   0.94188046 -5.459526  ]]

• class mindspore.nn.DistributedGradReducer(parameters, mean=True, degree=None)[source]
• A distributed optimizer.

Constructs a gradient reducer Cell, which applies communication and average operations onsingle-process gradient values.

• Parameters

• • parameters (list) – the parameters to be updated.

  • mean (bool) – When mean is true, the mean coefficient (degree) would apply on gradients. Default: False.

  • degree (int) – The mean coefficient. Usually it equals to device number. Default: None.

• Raises

• ValueError – If degree is not a int or less than 0.

Examples

Copy>>> from mindspore.communication import init, get_group_size
>>> from mindspore.ops import composite as C
>>> from mindspore.ops import operations as P
>>> from mindspore.ops import functional as F
>>> from mindspore import context
>>> from mindspore import nn
>>> from mindspore import ParallelMode, ParameterTuple
>>>
>>> device_id = int(os.environ["DEVICE_ID"])
>>> context.set_context(mode=context.GRAPH_MODE, device_target="Davinci", save_graphs=True,
>>>                     device_id=int(devie_id), enable_hccl=True)
>>> init()
>>> context.reset_auto_parallel_context()
>>> context.set_auto_parallel_context(parallel_mode=ParallelMode.DATA_PARALLEL)
>>>
>>>
>>>
>>>
>>> class TrainingWrapper(nn.Cell):
>>>     def __init__(self, network, optimizer, sens=1.0):
>>>         super(TrainingWrapper, self).__init__(auto_prefix=False)
>>>         self.network = network
>>>         self.network.add_flags(defer_inline=True)
>>>         self.weights = ParameterTuple(network.trainable_params())
>>>         self.optimizer = optimizer
>>>         self.grad = C.GradOperation('grad', get_by_list=True, sens_param=True)
>>>         self.sens = sens
>>>         self.reducer_flag = False
>>>         self.grad_reducer = None
>>>         self.parallel_mode = context.get_auto_parallel_context("parallel_mode")
>>>         if self.parallel_mode in [ParallelMode.DATA_PARALLEL,
>>>                                            ParallelMode.HYBRID_PARALLEL]:
>>>             self.reducer_flag = True
>>>         if self.reducer_flag:
>>>             mean = context.get_auto_parallel_context("mirror_mean")
>>>             if mean.get_device_num_is_set():
>>>                 degree = context.get_auto_parallel_context("device_num")
>>>             else:
>>>                 degree = get_group_size()
>>>             self.grad_reducer = nn.DistributedGradReducer(optimizer.parameters, mean, degree)
>>>
>>>     def construct(self, *args):
>>>         weights = self.weights
>>>         loss = self.network(*args)
>>>         sens = P.Fill()(P.DType()(loss), P.Shape()(loss), self.sens)
>>>         grads = self.grad(self.network, weights)(*args, sens)
>>>         if self.reducer_flag:
>>>             # apply grad reducer on grads
>>>             grads = self.grad_reducer(grads)
>>>         return F.depend(loss, self.optimizer(grads))
>>>
>>> network = Net()
>>> optimizer = nn.Momentum(network.trainable_params(), learning_rate=0.1, momentum=0.9)
>>> train_cell = TrainingWrapper(network, optimizer)
>>> inputs = Tensor(np.ones([16, 16]).astype(np.float32))
>>> label = Tensor(np.zeros([16, 16]).astype(np.float32))
>>> grads = train_cell(inputs, label)

• class mindspore.nn.Dropout(keep_prob=0.5, seed0=0, seed1=0, dtype=mindspore.float32)[source]
• Dropout layer for the input.

Randomly set some elements of the input tensor to zero with probability

during trainingusing samples from a Bernoulli distribution.

Note

Each channel will be zeroed out independently on every construct call.

The outputs are scaled by a factor of

during training sothat the output layer remains at a similar scale. During inference, thislayer returns the same tensor as the input.

This technique is proposed in paper Dropout: A Simple Way to Prevent Neural Networks from Overfitting and proved to be effective to reduceover-fitting and prevents neurons from co-adaptation. See more details in Improving neural networks bypreventing co-adaptation of feature detectors.

• Parameters

• • keep_prob (float) – The keep rate, greater than 0 and less equal than 1. E.g. rate=0.9,dropping out 10% of input units. Default: 0.5.

  • seed0 (int) – The first random seed. Default: 0.

  • seed1 (int) – The second random seed. Default: 0.

  • dtype (mindspore.dtype) – Data type of input. Default: mindspore.float32.

• Raises

• ValueError – If keep_prob is not in range (0, 1).

• Inputs:

• • input (Tensor) - An N-D Tensor.

• Outputs:

• Tensor, output tensor with the same shape as the input.

Examples

Copy>>> x = Tensor(np.ones([20, 16, 50]), mindspore.float32)
>>> net = nn.Dropout(keep_prob=0.8)
>>> net(x)

• class mindspore.nn.DynamicLossScaleUpdateCell(loss_scale_value, scale_factor, scale_window)[source]
• Dynamic Loss scale update cell.

For loss scaling training, the initial loss scaling value will be set to be loss_scale_value.In every training step, the loss scaling value will be updated by loss scaling value/scale_factor_when there is overflow. And it will be increased by loss scaling value * _scale_factor if there is nooverflow for a continuous scale_window steps. This cell is used for Graph mode training in which alllogic will be executed on device side(Another training mode is feed mode in which some logic will beexecuted on host).

• Parameters

• • loss_scale_value (float) – Init loss scale.

  • scale_factor (int) – Coefficient of increase and decrease.

  • scale_window (int) – Maximum continuous training steps that do not have overflow.

• Inputs:

• • inputs (Tensor) - Tensor of shape

.

- 

label (Tensor) - Tensor of shape

.

• Outputs:
• Tensor, a scalar Tensor with shape

.

Examples

Copy>>> net_with_loss = Net()
>>> optimizer = nn.Momentum(net_with_loss.trainable_params(), learning_rate=0.1, momentum=0.9)
>>> manager = nn.DynamicLossScaleUpdateCell(loss_scale_value=2**12, scale_factor=2, scale_window=1000)
>>> train_network = nn.TrainOneStepWithLossScaleCell(net_with_loss, optimizer, scale_update_cell=manager)
>>> train_network.set_train()
>>>
>>> inputs = Tensor(np.ones([16, 16]).astype(np.float32))
>>> label = Tensor(np.zeros([16, 16]).astype(np.float32))
>>> output = train_network(inputs, label)

• class mindspore.nn.Embedding(vocab_size, embedding_size, use_one_hot=False, embedding_table='normal', dtype=mindspore.float32)[source]
• A simple lookup table that stores embeddings of a fixed dictionary and size.

This module is often used to store word embeddings and retrieve them usingindices. The input to the module is a list of indices, and the output isthe corresponding word embeddings.

Note

When ‘use_one_hot’ is set to True, the input should be of type mindspore.int32.

• Parameters

• • vocab_size (int) – Size of the dictionary of embeddings.

  • embedding_size (int) – The size of each embedding vector.

  • use_one_hot (bool) – Specifies whether to apply one_hot encoding form. Default: False.

  • embedding_table (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the embeddingtable.Refer to class _initializer for the values of string when a stringis specified. Default: ‘normal’.

  • dtype (mindspore.dtype) – Data type of input. Default: mindspore.float32.

• Inputs:

• • input (Tensor) - Tensor of shape

.

• Outputs:
• Tensor of shape

.

Examples

Copy>>> net = nn.Embedding(20000, 768,  True)
>>> input_data = Tensor(np.ones([8, 128]), mindspore.int32)
>>>
>>> # Maps the input word IDs to word embedding.
>>> output = net(input_data)
>>> output.shape()
(8, 128, 768)

• class mindspore.nn.EvaluationBase(eval_type)[source]
• Base class of evaluation.

Note

Please refer to the definition of class Accuracy.

• Parameters

• eval_type (str) – Type of evaluation must be in {‘classification’, ‘multilabel’}.

• Raises

• TypeError – If the input type is not classification or multilabel.

• clear()[source]

• A interface describes the behavior of clearing the internal evaluation result.

Note

All subclasses should override this interface.

• eval()[source]
• A interface describes the behavior of computing the evaluation result.

Note

All subclasses should override this interface.

• update(*inputs)[source]
• A interface describes the behavior of updating the internal evaluation result.

Note

All subclasses should override this interface.

- Parameters
- 

inputs – The first item is predicted array and the second item is target array.

• class mindspore.nn.F1[source]
• Calculates the F1 score. F1 is a special case of Fbeta when beta is 1.Refer to class Fbeta for more details.

Examples

Copy>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]]))
>>> y = Tensor(np.array([1, 0, 1]))
>>> metric = nn.F1()
>>> metric.update(x, y)
>>> fbeta = metric.eval()

• class mindspore.nn.Fbeta(beta)[source]
• Calculates the fbeta score.

Fbeta score is a weighted mean of precison and recall.

• Parameters
• beta (float) – The weight of precision.

Examples

Copy>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]]))
>>> y = Tensor(np.array([1, 0, 1]))
>>> metric = nn.Fbeta(1)
>>> metric.update(x, y)
>>> fbeta = metric.eval()
[0.66666667 0.66666667]

• clear()[source]

• Clears the internal evaluation result.

• eval(average=False)[source]

• Computes the fbeta.

  • Parameters

  • average (bool) – Whether to calculate the average fbeta. Default value is False.

  • Returns

  • Float, computed result.

• update(*inputs)[source]

• Updates the internal evaluation result y_pred and y.

  • Parameters
  • inputs – Input y_pred and y. y_pred and y are Tensor, list or numpy.ndarray.y_pred is in most cases (not strictly) a list of floating numbers in range

and the shape is

, where

is the number of cases and

is the number of categories. y contains values of integers. The shape is

if one-hot encoding is used. Shape can also be

if category index is used.

• class mindspore.nn.FixedLossScaleUpdateCell(loss_scale_value)[source]
• Static scale update cell, the loss scaling value will not be updated.

For usage please refer to DynamicLossScaleUpdateCell.

• Parameters
• loss_scale_value (float) – Init loss scale.

Examples

Copy>>> net_with_loss = Net()
>>> optimizer = nn.Momentum(net_with_loss.trainable_params(), learning_rate=0.1, momentum=0.9)
>>> manager = nn.FixedLossScaleUpdateCell(loss_scale_value=2**12)
>>> train_network = nn.TrainOneStepWithLossScaleCell(net_with_loss, optimizer, scale_update_cell=manager)
>>> train_network.set_train()
>>>
>>> inputs = Tensor(np.ones([16, 16]).astype(np.float32))
>>> label = Tensor(np.zeros([16, 16]).astype(np.float32))
>>> output = train_network(inputs, label)

• class mindspore.nn.Flatten[source]
• Flatten layer for the input.

Flattens a tensor without changing dimension of batch size on the 0-th axis.

• Inputs:
• • input (Tensor) - Tensor of shape

to be flattened.

• Outputs:
• Tensor, the shape of the output tensor is

, where

isthe product of the remaining dimensions.

Examples

Copy>>> net = nn.Flatten()
>>> input = Tensor(np.array([[[1.2, 1.2], [2.1, 2.1]], [[2.2, 2.2], [3.2, 3.2]]]), mindspore.float32)
>>> input.shape()
(2, 2, 2)
>>> net(input)
[[1.2 1.2 2.1 2.1]
 [2.2 2.2 3.2 3.2]]

• class mindspore.nn.GELU[source]
• Gaussian error linear unit activation function.

Applies GELU function to each element of the input. The input is a Tensor with any valid shape.

GELU is defined as:

, where

is the cumulative distribution functionof standard Gaussian distribution and

is the element of the input.

• Inputs:
• • input_data (Tensor) - The input of Tanh.

• Outputs:

• Tensor, with the same type and shape as the input_data.

• class mindspore.nn.GetNextSingleOp(dataset_types, dataset_shapes, queue_name)[source]

• Cell to run get next operation.

  • Parameters

  • • dataset_types (list[mindspore.dtype]) – The types of dataset.

    • dataset_shapes (list[tuple[int]__]) – The shapes of dataset.

    • queue_name (str) – Queue name to fetch the data.

Detailed information, please refer to ops.operations.GetNext.

• class mindspore.nn.L1Loss(reduction='mean')[source]
• L1Loss creates a criterion to measure the mean absolute error (MAE) between

and

by element,where

is the input Tensor and

is the target Tensor.

For simplicity, let

and

be 1-dimensional Tensor with length

,the unreduced loss (i.e. with argument reduction set to ‘none’) of

and

is given as:

When argument reduction is ‘mean’, the mean value of

will be returned.When argument reduction is ‘sum’, the sum of

will be returned.

is the batch size.

• Parameters

• reduction (str) – Type of reduction to apply to loss. The optional values are “mean”, “sum”, “none”.Default: “mean”.

• Inputs:

• • input_data (Tensor) - Tensor of shape

.

- 

target_data (Tensor) - Tensor of shape

.

• Outputs:
• Tensor, loss float tensor.

Examples

Copy>>> loss = nn.L1Loss()
>>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32)
>>> loss(input_data, target_data)

• class mindspore.nn.LARS(optimizer, epsilon=1e-05, hyperpara=0.001, weight_decay=0.0, use_clip=False, decay_filter=>, lars_filter=>, loss_scale=1.0)[source]
• Implements the LARS algorithm with LARSUpdate Operator.

LARS is an optimization algorithm employing a large batch optimization technique. Refer to paper LARGE BATCHTRAINING OF CONVOLUTIONAL NETWORKS.

• Parameters

• • optimizer (Optimizer) – MindSpore optimizer for which to wrap and modify gradients.

  • epsilon (float) – Term added to the denominator to improve numerical stability. Default: 1e-05.

  • hyperpara (float) – Trust coefficient for calculating the local learning rate. Default: 0.001.

  • weight_decay (float) – Weight decay (L2 penalty). Default: 0.0.

  • use_clip (bool) – Whether to use clip operation for calculating the local learning rate. Default: False.

  • decay_filter (Function) – A function to determine whether apply weight decay on parameters. Default:lambda x: ‘LayerNorm’ not in x.name and ‘bias’ not in x.name.

  • lars_filter (Function) – A function to determine whether apply lars algorithm. Default:lambda x: ‘LayerNorm’ not in x.name and ‘bias’ not in x.name.

  • loss_scale (float) – A floating point value for the loss scale. Default: 1.0.

• Inputs:

• • gradients (tuple[Tensor]) - The gradients of params in optimizer, the shape isas same as the params in optimizer.

• Outputs:

• Union[Tensor[bool], tuple[Parameter]], it depends on the output of optimizer.

Examples

Copy>>> net = Net()
>>> loss = nn.SoftmaxCrossEntropyWithLogits()
>>> opt = nn.Momentum(net.trainable_params(), 0.1, 0.9)
>>> opt_lars = nn.LARS(opt, epsilon=1e-08, hyperpara=0.02)
>>> model = Model(net, loss_fn=loss, optimizer=opt_lars, metrics=None)

• class mindspore.nn.LSTM(input_size, hidden_size, num_layers=1, has_bias=True, batch_first=False, dropout=0, bidirectional=False)[source]
• LSTM (Long Short-Term Memory) layer.

Applies a LSTM to the input.

There are two pipelines connecting two consecutive cells in a LSTM model; one is cell state pipelineand another is hidden state pipeline. Denote two consecutive time nodes as

and

.Given an input

at time

, an hidden state

and an cellstate

of the layer at time

, the cell state and hidden state attime

is computed using an gating mechanism. Input gate

is designed to protect the cellfrom perturbation by irrelevant inputs. Forget gate

affords protection of the cell by forgettingsome information in the past, which is stored in

. Output gate

protects otherunits from perturbation by currently irrelevant memory contents. Candidate cell state

iscalculated with the current input, on which the input gate will be applied. Finally, current cell state

and hidden state

are computed with the calculated gates and cell states. The completeformulation is as follows.

Here

is the sigmoid function, and

is the Hadamard product.

are learnable weights between the output and the input in the formula. For instance,

are the weight and bias used to transform from input

to

.Details can be found in paper LONG SHORT-TERM MEMORY andLong Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling.

• Parameters

• • input_size (int) – Number of features of input.

  • hidden_size (int) – Number of features of hidden layer.

  • num_layers (int) – Number of layers of stacked LSTM . Default: 1.

  • has_bias (bool) – Specifies whether has bias b_ih and b_hh. Default: True.

  • batch_first (bool) – Specifies whether the first dimension of input is batch_size. Default: False.

  • dropout (float) – If not 0, append Dropout layer on the outputs of eachLSTM layer except the last layer. Default 0. The range of dropout is [0.0, 1.0].

  • bidirectional (bool) – Specifies whether this is a bidirectional LSTM. If set True,number of directions will be 2 otherwise number of directions is 1. Default: False.

• Inputs:

• • input (Tensor) - Tensor of shape (seqlen, batch_size, _input_size).

  • hx (tuple) - A tuple of two Tensors (h0, c_0) both of data type mindspore.float32 ormindspore.float16 and shape (num_directions * _num_layers, batchsize, _hidden_size).Data type of hx should be the same of input.

• Outputs:

• Tuple, a tuple constains (output, (h_n, c_n)).

  • output (Tensor) - Tensor of shape (seqlen, batch_size, num_directions * _hidden_size).

  • hx_n (tuple) - A tuple of two Tensor (hn, c_n) both of shape(num_directions * _num_layers, batchsize, _hidden_size).

Examples

Copy>>> class LstmNet(nn.Cell):
>>>     def __init__(self, input_size, hidden_size, num_layers, has_bias, batch_first, bidirectional):
>>>         super(LstmNet, self).__init__()
>>>         self.lstm = nn.LSTM(input_size=input_size,
>>>                             hidden_size=hidden_size,
>>>                             num_layers=num_layers,
>>>                             has_bias=has_bias,
>>>                             batch_first=batch_first,
>>>                             bidirectional=bidirectional,
>>>                             dropout=0.0)
>>>
>>>     def construct(self, inp, h0, c0):
>>>         return self.lstm(inp, (h0, c0))
>>>
>>> net = LstmNet(10, 12, 2, has_bias=True, batch_first=True, bidirectional=False)
>>> input = Tensor(np.ones([3, 5, 10]).astype(np.float32))
>>> h0 = Tensor(np.ones([1 * 2, 3, 12]).astype(np.float32))
>>> c0 = Tensor(np.ones([1 * 2, 3, 12]).astype(np.float32))
>>> output, (hn, cn) = net(input, h0, c0)

• class mindspore.nn.Lamb(params, decay_steps, warmup_steps=0, start_learning_rate=0.1, end_learning_rate=0.0001, power=1.0, beta1=0.9, beta2=0.999, eps=1e-06, weight_decay=0.0, decay_filter=>)[source]
• Lamb Dynamic LR.

LAMB is an optimization algorithm employing a layerwise adaptive large batchoptimization technique. Refer to the paper LARGE BATCH OPTIMIZATION FOR DEEP LEARNING: TRAINING BERT IN 76MINUTES.

• Parameters

• • params (list[Parameter]) – A list of parameter, which will be updated. The element in _params_should be class mindspore.Parameter.

  • decay_steps (int) – The steps of the lr decay. Should be equal to or greater than 1.

  • warmup_steps (int) – The steps of lr warm up. Default: 0.

  • start_learning_rate (float) – A floating point value for the learning rate. Default: 0.1.

  • end_learning_rate (float) – A floating point value for the end learning rate. Default: 0.0001.

  • power (float) – The power of the polynomial. Default: 1.0.

  • beta1 (float) – The exponential decay rate for the 1st moment estimates. Default: 0.9.Should be in range (0.0, 1.0).

  • beta2 (float) – The exponential decay rate for the 2nd moment estimates. Default: 0.999.Should be in range (0.0, 1.0).

  • eps (float) – Term added to the denominator to improve numerical stability. Default: 1e-6.Should be greater than 0.

  • weight_decay (float) – Weight decay (L2 penalty). Default: 0.0. Should be equal to or greater than 0.

  • decay_filter (Function) – A function to determine whether to apply weight decay on parameters. Default:lambda x: ‘LayerNorm’ not in x.name and ‘bias’ not in x.name.

• Inputs:

• • gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.

• Outputs:

• tuple[Parameter], the updated velocity value, the shape is the same as params.

Examples

Copy>>> net = Net()
>>> loss = nn.SoftmaxCrossEntropyWithLogits()
>>> optim = nn.Lamb(params=net.trainable_params(), decay_steps=10)
>>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None)

• class mindspore.nn.LayerNorm(normalized_shape, begin_norm_axis=-1, begin_params_axis=-1, gamma_init='ones', beta_init='zeros')[source]
• Applies Layer Normalization over a mini-batch of inputs.

Layer normalization is widely used in recurrent neural networks. It appliesnormalization over a mini-batch of inputs for each single training case as describedin the paper Layer Normalization. Unlike batchnormalization, layer normalization performs exactly the same computation at training andtesting times. It can be described using the following formula. It is applied across all channelsand pixel but only one batch size.

• Parameters

• • normalized_shape (Union__(tuple[int]__, list[int]) – The normalization is performed over axesbegin_norm_axis … R - 1 and centering and scaling parameters are calculated overbegin_params_axis … R - 1.

  • begin_norm_axis (int) – It first normalization dimension: normalization will be performed along dimensionsbegin_norm_axis: rank(inputs), the value should be in [-1, rank(input)). Default: -1.

  • begin_params_axis (int) – The first parameter(beta, gamma)dimension: scale and centering parameterswill have dimensions begin_params_axis: rank(inputs) and will be broadcast withthe normalized inputs accordingly, the value should be in [-1, rank(input)). Default: -1.

  • gamma_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight.The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’,‘he_uniform’, etc. Default: ‘ones’.

  • beta_init (Union__[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight.The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’,‘he_uniform’, etc. Default: ‘zeros’.

• Inputs:

• • input_x (Tensor) - The shape of ‘inputx’ is input_shape = (x1, x_2, …, x_R),and input_shape[begin_norm_axis:] is equal to normalized_shape.

• Outputs:

• Tensor, the normalized and scaled offset tensor, has the same shape and data type as the input_x.

Examples

Copy>>> x = Tensor(np.ones([20, 5, 10, 10], np.float32))
>>> shape1 = x.shape()[1:]
>>> m = nn.LayerNorm(shape1,  begin_norm_axis=1, begin_params_axis=1)
>>> m(x)

• extend_repr()[source]
• Display instance object as string.

• class mindspore.nn.LeakyReLU(alpha=0.2)[source]
• Leaky ReLU activation function.

LeakyReLU is similar to ReLU, but LeakyReLU has a slope that makes it not equal to 0 at x < 0.The activation function is defined as:

See https://ai.stanford.edu/~amaas/papers/relu_hybrid_icml2013_final.pdf

• Parameters

• alpha (float) – Slope of the activation function at x < 0. Default: 0.2.

• Inputs:

• • input_x (Tensor) - The input of LeakyReLU.

• Outputs:

• Tensor, has the same type and shape with the input_x.

• class mindspore.nn.LogSoftmax(axis=-1)[source]
• LogSoftmax activation function.

Applies the LogSoftmax function to n-dimensional input tensor.

The input is transformed with Softmax function and then with log function to lie in range[-inf,0).

Logsoftmax is defined as:

,where

is the

-th slice along the given dim of the input Tensor.

• Parameters

• axis (int) – The axis to apply LogSoftmax operation, -1 means the last dimension. Default: -1.

• Inputs:

• • x (Tensor) - The input of LogSoftmax.

• Outputs:

• Tensor, which has the same type and shape as the input as x with values in the range[-inf,0).

• class mindspore.nn.Loss[source]
• Calculates the average of the loss. If method ‘update’ is called every

iterations, the result ofevaluation will be:

Examples

Copy>>> x = Tensor(np.array(0.2), mindspore.float32)
>>> loss = nn.Loss()
>>> loss.clear()
>>> loss.update(x)
>>> result = loss.eval()
0.20000000298023224

• clear()[source]

• Clears the internal evaluation result.

• eval()[source]

• Calculates the average of the loss.

  • Returns

  • Float, the average of the loss.

  • Raises

  • RuntimeError – If the total number is 0.

• update(*inputs)[source]

• Updates the internal evaluation result.

  • Parameters

  • inputs – Inputs contain only one element, the element is loss. The dimension ofloss should be 0 or 1.

  • Raises

  • • ValueError – If the length of inputs is not 1.

    • ValueError – If the dimensions of loss is not 1.

• class mindspore.nn.MAE[source]
• Calculates the mean absolute error.

Creates a criterion that measures the mean absolute error (MAE)between each element in the input:

and the target:

.

Here

is the prediction and

is the true value.

Note

The method update must be called with the form update(y_pred, y).

Examples

Copy>>> x = Tensor(np.array([0.1, 0.2, 0.6, 0.9]), mindspore.float32)
>>> y = Tensor(np.array([0.1, 0.25, 0.7, 0.9]), mindspore.float32)
>>> error = nn.MAE()
>>> error.clear()
>>> error.update(x, y)
>>> result = error.eval()

• clear()[source]

• Clears the internal evaluation result.

• eval()[source]

• Computes the mean absolute error.

  • Returns

  • Float, the computed result.

  • Raises

  • RuntimeError – If the number of the total samples is 0.

• update(*inputs)[source]

• Updates the internal evaluation result

and

.

- Parameters
- 

inputs – Input y_pred and y for calculating mean absolute error where the shape ofy_pred and y are both N-D and the shape are the same.

- Raises
- 

ValueError – If the number of the input is not 2.

• class mindspore.nn.MSE[source]
• Measures the mean squared error.

Creates a criterion that measures the mean squared error (squared L2norm) between each element in the input:

and the target:

.

where

is batch size.

Examples

Copy>>> x = Tensor(np.array([0.1, 0.2, 0.6, 0.9]), mindspore.float32)
>>> y = Tensor(np.array([0.1, 0.25, 0.5, 0.9]), mindspore.float32)
>>> error = nn.MSE()
>>> error.clear()
>>> error.update(x, y)
>>> result = error.eval()

• clear()[source]

• Clear the internal evaluation result.

• eval()[source]

• Compute the mean squared error.

  • Returns

  • Float, the computed result.

  • Raises

  • RuntimeError – If the number of samples is 0.

• update(*inputs)[source]

• Updates the internal evaluation result

and

.

- Parameters
- 

inputs – Input y_pred and y for calculating mean square error where the shape ofy_pred and y are both N-D and the shape are the same.

- Raises
- 

ValueError – If the number of input is not 2.

• class mindspore.nn.MSELoss(reduction='mean')[source]
• MSELoss create a criterion to measures the mean squared error (squared L2-norm) between

and

by element, where

is the input and

is the target.

For simplicity, let

and

be 1-dimensional Tensor with length

,the unreduced loss (i.e. with argument reduction set to ‘none’) of

and

is given as:

When argument reduction is ‘mean’, the mean value of

will be returned.When argument reduction is ‘sum’, the sum of

will be returned.

is the batch size.

• Parameters

• reduction (str) – Type of reduction to apply to loss. The optional values are “mean”, “sum”, “none”.Default: “mean”.

• Inputs:

• • input_data (Tensor) - Tensor of shape

.

- 

target_data (Tensor) - Tensor of shape

.

• Outputs:
• Tensor, weighted loss float tensor.

Examples

Copy>>> loss = nn.MSELoss()
>>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32)
>>> loss(input_data, target_data)

• class mindspore.nn.MaxPool2d(kernel_size=1, stride=1, pad_mode='VALID', padding=0)[source]
• Max pooling operation for temporal data.

Applies a 2D max pooling over an input Tensor which can be regarded as a composition of 2D planes.

Typically the input is of shape

, MaxPool2d outputsregional maximum in the

-dimension. Given kernel size

and stride

, the operation is as follows.

Note

pad_mode for training only supports “same” and “valid”.

• Parameters

• • kernel_size (int) – Size of the window to take a max over.

  • stride (int) – Stride size of the window. Default: None.

  • pad_mode (str) –

Select the mode of the pad. The optional values are“same” and “valid”. Default: “valid”.

  - 

same: Adopts the way of completion. Output height and width will be the same asthe input. Total number of padding will be calculated for horizontal and verticaldirection and evenly distributed to top and bottom, left and right if possible. Otherwise, thelast extra padding will be done from the bottom and the right side.

  - 

valid: Adopts the way of discarding. The possibly largest height and width of output will be returnwithout padding. Extra pixels will be discarded.

- 

padding (int) – Now is not supported, mplicit zero padding to be added on both sides. Default: 0.

• Inputs:
• • input (Tensor) - Tensor of shape

.

• Outputs:
• Tensor of shape

.

Examples

Copy>>> pool = MaxPool2d(kernel_size=3, stride=1)
>>> x = Tensor(np.random.randint(0, 10, [1, 2, 4, 4]), mindspore.float32)
[[[[1. 5. 5. 1.]
   [0. 3. 4. 8.]
   [4. 2. 7. 6.]
   [4. 9. 0. 1.]]
  [[3. 6. 2. 6.]
   [4. 4. 7. 8.]
   [0. 0. 4. 0.]
   [1. 8. 7. 0.]]]]
>>> output = pool(x)
>>> output.shape()
(1, 2, 2, 2)
>>> output
[[[[7. 8.]
   [9. 9.]]
  [[7. 8.]
   [8. 8.]]]]

• class mindspore.nn.Metric[source]
• Base class of metric.

Note

For examples of subclasses, please refer to the definition of class MAE, ‘Recall’ etc.

• abstract clear()[source]
• A interface describes the behavior of clearing the internal evaluation result.

Note

All subclasses should override this interface.

• abstract eval()[source]
• A interface describes the behavior of computing the evaluation result.

Note

All subclasses should override this interface.

• abstract update(*inputs)[source]
• A interface describes the behavior of updating the internal evaluation result.

Note

All subclasses should override this interface.

- Parameters
- 

inputs – A variable-length input argument list.

• class mindspore.nn.Momentum(params, learning_rate, momentum, weight_decay=0.0, loss_scale=1.0, decay_filter=>)[source]
• Implements the Momentum algorithm.

Refer to the paper on the importance of initialization and momentum in deep learning for more details.

• Parameters

• • params (list[Parameter]) – A list of parameter, which will be updated. The element in _parameters_should be class mindspore.Parameter.

  • learning_rate (Union__[float, Tensor, Iterable]) – A value for the learning rate. When the learning_rate isIterable or a Tensor and the dims of the Tensor is 1,use dynamic learning rate, then the i-th step willtake the i-th value as the learning rate.When the learning_rate is float or learning_rate is a Tensorbut the dims of the Tensor is 0, use fixed learning rate.Other cases are not supported.

  • momentum (float) – Hyperparameter of type float, means momentum for the moving average.

  • weight_decay (float) – Weight decay (L2 penalty). Default: 0.0.

  • loss_scale (float) – A floating point value for the loss scale. Default: 1.0.

  • decay_filter (Function) – A function to determine whether to apply weight decay on parameters. Default:lambda x: ‘beta’ not in x.name and ‘gamma’ not in x.name.

• Inputs:

• • gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.

• Outputs:

• Tensor[bool], the value is True.

• Raises

• ValueError – If the momentum is less than 0.0.

Examples

Copy>>> net = Net()
>>> loss = nn.SoftmaxCrossEntropyWithLogits()
>>> optim = nn.Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9)
>>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None)

• class mindspore.nn.Norm(axis=(), keep_dims=False)[source]
• Computes the norm of vectors, currently including Euclidean norm, i.e.,

-norm.

• Parameters

• • axis (tuple) – The axis over which to compute vector norms. Default: ().

  • keep_dims (bool) – If True, the axis indicated in axis are kept with size 1. Otherwise,the dimensions in axis are removed from the output shape. Default: False.

• Inputs:

• • input (Tensor) - Tensor which is not empty.

• Outputs:

• Tensor, output tensor with dimensions in ‘axis’ reduced to 1 will be returned if ‘keep_dims’ is True;otherwise a Tensor with dimensions in ‘axis’ removed is returned.

Examples

Copy>>> net = nn.Norm(axis=0)
>>> input = Tensor(np.random.randint(0, 10, [4, 16]), mindspore.float32)
>>> net(input)

• class mindspore.nn.OneHot(axis=-1, depth=1, on_value=1.0, off_value=0.0, dtype=mindspore.float32)[source]
• Returns a one-hot tensor.

The locations represented by indices in argument ‘indices’ take value on_value,while all other locations take value off_value.

Note

If the input indices is rank

, the output will have rank

. The newaxis is created at dimension axis.

• Parameters

• • axis (int) – Features x depth if axis == -1, depth x featuresif axis == 0. Default: -1.

  • depth (int) – A scalar defining the depth of the one hot dimension. Default: 1.

  • on_value (float) – A scalar defining the value to fill in output[i][j]when indices[j] = i. Default: 1.0.

  • off_value (float) – A scalar defining the value to fill in output[i][j]when indices[j] != i. Default: 0.0.

  • dtype (mindspore.dtype) – Default: mindspore.float32.

• Inputs:

• • indices (Tensor) - A tensor of indices of data type mindspore.int32 and arbitrary shape.

• Outputs:

• Tensor, the one-hot tensor of data type ‘dtype’ with dimension at ‘axis’ expanded to ‘depth’ and filled withon_value and off_value.

Examples

Copy>>> net = nn.OneHot(depth=4, axis=1)
>>> indices = Tensor([[1, 3], [0, 2]], dtype=mindspore.int32)
>>> net(indices)
[[[0. 0.]
  [1. 0.]
  [0. 0.]
  [0. 1.]]
 [[1. 0.]
  [0. 0.]
  [0. 1.]
  [0. 0.]]]

• class mindspore.nn.Optimizer(learning_rate, parameters)[source]
• Base class for all optimizers.

This class defines the API to add Ops to train a model.

Note

This class defines the API to add Ops to train a model. Never usethis class directly, but instead instantiate one of its subclasses.

• Parameters

• • learning_rate (float) – A floating point value for the learning rate. Should be greater than 0.

  • parameters (list) – A list of parameter, which will be updated. The element in _parameters_should be class mindspore.Parameter.

• Raises

• • ValueError – If the learning_rate is a Tensor, but the dims of tensor is greater than 1.

  • TypeError – If the learning_rate is not any of the three types: float, Tensor, Iterable.

• class mindspore.nn.PReLU(channel=1, w=0.25)[source]
• PReLU activation function.

Applies the PReLU function element-wise.

PReLU is defined as:

, where

is an element of an channel of the input.

Here

is an learnable parameter with default initial value 0.25.Parameter

has dimensionality of the argument channel. If called without argumentchannel, a single parameter

will be shared across all channels.

• Parameters

• • channel (int) – The dimension of input. Default: 1.

  • w (float) – The initial value of w. Default: 0.25.

• Inputs:

• • input_data (Tensor) - The input of Tanh.

• Outputs:

• Tensor, with the same type and shape as the input_data.

• class mindspore.nn.ParameterUpdate(param)[source]
• Cell that updates parameters.

With this Cell, one can manually update param with the input Tensor.

• Parameters

• param (Parameter) – The parameter to be updated manually.

• Raises

• KeyError – If parameter with the specified name do not exist.

Examples

Copy>>> network = Net()
>>> param = network.parameters_dict()['learning_rate']
>>> update = nn.ParameterUpdate(param)
>>> update.phase = "update_param"
>>> lr = Tensor(0.001, mindspore.float32)
>>> update(lr)

• class mindspore.nn.Precision(eval_type='classification')[source]
• Calculates precision for classification and multilabel data.

The precision function creates two local variables,

and

, that are used to compute the precision. This value isultimately returned as the precision, an idempotent operation that simply divides

by the sum of

and

.

Note

In the multi-label cases, the elements of

and

should be 0 or 1.

• Parameters
• eval_type (str) – Metric to calculate accuracy over a dataset, for classification ormultilabel. Default: ‘classification’.

Examples

Copy>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]]))
>>> y = Tensor(np.array([1, 0, 1]))
>>> metric = nn.Precision('classification')
>>> metric.clear()
>>> metric.update(x, y)
>>> precision = metric.eval()
[0.5 1. ]

• clear()[source]

• Clears the internal evaluation result.

• eval(average=False)[source]

• Computes the precision.

  • Parameters

  • average (bool) – Specify whether calculate the average precision. Default value is False.

  • Returns

  • Float, the computed result.

• update(*inputs)[source]

• Updates the internal evaluation result with y_pred and y.

  • Parameters
  • inputs – Input y_pred and y. y_pred and y are Tensor, list or numpy.ndarray.y_pred is in most cases (not strictly) a list of floating numbers in range

and the shape is

, where

is the number of cases and

is the number of categories. For ‘multilabel’ evaluation type, y_pred can only be one-hotencoding with values 0 or 1. Indices with 1 indicate positive category. y contains valuesof integers. The shape is

if one-hot encoding is used. One-hot encodingshould be used when ‘eval_type’ is ‘multilabel’. Shape can also be

if categoryindex is used in ‘classification’ evaluation type.

- Raises
- 

ValueError – If the number of input is not 2.

• class mindspore.nn.ReLU[source]
• Rectified Linear Unit activation function.

Applies the rectified linear unit function element-wise. It returnselement-wise

, specially, the neurons with the negative outputwill suppressed and the active neurons will stay the same.

• Inputs:
• • input_data (Tensor) - The input of ReLU.

• Outputs:

• Tensor, with the same type and shape as the input_data.

• class mindspore.nn.ReLU6[source]
• Compute ReLU6 activation function.

ReLU6 is similar to ReLU with a upper limit of 6, which if the inputs are greater than 6, the outputswill be suppressed to 6.It computes element-wise as

. The input is a Tensor of any valid shape.

• Inputs:
• • input_data (Tensor) - The input of ReLU6.

• Outputs:

• Tensor, which has the same type with input_data.

• class mindspore.nn.Recall(eval_type='classification')[source]
• Calculate recall for classification and multilabel data.

The recall class creates two local variables,

and

,that are used to compute the recall. This value is ultimately returned as the recall, an idempotent operationthat simply divides

by the sum of

and

.

Note

In the multi-label cases, the elements of

and

should be 0 or 1.