L1Decay

  • paddle.fluid.regularizer.L1Decay(regularization_coeff=0.0)

L1Decay实现L1权重衰减正则化，用于模型训练，使得权重矩阵稀疏。

具体实现中，L1权重衰减正则化的计算公式如下：

L1Decay - 图1

  • 参数：
    • regularization_coeff (float) – L1正则化系数，默认值为0.0。

代码示例

  1. import paddle.fluid as fluid
  2.  
  3. main_prog = fluid.Program()
  4. startup_prog = fluid.Program()
  5. with fluid.program_guard(main_prog, startup_prog):
  6.     data = fluid.layers.data(name='image', shape=[3, 28, 28], dtype='float32')
  7.     label = fluid.layers.data(name='label', shape=[1], dtype='int64')
  8.     hidden = fluid.layers.fc(input=data, size=128, act='relu')
  9.     prediction = fluid.layers.fc(input=hidden, size=10, act='softmax')
  10.     loss = fluid.layers.cross_entropy(input=prediction, label=label)
  11.     avg_loss = fluid.layers.mean(loss)
  12. optimizer = fluid.optimizer.Adagrad(
  13.     learning_rate=1e-4,
  14.     regularization=fluid.regularizer.L1Decay(
  15.         regularization_coeff=0.1))
  16. optimizer.minimize(avg_loss)