Program

class paddle.static.Program [源代码]

注意:默认情况下,Paddle内部默认含有 default_startup_program default_main_program ,它们共享参数。 default_startup_program 只运行一次来初始化参数, default_main_program 在每个mini batch中运行并更新权重。

Program是Paddle对于计算图的一种静态描述,使用Program的构造函数可以创建一个Program。Program中包括至少一个 Block ,当 Block 中存在条件选择的控制流OP(例如 While 等)时,该Program将会含有嵌套着的 Block 即控制流外部的 Block 将包含着控制流内部的 Block ,而嵌套的 Block 的元素访问控制将由具体的控制流OP来决定。关于Program具体的结构和包含的类型请参阅 framework.proto

一个Program的集合通常包含初始化程序(startup_program)与主程序(main_program),初始化程序是一个包含一些初始化工作的Program,主程序将会包含用来训练的网络结构和变量,在使用同一个 执行引擎 执行时他们会共享初始化工作的结果,例如初始化的参数。一个Program的集合可以被用来测试或者训练,被用来训练时, Paddle 将会利用所有用户使用的OP和变量来搭建一个训练网络,被用来测试时, 可以通过调用Program相关的接口例如:clone 剪去一些与测试无关的OP和变量,比如反向传播的OP和变量。

返回

Program,创建的空的Program

代码示例

  1. import paddle
  2. import paddle.static as static
  3. paddle.enable_static()
  4. main_program = static.Program()
  5. startup_program = static.Program()
  6. with static.program_guard(main_program=main_program, startup_program=startup_program):
  7. x = static.data(name="x", shape=[-1, 784], dtype='float32')
  8. y = static.data(name="y", shape=[-1, 1], dtype='int32')
  9. z = static.nn.fc(name="fc", x=x, size=10, activation="relu")
  10. print("main program is: {}".format(main_program))
  11. print("start up program is: {}".format(startup_program))

to_string ( throw_on_error, with_details=False )

将Program转换为字符串

参数

  • throw_on_error (bool) - 是否在没有设置必需字段时抛出异常。

  • with_details (bool) - 值为true时,打印更多关于变量和参数的信息,如trainable, optimize_attr等

返回

str,由Program转换得到的字符串

抛出异常: ValueError - 当 throw_on_error == true ,当没有设置任何必需的字段时,抛出 ValueError

代码示例

  1. import paddle
  2. import paddle.static as static
  3. paddle.enable_static()
  4. prog = static.default_main_program()
  5. x = static.data(name="X", shape=[2,3], dtype="float32")
  6. pred = static.nn.fc(x, size=3)
  7. prog_string = prog.to_string(throw_on_error=True, with_details=False)
  8. prog_string_with_details = prog.to_string(throw_on_error=False, with_details=True)
  9. print("program string without detail: {}".format(prog_string))
  10. print("program string with detail: {}".format(prog_string_with_details))

clone ( for_test=False )

注解

  1. Program.clone() 方法不会克隆例如 DataLoader 这样的数据读取相关的部分,这可能会造成的数据读取部分在克隆后丢失;

  2. 此API当 for_test=True 时将会裁剪部分OP和变量。为防止错误的裁剪,推荐在 append_backward 和执行优化器之前使用; clone(for_test=True)

for_test=True 时创建一个新的、仅包含当前Program前向内容的Program。否则创建一个新的,和当前Program完全相同的Program

有些OP,在训练和测试之间的行为是不同的,比如 batch_norm 。它们有一个属性 is_test 来控制行为。当 for_test=True 时,此方法将把它们的 is_test 属性更改为True。

  • 克隆Program用于训练时,将 for_test 设置为False。

  • 克隆Program用于测试时,将 for_test 设置为True。虽然在这种情况下,如果在使用了优化器之后调用 clone 我们依旧会对Program当中反向执行以及优化器相关的内容进行自动裁剪,但是,我们强烈建议在使用优化器之前使用 clone 例如如果使用的是 Momentum 可以这样去使用:

代码示例

  1. import paddle
  2. import paddle.static as static
  3. paddle.enable_static()
  4. img = static.data(name='image', shape=[None, 784])
  5. pred = static.nn.fc(x=img, size=10, activation='relu')
  6. loss = paddle.mean(pred)
  7. # Here we use clone before Momentum
  8. test_program = static.default_main_program().clone(for_test=True)
  9. optimizer = paddle.optimizer.Momentum(learning_rate=0.01, momentum=0.9)
  10. optimizer.minimize(loss)

参数

  • for_test (bool) – 取值为True时,clone方法内部会把operator的属性 is_test 设置为 True, 并裁剪反向OP和参数优化OP,默认值为False

返回

Program,当 for_test=True 时返回一个新的、仅包含当前Program前向内容的Program。否则返回一个新的,和当前Program完全相同的Program

代码示例

注解

Program在clone后的顺序可能不同,这不会影响的训练或测试进程。在下面的示例中,我们提供了一个简单的方法print_prog(Program)来打印程序描述,以确保clone后仍能得到同样的打印结果:

  1. import six
  2. def print_prog(prog):
  3. for name, value in sorted(six.iteritems(prog.block(0).vars)):
  4. print(value)
  5. for op in prog.block(0).ops:
  6. print("op type is {}".format(op.type))
  7. print("op inputs are {}".format(op.input_arg_names))
  8. print("op outputs are {}".format(op.output_arg_names))
  9. for key, value in sorted(six.iteritems(op.all_attrs())):
  10. if key not in ['op_callstack', 'op_role_var']:
  11. print(" [ attrs: {}: {} ]".format(key, value))

1.克隆一个Program,示例代码如下。

  1. import six
  2. import paddle
  3. import paddle.static as static
  4. import paddle.utils as utils
  5. import paddle.nn.functional as F
  6. paddle.enable_static()
  7. def print_prog(prog):
  8. for name, value in sorted(six.iteritems(prog.block(0).vars)):
  9. print(value)
  10. for op in prog.block(0).ops:
  11. print("op type is {}".format(op.type))
  12. print("op inputs are {}".format(op.input_arg_names))
  13. print("op outputs are {}".format(op.output_arg_names))
  14. for key, value in sorted(six.iteritems(op.all_attrs())):
  15. if key not in ['op_callstack', 'op_role_var']:
  16. print(" [ attrs: {}: {} ]".format(key, value))
  17. train_program = static.Program()
  18. startup_program = static.Program()
  19. # startup_program is used to do some parameter init work,
  20. # and main program is used to hold the network
  21. with static.program_guard(train_program, startup_program):
  22. with utils.unique_name.guard():
  23. img = static.data(name='image', shape=[None, 784])
  24. hidden = static.nn.fc(x=img, size=200, activation='relu')
  25. hidden = F.dropout(hidden, p=0.5)
  26. loss = F.cross_entropy(
  27. input=static.nn.fc(x=hidden, size=10, activation='softmax'),
  28. label=static.data(name='label', shape=[1], dtype='int64'))
  29. avg_loss = paddle.mean(loss)
  30. test_program = train_program.clone(for_test=True)
  31. print_prog(test_program)
  32. # Due to parameter sharing usage for train and test, so we need to use startup program of train
  33. # instead of using test startup program, while nothing is in test's startup program
  34. # In Paddle we will share weights by using the same Tensor name. In train and test program
  35. # all parameters will have the same name and this can make train and test program sharing parameters,
  36. # that's why we need to use startup program of train. And for startup program of test, it has nothing,
  37. # since it is a new program.
  38. with static.program_guard(train_program, startup_program):
  39. with utils.unique_name.guard():
  40. sgd = paddle.optimizer.SGD(learning_rate=1e-3)
  41. sgd.minimize(avg_loss)

2.如果分别运行 train Program 和 test Program,则可以不使用clone。

  1. import six
  2. import paddle
  3. import paddle.static as static
  4. import paddle.utils as utils
  5. import paddle.nn.functional as F
  6. paddle.enable_static()
  7. def print_prog(prog):
  8. for name, value in sorted(six.iteritems(prog.block(0).vars)):
  9. print(value)
  10. for op in prog.block(0).ops:
  11. print("op type is {}".format(op.type))
  12. print("op inputs are {}".format(op.input_arg_names))
  13. print("op outputs are {}".format(op.output_arg_names))
  14. for key, value in sorted(six.iteritems(op.all_attrs())):
  15. if key not in ['op_callstack', 'op_role_var']:
  16. print(" [ attrs: {}: {} ]".format(key, value))
  17. def network():
  18. img = static.data(name='image', shape=[None, 784])
  19. hidden = static.nn.fc(x=img, size=200, activation='relu')
  20. hidden = F.dropout(hidden, p=0.5)
  21. loss = F.cross_entropy(
  22. input=static.nn.fc(x=hidden, size=10, activation='softmax'),
  23. label=static.data(name='label', shape=[1], dtype='int64'))
  24. avg_loss = paddle.mean(loss)
  25. return avg_loss
  26. train_program_2 = static.Program()
  27. startup_program_2 = static.Program()
  28. test_program_2 = static.Program()
  29. with static.program_guard(train_program_2, startup_program_2):
  30. with utils.unique_name.guard():
  31. avg_loss = network()
  32. sgd = paddle.optimizer.SGD(learning_rate=1e-3)
  33. sgd.minimize(avg_loss)
  34. # the test startup program is not used.
  35. with static.program_guard(test_program_2, startup_program_2):
  36. with utils.unique_name.guard():
  37. avg_loss = network()
  38. print_prog(test_program_2)

上边两个代码片段生成和打印的Program是一样的。

static parse_from_string ( binary_str )

通过对 protobuf 的反序列化,转换成Program

参数

  • binary_str_type (str) – protobuf 二进制字符串

返回

Program,反序列化后的 Program

代码示例

  1. import paddle
  2. import paddle.static as static
  3. paddle.enable_static()
  4. startup_prog = static.Program()
  5. main_prog = static.Program()
  6. with static.program_guard(startup_prog, main_prog):
  7. x = static.data(name='X', shape=[1000, 784], dtype='float32')
  8. y = static.data(name='Y', shape=[784, 100], dtype='float32')
  9. z = paddle.matmul(x=x, y=y)
  10. binary_str = static.default_main_program().desc.serialize_to_string()
  11. prog_restored = static.default_main_program().parse_from_string(binary_str)
  12. print(static.default_main_program())
  13. print(prog_restored)

num_blocks

该Program中的 Block 的个数

返回

int,该Program中的 Block 的个数

代码示例

  1. import paddle
  2. import paddle.static as static
  3. paddle.enable_static()
  4. prog = static.default_main_program()
  5. num_blocks = prog.num_blocks
  6. print(num_blocks)
  7. # print result:
  8. # 1

random_seed

注解

必须在相关OP被添加之前设置。

程序中随机运算符的默认随机种子。0意味着随机生成随机种子。

返回

int64,该Program中当前正在使用的random seed

代码示例

  1. import paddle
  2. import paddle.static as static
  3. import paddle.nn.functional as F
  4. paddle.enable_static()
  5. prog = static.default_main_program()
  6. random_seed = prog.random_seed
  7. x_var = static.data(name="X", shape=[3,3], dtype="float32")
  8. print(random_seed)
  9. ## 0
  10. ## the default random seed is 0
  11. # Here we need to set random seed before we use paddle.nn.functional.dropout
  12. prog.random_seed = 1
  13. z_var = F.dropout(x_var, 0.7)
  14. print(prog.random_seed)
  15. ## 1
  16. ## the random seed is change to 1

global_block ( )

获取该Program的第一个 Block

返回

Block,该Program的第一个 Block

代码示例

  1. import paddle
  2. import paddle.static as static
  3. paddle.enable_static()
  4. prog = static.default_main_program()
  5. gb_block = prog.global_block()
  6. print(gb_block)

block ( index )

返回该Program中 , index 指定的 Blockindex 类型为int

参数

  • index (int) - 需要获取的 Block 的index

返回

Block,该Program中index对应的那个 Block

代码示例

  1. import paddle
  2. import paddle.static as static
  3. paddle.enable_static()
  4. prog = static.default_main_program()
  5. block_0 = prog.block(0)
  6. print(block_0)

current_block ( )

获取当前 Block 。当前 Block 是用来添加OP的。

返回

Block,该Program中用户当前所在的 Block

代码示例

  1. import paddle
  2. import paddle.static as static
  3. paddle.enable_static()
  4. prog = static.default_main_program()
  5. current_blk = prog.current_block()
  6. print(current_blk)

list_vars ( )

获取当前Program中所有变量。返回值是一个可迭代对象(iterable object)。

返回

Generator,会yield每个Program中的变量

代码示例

  1. import paddle
  2. import paddle.static as static
  3. paddle.enable_static()
  4. prog = static.default_main_program()
  5. img = static.data(name='img', shape=[None, 1,28,28], dtype='float32')
  6. label = static.data(name='label', shape=[None,1], dtype='int64')
  7. for var in prog.list_vars():
  8. print(var)
  9. # var img : paddle.VarType.LOD_TENSOR.shape(-1, 1, 28, 28).astype(VarType.FP32)
  10. # var label : paddle.VarType.LOD_TENSOR.shape(-1, 1).astype(VarType.INT64)

all_parameters ( )

获取当前Program中所有的 模型参数 。返回值是一个列表。

返回

list[ 模型参数 ],一个包含当前Program中所有参数的列表。

代码示例

  1. import paddle
  2. import paddle.static as static
  3. paddle.enable_static()
  4. program = static.default_main_program()
  5. data = static.data(name='x', shape=[None, 13], dtype='float32')
  6. hidden = static.nn.fc(x=data, size=10)
  7. loss = paddle.mean(hidden)
  8. paddle.optimizer.SGD(learning_rate=0.01).minimize(loss)
  9. for param in program.all_parameters():
  10. print(param)
  11. # Here will print all parameters in current program, in this example,
  12. # the result is like:
  13. #
  14. # persist trainable param fc_0.w_0 : paddle.VarType.LOD_TENSOR.shape(13, 10).astype(VarType.FP32)
  15. # persist trainable param fc_0.b_0 : paddle.VarType.LOD_TENSOR.shape(10,).astype(VarType.FP32)
  16. #
  17. # Here print(param) will print out all the properties of a parameter,
  18. # including name, type and persistable, you can access to specific
  19. # property of a parameter, such as param.name, param.type