Printing/Drawing Theano graphs

Theano provides the functions theano.printing.pprint() andtheano.printing.debugprint() to print a graph to the terminal before orafter compilation. pprint() is more compact and math-like,debugprint() is more verbose. Theano also provides pydotprint()that creates an image of the function. You can read about them inprinting – Graph Printing and Symbolic Print Statement.

Note

When printing Theano functions, they can sometimes be hard toread. To help with this, you can disable some Theano optimizationsby using the Theano flag:optimizer_excluding=fusion:inplace. Do not use this duringreal job execution, as this will make the graph slower and use morememory.

Consider again the logistic regression example:

  1. >>> import numpy
  2. >>> import theano
  3. >>> import theano.tensor as T
  4. >>> rng = numpy.random
  5. >>> # Training data
  6. >>> N = 400
  7. >>> feats = 784
  8. >>> D = (rng.randn(N, feats).astype(theano.config.floatX), rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))
  9. >>> training_steps = 10000
  10. >>> # Declare Theano symbolic variables
  11. >>> x = T.matrix("x")
  12. >>> y = T.vector("y")
  13. >>> w = theano.shared(rng.randn(feats).astype(theano.config.floatX), name="w")
  14. >>> b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), name="b")
  15. >>> x.tag.test_value = D[0]
  16. >>> y.tag.test_value = D[1]
  17. >>> # Construct Theano expression graph
  18. >>> p_1 = 1 / (1 + T.exp(-T.dot(x, w)-b)) # Probability of having a one
  19. >>> prediction = p_1 > 0.5 # The prediction that is done: 0 or 1
  20. >>> # Compute gradients
  21. >>> xent = -y*T.log(p_1) - (1-y)*T.log(1-p_1) # Cross-entropy
  22. >>> cost = xent.mean() + 0.01*(w**2).sum() # The cost to optimize
  23. >>> gw,gb = T.grad(cost, [w,b])
  24. >>> # Training and prediction function
  25. >>> train = theano.function(inputs=[x,y], outputs=[prediction, xent], updates=[[w, w-0.01*gw], [b, b-0.01*gb]], name = "train")
  26. >>> predict = theano.function(inputs=[x], outputs=prediction, name = "predict")

Pretty Printing

  1. >>> theano.printing.pprint(prediction) 
  2. 'gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\dot w)) - b)))),
  3. TensorConstant{0.5})'

Debug Print

The pre-compilation graph:

  1. >>> theano.printing.debugprint(prediction) 
  2. Elemwise{gt,no_inplace} [id A] ''
  3.  |Elemwise{true_div,no_inplace} [id B] ''
  4.  | |InplaceDimShuffle{x} [id C] ''
  5.  | | |TensorConstant{1} [id D]
  6.  | |Elemwise{add,no_inplace} [id E] ''
  7.  |   |InplaceDimShuffle{x} [id F] ''
  8.  |   | |TensorConstant{1} [id D]
  9.  |   |Elemwise{exp,no_inplace} [id G] ''
  10.  |     |Elemwise{sub,no_inplace} [id H] ''
  11.  |       |Elemwise{neg,no_inplace} [id I] ''
  12.  |       | |dot [id J] ''
  13.  |       |   |x [id K]
  14.  |       |   |w [id L]
  15.  |       |InplaceDimShuffle{x} [id M] ''
  16.  |         |b [id N]
  17.  |InplaceDimShuffle{x} [id O] ''
  18.    |TensorConstant{0.5} [id P]

The post-compilation graph:

  1. >>> theano.printing.debugprint(predict)  
  2. Elemwise{Composite{GT(scalar_sigmoid((-((-i0) - i1))), i2)}} [id A] ''   4
  3.  |...Gemv{inplace} [id B] ''   3
  4.  | |AllocEmpty{dtype='float64'} [id C] ''   2
  5.  | | |Shape_i{0} [id D] ''   1
  6.  | |   |x [id E]
  7.  | |TensorConstant{1.0} [id F]
  8.  | |x [id E]
  9.  | |w [id G]
  10.  | |TensorConstant{0.0} [id H]
  11.  |InplaceDimShuffle{x} [id I] ''   0
  12.  | |b [id J]
  13.  |TensorConstant{(1,) of 0.5} [id K]

Picture Printing of Graphs

The pre-compilation graph:

  1. >>> theano.printing.pydotprint(prediction, outfile="pics/logreg_pydotprint_prediction.png", var_with_name_simple=True)  
  2. The output file is available at pics/logreg_pydotprint_prediction.png

../_images/logreg_pydotprint_prediction2.png The post-compilation graph:

  1. >>> theano.printing.pydotprint(predict, outfile="pics/logreg_pydotprint_predict.png", var_with_name_simple=True)  
  2. The output file is available at pics/logreg_pydotprint_predict.png

../_images/logreg_pydotprint_predict2.png The optimized training graph:

  1. >>> theano.printing.pydotprint(train, outfile="pics/logreg_pydotprint_train.png", var_with_name_simple=True)  
  2. The output file is available at pics/logreg_pydotprint_train.png

../_images/logreg_pydotprint_train2.png

Interactive Graph Visualization

The new d3viz module complements theano.printing.pydotprint() tovisualize complex graph structures. Instead of creating a static image, itgenerates an HTML file, which allows to dynamically inspect graph structures ina web browser. Features include zooming, drag-and-drop, editing node labels, orcoloring nodes by their compute time.

=> d3viz <= ../_images/d3viz.png