Preprocessors

A preprocessor is simply a bit of code which gets run immediately after the book is loaded and before it gets rendered, allowing you to update and mutate the book. Possible use cases are:

  • Creating custom helpers like {{#include /path/to/file.md}}
  • Updating links so [some chapter](some_chapter.md) is automatically changed to [some chapter](some_chapter.html) for the HTML renderer
  • Substituting in latex-style expressions ($$ \frac{1}{3} $$) with their mathjax equivalents

Hooking Into MDBook

MDBook uses a fairly simple mechanism for discovering third party plugins. A new table is added to book.toml (e.g. preprocessor.foo for the foo preprocessor) and then mdbook will try to invoke the mdbook-foo program as part of the build process.

While preprocessors can be hard-coded to specify which backend it should be run for (e.g. it doesn’t make sense for MathJax to be used for non-HTML renderers) with the preprocessor.foo.renderer key.

[book] title = "My Book" authors = ["Michael-F-Bryan"] [preprocessor.foo] # The command can also be specified manually command = "python3 /path/to/foo.py" # Only run the `foo` preprocessor for the HTML and EPUB renderer renderer = ["html", "epub"]

In typical unix style, all inputs to the plugin will be written to stdin as JSON and mdbook will read from stdout if it is expecting output.

The easiest way to get started is by creating your own implementation of the Preprocessor trait (e.g. in lib.rs) and then creating a shell binary which translates inputs to the correct Preprocessor method. For convenience, there is an example no-op preprocessor in the examples/ directory which can easily be adapted for other preprocessors.

Example no-op preprocessor

  1. // nop-preprocessors.rs
  3. use crate::nop_lib::Nop;
  4. use clap::{App, Arg, ArgMatches, SubCommand};
  5. use mdbook::book::Book;
  6. use mdbook::errors::Error;
  7. use mdbook::preprocess::{CmdPreprocessor, Preprocessor, PreprocessorContext};
  8. use std::io;
  9. use std::process;
  11. pub fn make_app() -> App<'static, 'static> {
  12.     App::new("nop-preprocessor")
  13.         .about("A mdbook preprocessor which does precisely nothing")
  14.         .subcommand(
  15.             SubCommand::with_name("supports")
  16.                 .arg(Arg::with_name("renderer").required(true))
  17.                 .about("Check whether a renderer is supported by this preprocessor"),
  18.         )
  19. }
  21. fn main() {
  22.     let matches = make_app().get_matches();
  24.     // Users will want to construct their own preprocessor here
  25.     let preprocessor = Nop::new();
  27.     if let Some(sub_args) = matches.subcommand_matches("supports") {
  28.         handle_supports(&preprocessor, sub_args);
  29.     } else if let Err(e) = handle_preprocessing(&preprocessor) {
  30.         eprintln!("{}", e);
  31.         process::exit(1);
  32.     }
  33. }
  35. fn handle_preprocessing(pre: &dyn Preprocessor) -> Result<(), Error> {
  36.     let (ctx, book) = CmdPreprocessor::parse_input(io::stdin())?;
  38.     if ctx.mdbook_version != mdbook::MDBOOK_VERSION {
  39.         // We should probably use the `semver` crate to check compatibility
  40.         // here...
  41.         eprintln!(
  42.             "Warning: The {} plugin was built against version {} of mdbook, \
  43.              but we're being called from version {}",
  44.             pre.name(),
  45.             mdbook::MDBOOK_VERSION,
  46.             ctx.mdbook_version
  47.         );
  48.     }
  50.     let processed_book = pre.run(&ctx, book)?;
  51.     serde_json::to_writer(io::stdout(), &processed_book)?;
  53.     Ok(())
  54. }
  56. fn handle_supports(pre: &dyn Preprocessor, sub_args: &ArgMatches) -> ! {
  57.     let renderer = sub_args.value_of("renderer").expect("Required argument");
  58.     let supported = pre.supports_renderer(&renderer);
  60.     // Signal whether the renderer is supported by exiting with 1 or 0.
  61.     if supported {
  62.         process::exit(0);
  63.     } else {
  64.         process::exit(1);
  65.     }
  66. }
  68. /// The actual implementation of the `Nop` preprocessor. This would usually go
  69. /// in your main `lib.rs` file.
  70. mod nop_lib {
  71.     use super::*;
  73.     /// A no-op preprocessor.
  74.     pub struct Nop;
  76.     impl Nop {
  77.         pub fn new() -> Nop {
  78.             Nop
  79.         }
  80.     }
  82.     impl Preprocessor for Nop {
  83.         fn name(&self) -> &str {
  84.             "nop-preprocessor"
  85.         }
  87.         fn run(&self, ctx: &PreprocessorContext, book: Book) -> Result<Book, Error> {
  88.             // In testing we want to tell the preprocessor to blow up by setting a
  89.             // particular config value
  90.             if let Some(nop_cfg) = ctx.config.get_preprocessor(self.name()) {
  91.                 if nop_cfg.contains_key("blow-up") {
  92.                     return Err("Boom!!1!".into());
  93.                 }
  94.             }
  96.             // we *are* a no-op preprocessor after all
  97.             Ok(book)
  98.         }
  100.         fn supports_renderer(&self, renderer: &str) -> bool {
  101.             renderer != "not-supported"
  102.         }
  103.     }
  104. }

Hints For Implementing A Preprocessor

By pulling in mdbook as a library, preprocessors can have access to the existing infrastructure for dealing with books.

For example, a custom preprocessor could use the CmdPreprocessor::parse_input() function to deserialize the JSON written to stdin. Then each chapter of the Book can be mutated in-place via Book::for_each_mut(), and then written to stdout with the serde_json crate.

Chapters can be accessed either directly (by recursively iterating over chapters) or via the Book::for_each_mut() convenience method.

The chapter.content is just a string which happens to be markdown. While it’s entirely possible to use regular expressions or do a manual find & replace, you’ll probably want to process the input into something more computer-friendly. The pulldown-cmark crate implements a production-quality event-based Markdown parser, with the pulldown-cmark-to-cmark allowing you to translate events back into markdown text.

The following code block shows how to remove all emphasis from markdown, without accidentally breaking the document.

  2. #![allow(unused_variables)]
  3. fn main() {
  4. fn remove_emphasis(
  5.     num_removed_items: &mut usize,
  6.     chapter: &mut Chapter,
  7. ) -> Result<String> {
  8.     let mut buf = String::with_capacity(chapter.content.len());
  10.     let events = Parser::new(&chapter.content).filter(|e| {
  11.         let should_keep = match *e {
  12.             Event::Start(Tag::Emphasis)
  13.             | Event::Start(Tag::Strong)
  14.             | Event::End(Tag::Emphasis)
  15.             | Event::End(Tag::Strong) => false,
  16.             _ => true,
  17.         };
  18.         if !should_keep {
  19.             *num_removed_items += 1;
  20.         }
  21.         should_keep
  22.     });
  24.     cmark(events, &mut buf, None).map(|_| buf).map_err(|err| {
  25.         Error::from(format!("Markdown serialization failed: {}", err))
  26.     })
  27. }
  28. }

For everything else, have a look at the complete example.