Macro frame_benchmarking::benchmarks

macro_rules! benchmarks {
    (
		$( where_clause { where $( $where_ty:ty: $where_bound:path ),* $(,)? } )?
		_ {
			$(
				let $common:ident in $common_from:tt .. $common_to:expr => $common_instancer:expr;
			)*
		}
		$( $rest:tt )*
	) => { ... };
}

Construct pallet benchmarks for weighing dispatchables.

Works around the idea of complexity parameters, named by a single letter (which is usually upper cased in complexity notation but is lower-cased for use in this macro).

Complexity parameters (“parameters”) have a range which is a u32 pair. Every time a benchmark is prepared and run, this parameter takes a concrete value within the range. There is an associated instancing block, which is a single expression that is evaluated during preparation. It may use ? (i.e. return Err(…)`) to bail with a string error. Here’s a few examples:

// These two are equivalent:
let x in 0 .. 10;
let x in 0 .. 10 => ();
// This one calls a setup function and might return an error (which would be terminal).
let y in 0 .. 10 => setup(y)?;
// This one uses a code block to do lots of stuff:
let z in 0 .. 10 => {
  let a = z * z / 5;
  let b = do_something(a)?;
  combine_into(z, b);
}

Note that due to parsing restrictions, if the from expression is not a single token (i.e. a literal or constant), then it must be parenthesised.

The macro allows for a number of “arms”, each representing an individual benchmark. Using the simple syntax, the associated dispatchable function maps 1:1 with the benchmark and the name of the benchmark is the same as that of the associated function. However, extended syntax allows for arbitrary expresions to be evaluated in a benchmark (including for example, on_initialize).

The macro allows for common parameters whose ranges and instancing expressions may be drawn upon (or not) by each arm. Syntax is available to allow for only the range to be drawn upon if desired, allowing an alternative instancing expression to be given.

Note that the ranges are inclusive on both sides. This is in contrast to ranges in Rust which are left-inclusive right-exclusive.

Each arm may also have a block of code which is run prior to any instancing and a block of code which is run afterwards. All code blocks may draw upon the specific value of each parameter at any time. Local variables are shared between the two pre- and post- code blocks, but do not leak from the interior of any instancing expressions.

Any common parameters that are unused in an arm do not have their instancing expressions evaluated.

Example:

benchmarks! {
  where_clause {  where T::A: From<u32> } // Optional line to give additional bound on `T`.

  // common parameter; just one for this example.
  // will be `1`, `MAX_LENGTH` or any value inbetween
  _ {
    let l in 1 .. MAX_LENGTH => initialize_l(l);
  }

  // first dispatchable: foo; this is a user dispatchable and operates on a `u8` vector of
  // size `l`, which we allow to be initialized as usual.
  foo {
    let caller = account::<T>(b"caller", 0, benchmarks_seed);
    let l = ...;
  }: _(Origin::Signed(caller), vec![0u8; l])

  // second dispatchable: bar; this is a root dispatchable and accepts a `u8` vector of size
  // `l`. We don't want it pre-initialized like before so we override using the `=> ()` notation.
  // In this case, we explicitly name the call using `bar` instead of `_`.
  bar {
    let l = _ .. _ => ();
  }: bar(Origin::Root, vec![0u8; l])

  // third dispatchable: baz; this is a user dispatchable. It isn't dependent on length like the
  // other two but has its own complexity `c` that needs setting up. It uses `caller` (in the
  // pre-instancing block) within the code block. This is only allowed in the param instancers
  // of arms. Instancers of common params cannot optimistically draw upon hypothetical variables
  // that the arm's pre-instancing code block might have declared.
  baz1 {
    let caller = account::<T>(b"caller", 0, benchmarks_seed);
    let c = 0 .. 10 => setup_c(&caller, c);
  }: baz(Origin::Signed(caller))

  // this is a second benchmark of the baz dispatchable with a different setup.
  baz2 {
    let caller = account::<T>(b"caller", 0, benchmarks_seed);
    let c = 0 .. 10 => setup_c_in_some_other_way(&caller, c);
  }: baz(Origin::Signed(caller))

  // this is benchmarking some code that is not a dispatchable.
  populate_a_set {
    let x in 0 .. 10_000;
    let mut m = Vec::<u32>::new();
    for i in 0..x {
      m.insert(i);
    }
  }: { m.into_iter().collect::<BTreeSet>() }
}

Test functions are automatically generated for each benchmark and are accessible to you when you run cargo test. All tests are named test_benchmark_<benchmark_name>, expect you to pass them the Runtime Trait, and run them in a test externalities environment. The test function runs your benchmark just like a regular benchmark, but only testing at the lowest and highest values for each component. The function will return Ok(()) if the benchmarks return no errors.

You can optionally add a verify code block at the end of a benchmark to test any final state of your benchmark in a unit test. For example:

sort_vector {
	let x in 1 .. 10000;
	let mut m = Vec::<u32>::new();
	for i in (0..x).rev() {
		m.push(i);
	}
}: {
	m.sort();
} verify {
	ensure!(m[0] == 0, "You forgot to sort!")
}

These verify blocks will not affect your benchmark results!

You can construct benchmark tests like so:

#[test]
fn test_benchmarks() {
  new_test_ext().execute_with(|| {
    assert_ok!(test_benchmark_dummy::<Test>());
    assert_err!(test_benchmark_other_name::<Test>(), "Bad origin");
    assert_ok!(test_benchmark_sort_vector::<Test>());
    assert_err!(test_benchmark_broken_benchmark::<Test>(), "You forgot to sort!");
  });
}