You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

448 lines
66 KiB

4 months ago
<!DOCTYPE HTML>
<html lang="zh-CN" class="light" dir="ltr">
<head>
<!-- Book generated using mdBook -->
<meta charset="UTF-8">
<title>基准测试 benchmark - Rust语言圣经(Rust Course)</title>
<!-- Custom HTML head -->
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#ffffff">
<link rel="icon" href="../favicon.svg">
<link rel="shortcut icon" href="../favicon.png">
<link rel="stylesheet" href="../css/variables.css">
<link rel="stylesheet" href="../css/general.css">
<link rel="stylesheet" href="../css/chrome.css">
<link rel="stylesheet" href="../css/print.css" media="print">
<!-- Fonts -->
<link rel="stylesheet" href="../FontAwesome/css/font-awesome.css">
<link rel="stylesheet" href="../fonts/fonts.css">
<!-- Highlight.js Stylesheets -->
<link rel="stylesheet" href="../highlight.css">
<link rel="stylesheet" href="../tomorrow-night.css">
<link rel="stylesheet" href="../ayu-highlight.css">
<!-- Custom theme stylesheets -->
<link rel="stylesheet" href="../theme/style.css">
</head>
<body class="sidebar-visible no-js">
<div id="body-container">
<!-- Provide site root to javascript -->
<script>
var path_to_root = "../";
var default_theme = window.matchMedia("(prefers-color-scheme: dark)").matches ? "navy" : "light";
</script>
<!-- Work around some values being stored in localStorage wrapped in quotes -->
<script>
try {
var theme = localStorage.getItem('mdbook-theme');
var sidebar = localStorage.getItem('mdbook-sidebar');
if (theme.startsWith('"') && theme.endsWith('"')) {
localStorage.setItem('mdbook-theme', theme.slice(1, theme.length - 1));
}
if (sidebar.startsWith('"') && sidebar.endsWith('"')) {
localStorage.setItem('mdbook-sidebar', sidebar.slice(1, sidebar.length - 1));
}
} catch (e) { }
</script>
<!-- Set the theme before any content is loaded, prevents flash -->
<script>
var theme;
try { theme = localStorage.getItem('mdbook-theme'); } catch(e) { }
if (theme === null || theme === undefined) { theme = default_theme; }
var html = document.querySelector('html');
html.classList.remove('light')
html.classList.add(theme);
var body = document.querySelector('body');
body.classList.remove('no-js')
body.classList.add('js');
</script>
<input type="checkbox" id="sidebar-toggle-anchor" class="hidden">
<!-- Hide / unhide sidebar before it is displayed -->
<script>
var body = document.querySelector('body');
var sidebar = null;
var sidebar_toggle = document.getElementById("sidebar-toggle-anchor");
if (document.body.clientWidth >= 1080) {
try { sidebar = localStorage.getItem('mdbook-sidebar'); } catch(e) { }
sidebar = sidebar || 'visible';
} else {
sidebar = 'hidden';
}
sidebar_toggle.checked = sidebar === 'visible';
body.classList.remove('sidebar-visible');
body.classList.add("sidebar-" + sidebar);
</script>
<nav id="sidebar" class="sidebar" aria-label="Table of contents">
<div class="sidebar-scrollbox">
<ol class="chapter"><li class="chapter-item affix "><a href="../about-book.html">关于本书</a></li><li class="chapter-item affix "><a href="../into-rust.html">进入 Rust 编程世界</a></li><li class="chapter-item affix "><a href="../first-try/sth-you-should-not-do.html">避免从入门到放弃</a></li><li class="chapter-item affix "><a href="../community.html">社区和锈书</a></li><li class="spacer"></li><li class="chapter-item affix "><a href="../some-thoughts.html">Datav: 可编程的数据可视化平台和可观测性平台</a></li><li class="chapter-item affix "><li class="part-title">Rust 语言基础学习</li><li class="spacer"></li><li class="chapter-item "><a href="../first-try/intro.html"><strong aria-hidden="true">1.</strong> 寻找牛刀,以便小试</a><a class="toggle"><div></div></a></li><li><ol class="section"><li class="chapter-item "><a href="../first-try/installation.html"><strong aria-hidden="true">1.1.</strong> 安装 Rust 环境</a></li><li class="chapter-item "><a href="../first-try/editor.html"><strong aria-hidden="true">1.2.</strong> 墙推 VSCode!</a></li><li class="chapter-item "><a href="../first-try/cargo.html"><strong aria-hidden="true">1.3.</strong> 认识 Cargo</a></li><li class="chapter-item "><a href="../first-try/hello-world.html"><strong aria-hidden="true">1.4.</strong> 不仅仅是 Hello world</a></li><li class="chapter-item "><a href="../first-try/slowly-downloading.html"><strong aria-hidden="true">1.5.</strong> 下载依赖太慢了?</a></li></ol></li><li class="chapter-item "><a href="../basic/intro.html"><strong aria-hidden="true">2.</strong> Rust 基础入门</a><a class="toggle"><div></div></a></li><li><ol class="section"><li class="chapter-item "><a href="../basic/variable.html"><strong aria-hidden="true">2.1.</strong> 变量绑定与解构</a></li><li class="chapter-item "><a href="../basic/base-type/index.html"><strong aria-hidden="true">2.2.</strong> 基本类型</a><a class="toggle"><div></div></a></li><li><ol class="section"><li class="chapter-item "><a href="../basic/base-type/numbers.html"><strong aria-hidden="true">2.2.1.</strong> 数值类型</a></li><li class="chapter-item "><a href="../basic/base-type/char-bool.html"><strong aria-hidden="true">2.2.2.</strong> 字符、布尔、单元类型</a></li><li class="chapter-item "><a href="../basic/base-type/statement-expression.html"><strong aria-hidden="true">2.2.3.</strong> 语句与表达式</a></li><li class="chapter-item "><a href="../basic/base-type/function.html"><strong aria-hidden="true">2.2.4.</strong> 函数</a></li></ol></li><li class="chapter-item "><a href="../basic/ownership/index.html"><strong aria-hidden="true">2.3.</strong> 所有权和借用</a><a class="toggle"><div></div></a></li><li><ol class="section"><li class="chapter-item "><a href="../basic/ownership/ownership.html"><strong aria-hidden="true">2.3.1.</strong> 所有权</a></li><li class="chapter-item "><a href="../basic/ownership/borrowing.html"><strong aria-hidden="true">2.3.2.</strong> 引用与借用</a></li></ol></li><li class="chapter-item "><a href="../basic/compound-type/intro.html"><strong aria-hidden="true">2.4.</strong> 复合类型</a><a class="toggle"><div></div></a></li><li><ol class="section"><li class="chapter-item "><a href="../basic/compound-type/string-slice.html"><strong aria-hidden="true">2.4.1.</strong> 字符串与切片</a></li><li class="chapter-item "><a href="../basic/compound-type/tuple.html"><strong aria-hidden="true">2.4.2.</strong> 元组</a></li><li class="chapter-item "><a href="../basic/compound-type/struct.html"><strong aria-hidden="true">2.4.3.</strong> 结构体</a></li><li class="chapter-item "><a href="../basic/compound-type/enum.html"><strong aria-hidden="true">2.4.4.</strong> 枚举</a></li><li class="chapter-item "><a href="../basic/compound-type/array.html"><strong aria-hidden="true">2.4.5.</strong> 数组</a></li></ol></li><li class="chapter-item "><a href="../basic/flow-control.html"><strong aria-hidden="true">2.5.</strong> 流程控制</a></li><li class="cha
</div>
<div id="sidebar-resize-handle" class="sidebar-resize-handle">
<div class="sidebar-resize-indicator"></div>
</div>
</nav>
<!-- Track and set sidebar scroll position -->
<script>
var sidebarScrollbox = document.querySelector('#sidebar .sidebar-scrollbox');
sidebarScrollbox.addEventListener('click', function(e) {
if (e.target.tagName === 'A') {
sessionStorage.setItem('sidebar-scroll', sidebarScrollbox.scrollTop);
}
}, { passive: true });
var sidebarScrollTop = sessionStorage.getItem('sidebar-scroll');
sessionStorage.removeItem('sidebar-scroll');
if (sidebarScrollTop) {
// preserve sidebar scroll position when navigating via links within sidebar
sidebarScrollbox.scrollTop = sidebarScrollTop;
} else {
// scroll sidebar to current active section when navigating via "next/previous chapter" buttons
var activeSection = document.querySelector('#sidebar .active');
if (activeSection) {
activeSection.scrollIntoView({ block: 'center' });
}
}
</script>
<div id="page-wrapper" class="page-wrapper">
<div class="page">
<div id="menu-bar-hover-placeholder"></div>
<div id="menu-bar" class="menu-bar sticky">
<div class="left-buttons">
<label id="sidebar-toggle" class="icon-button" for="sidebar-toggle-anchor" title="Toggle Table of Contents" aria-label="Toggle Table of Contents" aria-controls="sidebar">
<i class="fa fa-bars"></i>
</label>
<button id="theme-toggle" class="icon-button" type="button" title="Change theme" aria-label="Change theme" aria-haspopup="true" aria-expanded="false" aria-controls="theme-list">
<i class="fa fa-paint-brush"></i>
</button>
<ul id="theme-list" class="theme-popup" aria-label="Themes" role="menu">
<li role="none"><button role="menuitem" class="theme" id="light">Light</button></li>
<li role="none"><button role="menuitem" class="theme" id="rust">Rust</button></li>
<li role="none"><button role="menuitem" class="theme" id="coal">Coal</button></li>
<li role="none"><button role="menuitem" class="theme" id="navy">Navy</button></li>
<li role="none"><button role="menuitem" class="theme" id="ayu">Ayu</button></li>
</ul>
<button id="search-toggle" class="icon-button" type="button" title="Search. (Shortkey: s)" aria-label="Toggle Searchbar" aria-expanded="false" aria-keyshortcuts="S" aria-controls="searchbar">
<i class="fa fa-search"></i>
</button>
</div>
<h1 class="menu-title">Rust语言圣经(Rust Course)</h1>
<div class="right-buttons">
<a href="../print.html" title="Print this book" aria-label="Print this book">
<i id="print-button" class="fa fa-print"></i>
</a>
<a href="https://github.com/sunface/rust-course" title="Git repository" aria-label="Git repository">
<i id="git-repository-button" class="fa fa-github"></i>
</a>
<a href="https://github.com/sunface/rust-course/edit/main/src/test/benchmark.md" title="Suggest an edit" aria-label="Suggest an edit">
<i id="git-edit-button" class="fa fa-edit"></i>
</a>
</div>
</div>
<div id="search-wrapper" class="hidden">
<form id="searchbar-outer" class="searchbar-outer">
<input type="search" id="searchbar" name="searchbar" placeholder="Search this book ..." aria-controls="searchresults-outer" aria-describedby="searchresults-header">
</form>
<div id="searchresults-outer" class="searchresults-outer hidden">
<div id="searchresults-header" class="searchresults-header"></div>
<ul id="searchresults">
</ul>
</div>
</div>
<!-- Apply ARIA attributes after the sidebar and the sidebar toggle button are added to the DOM -->
<script>
document.getElementById('sidebar-toggle').setAttribute('aria-expanded', sidebar === 'visible');
document.getElementById('sidebar').setAttribute('aria-hidden', sidebar !== 'visible');
Array.from(document.querySelectorAll('#sidebar a')).forEach(function(link) {
link.setAttribute('tabIndex', sidebar === 'visible' ? 0 : -1);
});
</script>
<div id="content" class="content">
<!-- Page table of contents -->
<div class="sidetoc"><nav class="pagetoc"></nav></div>
<main>
<h1 id="基准测试-benchmark"><a class="header" href="#基准测试-benchmark">基准测试 benchmark</a></h1>
<p>几乎所有开发都知道,如果要测量程序的性能,就需要性能测试。</p>
<p>性能测试包含了两种:压力测试和基准测试。前者是针对接口 API模拟大量用户去访问接口然后生成接口级别的性能数据而后者是针对代码可以用来测试某一段代码的运行速度例如一个排序算法。</p>
<p>而本文将要介绍的就是基准测试 <code>benchmark</code>,在 Rust 中,有两种方式可以实现:</p>
<ul>
<li>官方提供的 <code>benchmark</code></li>
<li>社区实现,例如 <code>criterion.rs</code></li>
</ul>
<p>事实上我们更推荐后者,原因在后文会详细介绍,下面先从官方提供的工具开始。</p>
<h2 id="官方-benchmark"><a class="header" href="#官方-benchmark">官方 benchmark</a></h2>
<p>官方提供的测试工具,目前最大的问题就是只能在非 <code>stable</code> 下使用,原因是需要在代码中引入 <code>test</code> 特性: <code>#![feature(test)]</code></p>
<h4 id="设置-rust-版本"><a class="header" href="#设置-rust-版本">设置 Rust 版本</a></h4>
<p>因此在开始之前,我们需要先将当前仓库中的 <a href="https://course.rs/appendix/rust-version.html#%E4%B8%8D%E7%A8%B3%E5%AE%9A%E5%8A%9F%E8%83%BD"><code>Rust 版本</code></a><code>stable</code> 切换为 <code>nightly</code>:</p>
<ol>
<li>安装 <code>nightly</code> 版本:<code>$ rustup install nightly</code></li>
<li>使用以下命令确认版本已经安装成功</li>
</ol>
<pre><code class="language-shell">$ rustup toolchain list
stable-aarch64-apple-darwin (default)
nightly-aarch64-apple-darwin (override)
</code></pre>
<ol start="3">
<li>进入 <code>adder</code> 项目(之前为了学习测试专门创建的项目)的根目录,然后运行 <code>rustup override set nightly</code>,将该项目使用的 <code>rust</code> 设置为 <code>nightly</code></li>
</ol>
<p>很简单吧,其实只要一个命令就可以切换指定项目的 Rust 版本,例如你还能在基准测试后再使用 <code>rustup override set stable</code> 切换回 <code>stable</code> 版本。</p>
<h4 id="使用-benchmark"><a class="header" href="#使用-benchmark">使用 benchmark</a></h4>
<p>当完成版本切换后,就可以开始正式编写 <code>benchmark</code> 代码了。首先,将 <code>src/lib.rs</code> 中的内容替换成如下代码:</p>
<pre><pre class="playground"><code class="language-rust edition2021"><span class="boring">#![allow(unused)]
</span>#![feature(test)]
<span class="boring">fn main() {
</span>extern crate test;
pub fn add_two(a: i32) -&gt; i32 {
a + 2
}
#[cfg(test)]
mod tests {
use super::*;
use test::Bencher;
#[test]
fn it_works() {
assert_eq!(4, add_two(2));
}
#[bench]
fn bench_add_two(b: &amp;mut Bencher) {
b.iter(|| add_two(2));
}
}
<span class="boring">}</span></code></pre></pre>
<p>可以看出,<code>benchmark</code> 跟单元测试区别不大,最大的区别在于它是通过 <code>#[bench]</code> 标注,而单元测试是通过 <code>#[test]</code> 进行标注,这意味着 <code>cargo test</code> 将不会运行 <code>benchmark</code> 代码:</p>
<pre><code class="language-shell">$ cargo test
running 2 tests
test tests::bench_add_two ... ok
test tests::it_works ... ok
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
</code></pre>
<p><code>cargo test</code> 直接把我们的 <code>benchmark</code> 代码当作单元测试处理了,因此没有任何性能测试的结果产生。</p>
<p>对此,需要使用 <code>cargo bench</code> 命令:</p>
<pre><code class="language-shell">$ cargo bench
running 2 tests
test tests::it_works ... ignored
test tests::bench_add_two ... bench: 0 ns/iter (+/- 0)
test result: ok. 0 passed; 0 failed; 1 ignored; 1 measured; 0 filtered out; finished in 0.29s
</code></pre>
<p>看到没,一个截然不同的结果,除此之外还能看出几点:</p>
<ul>
<li>单元测试 <code>it_works</code> 被忽略,并没有执行: <code>tests::it_works ... ignored</code></li>
<li>benchmark 的结果是 <code>0 ns/iter</code>,表示每次迭代( <code>b.iter</code> )耗时 <code>0 ns</code>,奇怪,怎么是 <code>0</code> 纳秒呢?别急,原因后面会讲</li>
</ul>
<h4 id="一些使用建议"><a class="header" href="#一些使用建议">一些使用建议</a></h4>
<p>关于 <code>benchmark</code>,这里有一些使用建议值得大家关注:</p>
<ul>
<li>将初始化代码移动到 <code>b.iter</code> 循环之外,否则每次循环迭代都会初始化一次,这里只应该存放需要精准测试的代码</li>
<li>让代码每次都做一样的事情,例如不要去做累加或状态更改的操作</li>
<li>最好让 <code>iter</code> 之外的代码也具有幂等性,因为它也可能被 <code>benchmark</code> 运行多次</li>
<li>循环内的代码应该尽量的短小快速,因为这样循环才能被尽可能多的执行,结果也会更加准确</li>
</ul>
<h4 id="谜一般的性能结果"><a class="header" href="#谜一般的性能结果">谜一般的性能结果</a></h4>
<p>在写 <code>benchmark</code> 时,你可能会遇到一些很纳闷的棘手问题,例如以下代码:</p>
<pre><pre class="playground"><code class="language-rust edition2021"><span class="boring">#![allow(unused)]
</span>#![feature(test)]
<span class="boring">fn main() {
</span>extern crate test;
fn fibonacci_u64(number: u64) -&gt; u64 {
let mut last: u64 = 1;
let mut current: u64 = 0;
let mut buffer: u64;
let mut position: u64 = 1;
return loop {
if position == number {
break current;
}
buffer = last;
last = current;
current = buffer + current;
position += 1;
};
}
#[cfg(test)]
mod tests {
use super::*;
use test::Bencher;
#[test]
fn it_works() {
assert_eq!(fibonacci_u64(1), 0);
assert_eq!(fibonacci_u64(2), 1);
assert_eq!(fibonacci_u64(12), 89);
assert_eq!(fibonacci_u64(30), 514229);
}
#[bench]
fn bench_u64(b: &amp;mut Bencher) {
b.iter(|| {
for i in 100..200 {
fibonacci_u64(i);
}
});
}
}
<span class="boring">}</span></code></pre></pre>
<p>通过<code>cargo bench</code>运行后,得到一个难以置信的结果:<code>test tests::bench_u64 ... bench: 0 ns/iter (+/- 0)</code>, 难道 Rust 已经到达量子计算机级别了?</p>
<p>其实,原因藏在<code>LLVM</code>中: <code>LLVM</code>认为<code>fibonacci_u64</code>函数调用的结果没有使用,同时也认为该函数没有任何副作用(造成其它的影响,例如修改外部变量、访问网络等), 因此它有理由把这个函数调用优化掉!</p>
<p>解决很简单,使用 Rust 标准库中的 <code>black_box</code> 函数:</p>
<pre><pre class="playground"><code class="language-rust edition2021"><span class="boring">#![allow(unused)]
</span><span class="boring">fn main() {
</span> for i in 100..200 {
test::black_box(fibonacci_u64(test::black_box(i)));
}
<span class="boring">}</span></code></pre></pre>
<p>通过这个函数,我们告诉编译器,让它尽量少做优化,此时 LLVM 就不会再自作主张了:)</p>
<pre><code class="language-shell">$ cargo bench
running 2 tests
test tests::it_works ... ignored
test tests::bench_u64 ... bench: 5,626 ns/iter (+/- 267)
test result: ok. 0 passed; 0 failed; 1 ignored; 1 measured; 0 filtered out; finished in 0.67s
</code></pre>
<p>嗯,这次结果就明显正常了。</p>
<h2 id="criterionrs"><a class="header" href="#criterionrs">criterion.rs</a></h2>
<p>官方 <code>benchmark</code> 有两个问题,首先就是不支持 <code>stable</code> 版本的 Rust其次是结果有些简单缺少更详细的统计分布。</p>
<p>因此社区 <code>benchmark</code> 就应运而生,其中最有名的就是 <a href="https://github.com/bheisler/criterion.rs"><code>criterion.rs</code></a>,它有几个重要特性:</p>
<ul>
<li>统计分析,例如可以跟上一次运行的结果进行差异比对</li>
<li>图表,使用 <a href="http://www.gnuplot.info"><code>gnuplots</code></a> 展示详细的结果图表</li>
</ul>
<p>首先,如果你需要图表,需要先安装 <code>gnuplots</code>,其次,我们需要引入相关的包,在 <code>Cargo.toml</code> 文件中新增 :</p>
<pre><code class="language-toml">[dev-dependencies]
criterion = &quot;0.3&quot;
[[bench]]
name = &quot;my_benchmark&quot;
harness = false
</code></pre>
<p>接着,在项目中创建一个测试文件: <code>$PROJECT/benches/my_benchmark.rs</code>,然后加入以下内容:</p>
<pre><pre class="playground"><code class="language-rust edition2021"><span class="boring">#![allow(unused)]
</span><span class="boring">fn main() {
</span>use criterion::{black_box, criterion_group, criterion_main, Criterion};
fn fibonacci(n: u64) -&gt; u64 {
match n {
0 =&gt; 1,
1 =&gt; 1,
n =&gt; fibonacci(n-1) + fibonacci(n-2),
}
}
fn criterion_benchmark(c: &amp;mut Criterion) {
c.bench_function(&quot;fib 20&quot;, |b| b.iter(|| fibonacci(black_box(20))));
}
criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);
<span class="boring">}</span></code></pre></pre>
<p>最后,使用 <code>cargo bench</code> 运行并观察结果:</p>
<pre><code class="language-shell"> Running target/release/deps/example-423eedc43b2b3a93
Benchmarking fib 20
Benchmarking fib 20: Warming up for 3.0000 s
Benchmarking fib 20: Collecting 100 samples in estimated 5.0658 s (188100 iterations)
Benchmarking fib 20: Analyzing
fib 20 time: [26.029 us 26.251 us 26.505 us]
Found 11 outliers among 99 measurements (11.11%)
6 (6.06%) high mild
5 (5.05%) high severe
slope [26.029 us 26.505 us] R^2 [0.8745662 0.8728027]
mean [26.106 us 26.561 us] std. dev. [808.98 ns 1.4722 us]
median [25.733 us 25.988 us] med. abs. dev. [234.09 ns 544.07 ns]
</code></pre>
<p>可以看出,这个结果是明显比官方的更详尽的,如果大家希望更深入的学习它的使用,可以参见<a href="https://bheisler.github.io/criterion.rs/book/getting_started.html">官方文档</a></p>
<div id="giscus-container"></div>
</main>
<nav class="nav-wrapper" aria-label="Page navigation">
<!-- Mobile navigation buttons -->
<a rel="prev" href="../test/ci.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
<a rel="next prefetch" href="../cargo/intro.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
<div style="clear: both"></div>
</nav>
</div>
</div>
<nav class="nav-wide-wrapper" aria-label="Page navigation">
<a rel="prev" href="../test/ci.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
<a rel="next prefetch" href="../cargo/intro.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
</nav>
</div>
<script>
window.playground_copyable = true;
</script>
<script src="../ace.js"></script>
<script src="../editor.js"></script>
<script src="../mode-rust.js"></script>
<script src="../theme-dawn.js"></script>
<script src="../theme-tomorrow_night.js"></script>
<script src="../elasticlunr.min.js"></script>
<script src="../mark.min.js"></script>
<script src="../searcher.js"></script>
<script src="../clipboard.min.js"></script>
<script src="../highlight.js"></script>
<script src="../book.js"></script>
<script type="text/javascript" charset="utf-8">
var pagePath = "test/benchmark.md"
</script>
<!-- Custom JS scripts -->
<script src="../assets/custom.js"></script>
<script src="../assets/bigPicture.js"></script>
</div>
</body>
</html>