理解总结报告
在每次测试运行结束时，OpenSearch Benchmark 会创建一份测试结果指标的总结，例如服务时间、吞吐量、延迟等。这些指标提供了关于所选工作负载在基准测试的 OpenSearch 集群上的表现的见解。
以下指南提供了关于如何理解总结报告结果的信息。
OpenSearch 基准测试运行
OpenSearch Benchmark 运行一系列针对 OpenSearch 开发集群的夜间测试。这些测试运行的结果可在 https://opensearch.org.cn/benchmarks 找到。结果显示了跨不同测试运行的指标，并针对 OpenSearch 的最新版本和未来版本。
总结报告指标
虽然 OpenSearch Benchmark 总结报告提供了与集群性能相关的指标，但如何比较和使用这些指标取决于您的用例。例如，一些用户可能对工作负载可以索引的文档数量感兴趣，而另一些用户可能对查询文档所需的延迟或服务时间感兴趣。以下示例总结报告显示了典型工作负载运行的指标
------------------------------------------------------
_______ __ _____
/ ____(_)___ ____ _/ / / ___/_________ ________
/ /_ / / __ \/ __ `/ / \__ \/ ___/ __ \/ ___/ _ \
/ __/ / / / / / /_/ / / ___/ / /__/ /_/ / / / __/
/_/ /_/_/ /_/\__,_/_/ /____/\___/\____/_/ \___/
------------------------------------------------------
| Metric | Task | Value | Unit |
|---------------------------------------------------------------:|-------------------------------------------:|------------:|-------:|
| Cumulative indexing time of primary shards | | 0.02655 | min |
| Min cumulative indexing time across primary shards | | 0 | min |
| Median cumulative indexing time across primary shards | | 0.00176667 | min |
| Max cumulative indexing time across primary shards | | 0.0140333 | min |
| Cumulative indexing throttle time of primary shards | | 0 | min |
| Min cumulative indexing throttle time across primary shards | | 0 | min |
| Median cumulative indexing throttle time across primary shards | | 0 | min |
| Max cumulative indexing throttle time across primary shards | | 0 | min |
| Cumulative merge time of primary shards | | 0.0102333 | min |
| Cumulative merge count of primary shards | | 3 | |
| Min cumulative merge time across primary shards | | 0 | min |
| Median cumulative merge time across primary shards | | 0 | min |
| Max cumulative merge time across primary shards | | 0.0102333 | min |
| Cumulative merge throttle time of primary shards | | 0 | min |
| Min cumulative merge throttle time across primary shards | | 0 | min |
| Median cumulative merge throttle time across primary shards | | 0 | min |
| Max cumulative merge throttle time across primary shards | | 0 | min |
| Cumulative refresh time of primary shards | | 0.0709333 | min |
| Cumulative refresh count of primary shards | | 118 | |
| Min cumulative refresh time across primary shards | | 0 | min |
| Median cumulative refresh time across primary shards | | 0.00186667 | min |
| Max cumulative refresh time across primary shards | | 0.0511667 | min |
| Cumulative flush time of primary shards | | 0.00963333 | min |
| Cumulative flush count of primary shards | | 4 | |
| Min cumulative flush time across primary shards | | 0 | min |
| Median cumulative flush time across primary shards | | 0 | min |
| Max cumulative flush time across primary shards | | 0.00398333 | min |
| Total Young Gen GC time | | 0 | s |
| Total Young Gen GC count | | 0 | |
| Total Old Gen GC time | | 0 | s |
| Total Old Gen GC count | | 0 | |
| Store size | | 0.000485923 | GB |
| Translog size | | 2.01873e-05 | GB |
| Heap used for segments | | 0 | MB |
| Heap used for doc values | | 0 | MB |
| Heap used for terms | | 0 | MB |
| Heap used for norms | | 0 | MB |
| Heap used for points | | 0 | MB |
| Heap used for stored fields | | 0 | MB |
| Segment count | | 32 | |
| Min Throughput | index | 3008.97 | docs/s |
| Mean Throughput | index | 3008.97 | docs/s |
| Median Throughput | index | 3008.97 | docs/s |
| Max Throughput | index | 3008.97 | docs/s |
| 50th percentile latency | index | 351.059 | ms |
| 100th percentile latency | index | 365.058 | ms |
| 50th percentile service time | index | 351.059 | ms |
| 100th percentile service time | index | 365.058 | ms |
| error rate | index | 0 | % |
| Min Throughput | wait-until-merges-finish | 28.41 | ops/s |
| Mean Throughput | wait-until-merges-finish | 28.41 | ops/s |
| Median Throughput | wait-until-merges-finish | 28.41 | ops/s |
| Max Throughput | wait-until-merges-finish | 28.41 | ops/s |
| 100th percentile latency | wait-until-merges-finish | 34.7088 | ms |
| 100th percentile service time | wait-until-merges-finish | 34.7088 | ms |
| error rate | wait-until-merges-finish | 0 | % |
| Min Throughput | match_all | 36.09 | ops/s |
| Mean Throughput | match_all | 36.09 | ops/s |
| Median Throughput | match_all | 36.09 | ops/s |
| Max Throughput | match_all | 36.09 | ops/s |
| 100th percentile latency | match_all | 35.9822 | ms |
| 100th percentile service time | match_all | 7.93048 | ms |
| error rate | match_all | 0 | % |
[...]
| Min Throughput | term | 16.1 | ops/s |
| Mean Throughput | term | 16.1 | ops/s |
| Median Throughput | term | 16.1 | ops/s |
| Max Throughput | term | 16.1 | ops/s |
| 100th percentile latency | term | 131.798 | ms |
| 100th percentile service time | term | 69.5237 | ms |
| error rate | term | 0 | % |
集群特有的指标从 index 任务行开始。以下是您可能感兴趣的指标示例
- 为了评估集群能处理多少负载，
index任务指标提供了在工作负载运行期间摄入的文档数量以及摄入错误率。 - 为了评估工作负载中查询的可测量延迟和服务时间，
match_all和term任务提供了每秒执行的查询操作数量、可测量的查询延迟以及查询操作错误率。
存储结果
OpenSearch Benchmark 结果存储在内存中或外部存储中。
当结果存储在内存中时，可以在 /.benchmark/benchmarks/test_executions/<test_execution_id> 目录中找到。结果的命名与最近工作负载测试的 test_execution_id 一致。
在运行测试时，您可以使用以下命令标志的任意组合来自定义结果的存储位置
--results-file：当提供文件路径时，将总结报告写入指定路径的文件。--results-format：定义总结报告结果的输出格式，可以是markdown或csv。默认值为markdown。--show-in-results：定义发布总结报告中显示的值，可以是available、all-percentiles或all。默认值为available。--user-tag：定义在指标记录中用作元信息的用户特定键值对，例如intention:baseline-ticket-12345。这在将指标和结果存储到外部存储时非常有用。