Back to the main page

TauBench: A Dynamic Benchmark for Graphics Rendering (v1.1)

2022 (v1.0), Alanko, J., Mäkitalo, M., and Jääskeläinen, P., In Proceedings of the 17th International Conference on Computer Graphics Theory and Applications (GRAPP).

2023 (v1.1), Momeni Yazdi, E., Mäkitalo, M., Ikkala, J., and Jääskeläinen, P., technical notes, arXiv:2305.04804.

TauBench v1.1 is a permissively licensed (CC-BY-NC-SA 4.0) dynamic benchmark for graphics rendering, especially for benchmarking temporal reuse methods. It contains two scenes in binary glTF 2.0 format: EternalValleyFPS and EternalValleyVR.

Download TauBench at https://doi.org/10.5281/zenodo.5729573. It contains the EternalValleyFPS and EternalValleyVR glTF files (.glb), and the example rendered videos as .mp4 (in total 2.0 GB compressed).

Changelog

Quick start

Import one of the .glb files in your 3D modelling software, renderer or glTF viewer (for example the newest version of blender).

Technical details
Paper abstract (v1.0)

Many graphics rendering algorithms used in both real time games and virtual reality applications can get performance boosts by reusing previous computations. However, the temporal reuse based algorithms are typically measured using trivial benchmarks with very limited dynamic features. To this end, we present two new benchmarks that stress temporal reuse algorithms: EternalValleyVR and EternalValleyFPS. These datasets represent scenarios that are common contexts for temporal methods: EternalValleyFPS represents a typical interactive multiplayer game scenario with dynamically changing lighting conditions and geometry animations. EternalValleyVR adds rapid camera motion caused by head-mounted displays popular with virtual reality applications. In order to systematically assess the quality of the proposed benchmarks in reuse algorithm stress testing, we identify common input features used in state-of-the-art reuse algorithms and propose metrics that quantify changes in the temporally interesting features. Cameras in the proposed benchmarks rotate on average 18.5x more per frame compared to the popular NVidia ORCA datasets, which results in 51x more pixels introduced each frame. In addition to the camera activity, we compare the number of low confidence pixels. We show that the proposed datasets have 1.6x less pixel reuse opportunities by changes in pixels’ world positions, and 3.5x higher direct radiance discard rate.

How to cite (BibTeX)

@inproceedings{alanko2022taubench,
	title={TauBench: Dynamic Benchmark for Graphics Rendering},
	author={Alanko, Joel and M\"{a}kitalo, Markku and J\"{a}\"{a}skel\"{a}inen, Pekka},
	booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - GRAPP},
	year={2022},
	pages={172--179},
	publisher={SciTePress},
	organization={INSTICC},
	doi={10.5220/0010819200003124}
}

Preprint of the v1.0 paper

Technical notes of the v1.1 release

Supplementary material (v1.0)

Temporal benchmark results (more details in the v1.1 PDF)
Benchmark results for various rendering algorithms in Tauray, in terms of target PSNR quality. The PSNR values (compared to the 16384 spp 1080p reference) are averages over all frames, with the 10 best and 10 worst frame-wise PSNR values removed before averaging. The average frame-wise rendering times (HOST times on Tauray) on an RTX 2070 Super are also reported. N/A means the target was not reached with any input spp.
EternalValleyFPS ≥18 dB ≥20 dB ≥22 dB ≥24 dB ≥26 dB ≥28 dB
Path Tracing (PT) 14 spp
0.67 s
22 spp
1.18 s
35 spp
1.89 s
59 spp
3.21 s
99 spp
5.41 s
167 spp
9.10 s
PT + TAA 3 spp
0.13 s
4 spp
0.17 s
7 spp
0.31 s
13 spp
0.61 s
36 spp
1.74 s
N/A
PT + SVGF 1 spp
0.064 s
N/A N/A N/A

EternalValleyVR ≥18 dB ≥20 dB ≥22 dB ≥24 dB ≥26 dB ≥28 dB
Path Tracing (PT) 12 spp
0.62 s
20 spp
1.04 s
33 spp
1.74 s
58 spp
3.07 s
101 spp
5.37 s
180 spp
9.52 s
PT + TAA 2 spp
0.086 s
4 spp
0.17 s
6 spp
0.27 s
11 spp
0.51 s
22 spp
1.01 s
61 spp
2.91 s
PT + SVGF 1 spp
0.060 s
4 spp
0.19 s
N/A
Example screenshots

Top: EternalValleyFPS, bottom: EternalValleyVR. Larger images here.

Example rendered frames of the TauBench dataset