Technical Notes
JSBenchmark is the next generation javascript benchmarking test. Its predecessor, which is still available, received quite a bit of attention. Since 2006, Safari for Windows and Google Chrome have been released, and Javascript speeds have increased dramatically. Over time, the need for an improved benchmarking tool became apparent.
Features of JSBenchmark
Web 2.0 Oriented
Many users were interested in posting and comparing their benchmark results against other users. If you don't want to install five competing browsers to benchmark, you can view the community's results to see how browsers perform among thousands of computers.
Store Results
If you create a free JSBenchmark.com account, your benchmark results are stored for future review. By storing results, you can longitudinally compare your results against the community.
Better Accuracy
There are two basic methods for benchmarking:
- Measure the time elapsed to execute a finite number of iterations (seconds per iteration)
- Measure the number of iterations over an elapsed period of time (iterations per second)
The older celtickane.com benchmark uses method #1, but this has become problematic because the run-times for this type of benchmark are approximately 0.3 seconds or less. This benchmarking method is more appropriate for code that takes a long time to execute (e.g. encoding a video file). The benchmark runs the risk of diminishing returns as browser performance increases.
JSBenchmark utilizes method #2, which is more appropriate for code that can execute multiple iterations in a short period of time. As browser performance improves, this benchmark will be able to demonstrate improvements better.
Description of Benchmark Tests
-
An array, increasing in size with each iteration, is populated with random numbers and quicksorted. This provides a real-world benchmark test of array manipulation.
-
With each iteration, the benchmark calculates i! (factorial) and 2^i (exponent), where i represents the number of iterations completed. This provides a good benchmark of recursion and exponent math, which is typically a CPU-expensive process.
-
Based on Conway’s Game of Life, an 8x8 matrix is generated and randomly populated with alive or dead cells. Each cycle within the game counts as an iteration. This provides a real-world example of a game that has heavy array manipulation.
-
With each iteration, a “nucleotide chain” of 1,000 base pairs is created and added on to a previous chain. As the chain grows, a random codon (triplet set) of nucleotides is selected to mutate into another random codon. This makes heavy use of regular expressions in a mock-simulation of a biological situation.
-
An MD5 encryption hash is generated from Hamlet’s famous soliloquy. At each iteration, the string changes slightly (to prevent potential unfair caching) and a new MD5 hash is created. This benchmark is extremely representative of real-world encryption hashing scenario.
-
Calculates as many prime numbers as possible within the given time allotment.
-
Uses a genetic algorithm to determine the optimum route between five randomly generated points. Each generation of the genetic algorithm counts as an iteration.
-
Classic benchmark test that adds random characters to an expanding array while shifting and pushing elements onto the end of the array. Each iteration ends by sorting and reversing the array.
-
A random date between 1/1/1050 and 12/31/2050 is generated using the Parse() method. This random date is compared against a static date (12/7/1941) and various date-based mathematical expressions are evaluated.
-
An error is thrown to the browser within a try/catch/finally statement. Error handling is typically an expensive CPU process; catching errors can consume large amounts of CPU time.
Test Coefficients
Each test’s results are multiplied by a test coefficient. This coefficient is designed to weigh all ten of the benchmark tests equally, thus not giving an unfair advantage to any one browser if it is especially good at executing a test that has a higher test weight.
The coefficients were determined by running the benchmark in five popular web browsers (Firefox, Opera, Internet Explorer, Chrome, Safari) and taking the average score of each test. The coefficient was calculated by normalizing the average score for each test to a single score. This coefficient reduces unfair browser advantage and standardizes each test.
Community-Based Results
Once a benchmark test is run, an AJAX query collects the browser’s user agent string, the IP address, and the test results. No other information is collected. These browser results are collected and aggregated at regular intervals to generate the community’s results. Although these results are less scientific due to the potential for error and bias, it does give you the ability to compare your results against the community.
Note that users are only allowed to contribute a certain number of test results into the community. Additionally, community results only display up to two standard deviations (a representative 95% sample) of scoring -- this reduces outliers that unfairly cloud the data pool.
Source Code Acknowledgements
The benchmark is using several pieces of code that are not my own. Proper credit has been given within the source code, but I also wanted to acknowledge credit below: