Skip to main content

Please Stop the jsPerf.com Abuse!


According to many of the tests on jsperf.com, jQuery is slow. Really slow. The jQuery team gets bug reports from web citizens who've discovered these egregious flaws via jsperf and want them fixed. Now, jsperf.com can be a great tool when used knowledgeably, but its results can also be misinterpreted and abused. For example:

Why doesn't jQuery use DOM classList for methods like .addClass()?

There are at least three good reasons.
  1. It doesn't make any practical performance difference given its frequency of use.
  2. It complicates code paths, since classList isn't supported everywhere.
  3. It makes jQuery bigger, which makes it load slower for everyone, every time.
But this jsperf shows classList is much faster! How can you say that?

Well it's possible that test is running afoul of microbenchmark issues, and not measuring what it is supposed to measure due to the increased sophistication of today's JavaScript compilers. This is a big problem with a lot of the jsperf tests out there, they are just not measuring anything useful, or at least not measuring what their creator thought they measured.

But let's assume that in this case, the JavaScript is actually doing just what the test case creator intended to do, and none of those microbenchmark oddities are occuring. The two test cases are not quite the same because the API behaviors are different:
  • The jQuery case works correctly for zero, one, or more elements in its collection. The classList case only works with one element selected, and will throw an error if no elements are selected.
  • The jQuery case can add or remove multiple classes at once. The pure DOM case will not, since classList does not support it.
  • The jQuery case runs on browsers that don't support classList. This includes the Android 2.3 browser (about 25 percent of all Android) and Internet Explorer before version 10 (about 36 percent of all IE).
Since jQuery provides wide browser support and a richer API for class operations, any attempt to use classList inside jQuery would still need supporting code around it to deal with older browsers plus the cases of multiple elements and multiple classes. That's more code to be downloaded and parsed each time any browser (including those without classList) loads a page with jQuery. That starts our space-vs-speed performance tradeoff in a hole that we may never dig out of.

As for the speed, let's look at how long it takes to do this block of operations, ignoring that they consist of things you'd never do all at once such as adding and then immediately removing the same class. Looking at the per-block execution times on Chrome, the jQuery API takes only about 16 microseconds (just 1/1000 of our 16 millisecond frame length) to do those class operations. Sure, the native methods do it roughly four times faster, but are these 12 additional microseconds too much considering the extra features jQuery offers?

Now the classList advocate might say, "What if I call this code 1,000 times? Now it's taking 12 milliseconds and eating up most of my frame budget!" Typical real-life code would only manipulate the class property a few times a second at most, usually in response to user interaction. If you're really manipulating 1,000 of any DOM element in JavaScript and expecting that to fit into a 16-millisecond chunk of work, you're likely running into all sorts of performance issues anyway. That's a problem with algorithms and design, unlikely to be magically solved by shaving a few microseconds off jQuery's class manipulation methods.

After just about any class change, the browser needs to recalculate styles and redraw at least part of the page. The cost of those operations will usually outweigh the cost of manipulating the class property, but they're not reflected in jsperf because the rendering engine runs on the same thread as the benchmark. The browser doesn't get a chance to re-render the page until after the timed block of code is complete. But even if it did, the trivial size of this document (just one test element) isn't typical of the workload the browser faces in recalculating styles and layouts.

In fringe cases where jQuery is too slow, you're in luck! jQuery doesn't place "Do Not Enter" signs around DOM methods. You can freely mix many DOM methods with jQuery. That's why event handlers have the target DOM element, not a jQuery collection, as their this keyword. In your checkbox event handler, you're free to use this.checked instead of $(this).prop("checked") -- and you should, it's shorter! Go ahead and use this.classList.add() directly if you don't care about old Android or old IE and have strong emotions about wasting microseconds.

Knuth said, "We should forget about small efficiencies, say about 97 percent of the time: premature optimization is the root of all evil." The classList case is one of those small efficiencies that makes little difference to the performance of real code. There is no good reason for jQuery to blindly optimize exclusively for the CPU performance dimension that jsperf measures, particularly when it has a cost in code size and complexity. But the programmer's mentality often won't let go of these issues because, darn it, jQuery should use the native API regardless.

Bottom Line: Don't obsess on speeding up code that isn't slow. As tempting as it seems, jsperf.com is usually not the right tool for identifying browser bottlenecks in application code. Ultimately, the best way to make web pages and apps fast is to use tools like WebPageTest, followed by a profiling session in your favorite browser tools to pinpoint slow JavaScript. Don't waste your development time on the 97 percent.

Comments

m_gol said…
Post to the point! A few notes:

1) Add & remove methods now allow mutliple parameters, though not all browsers support it (e.g. IE, both 10 & 11 IIRC).

2) There are bugs like: https://bugs.webkit.org/show_bug.cgi?id=111307 (Chrome equivalent has been fixed).

3) classList-based implementation would fail some strict jQuery unit tests checking for whitespace removal; they would need to be modified to support the classList way.

4) Some confusion might have been caused by the fact the classList-based implementations *used to* be much faster before browsers started updating their code bases to support multiple arguments. Nothing comes without cost.

5) There is some weirdness in the spec as only the add/remove methods accept multiple classes, the toggle one doesn't as the second optional boolean parameter is reserved for forcing toggle to behave as add/remove. This inconsistency is pretty sad.

And to conclude: I somehow understand those people that would prefer the native implementation at any cost as it seems more pure and removing code that seems unnecessary seems like a nice thing to do. I myself went into this trap when I submitted a pull request to jQuery adding the classList handling: https://github.com/jquery/jquery/pull/1190. It's good to be able to accept criticism and move past one's mistakes. (for me it was just the beginning of contributing to jQuery).

Of course it's even better to not make those mistakes in the first place. ;)
Unknown said…
+1. Nothing beats profiling.

Ironically, another perf benefit of .classList (and reason to use it) is to mitigate the unexpected recalc style bottlenecks you mention. In some cases, the browser can bypass the DOM altogether and do a lookup in its internal mapping [1]. In contrast, the nature of .className requires touching the DOM every time (e..g. DOMString needs to be [de]serialized into an HTML attribute).

[1]: https://plus.google.com/+PaulIrish/posts/APArpwWqew3
Dave Methvin said…
@Eric, If the class does change via either method, a recalc is necessary. The reason assigning to the class attribute causes a recalc is that some browsers don't check to see whether the assignment actually changes the existing string and they always demand a recalc even when it's not needed. We're avoiding that behavior in 1.11/2.1 via #14250 but it would still be handy for browsers to check since older versions won't have that.
Unknown said…
Exactly. It would be interesting to know if the browsers that do a recalc on assigning the same className also do a recalc using .classList for the same case...assuming it's supported (76% browsers).

BTW, I don't see a recalc in Chrome when setting the same class name. That's probably been fixed in the last 3+ years. Do you have a test case I can try?
Dave Methvin said…
@Eric, there are a few tickets, see if those test cases still cause it:

https://code.google.com/p/chromium/issues/detail?id=226854

https://code.google.com/p/chromium/issues/detail?id=278045

https://code.google.com/p/chromium/issues/detail?id=155000
Unknown said…
A great article Dave! I sat in on your talk at jQueryTO this weekend, and loved it! This part of the talk truly helps me drive home the topic for my fellow developers at my office. Take care!
LewisCowles said…
Hi Dave, just wanted to solidify our Twitter discussion and ask about a time-frame on the new lightweight jQuery that will fix all this ;)

Popular posts from this blog

Tragedy of the WebKit Commons

With Opera announcing that their future products will be based on WebKit, the Internet is abuzz with discussion about what that means and whether it's a good thing. Looking at it as a jQuery developer, it's a good thing if it gets WebKit participants to fix bugs and update older implementations. I can't be optimistic without some evidence that things are really going to change. We don't know how many of Opera's core developers will move to WebKit development, but the press release isn't encouraging:  "The shift to WebKit means more of our resources can be dedicated to developing new features and the user-friendly solutions."  I suspect they want some cost savings by eliminating Presto technical staff, or -- in the most optimistic case for their employees -- refocusing existing staff on the parts outside WebKit core that make browsers different. Opera did  land their first WebKit patch  so they wanted to make a statement that they weren't gettin

You Might Not Need Babel

Babel is great, and by all means use it if it makes it easier to develop your application. If you're developing a library on the other hand, please take a moment to consider whether you actually need Babel. Maybe you can write a few more lines of utility code and forego the need to pre-process. If you're only targeting some platforms, you may not need anything more than plain old ES6. At the very least, make sure you know what Babel is doing for you, and what it's not. Some developers believe that Babel is protecting us from a great demon of code complexity when, in truth, browsers and node are pretty easy to deal with on their own. Let's take a look at some of the strategies for not needing Babel. ES5 is pretty okay: In the days when we used tables for layout and the debugger was named alert() , we would have been so grateful if JavaScript had the features that ES5 has today. If you truly need Babel you are not only a weak programmer, but also an ingrate. Face