The results given by this benchmark are bothering me because they do not fit with what I've seen in production.
For example, the `for-of-array` is transpiled by babel to something that has 2 nested try/catch blocks, and unfortunately, V8 has a nasty deoptimisation when dealing with these, even if no exception is ever thrown.
This deopt led to a several orders of magnitude slowdown compare to a simple `for` loop in Google Chrome, forcing use to abandon the `for of` construction.
I suspect that the benchmark shown on this page does not expose this kind of behavior because it doesn't iterate enough for the JIT to kick in. If it's the case, it means that these values reflects only the behavior of the cold, interpreted code, and not the hot one. (which is quite sad for a performance benchmark, because the performance matter only for the former …)
Chrome doesn't have an interpreter, it has baseline and optimizing compiler. Also, I think try/catch preventing optimization was fixed.
Still your point could stand, or additionally there could be many optimizations that interfere with such simple micro benchmarks, turning all or parts of them into no-ops. If this is happening in ES5 and not ES6, the results could be drastic. I suspect this is what's happening with some of the larger differences in native ES6.
Agreed, and issues like this make me hugely suspicious of JS performance benchmarks. I'm only familiar with V8, but my experience has been that there are more or less three "tiers" of factors that affect performance:
1. whether the optimizing compiler gets used
2. random stuff that's impossible to know about unless you grab your code's internal representation and decompile it [1]
3. the raw performance of individual statements
The effects of (1) dominate (2), and the effects of (2) dominate (3). So when I see performance micro-benchmarks that purport to measure (3) without apparently paying any attention to (1) or (2), my first inclination is to assume the results are essentially random noise.
[1] E.g. whether statements get wrapped in type checks, whether an integer variable gets handled internally as a SMI (small int) or whether it keeps getting converted to a boxed Number and back. The cost of such things can easily exceed the cost of the statement you're trying to benchmark.
Yes, my understanding is the problem is essentially undefined behavior as exceptions can be caught at any point in the call stack which makes it basically impossible to optimize.
It's not impossible at all. try/catch in the non-throwing case is fast in both Firefox (SpiderMonkey) and IE (Chakra); I'm not sure what the state of things is in Safari (JavaScriptCore). In the throwing case you obviously have to do some work, but as long as that case is rare it's not a problem. Contrast with the V8 situation (which they are fixing), where simply having a try/catch in your function at all will deoptimize the function, even if an exception is never actually thrown.
Oh, and one optimization strategy for try catch is even pretty simple to describe in general terms: you need a cheap way to check whether an exception is thrown, and then after every operation that can throw (e.g. a call into the vm to a function that is allowed to throw) you check whether it did. If not, you just move along. If it did, you jump to an out of line path that does cleanup and callstack unwinding. The devil, as usual, is in the details.
I'm not a frontend developer but... is it still truly necessary to transpile stuff like for-of ?
It's supported by every desktop browser on the market, modulo some bugs (like inability to iterate over HTMLCollection and co. on Chrome)... as long as you only need to use it for arrays, things might work quite fine
Unless you're a startup and don't have to give a damn about what browsers your users use, IE marketshare is just too big to ignore. And depending on your audience, IE can mean anything from IE11 all the way back to IE8.
Theoretically IE is irrelevant now that Microsoft has started publicly killing it off, but you'd be surprised at the number of corporate users that are still on Windows XP (or maybe Vista/7) running IE 8 and 9.
Of course evergreen browsers are less of a problem but amazingly a lot of the same users who are stuck with IE also manage to have extremely outdated versions of Firefox.
This deopt led to a several orders of magnitude slowdown compare to a simple `for` loop in Google Chrome, forcing use to abandon the `for of` construction.
I suspect that the benchmark shown on this page does not expose this kind of behavior because it doesn't iterate enough for the JIT to kick in. If it's the case, it means that these values reflects only the behavior of the cold, interpreted code, and not the hot one. (which is quite sad for a performance benchmark, because the performance matter only for the former …)