Discussion about this post

User's avatar
Hisashi Miyaguni's avatar

The warning at the top is the right one, and I think the reason is structural rather than psychological.

A Pacesetter Index is the top quartile of a population, and that population is already a selection. So a reader comparing themselves to it is comparing against two stacked filters, and the second one is invisible in the chart.

That changes the question. Not am I below the line, but was I ever in the population the line was cut from. For most readers the answer is no, and that is not a performance gap.

We ran into the smaller version of this in our own first issue. Two statistics circulate constantly in AI-for-accounting writing, and when I went looking for the primary sources, both traced only to content marketing sites. No published method behind either that I could find. We named them and left them out, which cost us the two best numbers we had.

A benchmark you cannot trace to a method is not evidence, it is decoration. The Pacesetter numbers are almost certainly real. The open question is what population they were cut from, and whether the reader is in it.

Does ICONIQ publish the denominator?

No posts

Ready for more?