There are probably 100x and 1000x engineers, but their true value can't be easily captured with any metric, at least not on teams and projects of non-trivial size and duration.
The value of an engineer is not what they did on a given day. It lies in the architectural decisions they make. Those decisions have ramifications over the following years.
I have seen "genius 10x engineers" make egregious architecture decisions (both micro- and macro) that have cost thousands upon thousands of employee-hours of lost productivity in the ensuing decades. Those developers were often regarded as "geniuses" and "highly productive" because they spent thousands of hours simply fixing the very problems they caused in the first place! Those same developers have also made a number of decisions that avoided similar pitfalls, thus saving the company thousands upon thousands of lost hours. So were those good engineers or bad ones?
Research that all you want, but I've never seen anybody propose a metric that would even remotely capture this sort of long-term net value created (or harm done) by engineers.
To even begin to answer this question, we'd need to take an extremely long view. We'd need to look at multi-year projects, the architectural decisions made, and somehow correlate them with outcomes. I'm not sure how you'd quantitatively define the success of a software project in the first place, much less map actions to outcomes.
Especially since the evaluation of software architecture decisions really involves comparing roads taken versus the roads not taken. Suppose engineer A chose language XYZ for a particular solution. Was it the right call? Well, what you're really trying to evaluate is the choice of XYZ against other choices that might hypothetically be made.
And, on top of that, engineering decisions are typically constrained by outside factors. Engineers often make bad decisions because they're forced to. Perhaps XYZ was an utterly terrible choice for the task at hand, but this engineer had nothing but a team of XYZ developers at their disposal. Management had a hiring freeze in place and deadlines were tight, making the choice of anything but XYZ nearly impossible. So how do you judge the effectiveness of that engineer?
I (like most people reading this) have made some absolutely disgusting engineering decisions in my life simply due to constraints like this. I once hardcoded several megabytes of data into a PHP array instead of using a database like a normal human being. This is, by any reasonable standard, a pretty awful decision. You won't find it in any books.
However, this was a very short-term project. The data was read-only. The deployment environment had no database available. And we were facing a deadline measured in hours. (This was before the days of services like AWS / Digital Ocean / etc making it easy to spin up a server stack) After a bit of testing I discovered my kludge didn't perform too badly after the first page load since I guess PHP execution plans get cached somewhere by some app servers. So I shipped it, the site worked, and we got paid.
Was I a 10x engineer there? Or 0.1x? Or 1x? I have no idea, and frankly the question itself seems silly.
The value of an engineer is not what they did on a given day. It lies in the architectural decisions they make. Those decisions have ramifications over the following years.
I have seen "genius 10x engineers" make egregious architecture decisions (both micro- and macro) that have cost thousands upon thousands of employee-hours of lost productivity in the ensuing decades. Those developers were often regarded as "geniuses" and "highly productive" because they spent thousands of hours simply fixing the very problems they caused in the first place! Those same developers have also made a number of decisions that avoided similar pitfalls, thus saving the company thousands upon thousands of lost hours. So were those good engineers or bad ones?
Research that all you want, but I've never seen anybody propose a metric that would even remotely capture this sort of long-term net value created (or harm done) by engineers.
To even begin to answer this question, we'd need to take an extremely long view. We'd need to look at multi-year projects, the architectural decisions made, and somehow correlate them with outcomes. I'm not sure how you'd quantitatively define the success of a software project in the first place, much less map actions to outcomes.
Especially since the evaluation of software architecture decisions really involves comparing roads taken versus the roads not taken. Suppose engineer A chose language XYZ for a particular solution. Was it the right call? Well, what you're really trying to evaluate is the choice of XYZ against other choices that might hypothetically be made.
And, on top of that, engineering decisions are typically constrained by outside factors. Engineers often make bad decisions because they're forced to. Perhaps XYZ was an utterly terrible choice for the task at hand, but this engineer had nothing but a team of XYZ developers at their disposal. Management had a hiring freeze in place and deadlines were tight, making the choice of anything but XYZ nearly impossible. So how do you judge the effectiveness of that engineer?
I (like most people reading this) have made some absolutely disgusting engineering decisions in my life simply due to constraints like this. I once hardcoded several megabytes of data into a PHP array instead of using a database like a normal human being. This is, by any reasonable standard, a pretty awful decision. You won't find it in any books.
However, this was a very short-term project. The data was read-only. The deployment environment had no database available. And we were facing a deadline measured in hours. (This was before the days of services like AWS / Digital Ocean / etc making it easy to spin up a server stack) After a bit of testing I discovered my kludge didn't perform too badly after the first page load since I guess PHP execution plans get cached somewhere by some app servers. So I shipped it, the site worked, and we got paid.
Was I a 10x engineer there? Or 0.1x? Or 1x? I have no idea, and frankly the question itself seems silly.