by Brandon Ally, S2 Cognition
Cognitive Skills That Separate the Best QBs From the Rest
HIGHLIGHTS:
QBs with Career Passer Ratings above 90 scored 40 percentile points higher on S2 than QBs with Career Passer Ratings below 90
S2 Overall Score accounted for 28.7% of Career Passer Rating (r = .536)
Wonderlic scores for these same QBs accounted for less than 0.10% of Career Passer Rating (r = .027)
The top 13 QBs in the NFL scored significantly higher on the S2 measure of Tracking Capacity, Instinctive Learning, Decision Complexity, and Distraction Control
As of the 2022 draft, S2 has tested 117 QBs who were considered draft-eligible NFL prospects or already playing in the NFL. The NFL group consists of veteran NFL QBs who play on teams that work directly with S2, NFL QBs who reached out to us independently to be evaluated, and prospects that were ultimately drafted into the NFL. More specifically, of the 117 QBs, 27 have started in more than 16 games or currently start in the NFL, 42 have made an NFL roster for more than one year but have never started, and 48 never signed or rostered in the NFL. These are large enough sample sizes to begin looking at key cognitive separators, such as what cognitive skills separate those that make the NFL versus those that don't or what separates the best NFL QBs from the rest.
Diving into the data
In an earlier article, we covered cognition in football, with some basics at the QB position, and how cognition differs from IQ or intelligence (read the article HERE). Measures of cognition include split-second processing of information and decision making, not classic measures of IQ, intellect, or concussion evaluation. The S2 Eval for the QB position examines 9 different cognitive skills: Perception Speed, Search Efficiency, Tracking Capacity, Visual Learning, Instinctive Learning, Decision Complexity, Distraction Control, Impulse Control, and Improvisation. Each of these skills receives a score that is normed to a database of 2,500 NFL draft-eligible prospects across all positions. The Overall Score is an average of all 9 z-scores, then transformed into a percentile ranking ranging from 1 to 99.
As an interesting side note, the average S2 Overall Score for all 117 QBs we've tested was 68 (compared to the average of 50 for all positions). Having established that QBs are wired to make fast decisions, we wanted to examine group differences in all the QBs we've tested. The first step was to split the 27 QB starters we tested into two groups (top-tier starters and lower-tier starters) based on an objective and commonly used measure of on-field performance, Career Passer Rating (CPR). CPR takes into account completion percentage, touchdown percentage, interception percentage, and yards per attempt. Using these 4 variables, a CPR score is produced, ranging from 0 to 158.3. From talking with front offices and QB coaches, most people in the sport consider a CPR of 90 and above to be elite. Based on this 90 CPR "cut point", 13 NFL QB starters in our database performed higher than the cut point and were designated as "top-tier" QBs, and 14 NFL QB starters had CPR values below the cut point and were designated "lower-tier" QBs. When we compared groups, here's what we found:
GROUP | S2 OVERALL SCORE | RANGE |
|---|---|---|
Top Tier Starters | 91 | 74 - 99 |
Lower Tier Starters | 51 | 8 - 90 |
Backup/Roster Players | 69 | 15 - 99 |
Never Signed/Rostered | 62 | 1 - 99 |
Looking at the data above, it's quite remarkable. It's a good perspective to state that all NFL prospects have superior cognitive skills on these tasks compared to non-athlete undergraduates. So, to have a group (top-tier starting QBs) at any position to score better than 90% of all athletes taking the evaluation is mind-blowing. Really digging into the data, we wanted to look at the predictive value of the S2 Overall Score. To do so, we performed a Pearson's correlation to examine the relationship between the S2 Overall score and CPR. The analysis revealed an r-value of 0.536 (p = .002), which is a moderately positive correlation. Using R2 generated by a regression model, the Overall S2 Score accounts for 28.7% of the Career Passer Rating. That's well over a quarter of CPR can be explained or predicted by the S2 Overall Score. To put this in perspective, Ayush Batra at bestballstats.com examined the relationships between physical characteristics and college performance stats with NFL Passer Rating (read the article HERE). College completion percentage was the best predictor of CPR in the NFL (r = 0.368, which explains 13.5% of performance) and college interception percentage was second on the list (r = 0.358, which explains 12.8% of performance). These findings aren't terribly surprising, given there will likely be a fair amount of shared variance between college performance and pro performance, but not a one-to-one translation. Interestingly, there was very little relationship with QB height. QBs 6'2" and shorter actually have a CPR 3 points higher than QBs 6'3" and taller. Our own independent statistical analysis examining the relationship between height and CPR confirmed Batra’s original findings. A Pearson's correlation revealed an inverse relationship between height and CPR (r-value of -.185, p = .178), suggesting shorter QBs have slightly higher CPRs.
We also thought it would be interesting to examine the NFL's version of intelligence, the Wonderlic, to see if there was any relation to performance. We've certainly heard NFL executives complain about it being meaningless to performance, but when we looked in the literature, there hasn't been much data published to support this claim. We ran a Pearson's correlation analysis to examine the relationship between Wonderlic score and CPR. The results revealed an r-value of .027 (p = .449). Using R2 provided by a regression model, Wonderlic score accounts for less than .01% of CPR. These results highlight the difference between IQ or intellect and cognition. IQ, as measured by the Wonderlic, may tell you how that athlete would fare in a school setting, while cognition, as measured by the S2 Overall Score, will tell you how the athlete will perform on the field when making split-second, in-game decisions.
Up to this point, we have been focused on S2’s overall score. As we pointed out earlier, the S2 evaluation consists of 9 different cognitive skills. When we looked at which specific cognitive skills separated the top tier and lower tier NFL QBs, there were 4 skills that top tier starters performed more than 25 percentile points better than the lower tier starters: Tracking Capacity (45 points higher), Instinctive Learning (31 points higher), Decision Complexity (29 points higher), and Distraction Control (26 points higher).
Breaking down the concepts
Let's break down these cognitive separators into football concepts. Tracking Capacity measures a QB's ability to broaden his attention and see the entire field. Players with high scores have little difficulty anticipating where defenders and receivers will move on the field. Instinctive Learning measures a QB's ability to diagnose unscripted looks and subtle and less obvious tendencies of the defense. Players with high scores here are quick to discover new tendencies of the defense on the fly and adapt their performance accordingly. Parenthetically, Drew Brees is one of the highest scores we've ever seen on Instinctive Learning across tens of thousands of athletes in 9 different sports. Decision Complexity measures a QB's speed at executing complex decision rules. Players with high scores can quickly filter through "if-then" rules and execute with split-second performance (e.g., if the corner drops into zone and the backer blitzes, then I dump to slot, but if the corner covers and the backer drops back in coverage, I hit the RB on a bubble screen). Finally, Distraction Control measures a QB's ability to shield his attention and motor performance in the face of distracting stimuli (e.g., hands in the face, hearing footsteps, lineman rolling up on the back of the leg). QBs with high scores here have that steely focus and can make the throw despite the pocket collapsing around them and 100dbs of crowd noise.
Takeaways
So, what are the take-home points? The best QBs in the league are superhuman in their ability to process information in split-second, dynamic situations. They have better awareness of where everyone is moving on the field, notice things faster than other players, make exceptionally rapid read-react decisions, and have an uncanny ability to block out the noise and distractions. In fact, these specialized decision skills appear to account for a lion's share of on-field performance. While success as an NFL quarterback is unquestionably multi-factorial (e.g., arm talent, psychological makeup, tactical game knowledge, having talent around him), knowing a QBs cognitive skill set is a critical piece of the puzzle. Although it's not the purpose of this article to understand why lower-tier starters have the lowest overall scores of these 4 groups, one hypothesis comes to mind. The NFL seems to be overly attracted to physical characteristics! Undoubtedly physical characteristics account for a large part of performance, especially height, arm talent, and throw accuracy, but it would seem like decision-making either hasn't been a top priority or, more likely, has been a guessing game in the absence of measures to quantify an athlete's game-like cognitive skills. And it's clear from the analysis above, the Wonderlic isn't adding anything of value to the scouting puzzle.
The S2 data clearly shows that when you measure a QB's split-second decision skills using metrics derived from the cognitive sciences, we can account for a significant chunk of what it takes to be a successful QB in the NFL.
About Brandon Ally
Brandon has a PhD in neuropsychology and cognitive neuroscience and has published more than 50 peer-reviewed articles and book chapters on brain mechanisms underlying visual attention, perception, and memory. He is the co-founder and Vice President of S2 Cognition, a sports-science company that delivers a leading cognitive evaluation and technology platform to teams and athletes across all major sports at every level of play.
Comentarios