ChatGPT Picks an All-Time Yankees Dream Team
How would the popular AI chatbot ChatGPT answer a request for an All-Time Yankees Dream Team? Would it do pretty well or make some obvious mistakes?
Issue #68
Even if you’ve not used it yet, you’ve likely at least heard of ChatGPT, the chatbot developed by OpenAI and which grew quickly in popularity over the past few months. According to Wikipedia, “It is built on top of OpenAI's GPT-3 family of large language models and has been fine-tuned using both supervised and reinforcement learning techniques.”
I was curious: how would ChatGPT do when asked complicated sports questions that involve a lot of factors, including how you weigh those factors in arriving at an answer? Given his recent breaking of the all-time NBA scoring record, I asked ChatGPT for the pros and cons of whether Lebron James should now be considered the greatest NBA player of all time. Of course, that question is vague—do we mean greatest based on career accomplishments or greatest at their peak? Or a combination of both? And as I noted above, such sports questions involve a lot of factors, where one factor might clearly favor one player over another, and then the inverse for some other factor(s). Partly because of this, I didn’t ask ChatGPT to tell me who the greatest NBA player was, but rather the pros and cons of the argument for Lebron James. I won’t go into the details of what it said, but suffice to say I think it did a credible job giving me three or four pros, and three or four cons. Nice work ChatGPT!
I then went to the kind of question I’m most interested in: baseball all-time dream teams. I’m the author of Now Taking the Field: Baseball’s All-Time Dream Teams for All 30 Franchises (2019, ACTA Sports), where as the subtitle says I created robust, 30-man rosters for all current 30 MLB franchises (including their entire history, spanning across location changes and name changes). I used several criteria in creating my rosters and starting lineups, including total WAR, peak WAR, traditional statistics, postseason statistics, and honors and awards. It is a fun subject, and one that opens up all kinds of questions and criticisms, from the actual choices I made, to how I weighed the various factors involved, and more.
So how would ChatGPT do if given the same task? I started with just one team for now, the New York Yankees, and asked it: “Who should be on an all-time New York Yankees franchise dream team?” By that, ChatGPT took me to be asking just for a starting nine, which was fine. Here is what it said initially in response:
ChatGPT: “The New York Yankees have a rich history and have had many talented players throughout the years, so it's difficult to narrow it down to just one dream team. However, here's a lineup who could be considered for an all-time New York Yankees dream team:
Babe Ruth - Right Field
Lou Gehrig - First Base
Joe DiMaggio - Center Field
Mickey Mantle - Left Field
Derek Jeter - Shortstop
Yogi Berra - Catcher
Alex Rodriguez - Third Base
Robinson Cano - Second Base
Whitey Ford - Pitcher
This lineup includes some of the greatest players in Yankees history, including Babe Ruth, who helped establish the team as a powerhouse in the 1920s, Lou Gehrig, who is widely considered one of the greatest first basemen in the history of the game, Joe DiMaggio, who had a legendary 56-game hitting streak, and Mickey Mantle, who won seven World Series titles with the Yankees.
Derek Jeter is one of the greatest shortstops of all time, and Yogi Berra is considered one of the best catchers in baseball history. Alex Rodriguez and Robinson Cano were both standout players during their time with the Yankees, and Whitey Ford is one of the most successful pitchers in the team's history. Of course, there are many other players who could also be included on this dream team, but this is just one possible lineup.”
The first thing to note is that I highly doubt that ChatGPT attempted to place these selections into an actual starting lineup that would make sense according to most people’s lineup strategies. While it has become increasingly popular to bat power hitters leadoff, such as Aaron Judge and Kyle Schwarber in recent years, I doubt ChatGPT is “thinking” that way with the above list. With the exception of listing the pitcher last, I suspect it went in order by the most obvious players that needed to be included in the starting nine, and then made sure that it only listed one player per position.
Overall, I think this is a good starting nine—you could argue for Graig Nettles over A-Rod at 3B, or Tony Lazzeri over Cano at 2B, but I think this list is pretty solid. In fact, it is identical to the starting nine that I chose in my book, with one nuance. Mickey Mantle of course played very little LF: only 130 games, mostly in 1965, late in his career, before his final shift to 1B. Now, the issue that both I and the ChatGPT had to confront was how do you choose between Joe DiMaggio and Mickey Mantle in CF? Both players are such all-time greats, not just for the Yankees, but for baseball’s history overall.
In my book, I created two starting lineups for each all-time dream team, one against LHPs and one against RHPs. Given that I also used a DH spot in my lineups, and the pure-leftfielder candidates for the Yankees are relatively weak compared to greats like DiMaggio, Mantle, and of course Babe Ruth, I had two options. I could have slotted Mantle as the DH (to preserve his knees of course!) in both lineups, or I could shift him over to LF some of the time (since he did play there a little) and play someone else in LF and at DH some of the time. I did the latter, listing Mantle as the DH against RHPs, and as the LF against LHPs. Against RHPs, I went with left-handed slugger Charlie Keller for LF, as he played for the Yankees for 6 full seasons and parts of five others for the Yankees, hitting 184 HR with an impressive 153 OPS+. Against LHPs, I had a few options, but inserted right-handed hitter Thurman Munson in the lineup, as I had included him as a third catcher in my full 30-man roster.
Getting back to ChatGPT, here is what I asked it after it presented me the above starting-nine and rationale for each player: “Good work. Except Mickey Mantle played less than ten percent of his games in LF. So that seems like cheating!” Here is what ChatGPT said in response:
ChatGPT: “You are absolutely right! I apologize for the mistake in my previous response. Mickey Mantle is most well-known for his time in center field, where he won seven Gold Gloves and patrolled the outfield with his incredible range and arm strength. In this case, I should have included him as the center fielder on the dream team. Here's an updated lineup that should correct that mistake:
Babe Ruth - Right Field
Lou Gehrig - First Base
Joe DiMaggio - Center Field
Mickey Mantle - Center Field
Derek Jeter - Shortstop
Yogi Berra - Catcher
Alex Rodriguez - Third Base
Robinson Cano - Second Base
Whitey Ford - Pitcher
Thank you for bringing this to my attention, and please let me know if you have any other questions or concerns!”
First off, I feel like ChatGPT got a little, ahem, defensive in its description of how great Mantle was as a centerfielder. So much so that it made a massive factual error, as Mantle only won one Gold Glove, in 1962, not seven!
And further, obviously this doesn’t work as a starting nine, since you can’t have two CF and no LF. So I told ChatGPT: “Well, now there are two Centerfielders and no Left Fielder. So you need to choose between DiMaggio and Mantle in Center Field, and choose someone else for Left Field.” Without a DH spot in its lineups, ChatGPT had to make the hard CF choice between DiMaggio and Mantle. It went with DiMaggio:
ChatGPT: “You are absolutely right, and I apologize for the error in my previous response. Thank you for pointing it out. Here's an updated version of the lineup that should fix the problem:
Babe Ruth - Right Field
Lou Gehrig - First Base
Joe DiMaggio - Center Field
Dave Winfield - Left Field
Derek Jeter - Shortstop
Yogi Berra - Catcher
Alex Rodriguez - Third Base
Robinson Cano - Second Base
Whitey Ford - Pitcher
In this updated lineup, I’ve replaced Mickey Mantle with Dave Winfield in left field, and I've moved Joe DiMaggio to center field, where he spent the majority of his career. Thank you for bringing this to my attention, and please let me know if you have any other questions or concerns.”
I would have preferred Charlie Keller, but Winfield isn’t a bad choice. He played for the Yankees for eight full seasons, before being traded early in 1990 to the Angels for pitcher Mike Witt. He was an All-Star all eight of those full seasons, won five Gold Glove Awards, and provided 100+ RBIs six times. He had 205 HR for the Yankees, with a 134 OPS+. So that is a solid resume here, though he admittedly played far more games in RF (827) than in LF (351).
Yankees Dream Team Top 5 Starting Pitchers
I was next curious who else would ChatGPT select as a Yankees dream team five-man rotation, beyond White Ford as the first selection. When I asked, here is the detailed response I got back:
ChatGPT: “The New York Yankees have had many legendary starting pitchers throughout their history, so narrowing it down to just five is quite challenging. However, here are five of the greatest starting pitchers in New York Yankees history, in no particular order:
Whitey Ford: Known as "The Chairman of the Board," Whitey Ford is widely considered one of the greatest Yankees pitchers of all time. He spent his entire 16-year career with the team and won 236 games, which is still the most in franchise history. Ford was a 10-time All-Star and won six World Series championships.
Mariano Rivera: While Rivera is best known for his incredible career as a closer, he began his Yankees career as a starting pitcher. He made 10 starts in 1995 before transitioning to the bullpen, where he became the greatest closer in baseball history. However, even in his short stint as a starter, Rivera showed flashes of his future greatness, including a complete game shutout in his third career start.
Ron Guidry: Nicknamed "Louisiana Lightning," Guidry spent his entire 14-year career with the Yankees and won 170 games. He was a two-time 20-game winner, a four-time All-Star, and won the 1978 AL Cy Young Award.
Andy Pettitte: Pettitte is the winningest pitcher in postseason history, with 19 wins, and played a major role in the Yankees' late-1990s dynasty. He spent 15 seasons with the Yankees over two stints and won 219 games. Pettitte was a three-time All-Star and won five World Series championships.
Lefty Gomez: Gomez played for the Yankees from 1930 to 1942 and won 189 games in his career. He was a seven-time All-Star, a two-time ERA leader, and won five World Series championships. He was also the winning pitcher in the first ever MLB All-Star Game in 1933.
Honorable mentions include Roger Clemens, who won two Cy Young Awards and two World Series titles with the Yankees, and CC Sabathia, who won 251 games in his career and played a key role in the team's 2009 championship.”
OK, so here ChatGPT made several mistakes. The most glaring from a player selection standpoint is the inclusion of Mariano Rivera on this list of all-time great New York Yankees starting pitchers. He is of course their all-time greatest relief pitcher, and most would say the best reliever in baseball history. ChatGPT included him in this top-five list with the caveat that he didn’t pitch much as a starter in the majors—indeed only 10 games in his rookie season (after he had been a pretty solid starter for a few years in the minors). That is not really aligned with the intention of the question to include Rivera on the basis of just 10 starts.
Worse than that, there appears to be an error in the last sentence describing Rivera’s brief time in the Majors as a starter: “However, even in his short stint as a starter, Rivera showed flashes of his future greatness, including a complete game shutout in his third career start.” I was surprised to read that, so I checked and Rivera does not have any complete games. In his 10 starts as a rookie, he mostly didn’t pitch well, and certainly did not throw a CG in his third start. His best start was his fifth, when he through 8 IP with 11 strikeouts, allowing 2 hits and 0 runs before giving way to John Wetteland for the ninth inning.
Beyond that, I’ll give ChatGPT credit for the other three listed as they are solid selections. However, I again checked some of the remarks and there are some factual errors. I found this surprising, since it is one thing to not quite understand the question as humans usually mean it, e.g., how to weight various factors that should be considered in choosing an all-time dream team, but quite another to just get historical facts wrong. For example, Ron Guidry didn’t just win 20+ games twice, he did so three times (1978, 1983, 1985). I guess it is true then that he was a two-time 20-game winner, but usually that implies only two-times!
The info on Andy Pettitte looks correct, but then I would argue that ChatGPT got something wrong about Lefty Gomez. This could be argued I suppose, but Lefty Gomez was a key part of six Yankees World Series Champions, not just five. I suppose ChatGPT was not counting 1941, when the Yankees won the fall classic but Gomez didn’t participate. Does he not get credit as being a six-time champion? He went 15-5 in 1941, but wasn’t used in the World Series, as the Yankees had seven pitchers who had start ten or more games that year. Baseball-Reference.com still lists him as “6x World Series” at the top of his profile page.
I liked the mention of Roger Clemens and CC Sabathia as deserving honorable mentions. Though again there is an error in the claim that Clemens won two Cy Young Awards with the Yankees: he won seven overall, but only in New York (plus three with Boston, two with Toronto, and one with Houston). With Sabathia the info given is correct, though it is worth mentioning that only 134 of his 251 career wins came while playing with the Yankees.
The Yankees have had lots of great starting pitchers of course, but the one I think was most left out from ChatGPT’s response was Red Ruffing. I would include him, even over Clemens and Sabathia, in place of Rivera in the list given above. Ruffing really struggled to start his career with the Red Sox, but then was a key member of the Yankees from 1930-1942. He was away for two years for WWII military service, and returned at age 40 for two more partial, though effective, seasons for the Yankees. Overall he was a six-time All-Star for the New York, posting a 231-124 (.651) record with a 3.47 ERA and 119 ERA+.
Round Two: Asking ChatGPT again one week later
It took me several days to write-up this article with the above results. After doing so, I thought I’d ask ChatGPT my initial question again, to see if it gave me the same, or a somewhat different response: “Who should be on an all-time New York Yankees franchise dream team?” Interestingly, this time it gave me a more complete roster:
ChatGPT: “The New York Yankees have a rich history of baseball excellence, with many great players having played for the team over the years. Here is a possible dream team roster of the all-time New York Yankees franchise:
Starting Lineup:
Derek Jeter - SS
Babe Ruth - RF
Lou Gehrig - 1B
Joe DiMaggio - CF
Mickey Mantle - LF
Yogi Berra - C
Alex Rodriguez - 3B
Robinson Cano - 2B
Bench:
Thurman Munson - C
Don Mattingly - 1B
Bernie Williams - OF
Tony Lazzeri - 2B
Starting Rotation:
Whitey Ford - LHP
Ron Guidry - LHP
Lefty Gomez - LHP
Andy Pettitte - LHP
Red Ruffing - RHP
Bullpen:
Mariano Rivera - RHP (Closer)
Goose Gossage - RHP
Dave Righetti - LHP
Sparky Lyle - LHP
Dellin Betances - RHP
David Robertson - RHP
Manager: Joe Torre
Note: This is just one possible lineup and there are many other great players who could also be included in the team.”
Very interesting! This time it seems to have arranged the starting eight into an actual lineup, with Jeter leading off and Ruth batting second, perhaps following the modern approach of having your best all-around hitter bat second (e.g., Mike Trout on the Angels). It once again listed Mantle in LF, not wanting to omit such an all-time great due to the logjam with DiMaggio in CF.
The four chosen for the bench are respectable. One could argue about the choice of Thurman Munson over Bill Dickey as the backup Catcher, as Dickey played longer and had a higher OPS+ of 127 vs. Munson’s 116 (and that number likely would have decreased further had he not been tragically killed in a plane crash, as by 1979 his power had almost completely faded.) But Munson, along with OF Williams, 1B Mattingly, and 2B Lazzeri are definitely solid choices—especially Lazzeri since he mostly played 2B, but also played a little at 3B and SS and so could cover for Jeter and A-Rod if necessary.
I definitely like the updated five-man rotation here, as ChatGPT I think correctly included Red Ruffing in place of listing Mariano Rivera as a starter. The bullpen selections are great too, as Rivera, Gossage, Righetti, and Lyle are the same four closers that I chose in my book. I also like ChatGPT’s picks of Betances and Robertson, which I see as the AI engine saying “there have been good relievers besides just famous closers!” Robertson was a great setup man for Rivera, until his final year (during his first stint in New York) when he had the daunting task of taking over as closer upon Rivera’s retirement. And Betances had several great years as a Yankee reliever, including two with a sub-2.00 ERA.
As for the selection of Joe Torre as manager, I could accuse ChatGPT of recency bias I suppose, but you can make an argument for four great managers in Yankees history: Joe Torre, Miller Huggins, Joe McCarthy, and Casey Stengel. No matter who you like as skipper, that would be a lot of baseball knowledge to have as assistant coaches.
Did you know? I wrote a book with the same title as this Substack newsletter / blog: Now Taking the Field: Baseball’s All-Time Dream Teams for All 30 Franchises. It was published in early 2019, by ACTA Sports, the publisher of the annual Bill James Handbook and other popular titles. You can learn more about it at www.NowTakingTheField.com, or buy directly at Amazon and other booksellers.
