When I teach writing classes or give a book presentation, people are increasingly asking me what I think about artificial intelligence (AI) in connection with writing. The reality is that I’m neither adamantly against its usage nor wildly enthusiastic about the possibilities. Some of my writing clients want me to completely avoid AI while others give me free reign—while still others prefer AI usage to be limited to generating title ideas and article outlines. My personal preferences resonate with the middle ground. AI does a fantastic job of recommending titles and a reasonable one of creating outlines. The caveat for the second: you can’t assume that information provided is accurate.
People can get quite emotional about AI discussions, so I decided that I’d use my latest in-depth research project to get an objective sense of how accurate the technologies really are. I used ChapGPT.com for my deep dive.
ChatGPT Testing Context
First, some context. I’d been working on my book, Wells Waite Miller and Me: An 8th OVI Civil War Biography since June 2018. So, I have a good sense of what online material was available on the subject from that moment on. That’s important because, at one point, the technology behind ChatGPT was only trained to use information available in September 2021 or earlier to provide its answers. Since then, the respected news organization BBC has noted how ChatGPT can now use “up to date” information, making that statement in September 2023. So, for my purposes, I’m stipulating that ChatGPT’s technology can at least access information up to September 2023. Any information added to the internet after that date, I’ll consider that ChatGPT didn’t have it at its disposal.
Specific ChatGPT Testing
On June 19, 2025, I asked the chatbot this question: “Who was Wells Waite Miller?” The answer that I got was astoundingly . . . incorrect. Later in this post, I’ll share the many ways in which the information was just plain dead wrong.
So, What Went Wrong?
Let’s play devil’s advocate. The first argument defending ChatGPT could be that Google simply doesn’t have enough good information to feed the AI technology. However, a quick look at today’s availability on Google disproves that.
Next up, you might argue, this information was not available in September 2023. But, it was. Looking at the Antietam on the Web entry, that information was available to me when I began researching in 2018. And, you don’t have to take my word for it. Looking at the page source, you can see this:
Everything on that site was created between 2004-2021, well within our time limit for ChatGPT being able to access the information. Anticipating the argument that the page source for Antietam on the Web’s page notes how the website was updated on June 8, 2025:
I went back to the Wayback Machine where you can see information archived on specific dates in the past. So, this shows how information about Wells Waite Miller was readily available on June 24, 2021:
(Besides the images, I’ve also included links so that you, too, can verify the information I’m providing.)
I also relied upon the Find-a-Grave page for Wells Waite Miller early on.
This was created in 2008:
Here’s a Wayback Machine version from May 31, 2021.
Plus, I was also already blogging about him in February 2022:
So, ChatGPT had access to information on Google about Wells Waite Miller.
How Far Off Was ChatGPT Testing?
Here’s the image of their answer again.
Here are a few key examples of what the technology got wrong.
ChatGPT: The answer lists birth and death dates of 1839-1928.
Reality: Correct dates are 1842-1906.
ChatGPT: The answer calls him a politician and lawyer from New York.
Reality: He became a politician. He was never a lawyer. He was born in New York but lived in Ohio from age ten on.
ChatGPT: “He is best known for serving as the Speaker of the New York State Assembly in 1892. Miller had a long legal and political career, practicing in law and participating in state politics as a member of the Republican Party.”
Reality: This would be startling news to Robert P. Bush who beat James W. Husted to serve in that position that year. Wells never served in a New York political position. He was in the Republican Party (that part is correct) and served in elected county positions and appointed state ones in Ohio.
Giving ChatGPT Another Chance
So, I thought, maybe I’ll give ChatGPT a hint, adding that Wells Waite Miller was a Civil War soldier. Initially, the technology could not find any information on a Civil War soldier with that name.
Sigh.
So, I tried again on June 20. Here’s the response:
Here We Go Again
ChatGPT: Dates of his life are listed as August 2, 1841-February 26, 1925.
Reality: Correct dates are February 19, 1842-April 8, 1906.
ChatGPT: The technology has him being born and dying in Clyde, Ohio.
Reality: He was born in Enfield, New York and died in Castalia, Ohio.
ChatGPT: The technology lists his burial place as McPherson Cemetery in Clyde.
Reality: The correct answer is the Castalia Cemetery in Castalia.
As far as military service, ChatGPT is correct that he fought in the Union Army. Then things go a bit wonky.
ChatGPT: They list him as a private in Company D, 8th OVI, enlisting in June 1861.
Reality: He enlisted as the unit’s fourth sergeant, receiving promotions through captain. He never served as a private. He fought in the 8th OVI, first in Company E and then in Company H. He never fought in Company D. He enlisted in April 1861 (for three months) not June 1861. He did, however, re-enlist in June 1861 for a three-year period.
ChatGPT then provides more info to my original question:
Although service highlights are high level and general, they are correct. As far as his postwar life:
ChatGPT: They list him as a prominent citizen in Clyde.
Reality: He became a prominent citizen in Castalia.
ChatGPT: They note his career as a writer and speaker.
Reality: He worked as a teacher, a farmer, and a politician. He also become involved in business and banking.
ChatGPT: The technology claims that he wrote his memoirs and gave talks about the war.
Reality: He never wrote his memoirs although he did write Antietam. Report of the Ohio Antietam Battlefield Commission for the government. If he gave talks about the war, I am unaware of them after reading countless contemporary newspaper articles about his life.
ChatGPT: The technology says that he was active in veterans’ organizations like GAR.
Reality: He was highly active in farming organizations like the Grange and in 8th OVI Civil War reunions.
ChatGPT: The chatbot says he’s not a well-known national figure, but his writing and speeches capture the Ohio Civil War experience.
Reality: It’s true that he isn’t widely known as a national figure. It’s true that his report on Antietam shared where men from Ohio fought in that particular battle. Otherwise, no.
ChatGPT: The technology cites Sandusky County materials.
Reality: He lived in Erie County near the city of Sandusky.
Let’s Try Another Followup for Our ChatGPT Testing
I then informed ChatGPT that, yes, I’d like to see excerpts from his writing! After all, the technology made that offer. Here’s the response:
Unfortunately, while promising their presence, ChatGPT couldn’t find any.
At this point, proceed with caution with AI when researching and writing material.