The most famous name in American ornithology is that of a slave owner, grave robber, and fraud who invented birds and falsified scientific results. Birders and ornithologists are grappling with John James Audubon’s legacy today, but problematic behavior doesn’t stop at a single 19th-century naturalist. Some in the field are considering a complete rethinking of which naturalists we idolize with honorific bird names.

Birdwatchers and scientists alike have been aware of the complexity and unsavory details of Audubon’s work and legacy since before he died, but they have long swept the negative details under the rug, given his transformative impact on the field of ornithology and the fact that his name has become a symbol for birdwatching across the United States. Today, the birding community is once again weighing the man’s contributions against his faults amid a greater conversation surrounding the history of racism, imperialism, and colonialism in science, questioning which past scientific figures we idolize and why. But despite all of Audubon’s fraud and faults, it’s impossible to think critically about him without also tackling the broader issue of how American science developed in the early 19th century.

Today, Audubon lends his name to societies, towns, and even a meteor for his near-mythical contributions to our understanding of America’s birds. From 1827 to 1838, Audubon published The Birds of North America, a series of illustrations documenting the appearance and behaviors of birds that he and others shot and collected in expeditions across the expanding United States. Through the tome, Audubon offered his international sponsors descriptions of birds that are extinct today, like the ivory-billed woodpecker and Carolina parakeet, and laid much of the groundwork for the modern understanding of North America’s avifauna.

Scientists have long known that Audubon’s published work contained quirks—facts that don’t line up with what we know about certain species today, or species that don’t appear anywhere except in Audubon’s book, such as a small songbird with distinct black and red facial markings called a “Cuvier’s kinglet” and a black-capped yellow songbird called the “carbonated warbler;” perhaps they were aberrant individuals, or perhaps they were contrived. Some ornithologists and biographers have chalked these inaccuracies up to the honest mistakes of a pioneering scientist. But researchers today wonder whether these quirks were straight-up fraud.

Perhaps the most notable of these cases was Audubon’s “Bird of Washington,” a species of eagle that he described in 1827 and the first new species that Audubon claimed to have discovered. But research published this past summer by historian Matthew Halley, featuring all-new primary sources, reveals “beyond reasonable doubt” that the bird was the work of invention and that Audubon had likely crafted the bird both by plagiarizing past images from a text called The Cyclopædia and by making up the particulars surrounding his own supposed encounter with the bird. Halley posited that, in dire financial straits, Audubon used the bird to win the favor of English nobility in order to secure funding for his Birds of North America project. Audubon’s untruthfulness doesn’t stop at the Bird of Washington, either; he invented species in order to prank the naturalist Constantine Rafinesque and likely fabricated some or all of the results of his famous bird-banding experiment.

These aren’t new revelations; even some of Audubon’s contemporaries didn’t consider him one to be trusted, though this has been downplayed in more recent Audubon biographies. After learning of Audubon’s death in 1850, ornithologist George Ord wrote, “Audubon is fairly entitled to the merit of perseverance and industry. His elephant folio is a proof of this. [However] As a naturalist and a traveller, but little reliance can be placed on his narratives, in consequence of an inveterate habit of mendacity, which should seem to have been the premium mobile [prime mover] of his intellect,” as reported in Halley’s paper.

During this time, American science was just getting started, and American scientists were attempting to establish their scientific authority in the new country, Rutgers University professor Ann Fabian told Gizmodo. Audubon aimed to promote a brand of newly developing American science and rushed to catalogue as much North American natural wonder as he could while simultaneously trying to stand out as a writer, artist, and nature promoter.

Atop his scientific dishonesty was his inexcusable racism. Historian Greg Nobles argues in a piece in Audubon Magazine that Audubon wasn’t a “man of his time,” given the number of his contemporaries who stood in opposition to slavery. Audubon himself was a slave owner as well as slave buyer and seller, depending on his finances. He spun tales about returning escaped slaves to their original masters in order to reunite them with their family, and in 1843, disrupted a Native American gravesite to collect skulls.

“These conversations might seem like niche issues. But it’s one brick in a very big wall that’s overdue to come down.”

If you remove Audubon’s mythical aura, his fraud and racism are simply part of the story of an eccentric 19th-century self-promoter. Audubon was born to a plantation owner in 1785 on what is now Haiti. His mother, cited as a servant but whose race and status is still debated, died shortly after his birth, and he was sent to France to be raised by his father’s wife, where he received the education and upbringing of a wealthy family. But at 18, his family sent him to Pennsylvania in order to avoid him being drafted to the French army. There, he began observing birds while starting a family and running a general store. He was rejected by the Academy of Natural Sciences of Philadelphia in the mid-1820s, and the store failed and put him in debt; he began to take on a showman-like persona of a Davy Crockett-type frontiersman, claiming he was born in New Orleans. He traveled between Europe and the United States, showing off his paintings and telling tales of America’s wildlife to win favor and funds from the wealthy, including President Andrew Jackson and King George IV.

Scholars wonder whether Audubon’s racism and even his work were a product of his upbringing. Without knowing his mother, rumors abound today that Audubon was part Black; it’s possible that Audubon himself was anxious about his own identity, working extra hard to pass as white for the white society he lived in. Meanwhile, Audubon was an artist, showman, and writer attempting to present himself as a naturalist despite the rejection from the scientific field.

“I always feel a little uneasy at looking back at people creating things with so little support and pre-established protocols, without necessarily thinking about the social, racial, and colonial motivations that made them act the way that they did,” explained Antoine Traisnel, assistant professor of comparative literature at the University of Michigan currently exploring 19th-century science’s relationship with animals. “This isn’t a way to excuse him, as opposed to understanding what made [his writings] the way they were.”

Audubon’s name grew to represent bird conservation as a whole, despite his well-known fraud and racism. Halley writes in a blog for the British Ornithologists’ Union that some have taken great effort to protect Audubon’s legacy by sweeping the plagiarism and lies under the rug. “Even the primary record is not trustworthy,” he writes. “Audubon’s granddaughter destroyed his journals after publishing bowdlerized excerpts that showed ‘what [she believed] he was and not what others thought he was.’” She told a biographer in 1895 that she burned them herself after copying all she wanted the public to see.

Socially conscious birders are rethinking the legacy of Audubon—but you can’t just take one historical figure into account. The very foundation of science in the 18th and 19th century was rooted in arriving at a new place, killing, and collecting—in other words, stealing from the indigenous people—for the sake of knowledge. Nineteenth-century scientists were only just beginning to become familiar with the transient nature of species and the idea that they could go extinct; they applied that idea to intentionally doing things that would accelerate the extinction of groups of people like Native Americans, said Traisnel. Audubon’s contemporaries, like Georges Cuvier (after whom he named the bogus Cuvier’s kinglet), baked racism into science, helping to lay the groundwork for the modern eugenics movement.

You could say that Audubon is a uniquely reprehensible character and not someone we should be idolizing by today’s standards. But given the colonialism and racism inherent in 19th-century science, you could say the same for every other naturalist at the time for whom birds and other animals are named.

Plenty of birders and ornithologists realize that, too. This past August, the American Ornithological Society agreed to rename the McCown’s Longspur the Thick-Billed Longspur, as the bird’s former namesake was a Confederate general, after pressure from the birding community. More than 2,500 people have signed a petition to remove honorifics from all common bird names, not just the ones named for despicable people, as a way to detach birds from 19th-century colonizers

“These conversations might seem like niche issues,” said Jordan Rutter, one of the organizers of the Bird Names for Birds petition. “But it’s one brick in a very big wall that’s overdue to come down.”

You could equate this to toppling metaphorical statues—but it’s not so simple. It’s more about finding and squashing a racist, colonialist mindset still prevalent in ornithology and other scientific communities that alienate Black, indigenous, and other people in their fields, something that starts with rethinking who we idolize and what they stand for. After all, as late as 2011, a proposal to officially rename the Maui Parrotbill with its Hawaiian name kiwikiu was rejected by the American Ornithological Society’s North American Classification and Nomenclature Committee with comments such as, “It seems contrived, unfamiliar, unpronounceable, and lacks a long history of usage. No one has come up with Navajo, Chippewa, etc. names for mainland species, so why expunge English from all Hawaiian bird names?”

The people we chose to idolize matter. “If I was 8 or 9 years old and had a basic interest in wildlife and learned that there was a bird named after a Confederate general, that would give me pause,” said Jason Ward, a Black, Atlanta-based birder and host of the web series Birds of North America. “We can change organization names; we can change bird names. There are billion-dollar organizations out there talking about sports who change their logos and their names. We can change a couple of bird names.”

For its part, the National Audubon Society recently published the article by Nobles exploring Audubon’s racist legacy and pointed me toward a statement by its president David Yarnold when I asked for a comment, explaining how the organization has been working to apply antiracist principles. They did not comment specifically regarding any plans for their own organization’s name.

“I totally support the mission of the National Audubon Society and all of the regional Audubon societies to promote bird conservation through education and action,” Halley said. “We have a great thing going here. But J.J. Audubon’s record of scientific misconduct is extensive and was sustained throughout his career.”

This discussion mirrors similar ones across the United States, as some groups replace past problematic symbols like Confederate statues. You might argue that bird names represent a piece of history—but birds are living things, typically studied and appreciated by indigenous peoples long before European naturalists showed up in a foreign land and decided to categorize the things they saw using their own names and systems (not to mention the fact that plenty of birds have tenuous relationships at best with the person they’re named after).

Still, bird names represent just one small issue that has drawn attention recently, and they hide a much bigger conversation. The very foundations of Western science were in many cases built on past ideologies tarnished by colonialism, racism, and, in some cases, fraud. As these conversations continue, so too will calls for deeper, structural changes to these fields.