Your old friend, numbers.

Ah, the topic of book sales — among authors, it can produce a hushed, nervous silence. Selling a lot of books implies success, far and above other metrics (like critical praise or cultish fandom). The reason book sales remain the primary measure of a title’s success comes down to, well, data: using tools like Nielsen BookScan, publishers can get a concrete, tangible sense of how far the books they’re publishing have ventured.

But what happens when the data is inaccurate? It’s widely known that neither direct sales (books purchased directly from the publisher) nor sales from many independent bookstores figure into Nielsen numbers. Recently, Huw Thomas writes for the BBC, a dispute bubbled up when it was revealed that some winners of the Wales Book of the Year award had sold “fewer than 100 copies.”

For example, the Nielsen data revealed that poet Robert Minhinnick’s prize-winning collection Diary of the Last Man has sold 202 copies. But the figure appears to be incorrect — according to its publisher, the book has sold almost 4,000 copies.

Minhinnick’s book was far from the only one affected — according to Garmon Gruffudd, managing director of Y Lolfa, all of their shortlisted titles have “sold more than 600 copies, and we have already reprinted three of them.” He added that Nielsen had missed many of the sales from smaller bookshops.

Something’s clearly amiss. Author Lloyd Markham’s novel Bad Ideas/Chemicals, shortlisted for the prize, has sold twenty copies. You read that right: twenty. The publisher, Parthian, disputed the sales figure, stating that they had sold “significantly more,” though they didn’t provide a definitive figure. But the situation seems similar, huh?

It’s an issue of stores’ reporting their sales to Nielsen—which some do and some simply don’t—as well as how much importance is placed on these numbers (it varies depending on who you ask, but the answer, generally, is “a lot”). Data is useful and becomes important in an industry where so much of what is bought and sold might not make a lot of sense. Publishers are trying to solve problems and spot trends at the same time. Somewhere the idea of a book as cultural capital gets lost. BookScan offers, at best, an idea of sales performance. The real factors that determine whether a book sells well are word of mouth, visibility in the world, and reading by friends and colleagues. Still, in the end, an author will be judged by their sales figures, the very data that can be so inaccurate. Maybe we need to look at the bigger picture in asking what makes a book a success.