taking the 'self' out of 'self-publishing'
May 18, 2025 7:05 AM Subscribe
From BookBub, a survey of authors on AI: "Overall, opinions among authors are deeply divided — many consider any use of generative AI unethical and irresponsible, while others find it a helpful tool to enhance their writing and business processes. Some authors remain conflicted, and are still negotiating their own feelings about the utility and morality of this technology."
Importantly (and understandably, given BookBub's role), ~94% of respondents are self-published. 45% are using AI, and nearly 70% of those who use AI do so to write (more use it for research). 74% of the users do not disclose their AI use to readers.
(seen on this Lincoln Michel tweet: "Survey of (almost entirely self-pub) authors suggests AI books are already flooding the self-pub world. Expected but... you do wonder when that ecosystem will just implode under the ocean of AI text" (xcancel link))
Importantly (and understandably, given BookBub's role), ~94% of respondents are self-published. 45% are using AI, and nearly 70% of those who use AI do so to write (more use it for research). 74% of the users do not disclose their AI use to readers.
(seen on this Lincoln Michel tweet: "Survey of (almost entirely self-pub) authors suggests AI books are already flooding the self-pub world. Expected but... you do wonder when that ecosystem will just implode under the ocean of AI text" (xcancel link))
Yeah, for me the problem isn't that people are cheating at writing. I'm not even sure that's possible. What they're doing is cheating at writing badly, and I somehow doubt that it's redounding to sales. Quality is no guarantee of success, but mediocrity will generally leave you in obscurity.
posted by jy4m at 7:44 AM on May 18 [2 favorites]
posted by jy4m at 7:44 AM on May 18 [2 favorites]
I am sympathetic to the idea of using AI to refine word choices, etc.; frankly, writers have been using AI for decades in this way, although it used to have a charming and personable face that looked like a paperclip (and I suspect resurrecting Clippy would go a long way toward endearing AI to the hearts of users everywhere). I am extremely skeptical in terms of using it to compose and certainly to dictate plot. Less for ethical reasons than because AI is inherently derivative and my guess is that "its" best ideas would inevitably be stolen from actual writers.
posted by kittens for breakfast at 7:44 AM on May 18
posted by kittens for breakfast at 7:44 AM on May 18
Having been in and around the personal computer industry since the late 70’s, one of the biggest promises, especially from Apple, was that the computer would assist creative people to be more creative. But what ended up happening was that the idea became that the computer would make anyone a creative person. Now everyone can be an artist, a musician, a graphics designer, a film maker, a photographer, and so on. Here, they are now adding being a writer. The problem is that they are confusing the production of a finished image, film, piece of music, story, essay, etc. with the creation of said article. This whole process ignores having all the training, expertise, ideas, etc. required to actually make something good. Just knowing how to make a PowerPoint presentation does not mean that you can create a good, effective presentation. I spent over 16 years trying to convince the faculty at a school that they needed to be trained how to effectively communicate to their students. Very few thought they needed to learn how to teach. The computer is just a tool, there has to be someone behind it. But just because you own a screwdriver doesn’t mean you can design and build a car.
posted by njohnson23 at 8:21 AM on May 18 [7 favorites]
posted by njohnson23 at 8:21 AM on May 18 [7 favorites]
About a year and half ago, so maybe LLMs have gotten significantly better since then, my boss was making me use ChatGPT to generate puff pieces for the in-house newsletter to promote our work. It was kind of fine, as I was (after working in the department 14 years) kind of a subject matter expert and could spot errors, and then we ran the pieces past the doctors and scientists being discussed to make sure there were no errors that I, a dummy, might have missed.
Out of curiosity/threat assessment, I tested it on a couple simple prompts to see if it could write even a little bit decently. It could not, for exactly the reason mittens outlines. It had no theory of mind for its own characters. I asked it to write a spy story and the character immediately walked up to their target and introduced themselves as a spy. Hahahaha. I decided not to worry about LLMs threatening my career as a writer. People have told me they've gotten better, but I'm not interested in doing much investigation.
I would absolutely love to use AI for one purpose though, and that's to feed it a manuscript I've already written and have it spit out a perfectly polished book summary to use in pitches. I hate writing those things. However as I understand it, if I did that, even if it did a good job, I'd have just added my ms to its training data and I don't want my works used to feed these things.
posted by joannemerriam at 8:25 AM on May 18 [1 favorite]
Out of curiosity/threat assessment, I tested it on a couple simple prompts to see if it could write even a little bit decently. It could not, for exactly the reason mittens outlines. It had no theory of mind for its own characters. I asked it to write a spy story and the character immediately walked up to their target and introduced themselves as a spy. Hahahaha. I decided not to worry about LLMs threatening my career as a writer. People have told me they've gotten better, but I'm not interested in doing much investigation.
I would absolutely love to use AI for one purpose though, and that's to feed it a manuscript I've already written and have it spit out a perfectly polished book summary to use in pitches. I hate writing those things. However as I understand it, if I did that, even if it did a good job, I'd have just added my ms to its training data and I don't want my works used to feed these things.
posted by joannemerriam at 8:25 AM on May 18 [1 favorite]
ChatGPT4.0 is really quite good at genre and commercial plotting and characterization, with "genre" being broadly construed. (I just got a very useful outline from the prompt: "Tell me the story (in a four paragraph plot outline) of two Cornell professors and their beautiful wives living in the 1950s anomie of Ithaca, each of whom is hiding terrible secrets, sometimes from the others, sometimes from themselves, too.")
When actually asked to write the full story, it is weak at the blocking and tackling of text. Incidents are poorly described, rhythm is choppy at best, dialog generally stinks. Directions for stage plays and screenplays is usually poor.
posted by MattD at 8:59 AM on May 18
When actually asked to write the full story, it is weak at the blocking and tackling of text. Incidents are poorly described, rhythm is choppy at best, dialog generally stinks. Directions for stage plays and screenplays is usually poor.
posted by MattD at 8:59 AM on May 18
If you use AI to write any part of your creative work, you are not a writer.
posted by goatdog at 9:02 AM on May 18 [11 favorites]
posted by goatdog at 9:02 AM on May 18 [11 favorites]
Seems relevant that the question about AI use in the survey is "Do you use generative AI to assist with your writing, marketing, or other aspects of your work as an author?" And then the top four use cases are:
-Research
-Copy or Art for Marketing Purposes
-Outlining or Plotting
-Editor or Proofreading
And when it comes to the outlining/plotting, one person writes "I use it for generating plot and plot twist ideas...But none of this is just blind “generate me some text so I can use it for my novel” — it’s all reviewed/accepted/rejected to stay within my voice and vision. In the past, I might find myself stuck for days trying to resolve problems — now, I have a writing partner who always has advice and suggestions."
So it seems less that a lot of people are, at least at this stage, using it to write and so much feeding prompts into ChatGPT like "[Summary of plot so far] Give me four options for what [x character] might do next" and then using that to help them brainstorm further. It's easy to imagine how that would give rise to an increase in certain formulas - but it's maybe not so different than the phenomenon of the MFA novel. And perhaps just as authors embedded in MFA programs can, indeed, still be inventive, I can imagine some authors using AI sparingly and it's fine. On the other hand, it seems only a matter of time before someone starts publishing pulp fiction that's almost entirely AI-generated.
posted by coffeecat at 9:20 AM on May 18 [3 favorites]
-Research
-Copy or Art for Marketing Purposes
-Outlining or Plotting
-Editor or Proofreading
And when it comes to the outlining/plotting, one person writes "I use it for generating plot and plot twist ideas...But none of this is just blind “generate me some text so I can use it for my novel” — it’s all reviewed/accepted/rejected to stay within my voice and vision. In the past, I might find myself stuck for days trying to resolve problems — now, I have a writing partner who always has advice and suggestions."
So it seems less that a lot of people are, at least at this stage, using it to write and so much feeding prompts into ChatGPT like "[Summary of plot so far] Give me four options for what [x character] might do next" and then using that to help them brainstorm further. It's easy to imagine how that would give rise to an increase in certain formulas - but it's maybe not so different than the phenomenon of the MFA novel. And perhaps just as authors embedded in MFA programs can, indeed, still be inventive, I can imagine some authors using AI sparingly and it's fine. On the other hand, it seems only a matter of time before someone starts publishing pulp fiction that's almost entirely AI-generated.
posted by coffeecat at 9:20 AM on May 18 [3 favorites]
really looking forward to the 'you fucked up, you trusted us' phase of this, in the same way i look forward to waking up every day now though
posted by lescour at 9:40 AM on May 18 [4 favorites]
posted by lescour at 9:40 AM on May 18 [4 favorites]
others find it a helpful tool to enhance their writing and business processes
If I were being ssnarky, I would say that there are no authors who find it a helpful tool to enhance their writing. That's just a tautology.
But the survey hedges its bets considerably, and adds "marketing or other aspects of your work as an author," to the question, which basically makes the question "have you ever used generative AI to help you with anything, ever." And I am surprised the number is so low, given the large number of self published authors, and how it can be fun to use generative AI to do things like generate lists of weird names or whatever. Ethical concerns aside, the actual use case of genAI is to do the kind of bullshit work that you hate, suck at, or that is too expensive, and where you don't care if the job is half-assed.
From that perspective, it is interesting to see what, exactly, survey respondents find difficult and/or don't care about. Translation is the number one thing. Audiobook narration is not surprising. I'm surprised that cover art is so low. (It has been several years now since I first had the disheartening experience of someone sending me an AI version of what they want.) I'm surprised that writing copy and using art for marketing is so high on the list, since that strikes me as something an author might think of as the kind of bullshit work that is a good candidate for generative AI as labor saving device, and it doesn't directly affect their readers.
posted by surlyben at 10:22 AM on May 18 [1 favorite]
If I were being ssnarky, I would say that there are no authors who find it a helpful tool to enhance their writing. That's just a tautology.
But the survey hedges its bets considerably, and adds "marketing or other aspects of your work as an author," to the question, which basically makes the question "have you ever used generative AI to help you with anything, ever." And I am surprised the number is so low, given the large number of self published authors, and how it can be fun to use generative AI to do things like generate lists of weird names or whatever. Ethical concerns aside, the actual use case of genAI is to do the kind of bullshit work that you hate, suck at, or that is too expensive, and where you don't care if the job is half-assed.
From that perspective, it is interesting to see what, exactly, survey respondents find difficult and/or don't care about. Translation is the number one thing. Audiobook narration is not surprising. I'm surprised that cover art is so low. (It has been several years now since I first had the disheartening experience of someone sending me an AI version of what they want.) I'm surprised that writing copy and using art for marketing is so high on the list, since that strikes me as something an author might think of as the kind of bullshit work that is a good candidate for generative AI as labor saving device, and it doesn't directly affect their readers.
posted by surlyben at 10:22 AM on May 18 [1 favorite]
When I saw that 45% I was horrified, but the breakdown isn't quite so bad. Though, I think it's still pretty bad.
The top category for AI use in writing was.... research.
An area where we KNOW that AI tends to be somewhere between mediocre and actively awful depending on how niche the topic is. If you use AI for research you'll find yourself getting important things wrong.
Where things seem to jump from maybe, possibly being justifiable to outright wrong is that apparently 25% of so called "authors" use AI to do the writing.
Dude if the AI is doing the writing then a) it's shit, and b) YOU didn't write it. You hired a ghostwriter and you're committing plagiarism by claiming the work of the AI as your own. It's the writer, you're just the doofus who pushed some buttons. A properly credited work by such people should have the author listed as "ChatGPT" and the human's name omitted or listed only as "consultant" or something.
And apparently while "only" 25% of people who claim to be self published writers use the AI to do the actual writing "frequently", it looks like only around 30ish% of people they polled always did the writing instead of farming it out to a computer ghostwriter. What the actual fuck?
Let me repeat: a person passing off text generated by an LLM is committing plagiarism against the LLM, because it was the real writer.
And it's already flooding the whole space of self published writing with a ceaseless torrent of AI generated garbage that's overwhelming actual people who write.
Someone who desperately believes that they're the "real" writer despite having GPT come up with the plot, the characters, the setting, the outline, and also doing the writing, but they "clean it up" or "add their own touch" or whatever is sad. But not a villain.
The real problem is people who spin up an instance and have it churn out dozens, maybe hundreds, of books that they then put up for sale. It pushes out actual books written by real people, and it diverts reader money to fraudsters who callously destroy the self published marketplace for short term profit.
I strongly suspect none of those people answered this survey.
posted by sotonohito at 10:42 AM on May 18 [4 favorites]
The top category for AI use in writing was.... research.
An area where we KNOW that AI tends to be somewhere between mediocre and actively awful depending on how niche the topic is. If you use AI for research you'll find yourself getting important things wrong.
Where things seem to jump from maybe, possibly being justifiable to outright wrong is that apparently 25% of so called "authors" use AI to do the writing.
Dude if the AI is doing the writing then a) it's shit, and b) YOU didn't write it. You hired a ghostwriter and you're committing plagiarism by claiming the work of the AI as your own. It's the writer, you're just the doofus who pushed some buttons. A properly credited work by such people should have the author listed as "ChatGPT" and the human's name omitted or listed only as "consultant" or something.
And apparently while "only" 25% of people who claim to be self published writers use the AI to do the actual writing "frequently", it looks like only around 30ish% of people they polled always did the writing instead of farming it out to a computer ghostwriter. What the actual fuck?
Let me repeat: a person passing off text generated by an LLM is committing plagiarism against the LLM, because it was the real writer.
And it's already flooding the whole space of self published writing with a ceaseless torrent of AI generated garbage that's overwhelming actual people who write.
Someone who desperately believes that they're the "real" writer despite having GPT come up with the plot, the characters, the setting, the outline, and also doing the writing, but they "clean it up" or "add their own touch" or whatever is sad. But not a villain.
The real problem is people who spin up an instance and have it churn out dozens, maybe hundreds, of books that they then put up for sale. It pushes out actual books written by real people, and it diverts reader money to fraudsters who callously destroy the self published marketplace for short term profit.
I strongly suspect none of those people answered this survey.
posted by sotonohito at 10:42 AM on May 18 [4 favorites]
Earlier this year I spent time considering starting an all AI literary journal and spoke to many writer friends about this very issue. Most literary journals will not take work that has an AI element, but since I write speculative fiction, I occasionally use AI as a sounding board for some creative conversations as part of my practice. My focus for this all-AI literary magazine was going to be about writers discussing 1) which AI they used 2) how they worked with prompts, 3) what they believed AI brought to the creative work and 4) any ethical considerations whether of intellectual property or environmental degradation. So more "about using AI in creative writing" than "the actual AI-generated writing."
After speaking with writing friends who deeply feel the existential threat and reading the Alex Reisner article in the Atlantic (gift link here) I could not move forward. I not strongly believe that the AI power grab will be viewed in the future much like a land acknowledgement - a preface that notes how much was stolen that we will never be able to get back.
posted by Word_Salad at 10:55 AM on May 18 [1 favorite]
After speaking with writing friends who deeply feel the existential threat and reading the Alex Reisner article in the Atlantic (gift link here) I could not move forward. I not strongly believe that the AI power grab will be viewed in the future much like a land acknowledgement - a preface that notes how much was stolen that we will never be able to get back.
posted by Word_Salad at 10:55 AM on May 18 [1 favorite]
That's what I've seen as well, coffeecat, writers using AI not to generate writing per se but as brainstorm partners, alpha readers, research assistants, sounding boards.
Here's a UNESCO paper outlining different roles AI can play for students along these lines.
posted by doctornemo at 11:09 AM on May 18
Here's a UNESCO paper outlining different roles AI can play for students along these lines.
posted by doctornemo at 11:09 AM on May 18
If you don't want to actually write anything - no one is holding a gun to your head, you are free to just not do that.
posted by Gyre,Gimble,Wabe, Esq. at 12:09 PM on May 18 [5 favorites]
posted by Gyre,Gimble,Wabe, Esq. at 12:09 PM on May 18 [5 favorites]
> The top category for AI use in writing was.... research.
This is a distressing and misguided use of AI, inasmuch as many of the materials of good research--meaningful, thorough, laborious, time-intensive research--cannot be accessed by LLMs, and will be inaccessible to them for decades, and centuries in some cases.
Because these materials are created from non-digital substances--leather, vellum, glue, thread, ink, paper.
Sure, hundreds of thousands of physical books have been digitalized and placed in Internet repositories that are scrapable by LLMs. And this is a good thing overall. But, even today, their numbers are a tiny fraction of the physical books stored in the Library of Congress.
Add to these the countless books in libraries across the world--modern libraries in cities, obscure libraries in rural districts--and you have an astronomical sum of physical books storing the breadth of human knowledge.
LLMs, in their present form, won't access these books. They lack the eyes to read them, and the means of propulsion to reach them. And will lack these things for decades to come.
Meanwhile, dedicated researchers, with enough gumption, can cross continents, conquer mountains, and go the distance to reach whatever books they wish to partake of.
Libraries are a final fortress of knowledge that lie outside the domain of LLMs, and we can take comfort in this.
posted by Gordion Knott at 12:14 PM on May 18 [3 favorites]
This is a distressing and misguided use of AI, inasmuch as many of the materials of good research--meaningful, thorough, laborious, time-intensive research--cannot be accessed by LLMs, and will be inaccessible to them for decades, and centuries in some cases.
Because these materials are created from non-digital substances--leather, vellum, glue, thread, ink, paper.
Sure, hundreds of thousands of physical books have been digitalized and placed in Internet repositories that are scrapable by LLMs. And this is a good thing overall. But, even today, their numbers are a tiny fraction of the physical books stored in the Library of Congress.
Add to these the countless books in libraries across the world--modern libraries in cities, obscure libraries in rural districts--and you have an astronomical sum of physical books storing the breadth of human knowledge.
LLMs, in their present form, won't access these books. They lack the eyes to read them, and the means of propulsion to reach them. And will lack these things for decades to come.
Meanwhile, dedicated researchers, with enough gumption, can cross continents, conquer mountains, and go the distance to reach whatever books they wish to partake of.
Libraries are a final fortress of knowledge that lie outside the domain of LLMs, and we can take comfort in this.
posted by Gordion Knott at 12:14 PM on May 18 [3 favorites]
I've been actively experimenting with LLMs for fiction writing. Some observations:
* if your experience with ChatGPT is older than 6 months, your impression of its capabiities is badly out of date.
* having said that when people who are not writers think about writers using AI, they're likely picturing someone putting in a prompt, and having a coherent novel pop out. No LLM can do that yet.
*They can do a creditable job of putting out flash fiction, if prompted in the right way.
* when authors are saying they are using it for research, these are fiction authors and in particular genre fiction authors. They aren't largely talking about original archival research, they're talking about period details ("give me five types of clothing a middle class person would wear in the 1920s") or real-world minutiae ("what kind of handgun ammo would be quietest?"). Despite the popular notion of AI hallucinations, LLMs are very good for this.
* an interesting question for those who refuse to use AI. Is it because it threatens one's self-image as a creative?
* LLMs are very good at copyediting. Perfect spelling and grammar. It can de-clunkify like nobody's business. For genre writers and the large majority of fiction writers this is a huge boon. For the rarefied authors with the special skill/talent of truly original style, LLMs are not as much help.
posted by storybored at 1:00 PM on May 18 [1 favorite]
* if your experience with ChatGPT is older than 6 months, your impression of its capabiities is badly out of date.
* having said that when people who are not writers think about writers using AI, they're likely picturing someone putting in a prompt, and having a coherent novel pop out. No LLM can do that yet.
*They can do a creditable job of putting out flash fiction, if prompted in the right way.
* when authors are saying they are using it for research, these are fiction authors and in particular genre fiction authors. They aren't largely talking about original archival research, they're talking about period details ("give me five types of clothing a middle class person would wear in the 1920s") or real-world minutiae ("what kind of handgun ammo would be quietest?"). Despite the popular notion of AI hallucinations, LLMs are very good for this.
* an interesting question for those who refuse to use AI. Is it because it threatens one's self-image as a creative?
* LLMs are very good at copyediting. Perfect spelling and grammar. It can de-clunkify like nobody's business. For genre writers and the large majority of fiction writers this is a huge boon. For the rarefied authors with the special skill/talent of truly original style, LLMs are not as much help.
posted by storybored at 1:00 PM on May 18 [1 favorite]
I spent over 16 years trying to convince the faculty at a school that they needed to be trained how to effectively communicate to their students.
Oh my gods, they're so bad at it, too. I've been a fulltime uni professor for 28 years now, and the consistent, routine horror stories my students tell me about how their other profs use slideshows ought to stop gobsmacking me but still does. I cannot believe that in the current day any faculty member would stand there and read the bullet points on the slides out loud, but evidently nearly all of them still do.
And nothing has changed my mind that generative "AI" and all it proponents should be launched into the sun. I don't care if it's any good at producing anything: it's ripoff garbage designed to line the pockets of the oligarchy at our expense. You can produce fiction and NOT use the Bad Facts and Lake-Draining Racist Plagiarism Machine.
posted by outgrown_hobnail at 1:33 PM on May 18 [2 favorites]
Oh my gods, they're so bad at it, too. I've been a fulltime uni professor for 28 years now, and the consistent, routine horror stories my students tell me about how their other profs use slideshows ought to stop gobsmacking me but still does. I cannot believe that in the current day any faculty member would stand there and read the bullet points on the slides out loud, but evidently nearly all of them still do.
And nothing has changed my mind that generative "AI" and all it proponents should be launched into the sun. I don't care if it's any good at producing anything: it's ripoff garbage designed to line the pockets of the oligarchy at our expense. You can produce fiction and NOT use the Bad Facts and Lake-Draining Racist Plagiarism Machine.
posted by outgrown_hobnail at 1:33 PM on May 18 [2 favorites]
An area where we KNOW that AI tends to be somewhere between mediocre and actively awful depending on how niche the topic is.
DeepReseach is really good, and a much bigger improvement than promoting ChatGPT with ‘tell me about X’
posted by MisantropicPainforest at 1:59 PM on May 18
DeepReseach is really good, and a much bigger improvement than promoting ChatGPT with ‘tell me about X’
posted by MisantropicPainforest at 1:59 PM on May 18
I've written a novel and a few books worth of flash, and I'd say chatgpt can be useful, but ime it takes a lot of coaxing to produce something good. Feeding it all your stuff and then asking it for thoughts on a scene is genuinely no bullshit helpful, it will identify elements that work, suggest avenues for exploration and do rewrites you can use to mine for ideas.
The coaxing is mainly around giving it prompts that exclude glazing, teaching it how you want to be responded to and being very specific about what you actually want to know.
in terms of pure quality, I've had it produce paragraphs worth of genuinely excellent prose, gets a little shakey beyond there.
posted by Sebmojo at 2:36 PM on May 18
The coaxing is mainly around giving it prompts that exclude glazing, teaching it how you want to be responded to and being very specific about what you actually want to know.
in terms of pure quality, I've had it produce paragraphs worth of genuinely excellent prose, gets a little shakey beyond there.
posted by Sebmojo at 2:36 PM on May 18
* an interesting question for those who refuse to use AI. Is it because it threatens one's self-image as a creative?
I'm a musician, writer, actor, director at various times and it really is just another tool like a drum machine or a word processor. I'm sympathetic to people who hate it, but think a lot of the arguments made against it don't make a lot of sense - you really can't straight-faced claim you're suddenly an IP warrior in 2025, while posting on the internet, you just can't. Also calling it 'auto correct' is like calling a car an 'explosion machine', technically correct but practically nonsensical.
But it's creepy! we've made something that passes the turing test and it's actually somehow simultaneously boring, annoying and enraging! it's absolutely going to distort public discourse! I actually support the people who hate it and rage against it, because i think the results need to be shamed until they're good enough that they're inarguable, which isn't now.
posted by Sebmojo at 2:43 PM on May 18
I'm a musician, writer, actor, director at various times and it really is just another tool like a drum machine or a word processor. I'm sympathetic to people who hate it, but think a lot of the arguments made against it don't make a lot of sense - you really can't straight-faced claim you're suddenly an IP warrior in 2025, while posting on the internet, you just can't. Also calling it 'auto correct' is like calling a car an 'explosion machine', technically correct but practically nonsensical.
But it's creepy! we've made something that passes the turing test and it's actually somehow simultaneously boring, annoying and enraging! it's absolutely going to distort public discourse! I actually support the people who hate it and rage against it, because i think the results need to be shamed until they're good enough that they're inarguable, which isn't now.
posted by Sebmojo at 2:43 PM on May 18
DeepReseach is really good, and a much bigger improvement than promoting ChatGPT with ‘tell me about X’
And it's still an exploitive pile of shit, trained on stolen material and pouring money into the oligarchy's pockets. Just do research like a human being.
and it really is just another tool like a drum machine or a word processor.
It's really not. It's just thievery and lack of creativity. You're not making music: some pattern-recognition machine is using stolen property to make something music-adjacent.
posted by outgrown_hobnail at 2:51 PM on May 18 [1 favorite]
And it's still an exploitive pile of shit, trained on stolen material and pouring money into the oligarchy's pockets. Just do research like a human being.
and it really is just another tool like a drum machine or a word processor.
It's really not. It's just thievery and lack of creativity. You're not making music: some pattern-recognition machine is using stolen property to make something music-adjacent.
posted by outgrown_hobnail at 2:51 PM on May 18 [1 favorite]
Gordion Knott Well, yes and no. On the one hand, yes there is data that's yet to be digitized. On the other hand, we're digitizing it as fast as possible because it's so much more useful and available that way. I'm not sure if you'd consider that a net positive or not, I do but I'm focused on the part where the info becomes available planetwide to people who aren't rich enough to travel to the places those analog archives are.
doctornemo Wow is that ever a totally godawful bit of fluff. I checked and neither of the authors are connected to OpenAI as far as I can find in a quick search, but they write like they're OpenAI's marketing department.
Worse, that was written back in 2021 when ChatGPT was significantly more prone to hallucination than it is today. And there's nothing about that. At all.
They very briefly, in two separate bullet points, do mention in passing that the AI has no way of knowing if it's information is accurate. And they bury that in a paragraph about the AI being possibly prone to various bias and other things.
Any serious document on use of AI in education would need an entire section devoted to how AI can produce misinformation, why you have to be paranoid and double check with real sources anything it says, and emphasizing that it is absolutely not a reliable source and you can and will be mislead and say completely incorrect things if you take GPT output as valid without scrutiny.
You'd have thought the first batch of lawyers who used GPT, and found it simply made up cases that backed their position, were found out in court, and then sanctioned would have warned other lawyers off. But nope, an increasing number of lawyers keep using GPT produced stuff in court, getting caught, and then pulling a surprised pikachu face when they realize their "research" was just made up.
storybored I'm fascinated by AI so I tend to play with several models, but mainly ChatGPT, therefore I'm quite up to date.
"They can do a creditable job of putting out flash fiction, if prompted in the right way."
Well, if you don't mind flashfiction that's obviously AI produced. The thing is, anyone who's experimented much with GPT learns its style. It's remarkably distinctive, though I would be unable to tell you exactly what it is I'm seeing that makes it so distinctive. But since you've been working with it, you know as well as I do that far from having no voice, it does have a voice, and a strong one, and you can spot it a mile away once you learn how to see it.
when authors are saying they are using it for research, these are fiction authors and in particular genre fiction authors. They aren't largely talking about original archival research, they're talking about period details ("give me five types of clothing a middle class person would wear in the 1920s") or real-world minutiae ("what kind of handgun ammo would be quietest?"). Despite the popular notion of AI hallucinations, LLMs are very good for this.
Well, except for when they're not.
The really bad part is that you're mostly right. For general knowledge type questions they are right more often than they're wrong. But that's the part that really bites a person who starts taking what it says without checking. Because they are wrong sometimes a person who comes to trust what it says will be in for an unpleasant shock.
Your first example, for instance, asking it about middle class clothing in the 1920's. I asked GPT and it included three piece suits for men. Which is interesting because in the 1920's men were increasingly forgoing the vest, and it doesn't mention this anywhere. In the early 1920's a typical man might indeed be wearing a vest, but by the mid to late 1920's that same man would probably only wear a vest on formal occasions and for everyday he'd skip it.
Admittedly if you asked specified that you wanted to know how (or if) the fashion it cites changed it may have given a better answer, but as written the very first thing on the list it gave me was, in fact, incomplete at absolute best and it wouldn't be unfair to call it wrong.
Your second example provided better info, I suppose there's a lot of chatter online from gun fandom so it's got a pretty good sized training set for such question.
But I notice that literally the first thing it gave me when testing out your sample questions of what you thought GPT would be good at was, at absolute best, incomplete. If you, and that's the generic you meaning everyone not you specifically, take GPT answers and use them without checking you are definitely going to make a humiliating mistake.
LMs are very good at copyediting. Perfect spelling and grammar.
This one I'll agree with. Maybe not PERFECT, but definitely better than mine.
It can de-clunkify like nobody's business. For genre writers and the large majority of fiction writers this is a huge boon. For the rarefied authors with the special skill/talent of truly original style, LLMs are not as much help.
???
My friend, if you think you don't have a style that's distinct, you're almost certainly wrong. Even GPT itself has a distinct style, and it's not even intelligent.
I'm not sure about the declunkify part, I can't say I've ever asked it to do that. Until just now anyway. I fed it a segment of something I've been working on and asked it to ID anything clunky and show me how it thought it should be cleaned up.
I think the result was mixed. It did find what I think actually are a couple of draggy segments that could be tightened, so that's a plus. On the other hand it ID's a few segments that I can't agree about and I don't THINK I'm just being a writer who thinks their every word is precious. And it missed a segment I thought was a bit draggy and had been intending to clean up.
I think this could be an interesting experiment if anyone feels like spending a bit of time deciding for themselves if they think GPT is good at declunkifying things.
So, assuming I'm not breaking MeFi rules (and a quick look seems to indicate I'm not) if anyone is interested in that experiment and feels like reading around 2,400 words to see if they agree with GPT or not:
Here's a link to a Google Docs copy of that segment
Here's a link to a Google doc of what GPT said should be tightened up.
I doubt the mods would appreciate this thread turning into a "test GPT's ability to identify clunky writing" thread, so if you do feel like investing your time field testing an AI, it's probably best to leave your conclusions in the docs themselves, I've left comments open on both.
posted by sotonohito at 3:13 PM on May 18 [2 favorites]
doctornemo Wow is that ever a totally godawful bit of fluff. I checked and neither of the authors are connected to OpenAI as far as I can find in a quick search, but they write like they're OpenAI's marketing department.
Worse, that was written back in 2021 when ChatGPT was significantly more prone to hallucination than it is today. And there's nothing about that. At all.
They very briefly, in two separate bullet points, do mention in passing that the AI has no way of knowing if it's information is accurate. And they bury that in a paragraph about the AI being possibly prone to various bias and other things.
Any serious document on use of AI in education would need an entire section devoted to how AI can produce misinformation, why you have to be paranoid and double check with real sources anything it says, and emphasizing that it is absolutely not a reliable source and you can and will be mislead and say completely incorrect things if you take GPT output as valid without scrutiny.
You'd have thought the first batch of lawyers who used GPT, and found it simply made up cases that backed their position, were found out in court, and then sanctioned would have warned other lawyers off. But nope, an increasing number of lawyers keep using GPT produced stuff in court, getting caught, and then pulling a surprised pikachu face when they realize their "research" was just made up.
storybored I'm fascinated by AI so I tend to play with several models, but mainly ChatGPT, therefore I'm quite up to date.
"They can do a creditable job of putting out flash fiction, if prompted in the right way."
Well, if you don't mind flashfiction that's obviously AI produced. The thing is, anyone who's experimented much with GPT learns its style. It's remarkably distinctive, though I would be unable to tell you exactly what it is I'm seeing that makes it so distinctive. But since you've been working with it, you know as well as I do that far from having no voice, it does have a voice, and a strong one, and you can spot it a mile away once you learn how to see it.
when authors are saying they are using it for research, these are fiction authors and in particular genre fiction authors. They aren't largely talking about original archival research, they're talking about period details ("give me five types of clothing a middle class person would wear in the 1920s") or real-world minutiae ("what kind of handgun ammo would be quietest?"). Despite the popular notion of AI hallucinations, LLMs are very good for this.
Well, except for when they're not.
The really bad part is that you're mostly right. For general knowledge type questions they are right more often than they're wrong. But that's the part that really bites a person who starts taking what it says without checking. Because they are wrong sometimes a person who comes to trust what it says will be in for an unpleasant shock.
Your first example, for instance, asking it about middle class clothing in the 1920's. I asked GPT and it included three piece suits for men. Which is interesting because in the 1920's men were increasingly forgoing the vest, and it doesn't mention this anywhere. In the early 1920's a typical man might indeed be wearing a vest, but by the mid to late 1920's that same man would probably only wear a vest on formal occasions and for everyday he'd skip it.
Admittedly if you asked specified that you wanted to know how (or if) the fashion it cites changed it may have given a better answer, but as written the very first thing on the list it gave me was, in fact, incomplete at absolute best and it wouldn't be unfair to call it wrong.
Your second example provided better info, I suppose there's a lot of chatter online from gun fandom so it's got a pretty good sized training set for such question.
But I notice that literally the first thing it gave me when testing out your sample questions of what you thought GPT would be good at was, at absolute best, incomplete. If you, and that's the generic you meaning everyone not you specifically, take GPT answers and use them without checking you are definitely going to make a humiliating mistake.
LMs are very good at copyediting. Perfect spelling and grammar.
This one I'll agree with. Maybe not PERFECT, but definitely better than mine.
It can de-clunkify like nobody's business. For genre writers and the large majority of fiction writers this is a huge boon. For the rarefied authors with the special skill/talent of truly original style, LLMs are not as much help.
???
My friend, if you think you don't have a style that's distinct, you're almost certainly wrong. Even GPT itself has a distinct style, and it's not even intelligent.
I'm not sure about the declunkify part, I can't say I've ever asked it to do that. Until just now anyway. I fed it a segment of something I've been working on and asked it to ID anything clunky and show me how it thought it should be cleaned up.
I think the result was mixed. It did find what I think actually are a couple of draggy segments that could be tightened, so that's a plus. On the other hand it ID's a few segments that I can't agree about and I don't THINK I'm just being a writer who thinks their every word is precious. And it missed a segment I thought was a bit draggy and had been intending to clean up.
I think this could be an interesting experiment if anyone feels like spending a bit of time deciding for themselves if they think GPT is good at declunkifying things.
So, assuming I'm not breaking MeFi rules (and a quick look seems to indicate I'm not) if anyone is interested in that experiment and feels like reading around 2,400 words to see if they agree with GPT or not:
Here's a link to a Google Docs copy of that segment
Here's a link to a Google doc of what GPT said should be tightened up.
I doubt the mods would appreciate this thread turning into a "test GPT's ability to identify clunky writing" thread, so if you do feel like investing your time field testing an AI, it's probably best to leave your conclusions in the docs themselves, I've left comments open on both.
posted by sotonohito at 3:13 PM on May 18 [2 favorites]
And it's still an exploitive pile of shit, trained on stolen material and pouring money into the oligarchy's pockets. Just do research like a human being.
Ok. It does two days of work in 10 minutes.
posted by MisantropicPainforest at 3:22 PM on May 18
Ok. It does two days of work in 10 minutes.
posted by MisantropicPainforest at 3:22 PM on May 18
But since you've been working with it, you know as well as I do that far from having no voice, it does have a voice, and a strong one, and you can spot it a mile away once you learn how to see it.
I can't say I'm confident in spotting it!
Here's one by Chat G. Pt:
"The shop light clicked off, leaving dusk to pool in the corners. Pete locked up slow, the metal door clattering like a question. His evening stretched ahead: leftover chili, a silent TV, maybe that paperback with the dog-eared sadness he kept rereading. The cat might come by. Or not.
He walked to his truck, fingers sore, back stiff, the ache of ordinary life. Somewhere in town, someone was laughing into a second glass of wine. Somewhere, music was playing.
He sat behind the wheel and let the silence hold him for a minute.
Then he drove home, like always."
----
Now another:
"Pete locked the garage with the same key he’d had since ‘83, teeth worn down like everything else. His knees cracked like cheap jokes. Across the street, kids kicked a ball against the rabbi’s fence again, and someone shouted in Yiddish—probably a blessing, though it didn’t sound like one.
Tonight: beans from a can, two beers if he didn’t feel guilty, and maybe Jeopardy if the cat didn’t sit on the remote.
His ex-wife used to say he smelled like old pennies and bad decisions. She wasn’t wrong.
Still, the Monte Carlo ran smooth today. There were worse lives."
posted by storybored at 4:45 PM on May 18
I can't say I'm confident in spotting it!
Here's one by Chat G. Pt:
"The shop light clicked off, leaving dusk to pool in the corners. Pete locked up slow, the metal door clattering like a question. His evening stretched ahead: leftover chili, a silent TV, maybe that paperback with the dog-eared sadness he kept rereading. The cat might come by. Or not.
He walked to his truck, fingers sore, back stiff, the ache of ordinary life. Somewhere in town, someone was laughing into a second glass of wine. Somewhere, music was playing.
He sat behind the wheel and let the silence hold him for a minute.
Then he drove home, like always."
----
Now another:
"Pete locked the garage with the same key he’d had since ‘83, teeth worn down like everything else. His knees cracked like cheap jokes. Across the street, kids kicked a ball against the rabbi’s fence again, and someone shouted in Yiddish—probably a blessing, though it didn’t sound like one.
Tonight: beans from a can, two beers if he didn’t feel guilty, and maybe Jeopardy if the cat didn’t sit on the remote.
His ex-wife used to say he smelled like old pennies and bad decisions. She wasn’t wrong.
Still, the Monte Carlo ran smooth today. There were worse lives."
posted by storybored at 4:45 PM on May 18
Generally I do need a longer passage to tell, single lines or even brief paragraphs don't always have whatever it is I'm seeing as tells.
posted by sotonohito at 4:53 PM on May 18 [1 favorite]
posted by sotonohito at 4:53 PM on May 18 [1 favorite]
My friend, if you think you don't have a style that's distinct, you're almost certainly wrong. Even GPT itself has a distinct style, and it's not even intelligent.
Yes, you're right, everyone has a distinctive style. I think I should have said "idiosyncratic style". LLMs are going to be less helpful if an author's style is so boldly unique that it flies outside the range of its training data.
---
Re: middle class clothing in the 1920s.
For this I would use Perplexity, which gives web citations. If worried about hallucinations, you can check on the sources it's quoting. Here's its take:
Middle-class men in the 1920s typically wore:
Suits: Well-tailored, single- or double-breasted suits were the standard. Early in the decade, jackets were high-waisted and narrow-lapelled, with trousers that were straight and relatively short (so socks often showed). By the mid-1920s, wider trousers known as "Oxford bags" became fashionable, and jacket lapels became broader.
Enough to get me started on my Roaring Twenties noir story :)
posted by storybored at 4:56 PM on May 18
Yes, you're right, everyone has a distinctive style. I think I should have said "idiosyncratic style". LLMs are going to be less helpful if an author's style is so boldly unique that it flies outside the range of its training data.
---
Re: middle class clothing in the 1920s.
For this I would use Perplexity, which gives web citations. If worried about hallucinations, you can check on the sources it's quoting. Here's its take:
Middle-class men in the 1920s typically wore:
Suits: Well-tailored, single- or double-breasted suits were the standard. Early in the decade, jackets were high-waisted and narrow-lapelled, with trousers that were straight and relatively short (so socks often showed). By the mid-1920s, wider trousers known as "Oxford bags" became fashionable, and jacket lapels became broader.
Enough to get me started on my Roaring Twenties noir story :)
posted by storybored at 4:56 PM on May 18
As for grammar and 'correctness', they tend to iron out anything distinctive or counterintuitive you might write in favor of 'standard' language. If you're not playing with words in unexpected ways, why are you writing in the first place?
posted by signal at 5:00 PM on May 18
posted by signal at 5:00 PM on May 18
Yeah, that is a good question. What isn't clear is how far an allowance LLMs will make for distinctive or counterintuitive language. I get the (totally subjective) impression that it will work fine with anything that is mainstream and perhaps a little of the way outside it? Finnegans Wake was one of the novels that was used as training data.
If you're not playing with words in unexpected ways, why are you writing in the first place?
What if LLMs are helping me play with words in that way?
posted by storybored at 5:11 PM on May 18
If you're not playing with words in unexpected ways, why are you writing in the first place?
What if LLMs are helping me play with words in that way?
posted by storybored at 5:11 PM on May 18
« Older Mystical | "Do you prefer it as an outrageous breakfast drink... Newer »
What surprises me is that so many people are using them, because I have yet to see an example of sustained competent fiction written by an LLM. I've expressed my thoughts on this here before, that a couple of the basic tools of fiction--portraying action, portraying thought--are impossible for LLMs because they don't understand how having a body or brain works. Their lack of a sense of time means that they don't understand events either. (I really am trying to keep this brief but I could talk about this last one all day--I always try to find a term for how LLM text feels, maybe a phrase to use is premature narrative closure, this sense that there can never be loose ends, we need to wrap everything up in the space of a comment.)
In other words, setting aside the ethics of it (which I know we've already talked to death), there's the basic question: Does this work? Can it work? And there's always a contingent of people who say, sure, it can work, and if it doesn't work today it'll work tomorrow--but I always want those people to give it a try. Really, really try to make an LLM give you sustained fiction--that is, not a joke, not a quip, not a brief vignette--but a story of enough length that you can see action, thought, development, and narrative openness.
posted by mittens at 7:19 AM on May 18 [5 favorites]