As a quirky marketing exercise, we used ChatGPT to write parody lyrics for a song about ONC g.10 compliance to the tune of "Like a G6" by Far East Movement.
The whole thing took about an hour of effort. It was easy to be fun and lighthearted about it because "AI wrote it" and we didn't experience the normal fear of publishing.
This is extremely niche marketing, but our core audience loved it. Great bang for buck.
that was such a great scene, I experience wonder, awe, and immense satisfaction when writers and performers come together and recreate an authentic depiction of something they've potentially never experienced themselves.
There is an AI site that lets you train a voice very quickly too, currently in waitlist access. Forgot the name.
By spring I think you won't even have to use Fiverr for the recording. Or the people on Fiverr will be offerring this for less and less money at faster and faster speeds, undercutting each other until its $5 again - "a fiver".
who is we? just asking because I'm wondering if you are musicians who experience normal reputational risk fear of publishing, or healthcare workers with a (healthy :) fear of publishing something outside their wheelhouse
I find it revealing that while somebody with experience in songwriting finds the result dull and mediocre, I who knows next to nothing in the domain finds its nice and totally fine.
Same goes for other domains: philosophy, coding, writing, ...
It's telling me that this AI can generate content in many domains way better than an untrained human would with minimal efforts, while not (yet?) reaching the level of experts in the domain.
It empowers all the non-experts in these many domains to touch things they never could have before. This is an amazing tool.
Sure. But in this case it’s worth remembering that you’re not just listening to ChatGPT’s work here. It wrote lyrics and some weird chord progressions.
The OP, who is himself the frontman for the Decembrists applied his best effort to instill that with a melody, performance style, and sonic character of his band. He could take material from a 5-year-old or a Markov chain and dress it up to sound like a fair Decembrists song.
If you took the same ChatGPT output to another musician, expert or otherwise, you might not have the same experience you’re having now.
I think your last point may hold for this genre of music but not necessarily for pop. Pop musicians are extremely versatile. They sort of have to be to stay relevant. Take Ariana Grande for example. She could sing basically anything in any style you want[0], but you may not even have to ask her because we have efficient zero-shot voice cloning now[1]. Before you wonder if this will work for music, check out this video of AI generated Eminem track[3].
And at the other end of the scale, regular old digital tooling (sequencers, samplers, VSTs, chord transposers, autotune, DAWs and beginner DAW alternatives) has been doing the more important stuff to allow people that are not skilled musicians to produce something resembling a song for a long time before LLMs. Some in a more user friendly manner than others.
Stringing together some cliches and googling "what are some common chord progressions?" was never the barrier to humans becoming singer-songwriters and computers lowered more important barriers some time ago, though credit where it's due, the LLM is reasonably good at picking lines that rhyme.
I know nothing about songwriting, but I listened to a lot of Decemberists back in the day. Compared to their usual output, this is painfully bland. Their songs generally have a clever or spooky underlying theme, a strong sense of perspective, and often bits of cheeky humor. I understand what led the AI to write this as a Decemberists song, but it's like a second-grader's understanding of one. If this weren't sung by Colin Meloy himself, I would never peg it as such.
ChatGPT can't do better than experts in terms of analysis because it doesn't have a theory of mind. In other words, it's not actually thinking "what would experts do", and try to do the same or better. It's normal that it can produce mediocre results, because that's what it does, in a way: It produces "normal-looking" text. I don't know enough to say if it could do better if specialized (?), but I doubt it, as it cannot actually have an original idea, and knowing what is original is part of what makes an expert.
Theory of mind is super important here! Not just that of experts, but that of consumers. An actual author is always thinking about the audience. About what they'll find novel or interesting. About the best ways to inform, entertain, or delight them. About how to make them feel.
Merely producing "normal-looking" text (great and accurate phrase!) is an unclosed feedback loop. The songwriter here has to stop himself from driving that loop forward because he's so used to working through multiple drafts as he makes things work for his internal simulation of his audience. And generally, once an artist has something that works internally, they'll then start running it by actual other people to get their reactions. Up to and including testing material on full audiences.
Because LLMs lack internal simulations of audience response, they'll always be limited to producing "normal-looking" work.
'Theory of mind (ToM), or the ability to impute unobservable mental states to others, is central to human social interactions, communication, empathy, self-consciousness, and morality. We administer classic false-belief tasks, widely used to test ToM in humans, to several language models, without any examples or pre-training. Our results show that models published before 2022 show virtually no ability to solve ToM tasks. Yet, the January 2022 version of GPT-3 (davinci-002) solved 70% of ToM tasks, a performance comparable with that of seven-year-old children. Moreover, its November 2022 version (davinci-003), solved 93% of ToM tasks, a performance comparable with that of nine-year-old children. These findings suggest that ToM-like ability (thus far considered to be uniquely human) may have spontaneously emerged as a byproduct of language models' improving language skills.'
I see a lot of confident assertions of this type (LLMs don’t actually understand anything, cannot be creative, cannot be conscious, etc.), but never any data to substantiate the claim.
This recent paper suggests that recent LLMs may be acquiring theory of mind (or something analogous to it): https://arxiv.org/abs/2302.02083v1
Some excerpts:
> We administer classic false-belief tasks, widely used to test ToM in humans, to several language models, without any examples or pre-training. Our results show that models published before 2022 show virtually no ability to solve ToM tasks. Yet, the January 2022 version of GPT- 3 (davinci-002) solved 70% of ToM tasks, a performance comparable with that of seven-year-old children. Moreover, its November 2022 version (davinci-003), solved 93% of ToM tasks, a performance comparable with that of nine-year-old children.
> Large language models are likely candidates to spontaneously develop ToM. Human language is replete with descriptions of mental states and protagonists holding divergent beliefs, thoughts, and desires. Thus, a model trained to generate and interpret human-like language would greatly benefit from possessing ToM.
> While such results should be interpreted with caution, they suggest that the recently published language models possess the ability to impute unobservable mental states to others, or ToM. Moreover, models’ performance clearly grows with their complexity and publication date, and there is no reason to assume that their it should plateau anytime soon. Finally, there is neither an indication that ToM-like ability was deliberately engineered into these models, nor research demonstrating that scientists know how to achieve that. Thus, we hypothesize that ToM-like ability emerged spontaneously and autonomously, as a byproduct of models’ increasing language ability.
> It empowers all the non-experts in these many domains to touch things they never could have before.
Nothing stops anyone from writing a mediocre song. Hell, it's one of the easiest things in creation to do. Hum a melody, make some words fit. Write it down, make up your own notation for the melody if you want; you don't even have to perform it. Congratulations! You're in the same field as Colin Meloy.
ChatGPT isn't even doing this much. It's read all the tabs and lyrics of every song within its reach until 2021, and then spits out an average response based on average similar requests. It's a songwriter in the same way a 7-year-old telling you a story about an alligator who ate a city is an author.
So basically the pitch deck for Godzilla? Even including “and then a HUGE MOTH appears and Godzilla shoots a huuuge lazer out of his mouth. And then the moth whose name is uhh Mothra screeches in pain and falls into a skyscraper.”
I think you’re doing seven year olds and GPT a disservice. Lots of iconic media is made by adults who have managed to hold on to their seven year old storytelling brain.
> It empowers all the non-experts in these many domains to touch things they never could have before. This is an amazing tool.
Possibly touch things they don’t know much or even anything about? That with factual errors is not really a recipe for good. What is good is what makes us think and that is certainly possible with LLMs and AI. Think interactive fiction, think collaborations of sorts with humans and so on. The current trend of generate a remix of a rehash is, all I hope, just a temporary muscle flexing of what’s possible and not what we’ll get once the hype dies down a bit.
I think it lulls people into a false sense of confidence that they're producing anything of lasting value. I'll concede that it certainly enables the layman to be more productive... at least in terms of sheer volume.
Given that GPT represents a distillation of the sum of human output on the internet, is it truly unsurprising that 99% of the generated stories lyrics etc. are rather mediocre?
I mean, if you've listened to a couple Decemberist albums, that criticism extends fully. There are plenty of dull, mediocre, and repetitive songs throughout. I'm listening and reacting to this one now. It would definately fell like a filler song. It lacks the kind of juxtoposition of positionality that Decemberist songs have. Its there but it comes in late and is a bit weak. The cord transitions are also a bit boring. But if you asked me to tell the difference between this and some other filler of theirs I know of, I probably couldn't.
Its also relentlessly positive in a way that Decemberist songs rarely are. Both acoustically and lyrically, which having played with ChatGPT a bit, seems to be a major part of its training. It avoids negativity in big ways. The Decemberists dont necessarily, and so it feels very gushy and bright in a way that it shouldnt.
Just made this same argument without reading your comment. I do wonder if musicians will start using this in studios, especially pop ones with one or two hits, to help fill the rest of the album.
We have to be clear on why Colin Meloy finds the song "mediocre" (lit. middle of the road, not bad but not special). I don't know Colin personally, so of course I can't really make a definitive statement, but my attitude of the same is informed by the song’s literalness. And this probably doesn't mean much to someone who has not tried to write a song or a book before, so I wanted to comment on it.
People always have these excited memories about their childhood favorites, their favorite songs, their favorite books, the things they really connected with. But the craft in design for these works is usually that, the thing you connected with, is not literally in the artwork. Instead the artwork sits around it, the artwork gestures at it. You have an idea of this character as this lonely solitary brute who 15-year-old-you really identified with because you had just come out of hitting puberty early or whatever, but if you really look at the character descriptions, it turns out that most of your impression is formed by other people in the narrative wanting the character to say something and they just don't.
This creates a really plastic space which people mold into their own viewpoint, they take it and make it relatable to them. Eco calls a novel a “lazy machine,” to make it really special I need to trust that laziness, I need to trust my reader to fill in the gaps that I do not. For instance, the hulking brute above—did you understand that to be a guy or a girl?
When Dave Matthews charted it was with “Take these chances / Place them in a box until a / Quieter time, lights down, you up and die.” What chances? What box? What quieter time? We get vignettes or a humdrum existence, of dreams of how it was simpler, and it resonated because it tapped into a nostalgic yearning for a simpler life, a fatalistic concern about the finitude of life... But the character is only seen in dotted-outline silhouette, a blank slate that we paint with our own stories.
Same when Colin sings “And nobody nobody knows / Let the yoke fall from our shoulders / Don't carry it all, don't carry it all / We are all our hands and holders / Beneath this bold and brilliant sun / And this I swear to all.” What yoke? What sun? Who is “we”? The song sketches its theme, a theme of goodbyes, of picking up where others left off, of being assured that they will pick up what you leave off, of community and family and transition. But again, the details are just gestured at, you are invited to set yourself into the narrative, personalize and connect.
ChatGPT’s chorus is “Of sailors brave, and adventures untold / Of a life on the waves, and a heart grown old / Of a world of discovery, waiting to be sought / In a song that will live, when he is not.” What sailors? Well, the companions of the Mariner who shall not be named, he apparently had splendid friends. What adventure? Again, the adventure of the Mariner, who apparently had a splendid time. The only hope for the song is that the “song that will live” is going to be presented in this work: that we are going to see a key change and the Old Man’s Song will turn out to be the climax of the piece, we will dramatically insert ourselves and become the Old Man, dramatically ending with some crescendo, “these splinters I will cling to / the battered sail fills up with might / these are the fallen friends I drink to: / I go out alone, but I'm not dying tonight!” and for just one hair-on-end minute we feel that Odyssean determination to set out on one last ambitious venture before we die. But of course we never get that because ChatGPT is trying to create something cohesive where we want something emotional.
I think Colin hasn't quite put his finger on what's not compelling about the output. He's interpreting it as a lack of "intuition", and claims that's what separates us from the machine, but that may just be what he was hoping to see in the first place.
Whats missing from its story telling is that it doesn't understand how conflict drives a narrative and what counts as resolving it. Often I find it tries to short circuit the process with "and then the main character learned to be happy with life. The end." No, not the end. In a decent story the characters are a vessel to elaborate on a wider idea. We don't care if it makes them happy or sad. Chat GPT seems excessively averse to any sort of "negativity". It tends to never write tragic endings or disagreeable characters.
It is missing something very concrete, not something nebulous like "intuition".
I think he hit on a good word. In his rendering of it, some transitions seemed a little awkward, flourishes seemed to be missing - there seemed to be a lack of fine tuning that an experienced musician would give it.
To me, this illustrates more how good artists can turn about any garbage into something workable.
It only has 4 chords, and as pointed out, ChatGPT couldn't really line them up into a melody or give any clues. It was more 'heres some rhyming lines and 4 chords I've chosen, good luck.' If played more literally, it would have sounded pretty terrible.
Absolutely, it helps develop an MVP with a totally new genre, that gives the sense of what it would sound like.
Which is perfect! Now one can determine whether to continue or not -- without having to start + create everything from scratch [much longer than an 1 hour], especially with the R&D component that CHATGPT already provides out-of-the-box.
> It has data, it has information, but it has no intuition.
Obviously we’re into inexact analogies here, but I would have said the opposite: it doesn’t have data or information because that’s all been compressed into an approximation. Actually, some form of intuition is all it has.
Yeah, if we're comparing with human abilities at all, LLMs are pretty much just pure intuition- they produce a startling rendition of how I wrote essays the night before the due date in college. I suspect that the lack the author is detecting here is decisive judgement calls. This could be an issue with the underlying model, but I suspect that it's just a side effect of the task it's actually performing- predicting the next token. It can kind of only wander into any actual "decision" accidentally, so the result is middle-of-the-road. Not sure what kind of loss function you'd need to get more interesting results though.
OK, but, due respect to Colin Meloy here, this might have been a fair fight. I love the Decemberists and have probably seen them live more than any other band, but lyrically, well, I just looked up when the first thesaurus was published (1852 or thereabouts) because I think Meloy probably uses it. There are some Decemberists songs with excellent lyrics, but there are others where I think an extremely well-trained LLM could hold its own replacing them.
With some extra careful prompting, you could probably get a GPT model to write lyrics that really would read like they belonged on Castaways; you just need more archaisms. You probably could take these prompts and just tell ChatGPT "ok, but replace some of these words with archaic almost-synonyms that sounds like they mean the same thing but really don't" and get 98% of the way there.
This Slate article does a decent job of capturing some of what I mean:
You might, like me, read this and get these sense that Meloy's lyrical style is probably very trainable/promptable.
PS
I get that I'm talking exclusively about the lyrics, which are of course the easy part of songwriting for ChatGPT. I just think this is a funny particular comparison to draw, this particular band. Which, again, I do love.
I took a whack. Here's a result given Meloy's prompt:
I've seen the sun rise over distant shores
Where the birds they sing and the wildflowers grow
I've traveled to places far and wide
Where the stories of old still echo in the night
Here's 2 additional prompts later:
Risen sun I've spied, beyond the unknown shore
Where sweet the birds sing, and wildflowers ever pour
To lands of ancient lore, I've journeyed far and wide
Where tales of olden times, still boom and bellow beside
It's veering into Tolkein, of course, but I think if I spent a half hour with this I could probably get a reasonably reliable prompt formula going. I got it to write "And we'll not sway, like Juliet at the welm." If we constructed a Colin Turing Test, that line would pass.
The next thing to do, of course, is to complete the song and then ask ChatGPT to write a review of it in the style of Brent DiCrescenzo-era Pitchfork.
Honestly, I think that Slate article is just pure lazy snark, zero analysis.
Literally just points out things like object-verb inversion and large words and pretends they are intrinsically bad.
Look, if you don't like it, that's fine, but you're going to have to do a lot more work than "Ha! That word isn't in common use!" to convince me that it's bad.
I'm not so much saying that it's great criticism, so much as that if you took those observations and built prompts around them, you could make a song that sounded much more like Colin Meloy.
The formula is something like:
* Start with a Smiths song
* Make it melancholy.
* Rewrite all the references and allusions so that they jarringly alternate between Chaucer era early English lit and Victorian Gothic.
* Deliberately invert subject-verb order in places to make things sound more poetic.
I love the decemberists, but I’m not going to lie, if you gave me a handful of their songs (not the most popular but fillers from the albums) and injected this as well, then asked me to tell you which one was written by chat gpt I’d fail. I think Colin over estimates intuition here. Intuition is just the ability to ability to know what’s best without completely understanding why. Which is precisely how chat GPT works. It doesn’t know why these patterns are common for humans to love, but neither do humans really.
I wonder, is it because the model is trying to produce the most song-ish song that makes it mediocre?
Take a look at the Stable Diffusion community: images generated from plain prompts like "a guy walking on the street at night" returns extremely blend images, while throw in a bunch of modifiers like "futuristic", "cyberpunk", "4K", "perfect face", "trending on Artstation", "realistically shaded", etc actually adds in real vibe of art.
Also, sometimes the ways are not obvious. In the first place the model having difficulty on drawing hands. But later people find negative prompts which essentially saying "I don't want ill-drawned hands" works surprisingly.
> I wonder, is it because the model is trying to produce the most song-ish song that makes it mediocre?
I think there’s a simpler explanation. Good creative work needs an element of intelligently targeted unpredictability. It’s mayhaps not the best idea to ask a prediction model about that.
I don’t get how it took people so long to figure this out. In the first week, I was trying to get it to write a story in the style of Dostoyevsky, and it just couldn’t. I’d ask it to be more wavering, to break the rules, but it did a shallow interpretation of that too. Every story was wrapped up in a “then everyone became friends and were happy”-style.
However, when writing a corporate HR letter threatening employees to discuss the CEOs inappropriate behavior at the holiday party, as well as writing an inspirational LinkedIn post, it was indistinguishable from the real deal, on first try.
Not sure if you tried this out on any other model, but I find that a lot of the tonal issues are pretty specific to ChatGPT. An untuned davinci-003 doesn't feel the same need to wrap things up happy every time, and produces much better imitations of style. There's still the usual LLM issues (slow divergence from whatever the prompt was, occasional loops, a sort of dreamy quality to every narrative), but ChatGPT always writes in the bland style of a corporate memo because it was deliberately retrained to do that.
Readit News
We then paid $50 on Fiverr to record it: https://soundcloud.com/medplum/medplum-g-10
The whole thing took about an hour of effort. It was easy to be fun and lighthearted about it because "AI wrote it" and we didn't experience the normal fear of publishing.
This is extremely niche marketing, but our core audience loved it. Great bang for buck.
By spring I think you won't even have to use Fiverr for the recording. Or the people on Fiverr will be offerring this for less and less money at faster and faster speeds, undercutting each other until its $5 again - "a fiver".
who is we? just asking because I'm wondering if you are musicians who experience normal reputational risk fear of publishing, or healthcare workers with a (healthy :) fear of publishing something outside their wheelhouse
Same goes for other domains: philosophy, coding, writing, ...
It's telling me that this AI can generate content in many domains way better than an untrained human would with minimal efforts, while not (yet?) reaching the level of experts in the domain.
It empowers all the non-experts in these many domains to touch things they never could have before. This is an amazing tool.
The OP, who is himself the frontman for the Decembrists applied his best effort to instill that with a melody, performance style, and sonic character of his band. He could take material from a 5-year-old or a Markov chain and dress it up to sound like a fair Decembrists song.
If you took the same ChatGPT output to another musician, expert or otherwise, you might not have the same experience you’re having now.
[0] https://www.youtube.com/watch?v=ss9ygQqqL2Q
[1] https://arxiv.org/pdf/2301.02111.pdf
[2] https://aka.ms/valle (demos from one)
[3] https://www.youtube.com/watch?v=WtFNOSTTPYg
[4] https://github.com/NVIDIA/tacotron2 (used to create audio for 3)
Wow, buried lede there. That should be in the title!
Stringing together some cliches and googling "what are some common chord progressions?" was never the barrier to humans becoming singer-songwriters and computers lowered more important barriers some time ago, though credit where it's due, the LLM is reasonably good at picking lines that rhyme.
Merely producing "normal-looking" text (great and accurate phrase!) is an unclosed feedback loop. The songwriter here has to stop himself from driving that loop forward because he's so used to working through multiple drafts as he makes things work for his internal simulation of his audience. And generally, once an artist has something that works internally, they'll then start running it by actual other people to get their reactions. Up to and including testing material on full audiences.
Because LLMs lack internal simulations of audience response, they'll always be limited to producing "normal-looking" work.
https://arxiv.org/abs/2302.02083
'Theory of mind (ToM), or the ability to impute unobservable mental states to others, is central to human social interactions, communication, empathy, self-consciousness, and morality. We administer classic false-belief tasks, widely used to test ToM in humans, to several language models, without any examples or pre-training. Our results show that models published before 2022 show virtually no ability to solve ToM tasks. Yet, the January 2022 version of GPT-3 (davinci-002) solved 70% of ToM tasks, a performance comparable with that of seven-year-old children. Moreover, its November 2022 version (davinci-003), solved 93% of ToM tasks, a performance comparable with that of nine-year-old children. These findings suggest that ToM-like ability (thus far considered to be uniquely human) may have spontaneously emerged as a byproduct of language models' improving language skills.'
This recent paper suggests that recent LLMs may be acquiring theory of mind (or something analogous to it): https://arxiv.org/abs/2302.02083v1
Some excerpts:
> We administer classic false-belief tasks, widely used to test ToM in humans, to several language models, without any examples or pre-training. Our results show that models published before 2022 show virtually no ability to solve ToM tasks. Yet, the January 2022 version of GPT- 3 (davinci-002) solved 70% of ToM tasks, a performance comparable with that of seven-year-old children. Moreover, its November 2022 version (davinci-003), solved 93% of ToM tasks, a performance comparable with that of nine-year-old children.
> Large language models are likely candidates to spontaneously develop ToM. Human language is replete with descriptions of mental states and protagonists holding divergent beliefs, thoughts, and desires. Thus, a model trained to generate and interpret human-like language would greatly benefit from possessing ToM.
> While such results should be interpreted with caution, they suggest that the recently published language models possess the ability to impute unobservable mental states to others, or ToM. Moreover, models’ performance clearly grows with their complexity and publication date, and there is no reason to assume that their it should plateau anytime soon. Finally, there is neither an indication that ToM-like ability was deliberately engineered into these models, nor research demonstrating that scientists know how to achieve that. Thus, we hypothesize that ToM-like ability emerged spontaneously and autonomously, as a byproduct of models’ increasing language ability.
Nothing stops anyone from writing a mediocre song. Hell, it's one of the easiest things in creation to do. Hum a melody, make some words fit. Write it down, make up your own notation for the melody if you want; you don't even have to perform it. Congratulations! You're in the same field as Colin Meloy.
ChatGPT isn't even doing this much. It's read all the tabs and lyrics of every song within its reach until 2021, and then spits out an average response based on average similar requests. It's a songwriter in the same way a 7-year-old telling you a story about an alligator who ate a city is an author.
I think you’re doing seven year olds and GPT a disservice. Lots of iconic media is made by adults who have managed to hold on to their seven year old storytelling brain.
Possibly touch things they don’t know much or even anything about? That with factual errors is not really a recipe for good. What is good is what makes us think and that is certainly possible with LLMs and AI. Think interactive fiction, think collaborations of sorts with humans and so on. The current trend of generate a remix of a rehash is, all I hope, just a temporary muscle flexing of what’s possible and not what we’ll get once the hype dies down a bit.
Given that GPT represents a distillation of the sum of human output on the internet, is it truly unsurprising that 99% of the generated stories lyrics etc. are rather mediocre?
Its also relentlessly positive in a way that Decemberist songs rarely are. Both acoustically and lyrically, which having played with ChatGPT a bit, seems to be a major part of its training. It avoids negativity in big ways. The Decemberists dont necessarily, and so it feels very gushy and bright in a way that it shouldnt.
We have to be clear on why Colin Meloy finds the song "mediocre" (lit. middle of the road, not bad but not special). I don't know Colin personally, so of course I can't really make a definitive statement, but my attitude of the same is informed by the song’s literalness. And this probably doesn't mean much to someone who has not tried to write a song or a book before, so I wanted to comment on it.
People always have these excited memories about their childhood favorites, their favorite songs, their favorite books, the things they really connected with. But the craft in design for these works is usually that, the thing you connected with, is not literally in the artwork. Instead the artwork sits around it, the artwork gestures at it. You have an idea of this character as this lonely solitary brute who 15-year-old-you really identified with because you had just come out of hitting puberty early or whatever, but if you really look at the character descriptions, it turns out that most of your impression is formed by other people in the narrative wanting the character to say something and they just don't.
This creates a really plastic space which people mold into their own viewpoint, they take it and make it relatable to them. Eco calls a novel a “lazy machine,” to make it really special I need to trust that laziness, I need to trust my reader to fill in the gaps that I do not. For instance, the hulking brute above—did you understand that to be a guy or a girl?
When Dave Matthews charted it was with “Take these chances / Place them in a box until a / Quieter time, lights down, you up and die.” What chances? What box? What quieter time? We get vignettes or a humdrum existence, of dreams of how it was simpler, and it resonated because it tapped into a nostalgic yearning for a simpler life, a fatalistic concern about the finitude of life... But the character is only seen in dotted-outline silhouette, a blank slate that we paint with our own stories.
Same when Colin sings “And nobody nobody knows / Let the yoke fall from our shoulders / Don't carry it all, don't carry it all / We are all our hands and holders / Beneath this bold and brilliant sun / And this I swear to all.” What yoke? What sun? Who is “we”? The song sketches its theme, a theme of goodbyes, of picking up where others left off, of being assured that they will pick up what you leave off, of community and family and transition. But again, the details are just gestured at, you are invited to set yourself into the narrative, personalize and connect.
ChatGPT’s chorus is “Of sailors brave, and adventures untold / Of a life on the waves, and a heart grown old / Of a world of discovery, waiting to be sought / In a song that will live, when he is not.” What sailors? Well, the companions of the Mariner who shall not be named, he apparently had splendid friends. What adventure? Again, the adventure of the Mariner, who apparently had a splendid time. The only hope for the song is that the “song that will live” is going to be presented in this work: that we are going to see a key change and the Old Man’s Song will turn out to be the climax of the piece, we will dramatically insert ourselves and become the Old Man, dramatically ending with some crescendo, “these splinters I will cling to / the battered sail fills up with might / these are the fallen friends I drink to: / I go out alone, but I'm not dying tonight!” and for just one hair-on-end minute we feel that Odyssean determination to set out on one last ambitious venture before we die. But of course we never get that because ChatGPT is trying to create something cohesive where we want something emotional.
Whats missing from its story telling is that it doesn't understand how conflict drives a narrative and what counts as resolving it. Often I find it tries to short circuit the process with "and then the main character learned to be happy with life. The end." No, not the end. In a decent story the characters are a vessel to elaborate on a wider idea. We don't care if it makes them happy or sad. Chat GPT seems excessively averse to any sort of "negativity". It tends to never write tragic endings or disagreeable characters.
It is missing something very concrete, not something nebulous like "intuition".
I think there's a lot of behind the scenes manipulation of the model to steer clear of anything that would bring negative criticism of its output.
Surely that leaks into what lyrics it chooses to write, what stories it can produce.
It only has 4 chords, and as pointed out, ChatGPT couldn't really line them up into a melody or give any clues. It was more 'heres some rhyming lines and 4 chords I've chosen, good luck.' If played more literally, it would have sounded pretty terrible.
Which is perfect! Now one can determine whether to continue or not -- without having to start + create everything from scratch [much longer than an 1 hour], especially with the R&D component that CHATGPT already provides out-of-the-box.
Obviously we’re into inexact analogies here, but I would have said the opposite: it doesn’t have data or information because that’s all been compressed into an approximation. Actually, some form of intuition is all it has.
With some extra careful prompting, you could probably get a GPT model to write lyrics that really would read like they belonged on Castaways; you just need more archaisms. You probably could take these prompts and just tell ChatGPT "ok, but replace some of these words with archaic almost-synonyms that sounds like they mean the same thing but really don't" and get 98% of the way there.
This Slate article does a decent job of capturing some of what I mean:
https://slate.com/culture/2009/04/the-eight-most-pretentious...
You might, like me, read this and get these sense that Meloy's lyrical style is probably very trainable/promptable.
PS
I get that I'm talking exclusively about the lyrics, which are of course the easy part of songwriting for ChatGPT. I just think this is a funny particular comparison to draw, this particular band. Which, again, I do love.
The next thing to do, of course, is to complete the song and then ask ChatGPT to write a review of it in the style of Brent DiCrescenzo-era Pitchfork.
Literally just points out things like object-verb inversion and large words and pretends they are intrinsically bad.
Look, if you don't like it, that's fine, but you're going to have to do a lot more work than "Ha! That word isn't in common use!" to convince me that it's bad.
The formula is something like:
* Start with a Smiths song
* Make it melancholy.
* Rewrite all the references and allusions so that they jarringly alternate between Chaucer era early English lit and Victorian Gothic.
* Deliberately invert subject-verb order in places to make things sound more poetic.
* Employ archaic synonyms.
* Employ multi-line alliteration.
Take a look at the Stable Diffusion community: images generated from plain prompts like "a guy walking on the street at night" returns extremely blend images, while throw in a bunch of modifiers like "futuristic", "cyberpunk", "4K", "perfect face", "trending on Artstation", "realistically shaded", etc actually adds in real vibe of art.
Also, sometimes the ways are not obvious. In the first place the model having difficulty on drawing hands. But later people find negative prompts which essentially saying "I don't want ill-drawned hands" works surprisingly.
I think there’s a simpler explanation. Good creative work needs an element of intelligently targeted unpredictability. It’s mayhaps not the best idea to ask a prediction model about that.
I don’t get how it took people so long to figure this out. In the first week, I was trying to get it to write a story in the style of Dostoyevsky, and it just couldn’t. I’d ask it to be more wavering, to break the rules, but it did a shallow interpretation of that too. Every story was wrapped up in a “then everyone became friends and were happy”-style.
However, when writing a corporate HR letter threatening employees to discuss the CEOs inappropriate behavior at the holiday party, as well as writing an inspirational LinkedIn post, it was indistinguishable from the real deal, on first try.