Computers write prose, but are still inferior to people. Why?



    Two years ago, in a Japanese literary contest, I almost won a story written by artificial intelligence. It is called "The Day the Computer Wrote the Novel." Even if it sounds like a stupid incarnation of an anecdote “the best way to make a million is to write a book, how to make a million,” the story passed to the final, beating about fifteen hundred texts written by people. But the breakthrough did not happen. The artistic texts are still written by people, non-fiction - mostly the same.

    A couple of months ago, the company Meanotek tried to make an AI assistant for writing and write a story with him in collaboration. Spoiler - he did not succeed. The creators of the project told us what the idea was and how it was implemented. And we tried to figure out why poets come out of cars better than prose writers.

    I understand that you do not understand that we do not understand anything


    In the summer of this year, Sistema Foundation held a competition for science fiction stories “The Future”. The participants were asked a theme - “immortality”, and the winner was promised a solid jackpot of half a million rubles. Thousands of stories were sent to the organizers, among which one was written using artificial intelligence.


    Best of all, AI can work in a short genre - poems, wise quotes, motivators, and memes. But everything does not always go as planned. For example, InspiroBot , which creates motivating posters, puts words into strange and sometimes creepy things.

    The story is called "Frog spreads its wings . " This is a story about the girl engineer Potino, the owner of a gentleman's set of qualities characteristic of the heroes of cyberpunk noir: drug addiction, dark past, mental abnormalities, suicidal tendencies and endless reflection about the illusory nature of the world. Between the hallucinogenic trips, Fotino is trying to fix the AI, managing the city.
    This is a story written not entirely by AI, but only with his participation. The experiment was that a special program was not used to create the text, but several blocks of code that were originally intended to solve other problems were bolted together.

    The system used a proprietary library, which, in turn, uses Pytorch. Dialogues were made up of a chat bot developed earlier; for headings and descriptions, a very large recurrent network trained on texts downloaded from lib.ru.

    • Denis Tarasov ( Durham ), executive director of Meanotek.

    The most important aspects of writing, the idea and the plot, were still taken by the person. According to Denis, they tried to create a tool that only helps the writer, and does not work for him.
    The system fully generated dialogues with AI and left in the form in which they were received. The dialogues between the characters are partially generated and the roughness is smoothed there. Removed repetitions from phrases like "I understand that you do not understand that we do not understand anything." Although some proportion of such expressions remained. Subheads are also all generated. We tried to correct spelling and grammar, but it is possible that we added more errors than corrected.

    • Denis Tarasov ( Durham ), executive director of Meanotek.

    In the final text, it is difficult to determine the amount of machine and human text. As Denis says, there are paragraphs where one sentence was composed by a person, and the second is an algorithm. “A series of episodes would simply not have been in a story without AI titles and“ primes ”. And in general, the style of the AI ​​fragments influenced the style in which the person wrote. ”

    Work reminds, if you can call it that, a certain expansion of consciousness. Having written a sentence or paragraph, a person addresses not only his imagination, but also looks at what a computer generated from his words. "The man tried to adapt to what was generated, and, if possible, put it all into a meaningful plot," says Denis.



    However, the success of the Japanese colleagues failed. The story did not even hit the long list. Jury member Nikolai V. Kononov wrote about Frog as follows:
    The editor's hand is too noticeable - and in general, the human hand. The developers of the bot, in collaboration with the algorithm, wrote a story, which is unlikely to cause rejection of fiction among readers. In this sense, the experiment was a success.

    “What most people like today is not necessarily a quality story,” says Denis. “The quality of the text can still be improved, but the main problem is the lack of plot. At the exit, it turns out that some parts of the text may even contain meaning, but they have no beginning and end, no common idea. Just something happens, someone does something, talks, and it does not end with anything. ”

    But you cannot understand that you are going to solve a problem that will allow you to come up with something


    When the word “Robot” was first invented, it was already used in the context of the philosophical question “Is it fair not to consider artificial people as people”. This is a play by Karel čapek RUR. Any word is also a thought, a meaning, and a context behind it. In a good way, the writer first finds ideas and meanings, and then selects letters for them.

    Still, the words themselves are no more than ink or pixel squiggles on a white background.



    The big problem with machine texts is that the process goes the other way. The system selects a certain small set of words from a larger set, and a person - reading - tries to find a meaning in them, and goes through these sets until he finds meaning.
    So far, computers have learned quite well how to write poetry. This can be seen on the site botpoet.com , where there is a test, clearly showing that it is difficult to distinguish between a “human” poet and a computer. But in verses many readers think up a lot of logical connections. In prose (especially in the plot), it is usually more pronounced, and the computer is still doing poorly. Often, neural networks are taught on an array of works by one or another author, but they can only produce outwardly similar text, that is, imitate the style, but do not create a new storyline.



    Yandex employees created a neural network that was taught in an array of Russian poetry, and then fed the texts of “Civil Defense” and issued poems in the style of Yegor Letov. They also recorded the musical album “404” and named the group “Neural Defense”. The

    plots are formally arranged as a long-standing topic for research. Until the heyday of IT, researchers manually searched through thousands of texts in search of common patterns. One of the most famous examples is the book by Joseph Campbell, A Hero with a Thousand Persons. This is an attempt to derive a general structure, a sequence of situations, which is present in a more or less similar form in the myths of most nations. Vladimir Propp worked on a similar task. In the book “The Morphology of a Magic Fairy Tale” he tried to describe the general components of folklore stories.

    According to this knowledge, people teach people to make stories. Studies of the form of plots fall into books on writing and screenwriting skills, where they become concrete instructions. For example, screenwriter Blake Snyder in the book “Save the Seal” described the clear structure of a good film in his opinion, where he distributed the necessary plot functions down to the place in the timekeeping and the number of pages.

    But instructions for people have not yet become instructions for cars.
    Understanding the plot structure of the text and plot twists is more difficult, because for a start it would be good to agree on how we formally present them and then think how to extract them from the text. The instructions from the textbooks for writers can probably be laid in the car in the form of rules with some randomly filled variables (for example, that she choose a city of action from the list of European capitals), but this will hardly bring us closer to creating original stories. I think, so far, you can teach a computer to analyze whether a particular text is closer to successful or unsuccessful old ones, and to give the author some recommendations, but no more.


    “The AI ​​is not yet able to assess the quality of the story, because, in principle, the quality criteria are not clear,” says Denis Tarasov. But if you very roughly put an equal sign between quality and success, then the AI ​​can cope with the task.

    Last year, the best-selling code book was published, where screenwriters Jody Archer and Matthew Jokers tried to describe the common features of successful books. They fed the texts of books to a computer system built on machine learning (which one, nevertheless, they did not describe), compared them with commercial success and named patterns that could contribute to this success - themes, settings, characteristics of the characters and the sequence in changing their state .
    It is easy to bring the topic: for this, there are algorithms for extracting keywords that build a frequency dictionary and look at which words are much more common than average in a language. True, in order to learn how to transform keywords into themes, we must still try: for example, if we know that the words “kiss”, “gentle”, “night” and “lilac” are found with unusually high frequency an algorithm that will reduce these words to the theme "love". Or you can pre-break the texts into classes according to topics (“love”, “adventures”) and then solve the problem of classifying a new text - how this is done, for example, in news aggregators, where you have to understand whether the text applies to economics, politics, etc. p. In general, this problem seems to be completely solved.

    Characteristics of the characters are also analyzed well. Digital Humanities are doing a lot of this, which often refers to digital literature studies. For example, the classic of this area Franco Moretti automatically built and then analyzed the network of connections between the characters of Hamlet . On the Russian material this is done, for example, by the group of Frank Fisher at the Higher School of Economics, which constituted the corpus of the Russian drama RusDraCor .



    To make a generator of such stories is not a big scientific problem. There are plot generators on this principle on the Internet, and even story generators, where you can choose parameters and get text of different quality. This is all a solvable task, you just need to spend time and energy.

    First, a sequence of top-level events will be generated, and then each event will unfold until it reaches the level of sentences and letters. Or, in general, you can make a "rewrite" of a popular story, change the details there and here's a new popular story.

    • Denis Tarasov ( Durham ), executive director of Meanotek.

    Do I have to do what you are going to not know?


    Another question is whether this makes sense. When people are trying to automate and robotize heavy production, the goal is clear - in many industries, the physical capabilities of people have long since come up against the ceiling. There are no such problems in writing - those problems are far from the physical burden of labor.

    On the contrary, readers often complain about any attempts to automate writing, to bring it to common patterns and patterns. Watching, for example, a movie, people are more likely to be disappointed if a predetermined plot twist occurs at a clearly defined minute each time. In expensive commercial cinema this is happening more and more often, but it does not seem to be called an incredible breakthrough of dramatic art.


    In my opinion, such a system in the literature will bring great harm, since it will propagate various literary trash, optimizing it for the reader's requests. Such garbage is perfectly produced without a generator, and with a generator it will have nothing to breathe with. To write a good story you need to understand the complexity of human relationships, understand biology, physics, and much more, at a good level - and not just be able to put chains of words that cause a given emotional response from the reader.

    This problem is not solved by any simple algorithm. You need a level of intelligence and knowledge of a person, and an outstanding person. AI should be able to build the models of the world itself and the ability to predict the results of various events, and this is a problem that is not solved now, and science is just beginning to approach to it.

    From here and our direction in the experiment is an attempt to combine the AI ​​and the person.

    • Denis Tarasov ( Durham ), executive director of Meanotek.

    On the one hand, automation can really spoil the literature. On the other hand, if AI becomes just an expander of imagination, if it provides something that the author himself would never have invented, then the computer can really become a way out of creative crises, personal and global.
    Tags:

    Also popular now: