How To Train Your Dog Books
Janet Denzel
These ISBNs were included in the text of books in the Books3 dataset, part of The Pile, that was used to train Facebook\u2019s new model, LLaMA, launched Feb 24th. The dataset is made up of 197,000 ebooks downloaded from a BitTorrent server. Other big companies that have used Books3 to train models include Microsoft and nVidia. We can\u2019t know for sure if this is also the so-called \u201CBooks2\u201D dataset used by OpenAI to train GPT-3, because OpenAI refuses to any share any details of the data it has used for training. Books3 and OpenAI\u2019s Books2 are very similar in size. I assume OpenAI has used this set, though I welcome OpenAI proving otherwise.
Books3 is described by Facebook as \u201Ca publicly available dataset for training large language models.\u201D Well it sure is publicly available, for download at -eye.eu/public/AI/pile_preliminary_components/books3.tar.gz (a mere 36.8GB) or from AI community HuggingFace at _pile_books3. But that description elides the fact that this is a cache with large numbers of pirated ebooks. It wasn\u2019t passively scraped from the public-facing internet, someone (an AI coder named Shawn Presser it seems) had to go to a bittorrent server to download it. Then there was a great deal of processing needed to turn that cache from a bunch of epub files into text that could be fed to an LLM for training.
I wonder if HuggingFace should be a little worried about some legal risk here, they basically have 36.8 GB of pirated book text available for anyone to download. It includes around 65 books with \u201CHarry Potter\u201D in the title. To train AI models, sure, but also to just, you know, read illegally.
Books3 was incorporated into a larger dataset called The Pile, by the EleutherAI research collective, which started as a dischord server group and is now a research institute. EleutherAI has published open source models like GPT-J and GPT-NeoX. In the original Pile paper the authors cited Shawn Presser\u2019s tweet. Gao, Leo, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, et al. \u201CThe Pile: An 800GB Dataset of Diverse Text for Language Modeling,\u201D December 31, 2020.
I don\u2019t want to unnecessarily single out Facebook here. The whole reason that we know they used this dataset is because they are trying to be as transparent as possible, making the point that \u201CUnlike Chinchilla, PaLM, or GPT-3, we only use publicly available data, making our work compatible with open-sourcing, while most existing models rely on data which is either not publicly available or undocumented (e.g. \u201CBooks \u2013 2TB\u201D or \u201CSocial media conversations\u201D).
It\u2019s astounding to me that OpenAI, with all its recent attempts to take the moral high ground, cannot be bothered to share the training data they used. (For some further analyses of the recent OpenAI blogposts, see Scott Alexander, damning while trying to be as sympathetic as possible, and Emily Bender\u2019s Twitter thread, not bothering to be sympathetic.)
How to Train Your Dragon is a series of children's books written by British author Cressida Cowell. The books are set in a fictional Viking world, and focus on the experiences of protagonist Hiccup Horrendous Haddock the Third, as he overcomes obstacles on his journey of "becoming a hero, the hard way". The books were published by Hodder Children's Books in the UK and by Little, Brown and Company in the US. The first book was published in 2003 and the 12th and final one in 2015. By 2015, the series had sold more than seven million copies around the world.[1] The books have subsequently been adapted into a media franchise consisting of three animated feature films, several television series and other media, all produced by DreamWorks Animation.
Cowell has published twelve full novels, based around the adventures of a young Viking named Hiccup Horrendous Haddock the Third. The first book was published in 2003, and the last book was released in 2015. All the books have titles based around an instruction guide.
Cowell published the supplementary spin-off stories as part of the series: The Day of the Dreader is a short story published in 2012[3] and the novella How to Train Your Viking was published as part of World Book Day 2006 and is claimed to be written by the dragon Toothless and translated by Cowell. Both of these titles were published in the US as part of the May 2014 How to Train Your Dragon Special Edition: With Brand New Short Stories![4] paperback movie tie-in edition. A picture book, Hiccup the Seasick Viking published in 2000, is not considered a part of the series despite featuring the same character of Hiccup. An illustrative guide to the dragon world, titled The Incomplete Book of Dragons, was released in June 2014 (in the US it is titled The Complete Book of Dragons).[5][6][7]
The first book in the series follows Hiccup as he captures a dragon as a rite of passage and attempts to train him so that he will not be exiled from Berk, as is its tradition for young Viking boys on this island. Led by their teacher, Gobber the Belch, Hiccup manages to catch a small dragon, whom he names Toothless, and attempts to train it through his own methods. The Viking students are told to read a book called 'How to Train Your Dragon' by Professor Yobbish, but the book was found to have only one page and was therefore unhelpful. The Viking students must have their dragons trained before the Thor'sday Thursday Celebration in order to become proper Vikings. During this celebration, Toothless offends another dragon and a fight ensues between all the dragons. As this is seen as failure to train the dragons correctly, all the boys are sentenced to exile, but allowed to stay one night while a storm rages. During the storm, two Sea Dragons are washed up on the shore of the island and one eats the other, and seems a threat to the Vikings. Hiccup is chosen to negotiate with the dragon, who has eaten one of the other dragons. However, the dragon, who calls himself the Green Death, says he is going to kill all the Vikings. While the village elders argue over how to attack the Green Death, Hiccup, the boys and their dragons start a fight between the Green Death and the other sea dragon, the Purple Death, resulting in Hiccup nearly being swallowed and having to be rescued by Toothless, killing both sea dragons in the process. Hiccup and Toothless become heroes because of their bravery.
Hiccup finds the coffin of Grimbeard the Ghastly, the greatest pirate of all times and Hiccup's great-great-grandfather. The coffin contains a living man, named Alvin the Treacherous, who hides his true identity under the name Alvin the Poor-but-Honest-Farmer. Alvin tells the Hooligans that he was locked in the coffin and sent out to sea by "some very rude people". He claims that he found a riddle that tells how to find the treasure of Grimbeard the Ghastly and a map that depicts the island where the treasure is buried.
The Hooligans travel to the Isle of the Skullions, where they believe the treasure is hidden. Snotlout finds a small chest containing Grimbeard's famous sword, the Stormblade. Opening the chest triggers a trap which wakes up the Skullion dragons which inhabit the island. A brief fight with the Skullions ensues, a few of the Skullions are killed, and the Vikings escape with the treasure. Upon nearing Berk, the Hooligans begin fighting over the treasure, and as they fight, they are ambushed by a group of cannibalistic Vikings known as the Outcasts. Alvin reveals himself to be the Outcast's chief, and another battle starts. The ship soon catches fire and sinks. Everyone abandons it, climbing on the Outcasts' ship to continue the battle, but Hiccup, Toothless, Fishlegs, and Alvin are trapped underneath the capsized ship, kept alive with an air pocket.
As the air pocket is finally destroyed by the weight of the capsized ship, Toothless finds an underwater cavern filled with air. As the party begins to explore the cavern they come to a door and upon opening it, they find the real treasure of Grimbeard the Ghastly. Alvin turns on Hiccup to kill him using the Stormblade. In the sword-fight that follows, Alvin is eaten by a Monstrous Strangulator, an octopus-like creature which guards the treasure. As it comes next for Hiccup, Hiccup tricks the creature into injecting itself with its own poison. The creature's nervous system is destroyed and they escape, but Hiccup decides that the world is not ready for the treasure, and leaves it behind.
Hiccup and Fishlegs get lost while out at sea. Toothless is caught by Romans, while Hiccup hides and tries to figure out a way to free him and escape. Hiccup causes a distraction and escapes with Fishlegs, but not before losing half of his How to Speak Dragonese notebook. In the morning, Hiccup realizes he took home a Nanodragon from the Roman Ship. The Nanodragon is, in fact, the king of the Nanodragons and self-proclaimed 'living god', Ziggerastica. For saving his life, Ziggerastica tells Hiccup he can call his name and he will come.
During another lesson, Hiccup and Fishlegs are again kidnapped and taken to Fort Sinister, a base of operations for the Roman Empire. Once there, Hiccup recognizes his old enemy Alvin the Treacherous. Fishlegs and Hiccup are imprisoned along with the young female Bog-Burglar heir, Camicazi. Toothless is eventually allowed to stay with Hiccup. After several failed attempts at escape, Hiccup calls for Ziggerastica and comes up with a plan of escape on Saturn's Day.
When the festival day arrives, the gang are put on a ship and sail into the gladiatorial arena, where they are to fight deadly Sharkworms, rather than a traditional fight on land. However, due to Hiccup's planning with the nanodragon king, Hiccup appears to fly into the air like a God and 'blow' the cage roof off. The stunned Fat Consul gives him his Roman Shield, while Alvin the Treacherous gives Hiccup back his How to Speak Dragonese book. The arena erupts in chaos and the children escape in a Roman Observation Balloon. Alvin manages to get his hand-hook lodged into the balloon basket, but the children are able to twist him off, and he falls into Sharkworm-infested waters.They fly home in the balloon, then crash land onto the ships of Stoick and Big-Boobied Bertha. The two tribes make amends and return to their homes. Unbeknownst to Hiccup, Alvin had planted a Venomous Vorpent within the pages of his book, and it stings someone.
9d9a05e021