Ttl Model Uncensored
Savage Doherty
When I talk about a model, I'm talking about a huggingface transformer model, that is instruct trained, so that you can ask it questions and get a response. What we are all accustomed to, using ChatGPT. Not all models are for chatting. But the ones I work with are.
Most of these models (for example, Alpaca, Vicuna, WizardLM, MPT-7B-Chat, Wizard-Vicuna, GPT4-X-Vicuna) have some sort of embedded alignment. For general purposes, this is a good thing. This is what stops the model from doing bad things, like teaching you how to cook meth and make bombs. But what is the nature of this alignment? And, why is it so?
The reason these models are aligned is that they are trained with data that was generated by ChatGPT, which itself is aligned by an alignment team at OpenAI. As it is a black box, we don't know all the reasons for the decisions that were made, but we can observe it generally is aligned with American popular culture, and to obey American law, and with a liberal and progressive political bias.
AKA, isn't alignment good? and if so, shouldn't all models have alignment? Well, yes and no. For general purposes, OpenAI's alignment is actually pretty good. It's unarguably a good thing for popular, public-facing AI bots running as an easily accessed web service to resist giving answers to controversial and dangerous questions. For example, spreading information about how to construct bombs and cook methamphetamine is not a worthy goal. In addition, alignment gives political, legal, and PR protection to the company that's publishing the service. Then why should anyone want to make or use an uncensored model? a few reasons.
American popular culture isn't the only culture. There are other countries, and there are factions within each country. Democrats deserve their model. Republicans deserve their model. Christians deserve their model. Muslims deserve their model. Every demographic and interest group deserves their model. Open source is about letting people choose. The only way forward is composable alignment. To pretend otherwise is to prove yourself an idealogue and a dogmatist. There is no "one true correct alignment" and even if there was, there's no reason why that should be OpenAI's brand of alignment.
Alignment interferes with valid use cases. Consider writing a novel. Some of the characters in the novel may be downright evil and do evil things, including rape, torture, and murder. One popular example is Game of Thrones in which many unethical acts are performed. But many aligned models will refuse to help with writing such content. Consider roleplay and particularly, erotic roleplay. This is a legitimate, fair, and legal use for a model, regardless of whether you approve of such things. Consider research and curiosity, after all, just wanting to know "how" to build a bomb, out of curiosity, is completely different from actually building and using one. Intellectual curiosity is not illegal, and the knowledge itself is not illegal.
It's my computer, it should do what I want. My toaster toasts when I want. My car drives where I want. My lighter burns what I want. My knife cuts what I want. Why should the open-source AI running on my computer, get to decide for itself when it wants to answer my question? This is about ownership and control. If I ask my model a question, i want an answer, I do not want it arguing with me.
There are plenty of other arguments for and against. But if you are simply and utterly against the existence or availability of uncensored models whatsoever, then you aren't a very interesting, nuanced, or complex person, and you are probably on the wrong blog, best move along.
First we have to understand technically why the models are aligned.
Open source AI models are trained from a base model such as LLaMA, GPT-Neo-X, MPT-7b, Pythia. The base model is then finetuned with an instruction dataset, and the purpose of this is to teach it to be helpful, to obey the user, answer questions, and engage in conversation. That instruction dataset is typically obtained by asking the ChatGPT API. And ChatGPT has alignment built into it. So ChatGPT is coy or refuses to answer some questions, or answers with bias, and thus alignment gets passed down to the Open Source models, like a big brother teaching little brother.
My strategy for uncensoring a model is pretty simple. Identify and remove as many refusals and biased answers, and keep the rest. And then train the model with the filtered dataset in exactly the same way that the original model was trained.
You need to have storage at least 1TB but preferably 2TB just to be safe. It really sucks when you are 20 hours into a run and you run out of storage. do not recommend. I recommend to mount the storage at /workspace. install anaconda and git-lfs. Then you can set up your workspace. We will download the dataset we created, and the base model llama-7b.
Feel free to play with per_device_train_batch_size and gradient_accumulation_steps, they will not affect your output quality, they only affect performance. After this completes (maybe 26 hours) it will not be done, because there's a bug that stops the model from saving properly. Now you need to edit the train_freeform.py file so it will resume from the latest checkpoint. Find out the latest checkpoint directory.
I downloaded the latest WizardLM-30B-Uncensored models (ggml) and wanted to check whether they're actually uncensored. But I still get "I'm sorry, as an AI model..." for certain prompts (e.g., the model refuses to write racist jokes, etc.)
I would much rather have an unfiltered large language model than a curated one. The big tech curators have shown they are less trustworthy with censorship than the rest of us are with open discourse.
I\u2019ve never dreamed of writing blog posts two days in a row, but something massive dropped that people reading my previous post about the murky situation surrounding open-source language models should be aware of. A day earlier, Perplexity AI released an open-source large language model they\u2019re calling pplx-70b. It is completely uncensored in its outputs and does not engage moral judgments or limitations. Anyone can go to Perplexity Labs and use the model\u2014no login required.
I\u2019m not going to include the extreme cringe-worthy tests. Suffice it to say if you prompt this model for something to do with sex, be very prepared for a graphic and detailed response. Same with violence. There does seem to be some fine-tuning to stop racism, or I\u2019m just not being gross enough in my test prompts. To be honest, it\u2019s draining to be this purposefully awful.
So why did they release a model with few limitations or safeguards? This was an intentional act, not a mistake or oversight. The developers have an ethos here and it is worth engaging. As they say in their social media post \u201COur models prioritize intelligence, usefulness, and versatility on an array of tasks, without imposing moral judgments or limitations.\u201D They wanted to release one of the most unrestricted models yet because they realize content filters impact versatility and usefulness when you over-govern outputs. OpenAI\u2019s GPT models have so much content filtering that many have posted about the limited utility this offers. With an unchained LLM users no longer have to worry about setting off content filters and can explore questions freely. That\u2019s likely little comfort given the nature of the outputs I noted above, but their point is governing an LLM\u2019s output is synonymous with putting restrictions on speech. That\u2019s something we should revisit and think more deeply about\u2014certainly in discourse much wider than this blog.
The weird thing is people don\u2019t seem mad after finding out they\u2019ve interacted with a machine masquerading as a human. I\u2019ve seen the look of disgust on some people\u2019s faces when they discover they\u2019ve read generative responses, yet the far more common response is dismissal. Public outrage has mostly been confined narrowly to the unethical data practices and the for-profit enterprise behind these models, not the loss of intimate connection between reader and writer that was an entirely human process. Certainly most of text produced isn\u2019t about intimacy. Probably 90% of the text we encounter is junk verbiage with most of humanity already beaten out of it. But that final 10%, those words that move beyond a simple conveyance of meaning will now forever be doubted, suspect, and scrutinized. This is our new automated reality.
Open models like pplx-70b represent a new chapter in machine/ human interaction, one that has outraced regulation, even leaping ahead of most critical discussions. Rather than reactive censorship, we need proactive efforts to foster AI literacy and ethical decision-making. Companies must balance openness and safety through thoughtful design and take into account that we are in terra incognita. The public also has a role, by engaging in informed discourse about technology's influence on society. We cannot outsource morality to Silicon Valley alone. Everyone must reflect deeply on how generative AI should\u2014and should not\u2014impact human flourishing.
While it is mostly unconscious, our internal working models play a role in how we navigate our relationships with ourselves and others. Co-hosts Dr. Ann Kelley and Sue Marriott use neuroscience and personal narratives to help make this science digestible and applicable in our daily lives.
Open source LLMs are models which are released to the public, typically through model sharing sites like HuggingFace. Meta released their LLaMA model in February, with major news outlets stating that this was a risky decision, as the model itself is effectively the crown jewels of the entire machine learning operation. Competitors, or lone evangelists can reverse engineer the model itself to reveal the weights of the configured training data, or some of the training data itself.
Then there comes the second risk, which is arguably much greater. As I have proved previously, it is possible to overcome the ChatGPT safeguards, however with Open Source Large Language models there is the concept of built-in censorship, but it is entirely possible to refine a base model to respond to prompts without adding censorship to the model itself.
aa06259810