Win Alignment
Francesca Cruiz
Right now alignment research progress is mostly bottlenecked by talent: we have a lot more ideas and projects we are excited about than people who can pursue them. Automating more and more parts of alignment research would unblock the talent bottleneck because it would enable organizations to transform compute (and thus capital) into alignment progress.
Our day-to-day work on the Alignment Team at OpenAI involves a variety of tasks that are ripe for automation to varying degrees. For example, recently there has been a lot of progress on automating the writing of code, and we could leverage these capabilities for an alignment research assistant. A sufficiently capable generative language model trained on alignment research and discussions scraped from the web should also be helpful in having discussions and generating ideas about alignment.1
Tl;dr: My currently favored approach to solving the alignment problem: automating alignment research using sufficiently aligned AI systems. It doesn\u2019t require humans to solve all alignment problems themselves, and can ultimately help bootstrap better alignment solutions.
The space of all problems is truly vast and the space of problems that humanity is currently able to solve is pretty tiny in comparison. This means that today we just aren\u2019t in a position to solve most problems. This is a key motivation to work on AI: progress in AI will significantly expand the space of problems humanity can solve.
Maybe a once-and-for-all solution to the alignment problem is located in the space of problems humans can solve. But maybe not. By trying to solve the whole problem, we might be trying to get something that isn\u2019t within our reach. Instead, we can pursue a less ambitious goal that can still ultimately lead us to a solution, a minimal viable product (MVP) for alignment:
While we clearly need AI systems that are more capable than today\u2019s to do good alignment research, we don\u2019t actually need a system that is smarter than the best human experts. Compared to humans, machines have a bunch of advantages: they can work a lot faster than humans and try more things in parallel. Moreover, evaluation is often easier than generation: I find it a lot easier to evaluate whether an alignment idea is promising than to come up with a good one. What would the best out of 100 alignment ideas sampled from a future language model look like? What about the best out of 1,000,000?
Essentially, an alignment MVP can allow us to bootstrap all other solutions to alignment problems and eventually allow us to reach a full solution (if it exists). Ultimately we shouldn\u2019t care how we arrive at a solution to the alignment problem, whether manually or using automation.
This goal is less ambitious than some other alignment research agendas since it does not require us to have a solution to all problems that come up when trying to align AI systems, including far-future ones. Don\u2019t get me wrong\u2013it\u2019s still a very ambitious project.
Importantly, this approach doesn\u2019t assume that humans (by themselves) will have any fundamentally new ideas about alignment. If it\u2019s true that humans can recognize (possibly with trusted AI assistance) a good alignment proposal as such, then existing techniques like reinforcement learning from human feedback and ideas like recursive reward modeling might suffice to make such a minimal viable product sufficiently aligned, given sufficiently capable AI systems. Even more, the converse is true too: if humans are incapable of evaluating the quality of an alignment proposal (even with AI assistance), then this will also obstruct adoption of alignment proposals humans come up with.
Moreover, an alignment MVP doesn\u2019t require us to fully align a generally capable AI system. Being very helpful for alignment research doesn\u2019t require interaction with the real world or the open internet. It also doesn\u2019t require the model to tell us everything it knows about a given topic, as long as it sometimes tells us helpful information (it doesn\u2019t need to be fully aligned).
One of the main downsides of this approach is that it\u2019s plausible that nearby in design space to an alignment MVP is a system that accelerates AI progress faster than alignment progress. In practice, most of the time spent on empirical alignment research is similar to time spent on ML research. This could mean that by the time our system makes significant alignment research contributions, ML research itself is already getting automated. It seems very likely to me that the future of ML research will end up looking that way anyway, and that this is mostly bottlenecked by the capabilities of our models. If that\u2019s true, then working on an alignment MVP wouldn\u2019t impact the overall rate of AI progress.
Needless to say, today\u2019s AI is nowhere near being able to do alignment research better than humans. Nevertheless, I understand this as a general strategy that we make iterative progress on today, which can ultimately lead to solutions to all future alignment problems.
Tire alignment, also known as wheel alignment, can help your tires perform properly and help them last longer. It can also improve handling and keep your vehicle from pulling in one direction or vibrating strangely on the road.
There are a couple ways to tell if your car needs a tire alignment. If you've noticed one or more of these indicators, you should have your alignment checked by a licensed service technician immediately.
This is the inward or outward angle of the tire when viewed from the front of the vehicle. Too much inward or outward tilt, also known as negative and positive camber, respectively, indicates improper alignment and will need to be adjusted. Worn bearings, ball joints, and other wheel-suspension parts may contribute to camber misalignment.
This happens when one side of your tread blocks wears down more quickly than the other in a circumferential direction. When you run your hand over the tread, it will look and feel like saw teeth when viewed from the side. Heel/toe wear could be a sign of under inflation and/or lack of rotation.
To begin balancing your tires, a technician will mount them on the correct rims and adjust the pressure to optimal inflation. Then each tire goes on the center bore of a balancing machine. The machine spins the tire at a high speed to measure the wheel/tire combination imbalance. It signals how much weight the tech should add to balance out the tire and the areas where said weight is needed.
Tire balancing is essential for proper tire care for the same reason as wheel alignment: prevention of premature tread wear. Having tires aligned and balanced every 5,000 to 6,000 miles can help maximize their lifespan and overall performance.
I'm trying to do something that I can't figure out how to do, I'm not even sure it's even possible. I'm trying to split the alignment on a single line so that a few words are left aligned and then a few words are right aligned, leaving a large gap in between. I have it set up right now with tabs, but the second set of words on the right-hand side are a different length and does not have the effect that's needed for the document. There are numerous lines that have this formatting and even thinking about the time required to set up customized tabs and spacing for each individual line threatens to give me an anxiety attack. Is there a way to split the alignment on a single line and I'm just missing how? Thank you in advance!
You could also try splitting the game into 3 columns and using Breaks (e.g. Continuous Break) to help manage the content, but it is a bit messy and will require some trial and error to see if that works for you.
Thank you both for replying. I just figured out a workaround, at 5:45 am, that actually is effective and time efficient. Each line that I am needing to split the alignment on is separated by one or more paragraphs, which was making the issue even more tricky.
What I ended up doing was highlighting the line that I wanted to have split alignment, went to Insert > Table > Insert Table. Since I already had the to sections split by a single tab, this created a 2 cell table. I then, under Table Design, chose No Borders, moved over to Table Layout > Cell Margins and reduced both the left and right margins to 0". This left me with the line I was editing looking like a normal line, not a table. Lastly, I chose the second cell and, under Home, clicked Align Right.
You just need to select the lines/paragraphs or if you want to split the whole document, you do not need to select anything. Then go to Layout and select Columns. From here you can split everything you have selected or the whole document into as many parts as you would like!
I've written a 3D vector class using a lot of SSE compiler intrinsics. Everything worked fine until I started to instatiate classes having the 3D vector as a member with new. I experienced odd crashes in release mode but not in debug mode and the other way around.
That seemed to solve the issue at first. But after changing some code my program started to crash in odd ways again. I searched the web some more and found a blog article. I tried what the author, Ernst Hot, did to solve the problem and it works for me too. I added new and delete operators to my classes like this:
c80f0f1006