Research update: Obsidian, Wikipedia, RL frameworks
Ram Rachum
Hi everyone!
It's been an exciting month for me. It was sad to put Google behind me, and there's a lot of ambiguity ahead of me. This has been my first month as a full-time researcher. I've produced some content that I'll link to below, but this is still a period of adjustment. It's especially difficult that I spend less of my time creating things and more of my time reading and looking things up. I find myself cooking for myself more than I used to, just because I have an urge to actually create things during the day. I hope that in the next few months I could channel that energy into making interesting experiments.
Retrospective on goals for March
In the March update email I outlined a list of goals. Here's my update on these goals:
Make my talk available on YouTube: ā Done.
My talk on YouTube. Some of our new subscribers have joined because they've seen that talk linked on my Facebook. I hope that I could this YouTube talk could serve as a good "business card" for me to present my goals when discussing my research with people.
Try to get a grant: ā· Postponed
I've been thinking about this one a lot. I was also offered a 50% position at a startup that sells an ML product to hospitals. This is a tempting offer, because it means I could work on my research for many years without financial worries. I turned down the offer, and also decided to not think about grants until April, May or even June.
My reasoning is that money isn't the weak link in the chain right now. I'm overwhelmed with my new life as a researcher. There are crucial things I need to learn and big questions I need to have good answers for in order to make my research work more effective and comfortable. Once I do apply for grants, I expect the process to be intense and emotionally draining. I can afford giving myself two or three months to get in the zone before I undertake this challenge.
I did come upon a few more organizations that might want to give me a grant. I haven't applied for them, but I have them listed out in a file, so I could approach all of them when the time is right.
Set up research infrastructure: ā Done and published
Oh, this is such nerdy fun. I'm grateful to my brother for teaching me how to use Obsidian. It's so hilariously exciting to have a tool for dumping out all of my thoughts and arranging them in a network.
Over the last year I've had so many discussions with Edgar where I solidified my positions about some concepts and approaches. It's important to have these positions written down and detailed. I might change my mind on some of these positions, and if I do, it'll be critical to know exactly why I held them.
The cherry on top is that Obsidian lets you easily publish your notes online, and I've done just that! Here's the home page of my knowledge base. There are lots of links there that you can follow. I've written notes about how my approach is different, autocurricula, the abstraction of agents and more. I'm keeping files for all my meetings with people in the private part of my knowledge base, and linking those meetings to papers I should read and to notes that I have about them.
I'll keep updating this knowledge base regularly for the indefinite future. This is important and fun; it's so much fun, that at some point I'll need to reevaluate whether I should spend less time maintaining this library of my mind, and more time innovating and coding. But for now, I can continue on the present course.
If you have feedback on any of the notes I've written in my knowledge base, please email it to me!
Take a vacation and clear my mind: š Just getting started!
I'm writing this email from an airplane flying over the Mediterranean. I bought a one-way ticket to London. I'm thinking I'll spend a week there, then 2 weeks in New York. But I'm keeping it open. I'm excited... I haven't taken a vacation abroad since before Covid started.
Stuff I've done beyond the goals
I've been talking about my research with a bunch of people. Some of them were familiar with Multi-Agent Reinforcement Learning and some weren't. When I explained to them about the field, I tried to look it up on Wikipedia to see whether there's any interesting content there that I could send them to. There were only two paragraphs, and under a wrong name.
Since part of my mission is to make MARL grow from a small niche of RL to a more popular field, I figured, why not give that Wikipedia page an overhaul?
I got permission from the admins to rename the article. I added a bunch of sections and I edited some of the existing text. This is the new article: Multi-Agent Reinforcement Learning.
If you find any mistakes or want to add anything, feel free to do that! You can also email me to discuss. I'm especially interested in finding relevant charts and animations that I can include there; that requires CC licensing, which is difficult for most academic texts.
My goals for April
Keep having fun on my vacation. I'm going to spend most of this month on vacation. Having fun and relaxing is my first priority. This is why there's only one other goal for what's left of this month.
Experiment with existing RL frameworks: I suspect that last year I rushed too quickly into writing my own code for running MARL experiments, such as GridRoyale and Marley. I did that because I didn't connect with the way that the existing frameworks are written, but I haven't done a good enough job in listing out my frustrations with them. What I should do is try out some of the popular frameworks: Gym, PettingZoo, Stable Baselines 3 and Griddly. I should go through the tutorials and try to run basic experiments. Right now I'm watching this tutorial.
Maybe my conclusions would be that these frameworks are a good choice for my research, which means I would save a lot of development time and spend that time researching. Maybe I'll find that I have problems with them, but this time I'm going to write these problems out so I could discuss them with other researchers.
That's it for now. See you in May!
Ram.