startups x alignment

observations from the last month

i spent my winter on the yc iap program, which was a mini version of the batch long yc accelerator. i’ve also recently gotten a lot more involved with mit’s ai alignment club, and i just came back from the organizer retreat. i couldn’t help but notice how similar the principles behind both groups of people are: those who found ai startups and those who work in ai alignment are fundamentally similar and differ only in motivation. i believe these characteristics will make them among the last to be replaced by ai. ¹

1. moving fast

both groups move on extremely fast timelines. founders race against all other competitors to occupy the market, and those in alignment race to prevent the threats of agi. when your timeline is this short, around the magnitude of 5 years, your mindset towards things drastically changes. in particular, both groups seem to value exploratory behavior over incremental improvements, because the world currently changes too fast for small tweaks to survive and stay relevant.

here also comes the notion of being “agentic”² and making things happen now. the idea of having the mindset of asking for forgiveness rather than waiting for permission:³ send that email and reach out to that person, organize that event, build the prototype, because it means you’ve tried more.

2. justifying the means with the end

the work that founders deal with is often quite monotonous and not intellectually stimulating. a lot of the work is quite unglamorous and “grindy”, and most founders wear many hats around the company and likely work jobs they don’t particularly enjoy. similarly, many people deeply embedded in alignment, even those who are highly technical, choose to work in policy or evals because it’s the highest-impact area they could be in. ⁴

however, they still choose to work long hours and often find a lot of fulfillment. i find it interesting that a large portion of founders and those in alignment have their own personal blog (and many are on substack too). they’re highly motivated by their view of what an ideal world could look like, and will do whatever it takes to make that a reality.

3. solving unstructured problems

i think this is the most important characteristic the groups share, and what makes them less replaceable. in particular, this is what differentiates these careers from quant/swe, which are quite well trodden and have clear pipelines; if you grind out whatever problems and take whatever courses that people tell you to do, you’ll probably be decently successful.

this is not true for founders or those in ai alignment. ai alignment is a very young field, and although there are programs such as MATS and ARENA that add some structure, most of the work is pretty sparse and new. with founders, there are accelerator programs that feed advice and VCs that connect them with other people, but every company is different and requires its own judgement.

ai is currently very good at solving well structured problems, but to be successful in these fields, one needs to be very good at dealing with these less defined spaces.⁵

---

i believe startup founders and people in ai alignment would be great collaborators, as long as the goal is set up through thoughtfully designed incentives for both sides.

1. in comparison to jobs such as swe, quant, etc.

2. i’ve probably heard this word every 15 minutes at both events

3. thank you lydia for this fire quote!

4. and many do not find it as interesting as interpretability research

5. i think it’s worth thinking about which careers and jobs will still be relevant in a few years, and what types of people they select for.