Joining Epoch AI and linkpost

Getting a real job

On Monday the 25th of May I will be joining Epoch AI as a senior researcher, with the remit of managing the Epoch Capabilities Index (subject of many previous substack posts, which I suppose makes this rather forseeable) as well as other projects tracking AI progress trends.

I enjoyed working as an independent statistical consultant, but over the past year I’ve become increasing excited about working on understanding AI capabilities and progress trends, and I think joining Epoch will be an excellent opportunity for this.

I have the (much delayed) third and final post in my series on the ECI largely written and will aim to publish it this week before I start at Epoch. The analysis for it was conducted before I started working with Epoch, so hopefully it doesn’t come across as too biased!

I will plan to still write posts on this substack (I already have a draft folder twice as large as the number of published posts), but this will likely be split with work I will publish directly through Epoch, that I will try to linkpost where appropriate.

I also want to use this as an opportunity to share some links to work I have done in the past few months with Epoch and METR:

• METR: Impact of modelling assumptions on time horizon results

  • This was the culmination of a few months of work on the model underlying time horizon for METR, and understanding how it (increasingly) impacts the results.

• Epoch AI: Have AI Capabilities Accelerated?

  • This makes the case that AI progress rates seemed to accelerate with reasoning models, and that there is now enough evidence for this to be pretty conclusive.

• Epoch AI: Claude overperforms at software engineering and underperforms at math (and the domain-specific ECI in general)

  • I worked with Epoch to develop a version of the ECI that only uses benchmarks from a specific domain (like maths or SWE), and then wrote up a single application of this showing the Claude models systematically overperform at SWE relative to other LLMs.