Dispatches 005: Musings on FI's Vision Weekend

We sure live in interesting times.

Transition periods are strange. So is living early on the takeoff curve of multiple general purpose technologies. AI, biotech, robotics, on the list goes. GPTs have always been around, but never this many and never in such quick succession. Industrial revolutions always spawn a whole gamut of ripple effects and adjacent advances around themselves, but this is closer to several concurrent paradigm shifts.

Title of the century: everything everywhere all at once.

Progress has become a system. I’m not sure who coined this first, but it has, both in the way it happens and how we shape it.

On one end, there is the deepening interconnection of technology with science, policy, governance, civil society. Something like current generation AI simply has too many knock-on effects for it to remain entirely a subject for the tech sector. It’s sociotechnical, socioeconomicpolitical. We don’t really have the adjectives. Think of the second- and third-order effects of advanced autonomous systems on the economy, employment, sector structure, politics, incomes, on the list goes. Advances in longevity have similar ripple effects; robotics, likewise.

On the other, we’re getting better and better and seeding progress. Accumulated collective learning, perhaps. The crop of organisations set up in the past ten years alone is impressive: ARIA, SPRIN-D, RenPhil, Convergent Research, a set of new FROs and the FRO as a form itself, and so on. Predating this wave, startup creation has also become more structured, with dozens of incubators, accelerators and fellowships being set up. Perhaps hundreds. Startups ecosystems are a thing now, a concept, not that the basic idea of agglomeration of knowledge is itself new — but our toolkit for speeding it up is.

It’s all cause-response perhaps. Or just two parallel, interlinked trajectories.

Our technology is getting more complex and is driving many more ripple effects. We are also getting better at generating the structured chaos that is innovation, from problem selection to funding to scaling.

Indeed an interesting time to be alive.

I’ve spent the weekend in-between the Barbican and Tate Modern in the company of ARIA, SPRIN-D, RenPhil, and a who’s who of the deep tech ecosystem across the UK and beyond. Foresight Institute have brought their first Vision Weekend to the UK. As far as conferences go, this is about as long-horizon as conversations get.

Naturally, I’ve been left with much to think about. Some thoughts below.

AI as Meta-Technology

Hardly the hottest take of the decade, but AI is having a massive impact on all kinds of sectors.

We pay a lot of attention to consumer-facing and enterprise products — new Claude, new version of GPT, and so on. (I’m experimenting with Fable 5 myself in-between writing this.)

What stood out to me throughout all the different talks — on bio, neurotech, energy — is how multifaceted the applications of AI have been. Take just Google DeepMind’s biological foundation models: AlphaMissence, AlphaGenome, and the much-talked-about AlphaFold. These and many others — Arc Institute's open Evo2, EvolutionaryScale's protein models, etc — are paving the way for a kind of computational biology engine. BioCompute infra. In effect, enabling technologies that are speeding up developments in other areas.

The crops of deep learning are many indeed.

The interesting thing here is the positive feedback loop. Further advances in foundation models are likely to translate into accelerating progress in other sectors. It’s not magic and neither is it instant; physical AI for robotics or biological models are all dedicated developments that take time, strands of LLMs or not. Sometimes the underlying technology isn’t even the same (As with World Models, them being a different strand of AI). Still, the point is that advances in AI are a general accelerant.

There’s a similar halo effect with funding. The amount of cash flowing into the AI infrastructure buildout is so massive that it is effectively creating supporting markets: better compute, better cooling, specialised inference chips, and so on.

Precarious, perhaps. Hyper-concentration. Having this much of the economy effectively funnelled into a single direction is itself a systemic risk. Last I checked, South Korea’s KOSPI alone is something like 50% driven by just two companies: SK Hynix and Samsung Electronics. The S&P 500 is in a similar boat with overwhelming concentration into the top 7 stocks.

A risk, then, but also a somewhat natural result of a General Purpose Technology that is slowly transforming society. The best play is probably like a counter-Dutch Disease strategy: ride the wave, diversify, guard against deflation in case there is a deflation.

A broad basket of programs, many strands of funding, decentralisation. Support other sectors alongside AI, harness its advances.

Interlinked, Interlinked

The sheer scale of impact being driven by the crop of technologies is of a different magnitude. Which begs the question: how do we deal with that? What does ‘deal with that‘ even mean in this context?

I think there is a reason you see a lot of very different kinds of people at tech events these days, as with Vision Weekend here. Policy, science, technology. Founders, funders, foresight and… and I don’t have a neat rhyme but point stands.

It’s a shared conversation, and it’s a shared conversation for a reason.

Technology always has some kind of transformation aspect to it. That isn’t new. After all, it’s like rewriting the firmware of reality in some sense. Oceanic travel made reliable voyages between continents possible. The airplane cut that by a hundredfold. That changes how ideas spread, how goods travel. It introduces a new opportunity curve to those who can afford it; and then your usual diffusion and adoption trajectory as the unit economics change.

Point being, the impact of technology isn’t new. Not even the scale necessarily.

It’s the speed and the parallelism.

Consider for a moment how the impact of AI on the job market is going to play out. It’s already eating into entry level roles. Increases in productivity mean organisational structures will change. Who gets hired, for which roles, and for how many positions. With a large enough effect, that translates into rising unemployment, which hits income tax which hits pensions which hits living standards. Cascade effects. And that’s just one example. Mind you, I’m not saying this to be a doomer — there are many benefits here as well.

Except that for the benefits to translate into improved well-being, we need to know how to respond. Will we need UBI? Do we deploy funding towards careers and reskilling programs? Do we support research into interpretability so that humans can better steer these advanced systems?

That is the sort of work that happens as the boundaries. Scientists and technologists working with policymakers. Economists working closely with AI labs. Cross-sectional communities on the ground ideating and experimenting with what these systems can do, what can be built, what the new collection of emerging roles looks like and what organisations can become.

I’m biased because I adore interconnection with all my heart — it’s a hobby — but I do genuinely believe this is the response we will need. A part of it, at least. And not just in AI. The new bio stack will be similarly transformational. Robotics in industry will change things slowly but surely. So on.

More specifically, understanding, exploring, modelling, and planning for the impact of transformation technologies has to be an ongoing process happening alongside technological progress. This isn’t even about regulation vs deregulation per se, I’m not qualified to speak on that discussion as much. Rather, this is about putting options on the table and designing responses before they are needed to prevent crisis management. Anticipatory governance of sorts.

No, we will not know everything ahead of time and not everything can be foreseen. But we can make a start. There’s no rule saying we can’t work out how to make tech benefit everyone as it is being deployed, bumpy road or not.

Funding, Seeding Progress, and Emergent Innovation

I am personally fascinated by ways of driving long-run impact through innovation. Second, third degree effects, positive cascade reactions. A good 50% of Meridian as a project is about this: working out designs & deployment strategies for innovation catalysts.

The tricky thing about funding innovation is that you are always dealing with probabilities.

Probability of success, failure, degree of impact, impact ROI per dollar spent, N of people impacted, potential knock-on effects, potential fallbacks. A kind of composite beneficial effect calculation. Long-Run Estimated Impact, L-REI.

How, then, do we deploy funding, direct programs, and design organisations in such a way so as to drive the highest net positive — across sectors, geographies, modalities?

The genius of ARPAs, I think, is that the state is perhaps the only entity capable of absorbing all positive externalities. Take the impact of funding longevity bio. A classic deep tech startup would have to capture it via its business model, whether that’s direct sales, licensing, or something else. With an ARPA, however, it’s much simpler: crunch the numbers on economic gains per median healthspan / lifespan gain across the population and you get your cost justification. The effect is very much real, it just has to bounce a little while before it reappears somewhere.

That is half the trick with funding the frontier: how to design the model such that either the entity itself absorbs the resulting gains, or the funder does.

It is why funding areas such as Responsible AI seems intuitively important and crucial to society, but creating a model for it is harder. Tracing the externalities is tricky. Who has the calculation on cutting existential risk by 10%? What’s the number on that? Trillions? Quintillions, applying a discounted rate on all of future human history?

We need organisations enabled and empowered to do such work. Whether they’re nodes, networks, labs, some kind of open exploratory studios or project networks. Structuring these is one of the emerging challenges we have to keep working on.

I suspect we will see many more organisations like Renaissance Philanthropy — plus RenPhil itself growing. They’re coordinated mobilisation mechanisms, brilliantly effective in RenPhil’s case considering how young it is. That’s part of the connective tissue we will need to drive the next generation of progress; ways of shaping programs that connect funders, driving technological development & frontier science while bringing in policy & foresight specialists to map their effects and trajectories.

Infrastructure of Technology

One particularly tricky area is what to do with infrastructure.

Take scientific publishing. It is already buckling under the flood of AI-produced papers, new hypotheses and not enough review capacity. Something similar is happening with clinical trials; medial ethics and safety exist for a reason, and yet the system simply is not designed to handle the volume of drug candidates being directed at it.

The point isn’t that these systems are old and terrible. They’re a consensus equilibrium that has worked. Rather, it’s how we deal with infrastructure no one really owns that is the challenge.

Reforms always happen under pressure, or tend to at least. As with AI economics, what we could perhaps use are organisations that work on putting forth proposals of what those reforms could be and transition plans.

I see this as a different kind of translation problems, perhaps: adapting infrastructure around technologies that rewrite the logic behind that infrastructure. Once again, with no clear ownership and hard-to-compute gains, this will be another thorny challenge.

My guess is what we will see are spearheads. New models of scientific collectives similar to what FROs are doing, re-organising around “moonshot science“ with its own delivery timelines and deployment vectors. That is perhaps less likely for bio — and with good reason — but we might see AI-driven drug discovery and design cycles hooking directly into the regulatory process and manufacturing to accelerate the upstream portions of the drug-to-market trajectory.

Also, while not the focus here, I suspect the focus of the economy itself — or rather the knowledge work economy specifically — will shift somewhat to exploratory work. This is of course deeply speculative, but if AI continues to eat away at knowledge work and software becomes massively faster to build, the effort will instead flow into other areas. It is part of why we are seeing more Deep Technology startups and funding, I suspect. Some of that reallocated effort might end up being these thorny problems, suddenly reprioritised as higher-ROI. Not a solution exactly, but helpful shifts in trends.

Moving Forward

Transformative technology is here to stay, probably long into the 21st century and right past it. Thankfully, there is much collective learning happening in how we advance, steer, govern and seed technology. New organisational forms, new coordination mechanisms, new funding models.

Work in this direction should continue. Progress studies, as it goes. Meta-innovation.

I’m glad events like the Vision Weekend exist. We need those conversations at the boundaries of disciplines. Convening spaces, catalysts, across all levels of seniority, across the world, across domains.

This is what Meridian itself is about, of course, comparatively small operation that it is. Cataloguing models, understanding them, designing new ones. Modelling futures and figuring out how to seed them.

Much to learn, much to explore. We’re hardly three decades into the century after all.