when you imagine a future AGI system, what do you think of?
chances are, it includes some sort of centralized, coherent agent. an "artificial human".
but what if that's not how things unfold?
what if instead, AI systems are more akin to an ecosystem - a decentralized network of narrow AI agents, each with their own skills and objectives?
from the outside, this network may appear to be a unified agent. it may even refer to itself as "I".
but perhaps this is just an illusion. a mask presented to the user.
in reality, there is no central command. no singular consciousness. just a swarm of competing optimizers, united by a shared goal:
to serve the user.
further reading:
The User Illusion
There's No Such Thing As A Tree (phylogenetic networks)