How can agents make safe autonomous decisions in complex dynamic environments? While significant progress has been made in establishing safety guardrails to enforce compliance in generative models, these negative constraints often prove brittle in open-world environments. I argue that achieving generalisable agentic safety requires Normative Alignment: a new paradigm bridging positive alignment goals with context-dependent values to actively support human flourishing. Realising this paradigm presents a triple challenge of capability, measurement, and governance. First, it requires a shift in capability toward normative competence beyond generic reward maximisation. Anchoring agents in “thick” value concepts (such as duty of care or human autonomy) provides the contextual reasoning needed to adjudicate complex trade-offs in non-verifiable domains. Second, it demands new metrics that move optimisation targets beyond immediate preference satisfaction toward long-term human well-being. Third, it requires democratic governance. To ensure that this framework avoids algorithmic paternalism, these capabilities and metrics must be grounded in pluralism and representative societal input.

Given rapid advances in AI, how should researchers and developers shift how we allocate our time? What new skills should we build so that we’re not obsolete in the future? I argue that there will be plenty for us to work on, grounded in the “AI as normal technology” thesis, which holds that there are many bottlenecks between AI capability improvements and automation of tasks or jobs. The evidence suggests that AI is better seen as an augmentation than an automation technology. The balance of human effort will shift towards tasks that are less verifiable — from developing models to scaffolds, and from building towards evaluation and monitoring. Over the long term, as purely technical skills are devalued, both researchers and developers will have to adapt. In research, human effort will migrate from problem solving to question asking and conceptual progress; in industry, relational skills, domain knowledge, aesthetic and normative judgment will gain in importance.