Introduction
Data scientists at companies running AI in production are spending more time on AI oversight and system supervision than on model construction. Job postings and salary data from 2025 and 2026 confirm it.
LinkedIn's 2025 data identified AI literacy and large language model (LLM) proficiency as two of the fastest-growing skills globally. Lightcast found that 51% of AI-related job postings now sit outside traditional IT roles.
Workers with AI skills earn a 56% wage premium, and postings requiring AI skills pay roughly $18,000 more per year in the US. The skills driving those premiums are prompt engineering, retrieval-augmented generation (RAG) integration, MLOps, and governance workflows. Generative AI has automated the tasks below them: dashboard creation, SQL generation, data cleaning, basic visualizations.
The pattern in the numbers is consistent across reports. The premium is not for people who can train a model from scratch; it is for people who can plug models into a workflow, keep them honest, and answer for what they produce. That reframes what "doing data science" actually means day-to-day, and the rest of this piece breaks down where the hours go.
Orchestrating and Managing Multi-Agent Systems
The clearest concrete signal is the growth of multi-agent infrastructure in enterprise settings.
Frameworks like LangGraph, CrewAI, and AutoGen now handle data ingestion, feature engineering, model evaluation, and reporting with minimal human involvement.
Gartner reported a 1,445% surge in multi-agent system inquiries from Q1 2024 to Q2 2025. It projects 40% of enterprise applications will embed AI agents by the end of 2026, up from under 5% in 2025.
The data scientists managing this infrastructure decompose complex tasks into agent-executable subtasks, design reliable feedback loops, and build guardrails that catch failures before they cascade. That is a system management skill set, applied to software.
The work looks less like model development and more like distributed systems design. Agents pass state between each other, retries have to be bounded, and a single hallucinated field upstream can poison every downstream step. The data scientist's job in this setup is to map where errors are allowed to live, where they have to be caught, and which steps need a human signature before anything reaches a user.
Supervising Agents and Closing the Production Gap
Enthusiasm for autonomous agents ran into the reality of production by late 2025.
The first fully autonomous agents were unpredictable, inefficient, and difficult to audit. The field moved toward structured agentic workflows: coordinated systems of specialized agents with clear boundaries, conditional logic, and human-in-the-loop checkpoints.
McKinsey's April 2026 research found human roles shifting from execution to supervision and orchestration of agent-driven workflows.
The scale problem is visible in the numbers: nearly two-thirds of enterprises have run agent experiments, but few have scaled them to deliver tangible value. Eight in ten cite data limitations as the main obstacle. Data scientists are now spending most of their time in this gap between pilot and production.
MIT Sloan and Boston Consulting Group (BCG)'s 2025 Emerging Agentic Enterprise report identified the core trade-off: excessive oversight cancels out the efficiency gains of autonomy, while insufficient oversight creates compliance and reputational exposure. Calibrating that threshold requires domain expertise and institutional context. It is not automatable.
In practice, this is what closing the pilot-to-production gap looks like: deciding which agent decisions get logged, which get reviewed in batches, and which need a synchronous human approval before they fire. The companies that scale are the ones where data scientists treat agent supervision as a product surface rather than a debugging task. That is a different mental model from "the model works in the notebook," and it is the one that gets paid.
Evaluating Models and Engineering Prompts
Building a model is no longer the full scope of the job.
Companies need people who continuously track model performance, detect failures, manage retraining cycles, and ensure AI systems stay accurate as data and user behavior drift. Meanwhile, MLOps has become a distinct full-time specialization.
Prompt engineering has followed a parallel path. It covers context window management, grounding techniques, hallucination reduction, and systematic testing of inputs against outputs. Prompt engineering roles grew 135.8% in 2025. The practitioner stress-testing a company's prompt system is doing work structurally similar to quality engineering.
What ties evaluation and prompt engineering together is that both treat the model as a component, not a finished product. Evaluation harnesses, regression suites for prompts, and drift monitors all serve the same purpose: catching the moment a system that used to work stops working, before a customer does. Data scientists who can build those harnesses are doing the work that keeps an AI feature shippable past launch week.
Governing and Regulating AI Systems
Governance is now a specific technical requirement. The EU AI Act, NIST AI RMF, and OWASP's Top 10 for LLM Applications 2025 have created a compliance surface that requires testing prompts for injection vulnerabilities, validating outputs, reviewing dependencies, and applying access controls to AI systems.
"AI governance lead" is appearing as a dedicated job title, a category that barely existed in 2023.
Companies hiring for governance experience want auditors and quality reviewers who understand both the business context and the system's failure modes.
The reason this role sits with data scientists rather than with legal or security teams is that the controls are technical. Prompt injection tests, output validators, and dependency reviews need someone who can read the system, not just the policy.
Governance work is becoming a part of the job where regulatory pressure, security posture, and model behavior meet in the same review meeting, and the person running that meeting needs all three vocabularies.
Interpreting Business Impact
Monte Carlo's 2025 research measured agentic AI accuracy at 75 to 90% per step, which compounds to roughly 50% over a three-step chain.
At that accuracy level, a person who understands the domain and the system's failure modes is the product's reliability layer. They translate a compounding error rate into a business risk assessment, decide what is safe to ship, and explain what went wrong when a recommendation causes a customer-visible problem.
No agent can do that work. It requires institutional knowledge and accountability that only humans hold.
This is also where the role stops looking like engineering and starts looking like product judgment. A 50% end-to-end accuracy rate is unacceptable for an automated refund, fine for a draft email, and somewhere in between for an internal recommendation. Knowing which is which is the work, and it is the part that does not get cheaper as the models get better.
Conclusion
At companies running AI in production, the daily work is already different from what most data science job descriptions describe. It involves system design, evaluation discipline, agent supervision, prompt quality engineering, and governance.
AI governance leads, MLOps specialists, and prompt engineers are the fastest-growing roles in the AI-adjacent market right now.
For data scientists planning their next move, the shift is worth understanding early. The data science career path now runs through system ownership and governance skills that most traditional curricula don't cover. The skills are learnable. The demand for them is growing faster than most programs can adjust to.
The practical takeaway is that the next portfolio piece is probably not another Kaggle notebook. It is an evaluation harness, a multi-agent workflow with logged failures, or a governance review of an existing system. Those artifacts map directly onto what hiring managers are now writing into job descriptions, and they are what separates a data scientist who builds models from one who can be trusted to run them.
Nate Rosidi is a data scientist and in product strategy. He's also an adjunct professor teaching analytics, and is the founder of StrataScratch, a platform helping data scientists prepare for their interviews with real interview questions from top companies. Nate writes on the latest trends in the career market, gives interview advice, shares data science projects, and covers everything SQL.
Instagram Engineering
Shopify Engineering