AI governance & compliance Implement and maintain governance controls for AI models and services on the AI Foundation Platform, so AI use is trusted, controlled and compliant. Ensure AI services comply with data security, data classification, privacy (PDPA) and Responsible AI requirements. Maintain compliance evidence for ICT&SS / IM8, the System Security Plan (SSP), PDPA and Responsible AI requirements for agentic AI, and support security reviews and ARB submissions.
Monitoring & observability Implement model monitoring covering accuracy, drift, hallucination, latency, cost and usage across AI applications. Implement audit logging and traceability of model inputs, outputs and usage to support accountability and audit requirements. Implement end-to-end agent observability and audit trails (traces of prompts, reasoning steps, tool calls, outputs and approvals), with dashboards and alerts mapped to SLIs/SLOs and risk thresholds.
Develop full-stack monitoring dashboards and reporting tools that give stakeholders visibility of AI performance, risks and compliance status. Set up alerts and incident response procedures for AI service anomalies and degradation. Evaluation & testing Develop and run accuracy validation and evaluation processes, including test datasets, benchmarking, regression testing and human-in-the-loop review for model and prompt changes.
Evaluate agent behaviour, including task success, tool-use correctness, trajectory review, groundedness and regression testing