AI Risk and Ethics Specialist

Educates those involved in the development of AI and conducts assessments on the technical and societal risks across the lifecycle of AI solutions from acquisition or design to deployment and use.

Tasks

The concrete work activities defined for this role in the DCWF v5.1 spreadsheet. Core tasks are required for the role; additional tasks are associated but not mandatory.

• T1000B additional Ensure that AI design and development activities are properly documented and updated.
• T537A additional Develop methods to monitor and measure risk and assurance efforts on a continuous basis.
• T5854 additional Collaborate with appropriate personnel to address Personal Health Information (PHI), Personally Identifiable Information (PII), and other data privacy and data reusability concerns for AI solutions.
• T5856 additional Communicate the results of AI risk assessments to relevant stakeholders.
• T5860 additional Coordinate with appropriate personnel to identify methods for users and developers to report concerns about the implementation of DoD AI Ethical Principles.
• T5863 additional Create and/or maintain processes to ensure data management efforts comply with AI ethical principles.
• T5873 additional Determine methods and metrics for quantitative and qualitative measurement of AI risks so that sensitivity, specificity, likelihood, confidence levels, and other metrics are identified, documented, and applied.
• T5878 additional Develop risk mitigation strategies to ensure enumerated risks are prioritized, mitigated, shared, transferred, and/or accepted.
• T5879 additional Direct and/or support organizational and project-level AI risk management activities.
• T5881 additional Ensure risk management responsibilities are clearly defined, assigned, and communicated to relevant stakeholders.
• T5889 additional Identify and submit exemplary AI use cases, best practices, failure modes, and risk mitigation strategies, including after-action reports.
• T5893 additional Implement Responsible AI best practices and standards within AI solutions according to the DoD AI Ethical Principles, Responsible AI Guidelines, and/or any other pertinent laws.
• T5896 additional Maintain current knowledge of advancements in DoD AI Ethical Principles and Responsible AI.
• T5900 additional Measure the compliance of AI tools with DoD AI Ethical Principles.
• T5904 additional Perform risk assessment on AI applications to identify technical, societal, organizational, and mission risks.
• T5905 additional Perform risk assessment whenever an AI application or AI-enabled system undergoes a major change, when emergent behaviors are detected, and/or unintended consequences are reported.
• T765B additional Perform AI architecture security reviews, identify gaps, and develop a risk management plan to address issues.
• T963A additional Ensure risk mitigation plans of action and milestones are in place.

Knowledge, Skills, and Abilities

KSA statements define what a person filling this role knows or can do. "Knowledge" is what they must know, "Skill" is what they can perform, and "Ability" is a durable capacity they bring to the work.

• K0952 knowledge core Knowledge of emerging security issues, risks, and vulnerabilities.
• K6311 knowledge core Knowledge of machine learning theory and principles.
• K7003 knowledge core Knowledge of AI security risks, threats, and vulnerabilities and potential risk mitigation solutions.
• K7020 knowledge core Knowledge of DoD AI Ethical Principles (e.g., responsible, equitable, traceable, reliable, and governable).
• K7021 knowledge core Knowledge of emerging trends and future use cases of AI.
• K7024 knowledge core Knowledge of how AI is developed and operated.
• K7034 knowledge core Knowledge of interactions and integration of DataOps, MLOps, and DevSecOps in AI.
• K7036 knowledge core Knowledge of laws, regulations, and policies related to AI, data security/privacy, and use of publicly procured data for government.
• K7038 knowledge core Knowledge of metrics to evaluate the effectiveness of machine learning models.
• K7040 knowledge core Knowledge of Personal Health Information (PHI), Personally Identifiable Information (PII), and other data privacy and data reusability considerations for AI solutions.
• K7041 knowledge core Knowledge of remedies against unintended bias in AI solutions.
• K7045 knowledge core Knowledge of the AI lifecycle.
• K7048 knowledge core Knowledge of the benefits and limitations of AI capabilities.
• K7051 knowledge core Knowledge of the possible impacts of machine learning blind spots and edge cases.
• K7052 knowledge core Knowledge of the principles, methods, and tools used for risk and bias assessment and mitigation, including assessment of failures and their consequences.
• S7056 skill core Skill in assessing AI capabilities for bias or ethical concerns.
• S7064 skill core Skill in developing solutions and/or recommendations to minimize negative impacts of machine learning, especially for edge cases.
• S7065 skill core Skill in explaining AI concepts and terminology.
• S7067 skill core Skill in identifying low-probability, high-impact risks in machine learning training data sets.
• S7068 skill core Skill in identifying organizational and project-level AI risks, including AI security risks and requirements.
• S7069 skill core Skill in identifying risk over the lifespan of an AI solution.
• S7075 skill core Skill in testing and evaluating machine learning algorithms and/or AI solutions.
• K0942 knowledge additional Knowledge of the organization's core business/mission processes.
• K7044 knowledge additional Knowledge of testing, evaluation, validation, and verification (T&E V&V) tools and procedures to ensure systems are working as intended.

Other roles in this element