WEBVTT

00:26.791 --> 00:27.875
Welcome to the introduction

00:27.875 --> 00:28.916
video for the Department

00:28.916 --> 00:31.833
of Defense Instruction 5000.98.

00:31.833 --> 00:35.041
Also known as the DoDI 5000.98.

00:36.208 --> 00:39.583
The DoDI 5000.98 is a key instruction

00:39.583 --> 00:42.583
that was published on December 9, 2024.

00:43.125 --> 00:46.375
This instruction: (1) establishes policy,

00:47.291 --> 00:49.625
(2) assigns responsibilities,

00:49.625 --> 00:50.375
and (3)

00:50.375 --> 00:51.791
prescribes procedures

00:51.791 --> 00:54.166
for Operational Test and Evaluation

00:54.166 --> 00:56.750
(or OT&E), as well as Live

00:56.750 --> 00:59.750
Fire Test and Evaluation (or LFT&E),

01:00.208 --> 01:00.875
of DoD

01:00.875 --> 01:02.375
systems and services

01:02.375 --> 01:03.541
acquired via

01:03.541 --> 01:05.541
the Defense Acquisition System

01:05.541 --> 01:06.083
or via

01:06.083 --> 01:09.083
other non-standard acquisition systems.

01:10.375 --> 01:11.791
It impacts all DoD

01:11.791 --> 01:12.500
components,

01:12.500 --> 01:14.291
including the Office of the Secretary

01:14.291 --> 01:15.083
of Defense,

01:15.083 --> 01:16.500
Military Departments,

01:16.500 --> 01:17.833
Joint Chiefs of Staff,

01:17.833 --> 01:19.208
Combatant Commands,

01:19.208 --> 01:19.833
the Inspector

01:19.833 --> 01:21.916
General, Defense Agencies,

01:21.916 --> 01:24.208
and all other DoD entities.

01:24.208 --> 01:25.041
It covers

01:25.041 --> 01:27.000
all DoD systems obtained

01:27.000 --> 01:29.041
through the Defense Acquisition System,

01:29.041 --> 01:31.041
following any pathway outlined

01:31.041 --> 01:33.000
in the Adaptive Acquisition Framework.

01:33.000 --> 01:36.500
The DoDI 5000.98

01:36.500 --> 01:40.541
supersedes the previous DoDI 5000.89

01:40.916 --> 01:43.583
regarding Operational Test and Evaluation

01:43.583 --> 01:45.333
and Life Fire Test and Evaluation.

01:46.333 --> 01:48.041
This update aligns policy

01:48.041 --> 01:48.750
with the rapidly

01:48.750 --> 01:49.625
evolving mission

01:49.625 --> 01:51.833
and technology landscape.

01:51.833 --> 01:53.041
At the end of the video,

01:53.041 --> 01:53.875
you’ll understand

01:53.875 --> 01:55.916
this key policy and its contents

01:55.916 --> 01:58.750
as it relates to the work at DOT&E.

01:58.750 --> 02:01.750
Let’s get started!

02:01.791 --> 02:04.458
Section 2: Responsibilities.

02:04.458 --> 02:05.875
Welcome to section 2.

02:05.875 --> 02:07.125
This section introduces

02:07.125 --> 02:08.958
the viewer to the Responsibilities

02:08.958 --> 02:10.041
detailed in the policy.

02:10.041 --> 02:12.375
Under Title 10 of the U.S.

02:12.375 --> 02:14.125
Code, key defense offices

02:14.125 --> 02:16.000
ensure the effective acquisition

02:16.000 --> 02:18.583
and sustainment of defense capabilities.

02:18.583 --> 02:20.000
Here we explore their roles

02:20.000 --> 02:22.750
and responsibilities. 2.1.

02:22.750 --> 02:26.791
DIRECTOR OF OPERATIONAL TEST

02:26.791 --> 02:29.833
AND EVALUATION (DOT&E).

02:30.083 --> 02:30.375
The Director of Operational Test

02:30.375 --> 02:30.500
and Evaluation,

02:30.500 --> 02:30.958
or DOT&E,

02:30.958 --> 02:32.291
is the principal official

02:32.291 --> 02:33.041
and advisor

02:33.041 --> 02:34.333
to the Secretary of Defense

02:34.333 --> 02:37.583
on all matters related to OT&E and LFT&E.

02:38.291 --> 02:39.291
In this role,

02:39.291 --> 02:39.958
DOT&E

02:39.958 --> 02:41.250
publishes and manages

02:41.250 --> 02:42.708
the T&E Oversight List,

02:42.708 --> 02:44.833
ensuring the comprehensive oversight

02:44.833 --> 02:48.916
of critical defense systems.
DOT&E

02:48.916 --> 02:50.500
also monitors, reviews,

02:50.500 --> 02:53.208
and independently reports on all OT&E

02:53.208 --> 02:54.791
and LFT&E activities

02:54.791 --> 02:56.416
and resources for systems

02:56.416 --> 02:58.583
on the T&E Oversight List.

02:58.583 --> 03:00.083
This ensures transparency

03:00.083 --> 03:01.083
and accountability in

03:01.083 --> 03:02.250
how these evaluations

03:02.250 --> 03:05.500
are conducted.
DOT&E also serves

03:05.500 --> 03:06.250
as an advisor

03:06.250 --> 03:07.416
to the Joint Requirements

03:07.416 --> 03:10.416
Oversight Council, also known as JROC.

03:10.500 --> 03:11.375
In this role,

03:11.375 --> 03:12.041
it provides

03:12.041 --> 03:13.000
expert advice

03:13.000 --> 03:15.208
to inform and influence decisions

03:15.208 --> 03:16.541
regarding requirements,

03:16.541 --> 03:17.583
concepts,

03:17.583 --> 03:19.375
capabilities-based assessments,

03:19.375 --> 03:21.458
and concepts of operations.

03:21.458 --> 03:24.250
Before moving onto additional relevant

03:24.250 --> 03:26.708
offices, subagencies, and groups

03:26.708 --> 03:28.208
it is critical to briefly

03:28.208 --> 03:30.000
dig into the T&E Oversight List

03:30.000 --> 03:32.833
that DOT&E is responsible for.

03:32.833 --> 03:35.291
About 300 programs are currently overseen

03:35.291 --> 03:36.375
by DOT&E,

03:36.375 --> 03:37.916
and the list encompasses 12

03:37.916 --> 03:39.166
components within the DoD.

03:40.208 --> 03:41.458
These programs range from

03:41.458 --> 03:43.000
oversight of aircraft

03:43.000 --> 03:45.375
to radar systems to munitions

03:45.375 --> 03:47.208
to operational processes –

03:47.208 --> 03:49.500
and the list goes on.

03:49.500 --> 03:52.333
DOT&E may place any program or system

03:52.333 --> 03:53.833
on the T&E oversight list

03:53.833 --> 03:54.875
at any time

03:54.875 --> 03:56.125
by using criteria

03:56.125 --> 03:58.000
related to dollar value,

03:58.000 --> 03:59.291
congressional or DoD

03:59.291 --> 04:01.000
interest, and critical mission

04:01.000 --> 04:03.000
warfighting capability.

04:03.000 --> 04:04.291
A formal notification

04:04.291 --> 04:06.583
will be provided to a Military Service

04:06.583 --> 04:07.791
when a program is being

04:07.791 --> 04:09.291
added to the T&E oversight list.

04:09.291 --> 04:10.625
2.2 UNDER SECRETARY OF DEFENSE FOR RESEARCH

04:10.625 --> 04:11.500
AND ENGINEERING (USD (R&E))

04:11.500 --> 04:12.791
The Under Secretary of Defense

04:12.791 --> 04:14.708
for Research and Engineering –

04:14.708 --> 04:20.583
or USD(R&E) –

04:20.625 --> 04:22.125
plays a pivotal role

04:22.125 --> 04:23.500
in overseeing the Department

04:23.500 --> 04:25.666
of Defense's acquisition programs,

04:25.666 --> 04:28.375
in accordance with Section 133a of Title

04:28.375 --> 04:29.416
10 of the U.S.

04:29.416 --> 04:30.458
Code.

04:30.458 --> 04:32.208
USD(R&E) is responsible

04:32.208 --> 04:33.333
for selecting specific

04:33.333 --> 04:34.750
DoD acquisition programs

04:34.750 --> 04:36.458
for DT&E oversight,

04:36.458 --> 04:38.000
and evaluating and approving

04:38.000 --> 04:39.250
DT&E strategies

04:39.250 --> 04:39.916
for programs

04:39.916 --> 04:41.708
on the T&E oversight list

04:41.708 --> 04:43.458
categorized under Acquisition

04:43.458 --> 04:47.166
Category 1D, or ACAT 1D.

04:47.166 --> 04:48.791
For other acquisition

04:48.791 --> 04:51.000
programs under DT&E oversight,

04:51.000 --> 04:52.708
the USD(R&E) provides

04:52.708 --> 04:54.458
guidance to the Milestone

04:54.458 --> 04:56.708
Decision Authority (or MDA)

04:56.708 --> 04:58.541
by conducting an independent

04:58.541 --> 05:00.375
analysis of test data,

05:00.375 --> 05:01.333
reports,

05:01.333 --> 05:02.625
and modeling and simulation

05:02.625 --> 05:04.750
results.
2.3.

05:04.750 --> 05:06.291
UNDER SECRETARY OF DEFENSE

05:06.291 --> 05:08.291
FOR ACQUISITION AND

05:08.291 --> 05:09.583
SUSTAINMENT (USD(A&S)):

05:11.125 --> 05:12.750
As mandated by Section

05:12.750 --> 05:14.750
133b, the Under Secretary of Defense

05:14.750 --> 05:18.333
– or USD(A&S) – establishes policies

05:18.333 --> 05:19.916
directing acquisition decision

05:19.916 --> 05:22.250
authorities and program managers.

05:22.250 --> 05:24.208
These policies ensure the intent

05:24.208 --> 05:25.625
and requirements of defense

05:25.625 --> 05:27.041
directives are implemented

05:27.041 --> 05:29.708
across all acquisition programs.

05:29.708 --> 05:31.208
When acting as the MDA,

05:31.208 --> 05:34.208
USD(A&S) evaluates OT&E

05:34.291 --> 05:37.208
and LFT&E activities and reports.

05:37.208 --> 05:38.250
This evaluation,

05:38.250 --> 05:40.416
in coordination with DOT&E,

05:40.416 --> 05:42.708
aids in making informed decisions about

05:42.708 --> 05:43.958
program acquisition

05:43.958 --> 05:46.833
and sustainment risks.

05:46.833 --> 05:47.458
2.4.

05:47.458 --> 05:50.541
DOD CHIEF INFORMATION OFFICER (CIO)

05:52.458 --> 05:54.708
The office of the DoD Chief Information

05:54.708 --> 05:56.500
Officer (CIO) plays

05:56.500 --> 05:58.791
a crucial role in aligning OT&E

05:58.791 --> 06:01.750
and LFT&E processes with the DoD’s

06:01.750 --> 06:03.333
Cybersecurity Program

06:03.333 --> 06:04.125
and Strategic

06:04.125 --> 06:05.375
Cybersecurity Program.
2.5.

06:05.375 --> 06:06.375
UNDER SECRETARY OF DEFENSE

06:06.375 --> 06:06.875
FOR INTELLIGENCE

06:06.875 --> 06:07.625
AND SECURITY (USD(I And S))

06:07.625 --> 06:08.958
The Under Secretary of Defense

06:08.958 --> 06:13.458
for Intelligence and Security –

06:13.458 --> 06:18.500
USD(I&S) –

06:18.875 --> 06:19.875
is responsible

06:19.875 --> 06:21.833
for overseeing intelligence support

06:21.833 --> 06:23.833
throughout the acquisition lifecycle,

06:23.833 --> 06:25.458
and advising DOT&E

06:25.458 --> 06:26.291
on intelligence

06:26.291 --> 06:27.750
supportability requirements.

06:28.958 --> 06:30.250
2.6.

06:30.250 --> 06:31.333
CHIEF DIGITAL AND

06:31.333 --> 06:34.333
ARTIFICIAL INTELLIGENCE OFFICE:

06:34.541 --> 06:35.500
The Chief Digital

06:35.500 --> 06:37.333
and Artificial Intelligence Office

06:37.333 --> 06:40.958
– or CDAO – issues guidance on T&E

06:40.958 --> 06:43.958
for AI capabilities within DoD systems,

06:44.208 --> 06:47.416
collaborates with the USD(R&E) and DOT&E

06:47.416 --> 06:48.083
to develop

06:48.083 --> 06:49.458
and utilize common tools

06:49.458 --> 06:51.416
and infrastructure for T&E,

06:51.416 --> 06:52.208
and conducts

06:52.208 --> 06:55.208
verification and validation – or V&V –

06:55.375 --> 06:58.750
of AI capabilities.

06:58.833 --> 07:01.833
2.7 DOD COMPONENT HEADS:

07:02.083 --> 07:04.208
Component heads are tasked with planning,

07:04.208 --> 07:07.208
funding, executing, and reporting on DoD

07:07.208 --> 07:10.208
systems’ OT&E and LFT&E.

07:10.208 --> 07:11.875
This is managed through their acquisition

07:11.875 --> 07:14.041
executives, program managers,

07:14.041 --> 07:15.625
LFT&E organizations,

07:15.625 --> 07:18.625
and designated operational test agencies.

07:21.125 --> 07:22.041
SECTION 3:

07:22.041 --> 07:25.041
OT&E AND LFT&E OVERVIEW

07:25.166 --> 07:26.958
Welcome to Section 3.

07:26.958 --> 07:27.541
This section

07:27.541 --> 07:28.541
introduces the viewer

07:28.541 --> 07:31.541
to an overview of OT&E and LFT&E,

07:31.666 --> 07:33.458
as detailed in the policy.

07:33.458 --> 07:35.083
3.1 SCIENCE- AND TECHNOLOGY-BASED OT&E

07:35.083 --> 07:41.916
AND LFT&E.
OT&E and LFT&E

07:42.375 --> 07:43.708
activities incorporate

07:43.708 --> 07:45.583
the latest scientific advancements,

07:45.583 --> 07:47.500
such as the design of experiments,

07:47.500 --> 07:48.833
statistical inference,

07:48.833 --> 07:51.000
and big data analytics.

07:51.000 --> 07:53.166
These methods are essential for:

07:53.166 --> 07:56.000
Providing vital live data, M&S

07:56.000 --> 07:59.000
results, and knowledge to stakeholders;

07:59.083 --> 08:01.583
and ensuring that tests effectively

08:01.583 --> 08:03.125
cover operationally relevant,

08:03.125 --> 08:04.666
multi-domain conditions.

08:06.041 --> 08:06.833
DOT&E

08:06.833 --> 08:08.125
utilizes cutting-edge

08:08.125 --> 08:11.125
technology in the planning, execution,

08:11.250 --> 08:14.250
analysis, and reporting phases.

08:14.250 --> 08:14.958
This includes

08:14.958 --> 08:16.250
artificial intelligence

08:16.250 --> 08:18.000
and machine learning tools

08:18.000 --> 08:19.208
to predict outcomes

08:19.208 --> 08:20.541
more accurately,

08:20.541 --> 08:21.916
alongside digital

08:21.916 --> 08:23.041
engineering and data

08:23.041 --> 08:23.958
management tools

08:23.958 --> 08:26.916
to manage and analyze large data sets.

08:26.916 --> 08:29.375
By integrating data

08:29.375 --> 08:30.958
from multiple sources,

08:30.958 --> 08:33.750
DOT&E can dynamically enhance

08:33.750 --> 08:34.791
the planning,

08:34.791 --> 08:35.791
execution,

08:35.791 --> 08:36.583
analysis,

08:36.583 --> 08:39.458
and reporting stages of integrated T&E,

08:39.458 --> 08:40.583
OT&E,

08:40.583 --> 08:41.666
and LFT&E

08:41.666 --> 08:44.666
throughout the acquisition lifecycle.

08:44.958 --> 08:45.291
3.2.

08:45.291 --> 08:46.333
OT&E AND LFT&E

08:46.333 --> 08:48.041
ACROSS THE ACQUISITION LIFE CYCLE

08:48.041 --> 08:51.041
Throughout the acquisition lifecycle,

08:51.208 --> 08:55.000
OT&E and LFT&E

08:55.000 --> 08:56.500
activities will leverage

08:56.500 --> 08:59.500
data and results from several sources.

08:59.541 --> 09:02.750
This includes:
Integrated T&E events

09:04.083 --> 09:05.916
Dedicated OT&E events

09:05.916 --> 09:07.666
to assess operational readiness

09:07.666 --> 09:11.666
and effectiveness
Dedicated LFT&E events

09:11.666 --> 09:13.625
to evaluate system survivability

09:13.625 --> 09:14.541
and lethality

09:14.541 --> 09:17.750
under live conditions
M&S results

09:17.750 --> 09:19.166
to predict outcomes

09:19.166 --> 09:20.750
and refine test conditions

09:22.416 --> 09:24.291
and
Data from Major Command

09:24.291 --> 09:26.916
or Combatant Command Exercises

09:26.916 --> 09:28.291
(Flag Events)

09:28.291 --> 09:29.458
to incorporate

09:29.458 --> 09:32.458
real-world operational scenarios.

09:33.250 --> 09:35.166
As a DoD system progresses

09:35.166 --> 09:36.291
from sub-components

09:36.291 --> 09:37.750
to systems-of-systems,

09:37.750 --> 09:38.666
the complexity

09:38.666 --> 09:40.583
and objectives of T&E events

09:40.583 --> 09:42.750
evolve significantly.

09:42.750 --> 09:44.666
This progression is meticulously planned

09:44.666 --> 09:45.375
and executed

09:45.375 --> 09:46.250
by the T&E

09:46.250 --> 09:48.000
Working-level Integrated Product

09:48.000 --> 09:49.791
Team (WIPT),

09:49.791 --> 09:50.458
also known

09:50.458 --> 09:53.458
as the Integrated Test Team (ITT).

09:54.041 --> 09:56.541
This team plays a crucial role in:

09:56.541 --> 09:57.750
Informing program

09:57.750 --> 10:00.750
requirements and acquisition contracts;

10:00.916 --> 10:03.083
Developing and updating the Test

10:03.083 --> 10:06.000
and Evaluation Master Plan – or TEMP –

10:06.000 --> 10:08.375
and T&E strategy to support sound

10:08.375 --> 10:10.375
acquisition decisions; and
Planning

10:10.375 --> 10:13.833
and executing risk-based test

10:13.833 --> 10:15.041
assessments and Mission-Based

10:15.041 --> 10:18.708
Risk Assessments – or MBRAs – to define

10:18.708 --> 10:21.708
a defensible scope of T&E.

10:23.458 --> 10:25.083
The TEMP, or T&E, strategy

10:25.083 --> 10:27.250
also includes an Integrated Decision

10:27.250 --> 10:29.708
Support Key – or IDSK –

10:29.708 --> 10:31.500
which enhances decision-making

10:31.500 --> 10:32.541
by doing the following:

10:33.625 --> 10:35.250
Identifying necessary data

10:35.250 --> 10:36.750
from various testing phases

10:36.750 --> 10:37.625
to evaluate

10:37.625 --> 10:38.708
technical requirements

10:38.708 --> 10:41.708
and operational capabilities.

10:42.250 --> 10:43.416
Facilitating early

10:43.416 --> 10:45.041
and frequent coordination

10:45.041 --> 10:46.708
stakeholders to ensure

10:46.708 --> 10:48.083
that operational realism

10:48.083 --> 10:48.875
is incorporated early.

10:48.875 --> 10:51.875
Implementing statistical methods

10:51.875 --> 10:53.333
and digital technologies

10:53.333 --> 10:54.541
to maximize the use

10:54.541 --> 10:56.166
of all relevant data –

10:56.166 --> 10:57.916
which increases the knowledge base

10:57.916 --> 10:59.708
regarding the system’s effectiveness,

10:59.708 --> 11:01.666
suitability, survivability,

11:01.666 --> 11:03.958
and lethality.

11:03.958 --> 11:04.666
3.3.

11:04.666 --> 11:08.041
OPERATIONAL TEST AND EVALUATION (OT&E).

11:09.416 --> 11:11.875
Comprehensive Evaluation Strategy:

11:11.875 --> 11:13.125
Operational Test Agencies

11:13.125 --> 11:15.500
– also known as OTAs –

11:15.500 --> 11:16.791
are tasked with planning

11:16.791 --> 11:18.791
and executing OT&E

11:18.791 --> 11:19.583
to evaluate

11:19.583 --> 11:20.875
the operational effectiveness

11:20.875 --> 11:22.708
and suitability of systems,

11:22.708 --> 11:23.500
while equally

11:23.500 --> 11:25.166
considering their survivability

11:25.166 --> 11:26.583
and lethality.

11:26.583 --> 11:27.500
Tests encompass

11:27.500 --> 11:29.083
a full spectrum of natural

11:29.083 --> 11:30.833
and man-made environments,

11:30.833 --> 11:32.625
covering extreme conditions

11:32.625 --> 11:35.625
to ensure robust system performance.

11:35.708 --> 11:36.916
Evaluations include

11:36.916 --> 11:38.541
all operationally relevant

11:38.541 --> 11:40.083
kinetic and non-kinetic

11:40.083 --> 11:41.750
threats and targets

11:41.750 --> 11:44.291
anticipated over the system’s lifecycle.

11:44.291 --> 11:46.083
Testing scenarios involve complex

11:46.083 --> 11:47.000
kill webs,

11:47.000 --> 11:48.583
mission threads, and essential

11:48.583 --> 11:49.708
task lists

11:49.708 --> 11:50.375
to ensure

11:50.375 --> 11:52.541
comprehensive readiness assessments.

11:52.541 --> 11:53.333
Additionally,

11:53.333 --> 11:55.250
tests assess the interoperability

11:55.250 --> 11:56.416
of DoD systems

11:56.416 --> 11:58.583
within multi-domain operations,

11:58.583 --> 12:00.375
decision-making processes,

12:00.375 --> 12:01.291
and compliance

12:01.291 --> 12:02.666
with standards to ensure

12:02.666 --> 12:05.666
seamless joint force operations.

12:05.750 --> 12:08.250
Categorization and Planning of OT&E

12:08.250 --> 12:09.291
Events:

12:09.291 --> 12:10.916
OT&E is categorized

12:10.916 --> 12:12.458
into five major types,

12:12.458 --> 12:13.708
with each tailored

12:13.708 --> 12:15.958
to different acquisition decisions.

12:15.958 --> 12:17.000
They are as follows:

12:18.083 --> 12:18.375
Early

12:18.375 --> 12:21.375
Operational Assessment, or EOA

12:21.708 --> 12:24.708
Operational Assessment, or OA

12:24.708 --> 12:26.791
Initial Operational Test and Evaluation,

12:26.791 --> 12:30.083
or IOT&E
Follow-on Operational

12:30.083 --> 12:33.541
Test and Evaluation, or FOT&E
and

12:33.958 --> 12:35.375
Operational Demonstration,

12:35.375 --> 12:39.958
or Ops Demo
EOA and OA assessments

12:39.958 --> 12:41.708
occur early in the system’s

12:41.708 --> 12:44.125
lifecycle, providing critical feedback

12:44.125 --> 12:46.041
on operational risks and system

12:46.041 --> 12:46.625
maturity

12:46.625 --> 12:48.333
to inform further development

12:48.333 --> 12:51.000
and fielding decisions.

12:51.000 --> 12:51.875
IOT&E

12:51.875 --> 12:53.250
assessments are conducted

12:53.250 --> 12:56.250
in realistic combat conditions to confirm

12:56.250 --> 12:57.500
operational effectiveness

12:57.500 --> 12:59.750
and suitability.

12:59.750 --> 13:00.416
FOT&E

13:00.416 --> 13:01.666
assessments are conducted

13:01.666 --> 13:03.291
post-deployment to address

13:03.291 --> 13:04.958
any operational changes

13:04.958 --> 13:06.083
or emerging threats

13:06.083 --> 13:09.083
that could impact system performance.

13:09.125 --> 13:11.333
Ops Demos assess the system’s readiness

13:11.333 --> 13:12.625
for initial production

13:12.625 --> 13:14.250
or transition to operations

13:14.250 --> 13:15.666
and sustainment.
3.4.

13:15.666 --> 13:19.708
LIVE FIRE TEST AND EVALUATION (or LFT&E)

13:22.000 --> 13:23.541
Comprehensive LFT&E

13:23.541 --> 13:25.791
Strategy:
LFT&E involves

13:25.791 --> 13:27.000
full-spectrum testing

13:27.000 --> 13:28.500
of combat-loaded systems

13:28.500 --> 13:30.000
in operationally relevant

13:30.000 --> 13:30.875
and representative

13:30.875 --> 13:32.625
contested, congested,

13:32.625 --> 13:35.000
and constrained environments.

13:35.000 --> 13:36.291
This includes the use of live

13:36.291 --> 13:37.708
kinetic and non-kinetic

13:37.708 --> 13:39.000
threats and targets,

13:39.000 --> 13:40.208
or their surrogates,

13:40.208 --> 13:42.375
to simulate actual combat conditions.

13:44.333 --> 13:47.250
Evaluating the Survivability Kill Chain:

13:47.250 --> 13:48.791
Several factors are considered

13:48.791 --> 13:50.333
when evaluating the survivability

13:50.333 --> 13:51.083
kill chain – including threat susceptibility,  vulnerability identification, recoverability,

13:51.833 --> 13:53.333
and coordinated attack effects.

14:00.958 --> 14:01.833
and coordinated attack effects.
Threat

14:01.833 --> 14:02.208
Susceptibility: Tests

14:02.208 --> 14:03.791
evaluate the system’s ability

14:03.791 --> 14:06.041
to detect and respond to threats,

14:06.041 --> 14:07.291
including the effectiveness

14:07.291 --> 14:08.875
of situational awareness,

14:08.875 --> 14:11.041
threat detection, signature control,

14:11.041 --> 14:12.833
and defensive measures.

14:12.833 --> 14:14.750
Vulnerability Identification:

14:14.750 --> 14:16.333
Focuses on identifying

14:16.333 --> 14:18.000
critical vulnerabilities

14:18.000 --> 14:19.250
in the system design

14:19.250 --> 14:20.208
and the effectiveness

14:20.208 --> 14:23.208
of any survivability enhancements.

14:23.333 --> 14:25.333
Recoverability: Assesses

14:25.333 --> 14:26.416
the system’s ability

14:26.416 --> 14:27.541
to recover

14:27.541 --> 14:29.250
from attacks and resume

14:29.250 --> 14:30.625
mission-critical functions,

14:30.625 --> 14:32.541
including the evaluation of force

14:32.541 --> 14:35.458
protection and user casualties.

14:35.458 --> 14:38.416
Coordinated Attack Effects: Examines

14:38.416 --> 14:40.083
the effects of coordinated kinetic

14:40.083 --> 14:43.083
and non-kinetic attacks on the system.

14:44.500 --> 14:45.458
3.5.

14:45.458 --> 14:48.208
CERTIFICATIONS.

14:48.208 --> 14:49.500
Testing certifications,

14:49.500 --> 14:50.833
including safety,

14:50.833 --> 14:52.875
airworthiness, interoperability,

14:52.875 --> 14:54.833
and nuclear certifications,

14:54.833 --> 14:56.333
are essential to ensure

14:56.333 --> 14:57.750
that our systems meet stringent

14:57.750 --> 14:59.875
operational and regulatory standards.

15:01.166 --> 15:03.833
Interoperability and Supportability:

15:03.833 --> 15:06.916
According to DoDI 8330.01,

15:06.916 --> 15:10.041
TEMPs and T&E strategies must include

15:10.041 --> 15:11.083
interoperability

15:11.083 --> 15:13.375
and supportability requirements.

15:13.375 --> 15:14.250
These are crucial

15:14.250 --> 15:15.666
for conducting thorough

15:15.666 --> 15:17.000
interoperability assessments

15:17.000 --> 15:19.916
and obtaining necessary certifications.

15:19.916 --> 15:22.958
These four requirements consist of: 1.

15:22.958 --> 15:25.291
Joint Interoperability Certification:

15:25.291 --> 15:26.708
For programs

15:26.708 --> 15:27.708
that exchange data

15:27.708 --> 15:29.166
externally beyond their DoD

15:29.166 --> 15:30.500
Component, a joint

15:30.500 --> 15:32.250
interoperability certification

15:32.250 --> 15:33.000
from the Joint

15:33.000 --> 15:34.250
Interoperability Test

15:34.250 --> 15:36.375
Command is mandatory.

15:36.375 --> 15:36.958
2.

15:36.958 --> 15:38.708
Lifecycle Evaluation:

15:38.708 --> 15:41.458
Interoperability must be evaluated early

15:41.458 --> 15:42.166
and frequently

15:42.166 --> 15:44.333
throughout the system’s lifecycle.

15:44.333 --> 15:45.875
This helps to capture and assess

15:45.875 --> 15:46.625
any changes

15:46.625 --> 15:48.208
that could affect interoperability

15:48.208 --> 15:48.833
in various

15:48.833 --> 15:50.375
operational environments,

15:50.375 --> 15:53.375
including platform-specific, joint,

15:53.375 --> 15:55.708
multinational, and interagency

15:55.708 --> 15:57.416
contexts.
3.

15:57.416 --> 15:59.166
Certification Before Fielding:

15:59.166 --> 16:00.791
Interoperability certification

16:00.791 --> 16:02.000
must be obtained

16:02.000 --> 16:04.041
before fielding any new capability

16:04.041 --> 16:06.333
or upgrading an existing one.

16:06.333 --> 16:07.416
4.

16:07.416 --> 16:08.791
Intra-service and Intra-agency

16:08.791 --> 16:11.416
Interoperability: For DoD systems

16:11.416 --> 16:13.000
without joint interoperability

16:13.000 --> 16:14.750
requirements, coordination

16:14.750 --> 16:16.458
with designated interoperability

16:16.458 --> 16:18.875
certification authorities is essential.

16:18.875 --> 16:20.166
3.6.

16:20.166 --> 16:21.500
MODELING AND SIMULATION (M&S)

16:21.500 --> 16:28.208
Modeling and Simulation – or M&S –

16:28.208 --> 16:29.583
plays a pivotal role

16:29.583 --> 16:30.291
in the testing

16:30.291 --> 16:32.000
and evaluation process,

16:32.000 --> 16:33.750
providing a digital representation

16:33.750 --> 16:34.666
of systems,

16:34.666 --> 16:36.291
environments, and phenomena

16:36.291 --> 16:37.583
that helps in achieving

16:37.583 --> 16:39.750
critical testing objectives.

16:39.750 --> 16:42.208
Let’s explore the various facets of M&S

16:42.208 --> 16:45.000
and the rigorous standards it must meet.

16:45.000 --> 16:45.750
Understanding

16:45.750 --> 16:47.416
Modeling and Simulation

16:47.416 --> 16:51.375
(M&S):
M&S encompasses a range of tools

16:51.375 --> 16:52.250
and technologies

16:52.250 --> 16:54.583
used to simulate physical, mathematical,

16:54.583 --> 16:56.833
or logical representations of systems,

16:56.833 --> 16:58.291
entities, environments,

16:58.291 --> 17:01.000
phenomena, or processes.

17:01.000 --> 17:02.125
These tools are crucial

17:02.125 --> 17:03.708
for delivering results-for-record

17:03.708 --> 17:05.541
that substitute for live data,

17:05.541 --> 17:07.041
which is essential for meeting

17:07.041 --> 17:09.291
the objectives of OT&E and LFT&E.

17:10.750 --> 17:11.583
Through the use of

17:11.583 --> 17:12.416
M&S, the

17:12.416 --> 17:13.583
DoD safeguards

17:13.583 --> 17:15.666
that systems are rigorously tested

17:15.666 --> 17:17.750
and evaluated in a controlled yet

17:17.750 --> 17:19.791
realistic digital environment,

17:19.791 --> 17:21.375
enhancing their predictability

17:21.375 --> 17:22.041
and safety

17:22.041 --> 17:24.291
before they are deployed in real-world

17:24.291 --> 17:25.958
scenarios. 3.7.

17:25.958 --> 17:28.958
PROGRAM MANAGEMENT.

17:29.666 --> 17:30.750
This section explores

17:30.750 --> 17:31.166
the roles

17:31.166 --> 17:32.125
and responsibilities

17:32.125 --> 17:34.166
within the T&E process for DoD

17:34.166 --> 17:35.541
acquisition programs.

17:35.541 --> 17:36.291
Let's dive

17:36.291 --> 17:37.250
into the key roles

17:37.250 --> 17:40.208
and their specific duties.

17:40.208 --> 17:42.875
The four main divisions are as follows:

17:42.875 --> 17:43.291
Section A:

17:43.291 --> 17:48.041
The Program Manager
Section B: T&E WIPT

17:48.041 --> 17:49.458
or ITT
Section

17:49.458 --> 17:52.625
C: Operational Test Agency (or OTA)

17:53.041 --> 17:56.125
Section D: LFT&E Organizations

17:59.083 --> 18:01.500
Section A: The Program Manager

18:01.500 --> 18:03.375
At the start of the acquisition program,

18:03.375 --> 18:05.291
the program manager charters

18:05.291 --> 18:06.500
the T&E Working-level

18:06.500 --> 18:09.583
Integrated Product Team (WIPT or ITT).

18:09.583 --> 18:13.708
They also ensure the availability

18:13.708 --> 18:15.458
and interoperability of systems

18:15.458 --> 18:16.250
intended to work

18:16.250 --> 18:19.291
together, integrating T&E WIPT or ITT

18:19.541 --> 18:21.416
in the development of strategies

18:21.416 --> 18:22.583
and requirements across

18:22.583 --> 18:23.833
the acquisition lifecycle.

18:25.250 --> 18:25.958
The program manager

18:25.958 --> 18:28.166
also confirms budgets and schedules,

18:28.166 --> 18:29.875
allowing for thorough testing

18:29.875 --> 18:30.625
and the correction

18:30.625 --> 18:31.708
of any deficiencies

18:31.708 --> 18:34.708
before moving to subsequent phases.

18:34.875 --> 18:36.125
They leverage technologies

18:36.125 --> 18:37.500
such as digital engineering

18:37.500 --> 18:40.500
and smart documentation to enhance T&E

18:40.500 --> 18:43.125
efficiency and maintain real-time

18:43.125 --> 18:45.875
communication. The program manager

18:45.875 --> 18:47.708
also establishes processes

18:47.708 --> 18:49.666
and metrics to assess changes

18:49.666 --> 18:51.083
in operational environments

18:51.083 --> 18:54.083
and system performance over time.

18:54.833 --> 18:56.125
Section B:

18:56.125 --> 18:59.125
T&E WIPT or ITT

18:59.125 --> 19:01.208
The next division is that of the T&E WIPT

19:01.208 --> 19:02.666
or ITT –

19:02.666 --> 19:03.500
which stands

19:03.500 --> 19:04.791
for the T&E Working-level

19:04.791 --> 19:06.958
Integrated Product Team.

19:06.958 --> 19:07.916
This team provides

19:07.916 --> 19:09.166
a collaborative environment

19:09.166 --> 19:09.583
for key

19:09.583 --> 19:11.000
T&E organizations,

19:11.000 --> 19:11.708
ensuring that

19:11.708 --> 19:13.708
all testing equities are represented

19:13.708 --> 19:14.666
in the strategies,

19:14.666 --> 19:15.666
plans,

19:15.666 --> 19:18.666
and modeling, and simulation events.

19:18.750 --> 19:20.583
The T&E WIPT or ITT

19:20.583 --> 19:22.375
plays a critical role

19:22.375 --> 19:24.583
in defining T&E requirements,

19:24.583 --> 19:25.916
developing the TEMP,

19:25.916 --> 19:28.166
and implementing formal data collection

19:28.166 --> 19:29.416
and scoring processes.

19:32.333 --> 19:33.166
Section C:

19:33.166 --> 19:36.166
Operational Test Agency (OTA)

19:36.750 --> 19:39.583
The next key player is the OTA.

19:39.583 --> 19:41.000
The OTA is responsible

19:41.000 --> 19:43.125
for planning and executing OT&E

19:43.125 --> 19:44.208
to provide data

19:44.208 --> 19:45.666
for an independent assessment

19:45.666 --> 19:48.500
of the system’s operational capabilities.

19:48.500 --> 19:49.416
They work closely

19:49.416 --> 19:51.416
with LFT&E organizations

19:51.416 --> 19:53.250
to coordinate data collection

19:53.250 --> 19:56.250
and support the accreditation of M&S.

19:56.708 --> 19:59.708
Section D: LFT&E Organizations

19:59.708 --> 20:00.750
And finally we have

20:00.750 --> 20:02.458
the LFT&E Organizations.

20:02.458 --> 20:03.958
These organizations

20:03.958 --> 20:06.000
conduct LFT&E activities,

20:06.000 --> 20:08.375
focusing on full-spectrum survivability

20:08.375 --> 20:09.958
and lethality.

20:09.958 --> 20:10.916
They are responsible

20:10.916 --> 20:11.666
for accrediting

20:11.666 --> 20:13.458
modeling and simulation tools

20:13.458 --> 20:15.041
and verifying and validating

20:15.041 --> 20:16.666
automated test capabilities

20:16.666 --> 20:19.666
that support operational evaluations.

20:20.958 --> 20:21.625
3.8.

20:21.625 --> 20:24.625
DATA MANAGEMENT

20:25.583 --> 20:27.541
The program manager adheres to the “DoD

20:27.541 --> 20:28.875
Data, Analytics,

20:28.875 --> 20:30.125
and Artificial Intelligence

20:30.125 --> 20:32.125
Adoption Strategy,”

20:32.125 --> 20:35.083
the five “DoD Data Decrees” as outlined

20:35.083 --> 20:36.458
in the Deputy SecDef

20:36.458 --> 20:39.500
Memorandum dated May 5, 2021,

20:40.166 --> 20:43.666
and DoDI 5015.02.

20:44.708 --> 20:45.458
These policies

20:45.458 --> 20:46.500
ensure that

20:46.500 --> 20:48.750
data handling within T&E processes

20:48.750 --> 20:50.291
meets the highest standards

20:50.291 --> 20:52.708
of security and efficiency.

20:52.708 --> 20:55.666
Data Management Plan and Repository:

20:55.666 --> 20:58.125
A robust data management plan alongside

20:58.125 --> 21:00.208
a common T&E data and program

21:00.208 --> 21:02.541
artifact repository ensures

21:02.541 --> 21:04.125
that all T&E data,

21:04.125 --> 21:05.541
including digital engineering

21:05.541 --> 21:06.500
models and related

21:06.500 --> 21:08.333
data, are visible,

21:08.333 --> 21:10.708
accessible, understandable,

21:10.708 --> 21:12.500
linked, trusted,

21:12.500 --> 21:15.500
interoperable, and secure.

21:15.958 --> 21:18.083
The plan includes detailed descriptions

21:18.083 --> 21:20.000
of test environments and conditions

21:20.000 --> 21:21.458
to promote commonality

21:21.458 --> 21:22.500
and usability

21:22.500 --> 21:25.500
across various stakeholder organizations.

21:25.916 --> 21:27.291
The plan addresses data

21:27.291 --> 21:29.333
protection laws and policies,

21:29.333 --> 21:30.875
focusing on environmental

21:30.875 --> 21:32.083
hardening and robust

21:32.083 --> 21:34.083
authentication, authorization,

21:34.083 --> 21:36.458
and access control measures.

21:36.458 --> 21:37.333
It also covers

21:37.333 --> 21:38.125
the identification

21:38.125 --> 21:39.916
of third-party libraries,

21:39.916 --> 21:42.916
cloud services, and DevSecOps practices

21:42.916 --> 21:44.041
to inform data

21:44.041 --> 21:45.708
consumers about the provenance

21:45.708 --> 21:46.875
and processing

21:46.875 --> 21:49.916
of datasets.
Data Formats and Pedigree:

21:50.750 --> 21:53.250
A robust data management plan alongside

21:53.250 --> 21:53.750
a common

21:53.750 --> 21:55.333
T&E data and program

21:55.333 --> 21:57.416
artifact repository ensures

21:57.416 --> 21:58.583
that all T&E data,

21:58.583 --> 21:59.875
including digital engineering

21:59.875 --> 22:00.833
models and related

22:00.833 --> 22:01.583
data,

22:01.583 --> 22:03.291
are visible, accessible,

22:03.291 --> 22:05.000
understandable, linked,

22:05.000 --> 22:06.541
trusted, interoperable,

22:06.541 --> 22:07.416
and secure.

22:07.416 --> 22:09.833
All relevant metadata,

22:09.833 --> 22:11.916
such as descriptions of the data,

22:11.916 --> 22:12.791
the conditions under

22:12.791 --> 22:13.750
which it was collected,

22:13.750 --> 22:15.416
and any deviations from test

22:15.416 --> 22:17.333
plans, are captured.

22:17.333 --> 22:18.750
A data dictionary provides

22:18.750 --> 22:20.208
plain-English descriptions

22:20.208 --> 22:21.958
of each data field, and unique

22:21.958 --> 22:23.375
identifiers are used

22:23.375 --> 22:25.833
for all entities captured in the data.

22:25.833 --> 22:27.416
Data that has passed through a data

22:27.416 --> 22:29.416
authentication group is clearly marked

22:29.416 --> 22:31.458
and segregated from unauthenticated

22:31.458 --> 22:32.833
data, and maintained

22:32.833 --> 22:34.583
unchanged.
Compliance

22:34.583 --> 22:35.750
with Records Management:

22:36.833 --> 22:38.166
All data and records

22:38.166 --> 22:40.375
produced from T&E activities are managed

22:40.375 --> 22:42.041
in accordance with approved records

22:42.041 --> 22:44.083
schedules, ensuring compliance

22:44.083 --> 22:46.041
with all regulatory requirements.

22:46.041 --> 22:47.125
Through these structured

22:47.125 --> 22:48.458
data management practices,

22:48.458 --> 22:51.125
the DoD ensures that T&E processes

22:51.125 --> 22:52.375
are not only compliant

22:52.375 --> 22:54.250
with current laws and policies

22:54.250 --> 22:55.458
but also optimized

22:55.458 --> 22:57.041
for security, efficiency,

22:57.041 --> 22:58.750
and interoperability.

22:58.750 --> 23:00.000
This approach enhances

23:00.000 --> 23:01.125
the overall effectiveness

23:01.125 --> 23:03.250
of testing and evaluation activities,

23:03.250 --> 23:05.000
supporting the successful deployment

23:05.000 --> 23:08.000
and operation of defense systems.

23:08.875 --> 23:10.541
3.9.

23:10.541 --> 23:13.541
DOT&E OVERSIGHT.

23:13.750 --> 23:16.750
Understanding the T&E Oversight List:

23:16.791 --> 23:19.083
DOT&E is responsible for managing

23:19.083 --> 23:19.791
and publishing

23:19.791 --> 23:22.791
the unclassified T&E Oversight List.

23:22.791 --> 23:24.625
Classified and sensitive programs

23:24.625 --> 23:25.208
are identified

23:25.208 --> 23:26.458
directly to their Milestone

23:26.458 --> 23:29.291
Decision Authorities, also known as MDAs.

23:29.291 --> 23:32.750
Criteria for Inclusion:
Programs

23:32.750 --> 23:34.000
that meet the statutory

23:34.000 --> 23:35.916
definition of major defense

23:35.916 --> 23:37.083
acquisition programs

23:37.083 --> 23:38.250
(MDAPs) as

23:38.250 --> 23:42.000
defined in Section 4201 of Title 10, U.S.

23:42.000 --> 23:43.541
Code.
Programs

23:43.541 --> 23:44.250
that exceed

23:44.250 --> 23:45.458
or have the potential

23:45.458 --> 23:46.833
to exceed the dollar value

23:46.833 --> 23:48.916
threshold for a major program.

23:48.916 --> 23:50.416
Programs with a high level

23:50.416 --> 23:52.250
of congressional or DoD interest.

23:52.250 --> 23:55.291
OR
DoD systems that provide

23:55.291 --> 23:56.625
or enhance critical mission

23:56.625 --> 23:58.666
warfighting capabilities or represent

23:58.666 --> 23:59.791
a significant change

23:59.791 --> 24:01.041
to an existing system.

24:02.208 --> 24:03.958
Concurrence and Reporting:

24:03.958 --> 24:06.625
DOT&E reviews and issues a concurrence

24:06.625 --> 24:09.250
decision on T&E concepts for OT&E

24:09.250 --> 24:09.833
and LFT&E.

24:09.833 --> 24:11.333
For programs

24:11.333 --> 24:13.750
on the T&E Oversight List, DOT&E

24:13.750 --> 24:14.750
submits an independent

24:14.750 --> 24:16.375
report to the SecDef,

24:16.375 --> 24:18.625
the MDA, the Joint Staff,

24:18.625 --> 24:19.666
congressional defense

24:19.666 --> 24:21.083
committees, Services,

24:21.083 --> 24:23.458
and other key stakeholders.

24:23.458 --> 24:24.958
This report supports

24:24.958 --> 24:26.958
critical acquisition decisions

24:26.958 --> 24:27.541
and ensures

24:27.541 --> 24:29.375
transparency and accountability

24:29.375 --> 24:31.375
in the defense acquisition process.

24:31.375 --> 24:34.500
As part of this thorough oversight

24:34.500 --> 24:35.708
and review process,

24:35.708 --> 24:37.375
DOT&E confirms

24:37.375 --> 24:39.416
that all programs under its purview

24:39.416 --> 24:41.000
achieve the necessary operational

24:41.000 --> 24:43.000
effectiveness, suitability,

24:43.000 --> 24:46.000
and survivability.

24:46.458 --> 24:48.125
SECTION 4: OT&E

24:48.125 --> 24:51.750
AND LFT&E PROCESS
Welcome to Section 4.

24:52.375 --> 24:53.708
Now that you have an overview

24:53.708 --> 24:56.458
of what OT&E and LFT&E are,

24:56.458 --> 24:57.791
it is time to introduce you

24:57.791 --> 25:00.916
to the process of OT&E and LFT&E as it is

25:00.916 --> 25:02.208
explained in the Instruction.

25:03.333 --> 25:04.166
4.1.

25:04.166 --> 25:06.625
TEST PLANNING.

25:06.625 --> 25:09.625
Integration and Strategy Development:

25:09.666 --> 25:11.750
OT&E and LFT&E planning

25:11.750 --> 25:13.416
begins with the acquisition program

25:13.416 --> 25:14.791
initiation, running

25:14.791 --> 25:16.166
parallel to the development

25:16.166 --> 25:17.833
of the acquisition strategy.

25:17.833 --> 25:20.500
Early integration is crucial

25:20.500 --> 25:21.958
for developing the initial TEMP

25:21.958 --> 25:23.458
and T&E strategy,

25:23.458 --> 25:24.541
which informs program

25:24.541 --> 25:27.125
requirements and contracts.

25:27.125 --> 25:28.000
Planning continues

25:28.000 --> 25:29.083
throughout the acquisition

25:29.083 --> 25:31.083
lifecycle, with statistical inference

25:31.083 --> 25:33.166
methods allowing for dynamic updates

25:33.166 --> 25:34.958
to planning products.

25:34.958 --> 25:36.541
This supports key acquisition

25:36.541 --> 25:38.083
decisions and maximizes

25:38.083 --> 25:39.291
the use of Contractor

25:39.291 --> 25:43.375
Testing (CT), Developmental Testing (DT),

25:43.375 --> 25:46.833
OT&E, LFT&E data, and M&S

25:46.833 --> 25:49.833
results.

25:49.958 --> 25:50.833
4.2.

25:50.833 --> 25:53.833
TEST PREPARATION.

25:54.166 --> 25:54.916
The Test Readiness

25:54.916 --> 25:55.375
Review –

25:55.375 --> 25:56.000
or TRR

25:56.000 --> 25:59.291
– process, ensures our defense systems

25:59.291 --> 26:00.958
meet the highest standards

26:00.958 --> 26:02.291
of operational readiness

26:02.291 --> 26:03.958
and effectiveness.

26:03.958 --> 26:05.833
It begins with a comprehensive review

26:05.833 --> 26:06.833
of testing results

26:06.833 --> 26:10.750
from CT&E, DT&E, and integrated T&E,

26:11.291 --> 26:13.208
and assessing their qualification

26:13.208 --> 26:16.208
for OT&E and LFT&E phases.

26:16.583 --> 26:17.250
This is followed

26:17.250 --> 26:18.666
by an in-depth assessment

26:18.666 --> 26:19.333
of the system's

26:19.333 --> 26:20.666
operational effectiveness,

26:20.666 --> 26:22.708
suitability, survivability,

26:22.708 --> 26:24.583
and lethality.

26:24.583 --> 26:25.750
A critical technical risk

26:25.750 --> 26:27.791
analysis is performed to verify

26:27.791 --> 26:30.541
that risks are appropriately managed.

26:30.541 --> 26:33.166
Certification reviews confirm adherence

26:33.166 --> 26:35.625
to safety standards and regulations.

26:35.625 --> 26:37.333
The process also evaluates

26:37.333 --> 26:38.250
entrance criteria

26:38.250 --> 26:41.166
specified in the TEMP or T&E strategy,

26:41.166 --> 26:43.250
ensuring alignment with strategic goals.

26:44.416 --> 26:45.166
Finally,

26:45.166 --> 26:47.833
a detailed examination of test plans,

26:47.833 --> 26:50.750
including objectives, schedules, methods,

26:50.750 --> 26:53.750
procedures, and other relevant factors,

26:53.750 --> 26:54.541
is conducted

26:54.541 --> 26:56.291
to ensure thorough preparation

26:56.291 --> 26:58.000
for operational testing.

26:58.000 --> 26:59.958
4.3.

26:59.958 --> 27:02.958
TEST EXECUTION.

27:03.416 --> 27:05.791
Adherence to Approved Plans:

27:05.791 --> 27:08.958
Execution of Tests: 
OTAs and LFT&E

27:09.000 --> 27:11.166
organizations must execute tests

27:11.166 --> 27:13.208
strictly according to approved test,

27:13.208 --> 27:14.125
Modeling & Simulation

27:14.125 --> 27:16.458
(M&S), and M&S Verification

27:16.458 --> 27:20.208
and Validation (V&V) plans,

27:20.750 --> 27:22.083
meeting all requirements

27:22.083 --> 27:23.500
by the end of the event.

27:23.500 --> 27:25.500
Handling Deviations:

27:25.500 --> 27:27.666
Any changes to OT&E, LFT&E,

27:27.666 --> 27:29.750
and M&S events

27:29.750 --> 27:30.916
must receive prior

27:30.916 --> 27:31.750
concurrence from DOT&E.

27:31.750 --> 27:34.333
M&S results

27:34.333 --> 27:36.583
must be accredited for their intended use

27:36.583 --> 27:38.375
unless waived by DOT&E.

27:38.375 --> 27:39.833
Urgent deviations

27:39.833 --> 27:41.750
must be communicated to DOT&E

27:41.750 --> 27:44.375
as soon as possible.
4.4.

27:44.375 --> 27:47.375
ANALYSIS AND EVALUATION.

27:47.500 --> 27:49.583
Independent and Collaborative Analysis:

27:49.583 --> 27:51.583
Both the OTA

27:51.583 --> 27:52.583
and DOT&E

27:52.583 --> 27:54.916
conduct independent analyses of test

27:54.916 --> 27:56.791
data and M&S results

27:56.791 --> 27:57.500
to ensure

27:57.500 --> 27:58.958
unbiased evaluations

27:58.958 --> 28:00.125
of a system's operational

28:00.125 --> 28:00.958
effectiveness,

28:00.958 --> 28:03.000
suitability, survivability,

28:03.000 --> 28:03.916
and lethality.

28:04.916 --> 28:05.291
They use

28:05.291 --> 28:06.416
appropriate statistical

28:06.416 --> 28:08.708
analysis and evaluation techniques

28:08.708 --> 28:10.416
to assess risks associated

28:10.416 --> 28:11.875
with full-rate production,

28:11.875 --> 28:13.875
minimal capability fielding,

28:13.875 --> 28:15.791
or early fielding decisions.

28:15.791 --> 28:17.000
Preliminary assessments

28:17.000 --> 28:18.083
evaluate risks

28:18.083 --> 28:21.083
for the next phase of OT&E and LFT&E,

28:21.291 --> 28:22.458
ensuring operational

28:22.458 --> 28:24.166
requirements are met for interim

28:24.166 --> 28:25.875
acquisition decisions.

28:25.875 --> 28:27.583
It is crucial to confirm

28:27.583 --> 28:28.791
that collected test data

28:28.791 --> 28:30.958
support OT&E and LFT&E

28:30.958 --> 28:32.875
objectives and established metrics.

28:32.875 --> 28:35.875
M&S events must meet their objectives,

28:35.916 --> 28:36.666
and M&S

28:36.666 --> 28:38.416
Verification, Validation,

28:38.416 --> 28:40.958
and Accreditation (VV&A)

28:40.958 --> 28:42.416
must be adequate before

28:42.416 --> 28:44.916
starting M&S runs-for-record.

28:44.916 --> 28:46.125
Lessons learned from data

28:46.125 --> 28:47.416
collection, analysis,

28:47.416 --> 28:50.250
and evaluation processes are recorded

28:50.250 --> 28:50.916
and applied

28:50.916 --> 28:51.916
to improve methods

28:51.916 --> 28:54.291
and processes continuously.

28:54.291 --> 28:55.583
System safety hazards,

28:55.583 --> 28:57.625
both hardware and software,

28:57.625 --> 28:58.708
are thoroughly assessed

28:58.708 --> 29:01.708
to ensure safety and reliability.

29:02.083 --> 29:02.916
Thorough these rigorous

29:02.916 --> 29:06.000
analytical processes, the OTA and LFT&E

29:06.375 --> 29:07.875
organizations validate

29:07.875 --> 29:09.166
that defense systems

29:09.166 --> 29:10.666
are effective and suitable,

29:10.666 --> 29:12.000
supporting the DoD’s

29:12.000 --> 29:13.125
strategic objectives

29:13.125 --> 29:14.833
and safeguarding our warfighters.

29:16.625 --> 29:17.500
4.5.

29:17.500 --> 29:19.875
TEST REPORTING

29:19.875 --> 29:21.458
Security and Distribution

29:21.458 --> 29:22.750
of Information:

29:22.750 --> 29:24.083
All reporting must adhere

29:24.083 --> 29:26.000
to applicable classification guidance

29:26.000 --> 29:27.583
when classifying and handling

29:27.583 --> 29:29.833
test data and reports.

29:29.833 --> 29:32.416
The program manager, OTAs,

29:32.416 --> 29:33.916
LFT&E organizations,

29:33.916 --> 29:34.833
and DOT&E

29:34.833 --> 29:36.875
are responsible for providing the Defense

29:36.875 --> 29:38.416
Technical Information Center

29:38.416 --> 29:39.875
with all reports,

29:39.875 --> 29:42.458
supporting data, and M&S results,

29:42.458 --> 29:44.166
ensuring they are archived

29:44.166 --> 29:46.208
and accessibility.

29:46.208 --> 29:48.250
Reports and data are also coordinated

29:48.250 --> 29:49.083
with the DoD

29:49.083 --> 29:50.583
CIO to integrate

29:50.583 --> 29:52.083
operational testing insights

29:52.083 --> 29:55.083
into broader security measures.

29:55.291 --> 29:57.333
To learn more about the DoDI

29:57.333 --> 29:59.291
5000.98 and view

29:59.291 --> 30:01.541
the policy in its entirety,

30:01.541 --> 30:02.375
please visit

30:02.375 --> 30:04.250
the Guidance section on the DOT&E

30:04.250 --> 30:08.875
website, at dote.osd.mil .

30:09.250 --> 30:09.625
Thank you.