System Autonomy Modeling during Early Concept Definition

—The current rapid systems engineering design methods, such as AGILE, significantly reduce the development time. This results in the early availability of incremental capabilities, increases the importance of accelerating and effectively performing early concept trade studies. Current system autonomy assessment tools are level based and are used to provide the levels of autonomy attained during field trials. These tools have limited applicability in earlier design definition stages. An algorithmic system autonomy tool is needed to facilitate trade off studies, analyses of alternatives and concept of operations performed during those very early phases. We developed our contribution to such a tool and described it in this paper. I. INTRODUCTION The United States Department of Defense (USDoD) is facing declining defense budgets for at least the next several years while adversary nation experience double digit defense budget increases1. In this fiscal environment, the USDoD must find new ways in meeting the goal of providing national security. A significant portion of the budget is for manpower in the operations and support phase of the system life cycle. Unmanned autonomous systems can provide this force multiplier 2 allowing a single operator to manage multiple unmanned systems[9][24]. Autonomy for unmanned systems is the needed technological innovation to reduce the workload of human operators. This technological demand is greatest in military operations where significant loss of life and extreme hazardous situations are common place.


I. INTRODUCTION
The United States Department of Defense (USDoD) is facing declining defense budgets for at least the next several years while adversary nation experience double digit defense budget increases1.In this fiscal environment, the USDoD must find new ways in meeting the goal of providing national security.A significant portion of the budget is for manpower in the operations and support phase of the system life cycle.Unmanned autonomous systems can provide this force multiplier 2 allowing a single operator to manage multiple unmanned systems [9] [24].Autonomy for unmanned systems is the needed technological innovation to reduce the workload of human operators.This technological demand is greatest in military operations where significant loss of life and extreme hazardous situations are common place.
Unmanned vehicles are a key component of the U. S. Navy (USN) defense transformation [28].The USN has several programs under development to address reduced manning with increased use of unmanned vehicles (UxV) [32].These unmanned vehicles require a significant amount of human interaction (HI) to control the UxV and interpret a significant amount of down linked data.Assessing intelligence, surveillance and reconnaissance (ISR) data to develop actionable security operations will continue to be a national priority.The amount of data collected is overwhelming the analysts.Current state of the art unmanned systems, like the Predator Unmanned Air Vehicle, require a sizeable team to operate the air vehicle, interpret sensory information, dynamically assess mission impacts and execute missions.The increasing demand for ISR missions are increasing crew support, counter to declining budget trends.

A. Background
The Congressional Budget Office in their FY2014 report anticipates that the portion of Gross Domestic Product (GDP) dedicated to the USDoD will continue to decrease over the next several decades [11].Future reduced funding for systems development will take a larger share of the operations, maintenance and personnel costs within the constrained budget [96].
To address this environment of declining defense budgets concurrent with increasing threats, the U. S. Navy is implementing unmanned technology in meeting the goal of providing national security at reduced cost [4].Autonomy is the needed technology to reduce manpower by allowing a single operator to manage multiple unmanned systems.
Autonomous systems results from complex integration of human intelligence and machine automation capable of adapting to unforeseen events [4].Autonomous systems could operate more independently and with lower focus levels of human interaction (HI), thus allowing for significant reductions in manpower.
The USN has several programs under development to address reduced manning through increased reliance of unmanned vehicles (UxV) and these systems require ever increasing levels of complex automation and autonomous capabilities.Proposed near-term maritime missions involve the use of collaborative unmanned autonomous systems.

B. Information Technology Acquisition Changes
The 2009 & 2011 National Defense Authorization Acts, Sec 804, mandated a new Information Technology (IT) Acquisition Process, Fig. 1 [30], was required because: The Defense acquisition process structured for weapon systems was ill-suited for information technology and  Systems take too long to deliver and inconsistent with technology cycles;  Documentation intensive, time consuming and process bound to respond effectively to end-user needs; www.ijacsa.thesai.org Funding process inconsistent with pace of evolving mission requirements;  Current metrics (financial, acquisition process) don't work well in measuring IT success;  Lack of meaningful trades between performance, cost and date-to-field;  Overly detailed requirements that are inconsistent with pace of technology change and need for rapid delivery;  Inability to prioritize requirements effectively;  Testing is integrated too late and serially;  Cyber-security is inadequately managed during the acquisition process;  Lack sufficient numbers of individuals with proven records of acquisition success;  Significant cultural impediments to change.
What is common across these definitions is the need to develop the "best" end product in response to a set of needs.This can be accomplished by execution of the systems engineering process where the requirements analyses and the allocation of those requirements to the Functional Analysis and Allocation are performed [10].Integrated tools to support analyses and assessments are critical at this early design phase because any shortfalls or miscalculations become costly if carried through the life cycle [1].The AGILE methodology is such an engineering process and focuses more on the collaborative efforts between the software developers and 'customers to allow for early capability releases [2][21].As a result, the releases are time driven rather than event driven which allows for maturing of the capabilities based on 'customer feedback'.This accelerated and iterative development release model is reliant on rational tools to support system analyses and requirements trade off studies as design deficits or errors become costly at later stages of the product life cycle.The AGILE methodology is appropriate for capabilities realized by software rather than implemented by hardware, which requires longer procurement and fabrication cycles [11] [33].

II. SYSTEM AUTONOMY ASSESSMENT
The USN has defined that autonomous systems results in a complex integration of human intelligence and machine automation capable of adapting to unplanned events changes encountered during mission operations [17].Current models assess system autonomy by assigning single numeric levels and do not support requirements trade off studies [4].On the low end of the autonomy scale, (tele-operations), a computer offers no assistance and the human operator must take make all decisions and actions [28].
Complete autonomy is at the other end of the scale as a computer decides everything and ignores the human being.As systems become more complex and the need for collaboration between these subsystems increases, a single numeric level describing autonomous capabilities is not adequate.Missions are becoming more complex and require systems of autonomous systems architectures that dynamically adapt to the varying levels of autonomous operations needed.Understanding the complex and dynamic relationship between human interaction, machine autonomy and the mission operational environment is critical in early candidate architecture trade studies.
AoA of system architecture designs have a significant impact on the mission concept of operations (CONOPS) and www.ijacsa.thesai.orgmust be efficiently done in those early stages of development [33].An approach to characterize autonomy in the early requirements modeling and trade-off studies is critical as large systems of systems development efforts are now quite commonplace [3].Although a significant body of work exists to assess autonomy, a mathematical relationship, as addressed by the Defense Science Board, does not exist to study the impacts of reduced manning and machine automation to meet mission success.Current methodologies and frameworks used have led to a misunderstanding of the level of autonomy required and developed.
While often interchanged, 'automation' and 'autonomy' are not synonymous and what is frequently referred to as a level of autonomy 'is a combination of human interaction and machine automation' (USN Chief of Naval Operations).The CNO continues to state that 'the degree of machine automation is not easily categorized' and not fully 'understanding autonomy has hindered development' of unmanned systems in the Navy.
As the USDoD acquisitions favor decreasing and rapid development cycles [4], the ambiguity in defining system autonomy, machine automation and human interaction contributes to alternate architecture assessment and trade studies leading to ambiguous requirements development.

A. Defense Science Board Task Force on Autonomy
The Defense Science Board (DSB) Task Force on Autonomy [4] 'reviewed many of the DoD-funded studies on "levels of autonomy" and concluded that they are not particularly helpful to the autonomy design process.These studies attempt to aid the development process by defining taxonomies and grouping functions needed for generalized scenarios.They are counter-productive because they focus too much attention on the computer rather than on the collaboration between the computer and its operator/supervisor to achieve the desired capabilities and effects.Further, these taxonomies imply that there are discrete levels of intelligence for autonomous systems and that classes of vehicle systems can be designed to operate at a specific level for the entire mission.' The DSB was asked to study relevant technologies, ongoing research and the current autonomy-relevant plans of the Military Services, to assist the USDoD in identifying new opportunities to more aggressively use autonomy in military missions, to anticipate vulnerabilities and to make recommendations for overcoming operational difficulties and systemic barriers to realizing the full potential of autonomous systems.
The DSB has concluded that autonomy technology is being underutilized as a result of obstacles within the USDoD inhibiting the acceptance of autonomy and unmanned systems.Key among these obstacles are a) poor design, b) lack of effective coordination of research and development and c) insufficient resources or time to refine concepts of operations [4].
The DSB states that 'Autonomy is a capability (or a set of capabilities) that enables a particular action of a system to be automatic or, within programmed boundaries, "selfgoverning."Unfortunately, the word "autonomy" often conjures images in the press and the minds of some military leaders of computers making independent decisions and taking uncontrolled action.While the reality of what autonomy is and can do is quite different from those conjured images, these concerns are in some cases limiting its adoption.It should be made clear that all autonomous systems are supervised by human operators at some level and autonomous systems' software embodies the designed limits on the actions and decisions delegated to the computer…Instead of viewing autonomy as an intrinsic property of an unmanned vehicle in isolation, the design and operation of autonomous systems needs to be considered in terms of human-system collaboration.' To address the issues that are limiting more extensive use of autonomy in USDoD systems, the DSB recommends [4] a crosscutting approach that includes the following key elements:  The DoD should embrace a three-facet (cognitive echelon, mission timelines and human-machine system trade spaces) autonomous systems framework to assist program managers in shaping technology programs, as well as to assist acquisition officers and developers in making key decisions related to the design and evaluation of future systems.
 The Joint Staff and the Military Services should improve the requirements process to develop a mission capability pull for autonomous systems to identify missed opportunities and desirable future system capabilities.

B. Mathematical Representation of System Autonomy
A system autonomy assessment tool must show a mathematical relationship between human interaction and machine automation [4].Being a software only model, this tool would be a good candidate for the AGILE development methodology.A workable and measurable definition of system autonomy (SA) is then defined as a functional of human interaction (HI) and machine automation (MA): If System Autonomy is considered as a vector, then the relationship between HI and MA would provide the scalar component.Mathematical assessment of SA as a vector representation is far more logical than using discrete integer levels.
The many unmanned air vehicles requires different levels of human interaction and supervisory control.Unmanned Air Vehicles range in sophistication and may need one or more human supervisors to successful carry out a surveillance mission.Equation 1 describes a single operator, single UMS configuration; the SA function from equation 1 above is modified as follows: HI = G[ HI 1 , HI 2 ,…HI n ], where n is the operators needed during the mission (2) www.ijacsa.thesai.orgAs the number of intelligence, surveillance and reconnaissance (ISR) missions increase, a single operator would control multiple UMS and in this scenario the equation is modified as: , where m is the number of UMS (3)

1) System Autonomy as a Vector
Fig. 2 graphically depicts System Autonomy as a vector in the MA/HI trade space.The magnitude of the SA vector is determined from the contributions of MA and HI component variables.The magnitude indicates whether the system architecture would meet mission objectives.The significance of the angle is discussed later.When the required value of the vector SA is set to a constant throughout the trade space, this defines the minimum autonomy levels needed to meet mission requirements.The dotted arc represents the Minimum Capability Threshold (MCT) where SA would meet this threshold.If the magnitude of the candidate system vector fell short of the MCT, then some mission objectives would not be accomplished.
Additional contributions from MA and HI would be needed to increase the magnitude of the SA vector.Magnitude exceeding the MCT indicates more than needed system autonomy to execute the mission.Normalizing the SA vector to a value of one (SA=1) allows further investigation to the relationship between MA and HI.Setting the SA vector to intersect with the HI axis sets the value for MA = 0 and HI = 1.This represents complete machine dependence on human interaction.Setting the SA vector to intersect with the MA axis sets the value for MA = 1 and HI = 0.This represents complete machine independence from human interaction.Maintaining SA=1 as the vector moves within this plane scribes the MCT arc and provide the mathematical relationship between SA, HI and MA.This spare capacity can be viewed as capability reserves or targeted for reduction as potential life cycle cost efficiencies.The magnitude of the vector becomes This allows the relationship between MA and HI to be defines as: Treating SA as a vector allows for analysis of candidate systems during the AoA and concept of operation activities where the systems design is developed.

2) SA Phase Angle
The angle Ψ, Fig. 3 provides an indication of the technology inherent in the configuration.The angle, with the scalar magnitude, describes SA as a vector.This allows vector mathematics when assessing system of autonomous system configurations.The angle is expressed as: The SA phase angle provides a relative comparison of the technology base for the candidate system.The smaller the difference in angles indicate that the candidate systems share the similar technology architectures and comparative analysis is relative straight forward.The greater the difference between the phase angles indicates that the systems have a diverse technological base making any comparison more complex.www.ijacsa.thesai.org

3) System Autonomy Trade Space
Expressing identifies the magnitude and phase angle of the vector.This provides the algorithmic assessment capabilities the current methods cannot provide.Fig. 4 shows two systems of equal magnitude.Both systems meet the MCT but the difference between the Ψs is 90 o and indicates an extreme divergence of technologies.One system is tele-operated, Level 1.The other system exhibits android behavior, Level 10 and does not depend on any human interaction [28].A comparison between the two systems architectures would not be straightforward because they operate in significantly diverse manners.For clarity, a systems candidate is shown in quadrant (Q1) unless uncooperative assessments are needed.Systems in different quadrants have vector components that would tend to negate, resulting in a smaller magnitude value.Systems in Q3 would be considered as countermeasures to systems in Q1 and are diametrically opposing forces.Systems in Q2 and Q4 have utility and assessments that may include fault, stress test or destabilizing scenarios.Future missions would include collaborative operations of more than one unmanned vehicle and the equation would be expanded to have two or more unmanned systems, UMS; where k is the number of UMS (8) Collaborative missions would include mixed UxV modes such as surface (USV), ground (UGV), air (UAV) and underwater (UUV) contributions.In the above relationship, UxV would be substituted by the appropriate type and number of UAVs, USV, UGVs and or UUVs as identified by the mission requirements.If one operator controls multiple UxVs, then the variable permutations of this model grow in complexity and a clear need for a model and methodology during AoA and CONOPS development becomes evident.The multiple combination UMS equation for SA becomes: The additive effect of SA from multiple subsystems is further described and depicted in section 0. Inclusion of dynamic variables like mission difficulty, meteorological impacts and many other probabilistic variations just increases the complexity of understanding and defining requirements.

4) Contextual System Autonomy
In previous sections, System Autonomy was discussed as a two dimensional vector.In more representative scenarios, system autonomy, human interaction and machine automation vary throughout the mission.Varying machine automation to meet mission needs is currently possible by commanding the machine to perform less than its maximum design capabilities allow.In some cases, new software can be downloaded to perform more efficiently.If the capability is not mechanically inherent in the machine, hardware reconfiguration by the machine itself is not supported by current technologies.The same may not be true of the human interaction element.Fig. 5 provides time as the third dimension to the trade space.Expanding the trade space to a third dimension should not infer a three dimensional SA vector.Instead the magnitude of the two dimensional SA vector is plotted against the third axis which represents the mission time.The mission phases may evolve and require a change from one type of UMS to another or a change of operator skills.In this case a Mission Phase would describe the system autonomy needed to conduct the mission phase peculiar activities.Mission Phase changes can appear as discontinuities in the SA level.www.ijacsa.thesai.org

5) A Dependency on Mission Phase
Time becomes a consideration in two ways.Complex mission scenarios may require several changes of system autonomy levels due to the changing phases of the mission like transit and area surveillance.This causes the SA vector to have a Mission Phase dependency.The combination of human supervision/interaction and the level of needed machine automation may need to vary within each discrete mission phase.This causes the SA Vector to have a time variant dependency.Fig. 6.Mission Phase Fig. 6 graphically represents the two dimensional SA vector throughout the notional mission duration.In this example, mission scenario with three phases -Φ1 is the transit to operation area, Φ2 is the surveillance reconnaissance and data gathering activity and Φ3 is the return transit.Each of these mission segments may require a specific level of SA.In this depiction, each SA is constant through the mission phases.This is not typical and most often observed is that there is some SA level variability with each mission phase. 3

SA Dependency on Time
System Autonomy or the Human Interaction can be a dynamic within a Mission Phase, Fig. 7.As an example, real time video could be collected during surveillance activity.The Human Interaction could be higher at the start of the data collection run as processor settings may need to be changed to accommodate the environmental conditions.The Human Interaction could be reduced during the data collection run and increase again at the end to verify data collection and processing.Human Interaction could also be a function of false positives that need to be interpreted and discounted.SA can be a linear or piece-wise aggregate of discrete action.As an example, during the launch phase, human interaction could be high at the very beginning, lower during system built in testing and high again just as the vehicle launches and separates from the cradle/gantry.
The SA and HI can be variable, distinctive and different for each Mission Phase.In the event that the system is composed of subsystems, then combined SA or HI is the accumulation of the individual subsystem contributions.The cumulative SA levels contributions from the subsystem involved in each Mission Phase is graphically shown in, Fig. 7.

III. COMPARATIVE AUTONOMY ASSESSMENTS
In 2007, Southwest Research Institute applied the Autonomy Levels For Unmanned Systems (ALFUS) framework to assess the achieved autonomy levels of eight www.ijacsa.thesai.orgunmanned ground vehicles (UGV) [23] 4 .ALFUS is a framework that has been developed by a consortium of government and non-government agencies during several workshops [18] [19].Note that in the ALFUS methodology, the variable HI is Human Independence not as used in this paper.The UGVs were categorized into four groups by market area and use.This allowed some narrowing of the definitions of Mission Complexity (MC) and Environmental Difficulty (ED) within each group.Even with this pre-filtering, some ambiguities in assessment existed and straight comparisons outside of the grouping are not straightforward.Within the Passenger Vehicle grouping, both UGVs require a human operator.The Human Operator in NavLab actuated the throttle and brake but not steering.The Operator in the NavLab had to monitor the UGVs unsafe lane positions and distances to vehicles.The ALFUS HI does not portray the involvement of the human operator as he would be required 100% of the time during these tests.Using the ALFUS methodology, SRI assessed the MC, ED, HI and ∑ for each UGV within the specific group constraints.In the ALFUS methodology, MC and ED seem to have some overlapping definitions.The mission complexity includes terrain and hostilities in the case of the Military grouping which spills into the environmental difficulty which also take into account terrain and hostilities.When SRI summed the three variables, the ALFUS autonomy assessments of the UGVs were very similar.
In the ALFUS methodology, the variable HI provides for human independence and not Human Interaction.If this variable is viewed as a form of machine automation (MA), then the algorithmic assessment can be applied, Error!Reference source not found..The algorithmic value of human interaction (HI and the technology base angle (Ψ) are calculated (normalized) for each UGV using equations 5 and 6.Although human operators were needed to operate some of the UGVs, no adjustments to the ALFUS derived levels were included.In the case of the NAVLAB UGV, the researched operated actuated the throttle and brake manually thereby increasing the human interaction to a greater level then indicated.When the UGVs are further segregated into subgroups, the algorithmic assessment inError!Reference source not found.Algorithmic (1) shows that the technology bases of the UGVs within each grouping maybe too diverse for straight comparisons.This is evident within the Passenger Vehicle category.Manual categorization into categories is not sufficient for assessment of system autonomy of between candidates of an AoA.Performance attenuating parameters such as terrain difficulty or hostilities can be applied in stochastic studies in developing concept of operations.The ALFUS methodology provides a combined label assessment and parametric sensitivity studies could not be performed easily.
In the analysis performed by SRI, the three ALFUS variables were summed and identified as Algorithmic (2) in the table .An alternate assessment of MA is done if the ALFUS variables are averaged and then applied as MA in a similar fashion done by SRI.As was found in the SRI assessments, the Ψs of the UGVs become numerically closer, indicating relative straight forward comparisons are possible.This could potentially increase the number of candidate systems during system requirement and AoA developments, not possible with current assessment methodologies and tools.As in the previous case, factors such as life cycle cost can be included for comparison.Parametric sensitivities and stochastic modeling can be performed to contribute to AoA, CONOPS and requirement development not possible with the ALFUS framework.Summarizing, a label based system autonomy tool has very limited usefulness in defining and developing system concepts.Label assessment tools do not provide visibility into system components or design contributors.Label assessment tools do not support parametric sensitivity or stochastic analyses.An algorithmic assessment tool can support design activities in developing system concepts.This is the inference reached by the DSB [4].
IV. SUMMARY AND CONCLUSIONS Autonomous systems result from a complex integration of human intelligence supervising machine automation to adapt to unforeseen events encountered during operations Although significant work has been undertaken, conventional SA assessment frameworks are not suited for trade studies in support of AoA, CONOPS and requirements development.Missions are becoming more complex and require everincreasing capabilities to adapt to varying unknown situations.Autonomy is a complex function of many dynamic and widely varying parameters and requires a mathematical relationship between Human Interaction and Machine Automation to provide the design tradeoff study capabilities needed during early development phases.The Defense Science Board stated that machine automation and human interaction assessments need algorithmic solutions instead of the label methodology.www.ijacsa.thesai.org The mathematical relationship described in this paper provides a basis for such a framework.Incremental and partial capabilities models can be developed using rapid design methodologies.Future development of System Autonomy Assessment tools would provide additional capabilities and mature the requirements refinement for the development of autonomous systems currently not available.

Fig. 1 .
Fig. 1.Long System Acquisition Cycle  Oversight process not aligned with rapid acquisitions (favors large programs, high-level oversight) and  Lack of accountability by personnel in the oversight process;  Complexity inherent in aligning three major USDoD processes -Requirements, Resourcing and Acquisition;

Fig. 2 .
Fig. 2. System Autonomy Trade SpaceAn alternative design for the system autonomy equation is a network of unmanned vehicles controlled by a single operator.This increases machine automation and facilitates a network of multiple UMS, operating concurrently and is supervised by a single controller.The alternate design would have multiple UMS operating sequentially and supervised by a single controller.

TABLE I .
ALGORITHMIC ASSESSMENT Comparisons of those UGVs with similar Ψs are straight forward and other factors such as life cycle cost can compared.