作為一名專業的自動化測試工程師，不應該僅僅侷限於對工具的掌握和使用，應該建立測試的自動化知識體系（ ABOK ）： http://www.automatedtestinginstitute.com/home/index.php?option=com_content&view=category&id=69&Itemid=95

 

 

Automation Body of Knowledge (ABOK) 自動化知識體系

The Automation Body of Knowledge is a tool-independent skill set designed to help software test automation professionals address automation challenges that are present in the world of software testing. （ABOK是獨立於工具的技巧集，設計的目的是幫助軟體測試自動化工程師處理現實世界中的軟體測試自動化的挑戰。）Automated Software Testing is a discipline that is separate from Manual Software Testing, and must be treated as such.（軟體自動化測試應該作為一個獨立於手工測試的學科出現。） The ABOK acknowledges this, and provides engineers with a way to assess, improve, and better market their test automation skills more effectively than tool-specific benchmarks can do alone. This body of knowledge may also be used by organizations as specific criteria for more effectively assessing resources and establishing career development tracks. Not every automation engineer is required to be proficient in every one of the ABOK skill categories, but knowledge of the different skill categories is essential in the continued improvement, growth and development. It is recommended that each test automation effort, however, have a team that represents most, if not all of these skills, whether the team is composed of one or many resources.

The Skill categories are listed below, and may be clicked on to get a summary of each category. Skill categories 1 through 7 (macroscopic) include skill concentrations for automated test leads, while skills 8 through 12 (microscopic) include skill concentrations for automated test engineers. Given the state of many software projects, the Test Lead depends on the Automation Engineer for answers regarding automation planning, therefore the Automation Engineer may also want to consider a concentration in skills 1 through 7 as well.  

 

Skill Category 1: Automation's Role in the STLC 自動化在軟體測試生命週期（ STLC ）中的角色

Being successful in test automation first requires an understanding of what test automation is, and where it fits into the overall testing lifecycle.  

Difference between software testing and software test automation 軟體測試自動化與軟體測試之間的區別

Software Testing is a discipline in which test logic is designed to be implemented by a human being. As opposed to, Software Test Automation designs script logic that allows that manual test logic to be implemented by a tool. An expansion of this concept asserts that software test automation involves tool support for all aspects of a test project, not just automation of test execution. With that being said, as a test automator, you should be aware of all phases of the testing lifecycle, along with tools that support those phases.  

Test Tool Acquisition and Integration 測試工具選購與整合

The Test Tool Acquisition is the process of identifying the tool goals and objectives, making an effective business case and narrowing down a group of candidate tools to the selection(s) that best meet(s) the organizational needs. Test Tool Integration is, conversely, the process of gradually addressing tool considerations and widening the tool’s acceptance, use and benefits.The various sources provide extensive directions on how to conduct the tool selection and acquisition process. Most sources will agree, however, that this process should at a minimum include the following:

• Making a well-informed decision to automate
• Conducting a test tool evaluation
• Addressing tool implementation considerations
• Piloting the tool implementation

Often, organizations are pushed into making a decision about the introduction or expansion of test automation based on the ability to buy a specific automated test tool. It’s important to expand your vision to be able to identify several options – both commercial and non-commercial – and to be knowledgeable on how to perform an effective evaluation of these options so that a more informed decision can be made about test automation implementation.   

Automation Benefits and Misconceptions 自動化的益處與誤解

Behind every decision to introduce test automation is a wide variety of stated or implied goals and expectations for what the automation must accomplish. Without a proper understanding of reasonable test automation goals, the automation effort will not survive, and will be destined to sit on the proverbial shelf of your test organization. Understanding the impact of test automation means having a firm grasp on the benefits of test automation, accompanied by an understanding of the common misconceptions surrounding test automation. Being equipped with this knowledge helps a Test Automator more effectively implement a test automation effort, “sell” the automation to the powers-that-be by presenting and tracking a business case, and manage effort-ending misconceptions and unrealistic expectations held by the organization decision makers.  

Automation Return-on-Investment (ROI) 自動化的 ROI 計算

Software test automation is an investment in time and money, and like any investment, the stakeholders – normally managers or customers – are expecting a positive return from that investment. If their expectations are not sufficiently met, stakeholders will probably discontinue the investing in automation. The investment is justified by identifying the potential return-on-investment (ROI), a ratio of benefits to costs. There are several approaches for calculating ROI including:

• Simple ROI Calculation – A percentage calculated in terms of the monetary savings that test automation provides.
• Efficiency ROI Calculation – Examines test automation benefits in terms of time that is saved on certain portions of the testing effort.
• Risk Reduction ROI Calculation – A percentage calculated in terms of the increased quality that results from the increased test coverage provided test automation.

 

Skill Category 2: Test Automation Types and Interfaces 測試自動化的型別和介面

Test Automation Types 測試自動化的型別

For simplicity the different types of test automation – functional (regression), unit, integration, performance, etc. – are often lumped into one ‘test automation’ category. It’s important to understand some of the basic differences among these different types, even if you don’t specialize in all of them, so that you can speak intelligently about them when questioned by members of the organization. Since all automators are often “painted with the same brush”, being totally oblivious to basic concepts in each type of automation can negatively impact all automation efforts in the organization. In addition, being able to display minimal knowledge in various types of automation may provide management with enough confidence in you to allow you to pilot a new automation effort, which would help you to ultimately expand your skills.  

Test Automation Interfaces 測試自動化的介面

The main interfaces made available for application automation are:

• Command Line Interface 命令列介面
• Application Programming Interface 應用程式程式設計介面
• Graphical User Interface 圖形使用者介面

The Graphical User Interface is probably the most popular interface for automation implementation, due to the fact if more closely simulates how a user might use the tool. The Command Line Interface and Application Programming Interface, however, are advantages given the fact that they make it simpler to implement test automation.

 

Skill Category 3: Automation Tools 自動化工具

A test automation professional also must be prepared to provide insight into tools that support all aspects of the testing lifecycle. This requires an understanding of the different types and categories of tools that support the testing lifecycle, as well as the criteria for assessing these different types of tools.

These different types of tools include:

• Software Configuration Management Tools
• Business/System Modeling Tools
• Requirements Management Tools
• Unit Testing Tools
• Test Management Tools
• Defect Tracking Tools
• Code Coverage Analyzer Tools
• Functional System Test Automation Tools
• Performance System Test Automation Tools

 

Skill Category 4: Test Automation Frameworks 測試自動化框架

Automation Scope 自動化的範圍

Total automation cost is composed of both development costs and maintenance costs.  （自動化的成本由開發成本和維護成本組成） As the automation framework becomes more defined, scripting increases in complexity. While this causes development costs to increase, it also causes maintenance costs to decrease. As the scope widens, maintenance becomes increasingly important. （隨著測試範圍的擴大，維護成本變得重要起來） Also, as maintenance becomes increasingly important, the automation framework must be more advanced in order to reduce total automation costs. Therefore, before making a determination of the framework that will be used for test automation, an evaluation must be conducted on the implied and/or stated scope of the organization’s automation effort.

 

Roles and Responsibilities 角色和職責

Each framework type will have one or more of the following roles for creation and implementation of the framework:

• Team Lead 測試主管
• Test Engineer 測試工程師
• Lead Automation Architect 自動化測試架構師
• Cross Coverage Coordinator 組織協調人員
• Automation Engineer (Automator) 自動化測試工程師

Very often, one person may hold multiple roles.

 

Framework Generations 框架的發展

Over the years, automated testing has evolved, becoming increasingly defined and sophisticated with each new evolutionary phase.  This evolution may be discussed in terms of three generations:

·         First Generation – This generation is what this book refers to as the Common Approach.  It is primarily driven by record and playback. 第一代 – 錄製回放

• Second Generation – The second-generation learns from the problems of the first generation.  It involves creating a framework in which to automate tests. It also incorporates the concepts of functional decomposition, which simply means creating modular functions for executing fundamental actions that occur in the execution of the tests in a given test bed.  These actions may then be called upon and executed independent of one another.  This generation also more greatly separates test data from automation code, through greater use of parameterization. 第二代 – 功能分解

·         Third Generation – The third-generation of automation builds upon the concepts of the second generation.  It strongly depends on functional decomposition, but evaluates the functions at a much higher level of abstraction, so that code may be reused by tests within the same application, and by tests in different applications.  This is accomplished by separating test data and specific application functionality from automation code. 第三代 – 資料與程式碼分離

During the design phase a determination must be made about which generation will define your automated test framework.

 

Skill Category 5: Automation Framework Design 自動化框架設計

The process of designing an automated test framework is not an exact science. It is possible to definitively identify the different types of frameworks, but the process used to select and implement a particular framework is a little more difficult to pinpoint in such a way that most of the industry experts agree. The most important thing however, at this point in automation history is to ensure that a well thought out approach based on common, successfully implemented industry practices is used and honed within a given organization. This skill category identifies an approach, from a high enough level that it will fit with where the IT industry currently is relative to test automation, while still remaining low-level enough to be useful for implementation. This approach involves the following steps:

1. Selecting a Framework Type 選擇框架型別
2. Identifying Framework Components 確定框架組成部分
3. Identifying Framework Directory Structure 確定框架目錄結構
4. Developing Implementation Standards 建立實施標準
5. Develop Automated Tests 開發自動化測試

 

Skill Category 6: Automated Test Script Concepts 自動化測試指令碼思想

Test Selection 測試用例選擇

Choosing what and when to automate involves an analysis of the following:

• Automate items that are tedious for manual execution, but relatively easy to automate. 對於那些手工測試比較繁瑣而又容易實現自動化的優先考慮自動化實現。
• Items that need to be executed over and over again. 對於那些需要重複又重複執行的測試用例實現自動化。
• Items that will increase coverage beyond what will realistically be done manually 對於那些能有效提高測試覆蓋率的測試用例實現自動化。

In addition, the decision depends on the goals of the organization such as cost goals, efficiency goals, and risk mitigation goals.  

Automated Test Design and Development 自動化測試的設計和開發

Automated test design and development are tied directly to the interface in which the tests will be created, the way quality attributes are applied at the test level, the elements that are common to automated tests, and the standards used for creation of tests.  

Automated Test Execution, Analysis and Reporting 自動化測試的執行、分析和報告

The biggest items to address regarding test execution are:

• How many machines will be used for automated test execution
• Error handling
• How the results are reported and analyzed

 

Skill Category 7: Quality Attribute Optimization 質量優化

This category defines various quality attributes of the automated test suite and identifies ways of addressing these attributes based on priorities and constraints surrounding each.

The following are some Quality Attributes that may be considered.

• Maintainability
• Portability
• Flexibility
• Robustness
• Scalability
• Reliability
• Usability
• Performance

 

Skill Category 8: Programming Concepts 程式設計思想

Whether you use a tool with a scripting language, tree structure and/or keywords, fundamental programming concepts remain paramount in effectively automating tests, and increasing the distance the test can cover through increased system coverage and test flexibility. Concepts such as variables, control flow statements (if..then..else, for..next, etc.), and modularity are discussed in this category.

 

Skill Category 9: Automation Objects 自動化物件

Recognizing Application Objects 識別應用程式物件

Once upon a time, functional software test automation primarily used an “analog” approach that relied on specific coordinate locations of a screen or application window for performing mouse and keyboard operations that simulated user activities on the application. This was slightly unreliable, and difficult to maintain on a large scale, because when the screen, window or application components moved ever so slightly, the automated test would fail because it was trying to operate on an element in a position in which it no longer existed. Modern test automation conversely takes a different approach that locates the objects on the screen based on properties that define that object and then performs the desired operations once the object is found.  

Object Maps 物件對映

One of the simplest ways to boost the maintainability and robustness of an automated test suite is through the introduction of Object Maps. Object Maps reduce the amount of information that is being maintained in an automated test by storing object properties in an external file and referencing objects according to variable names (also known as Logical Names) that are then tied to the properties. 

Object Models 物件模型

Many software applications are composed of several interrelated objects, and an object model is an abstract representation of the hierarchical group of related objects that define an application and work together to complete a set of application functions. The advantages offered by object models include: 

·   Increased application understanding

·   Increased scripting power

Dynamic Object Behavior 動態物件行為

One of the biggest challenges faced by automated test engineers is that of dynamic object behavior. People process information about the application based on visual inspection, while automated tests use object properties. People can, therefore, easily make adjustments when there is a slight change from a visual standpoint, and can ignore many property changes. The computer, however, cannot adjust as easily. The necessary adjustments have to be anticipated up front and programmed.

 

Skill Category 10: Debugging Techniques 除錯技巧

Types of Errors 錯誤型別

Regardless of how well automated tests are designed and created, problems will occur. Sometimes the problems are related to the scripts, and sometimes they are related to the application, but the root cause is not always simple to find. The inability to effectively debug scripts can severely delay schedules, and can even bring automation to a screeching halt.  The first step in script debugging is in understanding the types of errors that may be encountered. There are 4 main generic types of errors that may be encountered upon script execution:

• Syntax Errors 語法錯誤
• Run-time Errors 執行時錯誤
• Logic Errors 邏輯錯誤
• Application Errors 應用程式錯誤

Debugging Techniques 除錯技巧

The trickiest part of debugging is finding out the true source of an error. This involves ensuring the error is reproducible, and then the process of finding the main source of the error must begin. Very often what seems to be the source of the error is actually just a symptom of the true error, so there are several techniques that may be employed to find the source of an error. At that point a determination may be made on whether or not the error is due to an application failure or a script issue. Then solutions for fixing the error may be addressed. Effective debugging typically mirrors the following process: 

• Identifying error existence 識別錯誤
• Reproducing the error 重現錯誤
• Localizing the error 定位錯誤
• Fixing the error 修正錯誤

 

Skill Category 11: Error Handling 錯誤處理

Error Handling Implementation 錯誤處理的實現

Error handling is implemented in a variety of ways within automated test scripts, most of which can be summarized in the following three categories:

• Step Implementation
• Component Implementation
• Run Implementation

Error Handling Development 錯誤處理指令碼的開發

Error handling routines are critical for test automation, particularly for effectively resolving runtime errors and other unexpected system events, because excessive runtime errors are the enemy of test automation robustness. The figure below illustrates an error handling development process that may be used for implementing a successful error handling approach. These steps include:

• Diagnose Potential Errors 診斷出潛在的錯誤
• Define Error Capture Mechanism  定義錯誤捕獲機制
• Create Error Log Data 建立出錯日誌
• Create Error Handler Routine 建立錯誤處理函式

 

 

 

Skill Category 12: Automated Test Reporting 自動化測試報告

Test reporting and analysis is an extremely repetitive process, yet can be extremely time consuming. This category addresses what types of reports could be generated and how to effectively produce these reports, so that time may be saved in the analysis and reporting. The categories of reports that may be generated by an automated test framework include:

·   High-level (Suites/Tests) reports   高層（測試集/測試指令碼）報告

·   Low-level (Verification Points) reports  底層（驗證點）報告