Sikuli測試應用研究報告

 

最近Sikuli的作者發表了一份研究報告《GUI Testing Using Computer Vision》，在報告中闡述了Sikuli作為GUI測試工具的5個設計目標：

 

Based on the findings of this interview, we identified the following five design goals to guide the design and development of our new GUI testing tool:

• (G1) The tool should allow testers to write scripts to automate tests.

• (G2) The tool should not require testers to refer GUI components by names or by locations.

• (G3) The tool should minimize the instances when test scripts need to be modified due to design changes.

• (G4) The tool should minimize the effort of writing test scripts.

• (G5) The tool should support good testing practices such as unit testing, regression testing, and test-driven development.

 

  

在報告中同時還闡述了Sikuli應用在GUI自動化測試時的的可測性研究和可重用性研究。

  

表明大部分的GUI控制元件和操作模式都能得到支援。在選取的Capivara、jEdit等軟體作為測試用例可重用性研究表明隨著版本的演進，受到介面設計和互動設計改變的影響，測試用例的可重用性會逐步下降。

 

  

在報告的最後，作者也提到Sikuli Test目前存在的兩大主要限制，一個是對於非預期的視覺化反饋的檢測，一個是不能測試GUI的內部功能。因此建議不要把Sikuli作為現有測試工具的替代品，而是一個有益的補充。

 

However, Sikuli Test currently has two major limitations that can be improved upon in the future. First, while Sikuli Test can assert what visual feedback is expected to appear or to disappear, it is unable to detect unexpected visual feedback. For example, if a programmer accidentally places a random image in a blank area, it is an undetectable error since no one would have anticipated the need to test that area with assertions. One solution would be to run the visual feedback detector at the test time to see if there is any detected visual feedback not covered by an assertion statement. Second, Sikuli Test is designed to test a GUI’s outward visual feedback and is thus unable to test the GUI’s internal functionalities. For example, while Sikuli Test can check if a visual feedback is correctly provided to the user who clicks the save button, it does not know if the file is indeed saved. One solution would be to treat Sikuli Test not as a replacement of but a complement to an existing testing tool. Together they make sure both the outward feedback and inward functionalities of a GUI can be sufficiently tested, a task neither can

accomplish alone.