桌面應用自動化winappdriver

松勤吳老師發表於2023-03-29
APP

桌面應用自動化winappdriver

關於winappdriver

介紹

  • WinAppDriver全稱是Windows Application Driver，它提供了一些API，使得使用者可以像selenium操作web一樣來操作windows的應用程式
  • 它支援的系統是Windows 10 (Home and Pro) 和Windows Server 2016
  • 原始碼暫未開源
  • WinAppDriver可以獨立執行，也可以作為appium的一個外掛來使用

支援應用型別

  • UWP – Universal Windows Platform, also known as Universal Apps or Modern Apps, It's Microsoft’s latest desktop application technology. It's XAML based. Only runs on Windows 10 machines

  • WPF - also XAML based, much more mature, runs on any Windows version and has been around since 2006.

  • WinForms - one of the older technologies, now found mostly on legacy applications.

    WPF和WinForms 是兩套介面渲染方式。一個是對傳統windows介面元素的封裝，透過gdi繪製。另一個是全新的dx渲染繪製的介面，也脫離了對傳統windows控制元件的依賴，沒有歷史包袱，理論上可以展現更炫酷的介面。

  • MFC/Classic Windows - MFC is a UI library normally paired with Win32 applications. This option is normally chosen when more efficiency is needed with low-level C++ handling or when supporting non-Microsoft platforms.

資源

素材 地址 說明
FlaUInspect https://github.com/FlaUI/FlaUInspect/releases 定位工具
WinAppDriver https://github.com/microsoft/WinAppDriver/releases/tag/v1.2.1
UIRecorder https://github.com/microsoft/WinAppDriver/tree/master/Tools/UIRecorder 定位工具
inspect 微軟官方工具整合於 Windows SDK 定位工具
  1. UIRecorder(下文不涉及,僅供參考與備忘)
  1. Open WinAppDriverUIRecorder.sln in Visual Studio
  2. Select Debug > Start Debugging or simply Run

支援的定位方式

Client API Locator Strategy Matched Attribute in inspect.exe Example
FindElementByAccessibilityId accessibility id AutomationId AppNameTitle
FindElementByClassName class name ClassName TextBlock
FindElementById id RuntimeId (decimal) 42.333896.3.1
FindElementByName name Name Calculator
FindElementByTagName tag name LocalizedControlType (upper camel case) Text
FindElementByXPath xpath Any //Button[0]

配置

開啟windows的開發者模式

  • 你沒看錯，不是手機，windows也有
  • 第一步：搜開發者設定
  • 第二步：開啟開發人員模式

  • 第三步：確認啟用

啟動winappdriver

  • 不開啟開發人員模式的提示

    C:\Program Files (x86)\Windows Application Driver>WinAppDriver.exe
Developer mode is not enabled. Enable it through Settings and restart Windows Application Driver
Failed to initialize: 0x80004005

  • 開啟後啟動winappdriver

    C:\Program Files (x86)\Windows Application Driver>WinAppDriver.exe
Windows Application Driver listening for requests at: http://127.0.0.1:4723/
Press ENTER to exit.

  • 還可以這樣啟動

    WinAppDriver.exe 4727
WinAppDriver.exe 10.0.0.10 4725
WinAppDriver.exe 10.0.0.10 4723/wd/hub   # 推薦

例項

appium-python-client 版本不要用2.0+，此處是1.2.0

記事本

  • 比如記事本

    from appium import webdriver
des_cap = {}
des_cap['app'] = r'C:\Windows\System32\notepad.exe'
driver = webdriver.Remote(command_executor='http://127.0.0.1:4723/wd/hub',
                          desired_capabilities=des_cap)
driver.implicitly_wait(5)
driver.find_element_by_name('檔案(F)').click()
from time import sleep
sleep(2)
driver.find_element_by_name('儲存(S)	Ctrl+S').click()
# driver.find_element_by_name('退出(X)').click()
sleep(1)
import pyautogui
pyautogui.PAUSE = 0.5
pyautogui.typewrite(r'D:\hello.txt')
pyautogui.press('enter')

  • 這裡的難點是儲存(S) Ctrl+S的獲取

  • 這裡需要用到inspect.exe

計算器

  • 你可能會寫這樣的程式碼
from appium import webdriver
des_cap = {}
des_cap['app'] = r'C:\Windows\System32\calc.exe'
driver = webdriver.Remote(command_executor='http://127.0.0.1:4723/wd/hub',
                          desired_capabilities=des_cap)
driver.implicitly_wait(5)
  • 但會報錯
Traceback (most recent call last):
  File "D:/demo_calc.py", line 5, in <module>
    desired_capabilities=des_cap)
  File "D:\Python37\lib\site-packages\appium\webdriver\webdriver.py", line 157, in __init__
    AppiumConnection(command_executor, keep_alive=keep_alive), desired_capabilities, browser_profile, proxy
  File "D:\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "D:\Python37\lib\site-packages\appium\webdriver\webdriver.py", line 226, in start_session
    response = self.execute(RemoteCommand.NEW_SESSION, parameters)
  File "D:\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "D:\Python37\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to locate opened application window with appId: C:\Windows\System32\calc.exe, and processId: 4472

程式已結束，退出程式碼為 1
  • 開啟計算器，然後在powershell中執行如下命令
Get-StartApps |Select-String "計算器"
# 輸出
@{Name=計算器; AppID=Microsoft.WindowsCalculator_8wekyb3d8bbwe!App} # 你要的是這裡的AppID
  • 程式碼
from appium import webdriver
des_cap = {}
des_cap['app'] = r'Microsoft.WindowsCalculator_8wekyb3d8bbwe!App'
driver = webdriver.Remote(command_executor='http://127.0.0.1:4723/wd/hub',
                          desired_capabilities=des_cap)
driver.implicitly_wait(5)
driver.find_element_by_name('一').click()
driver.find_element_by_name('二').click()
driver.find_element_by_name('加').click()
driver.find_element_by_name('三').click()
driver.find_element_by_name('四').click()
driver.find_element_by_name('等於').click()
# 透過inspect 獲取 automationID 
print(driver.find_element_by_accessibility_id('CalculatorResults').text) # 得到的是   ·顯示為 46·  你仍然要處理才能做測試

driver.quit()

計算器測試(官網)

我沒跑，僅供參考，你可以認為是為了增加篇幅

# https://raw.githubusercontent.com/microsoft/WinAppDriver/master/Samples/Python/calculatortest.py
import unittest
from appium import webdriver

class SimpleCalculatorTests(unittest.TestCase):

    @classmethod

    def setUpClass(self):
        #set up appium
        desired_caps = {}
        desired_caps["app"] = "Microsoft.WindowsCalculator_8wekyb3d8bbwe!App"
        self.driver = webdriver.Remote(
            command_executor='http://127.0.0.1:4723',
            desired_capabilities= desired_caps)

    @classmethod
    def tearDownClass(self):
        self.driver.quit()

    def getresults(self):
        displaytext = self.driver.find_element_by_accessibility_id("CalculatorResults").text
        displaytext = displaytext.strip("Display is " )
        displaytext = displaytext.rstrip(' ')
        displaytext = displaytext.lstrip(' ')
        return displaytext


    def test_initialize(self):
        self.driver.find_element_by_name("Clear").click()
        self.driver.find_element_by_name("Seven").click()
        self.assertEqual(self.getresults(),"7")
        self.driver.find_element_by_name("Clear").click()

    def test_addition(self):
        self.driver.find_element_by_name("One").click()
        self.driver.find_element_by_name("Plus").click()
        self.driver.find_element_by_name("Seven").click()
        self.driver.find_element_by_name("Equals").click()
        self.assertEqual(self.getresults(),"8")

    def test_combination(self):
        self.driver.find_element_by_name("Seven").click()
        self.driver.find_element_by_name("Multiply by").click()
        self.driver.find_element_by_name("Nine").click()
        self.driver.find_element_by_name("Plus").click()
        self.driver.find_element_by_name("One").click()
        self.driver.find_element_by_name("Equals").click()
        self.driver.find_element_by_name("Divide by").click()
        self.driver.find_element_by_name("Eight").click()
        self.driver.find_element_by_name("Equals").click()
        self.assertEqual(self.getresults(),"8")

    def test_division(self):
        self.driver.find_element_by_name("Eight").click()
        self.driver.find_element_by_name("Eight").click()
        self.driver.find_element_by_name("Divide by").click()
        self.driver.find_element_by_name("One").click()
        self.driver.find_element_by_name("One").click()
        self.driver.find_element_by_name("Equals").click()
        self.assertEqual(self.getresults(),"8")

    def test_multiplication(self):
        self.driver.find_element_by_name("Nine").click()
        self.driver.find_element_by_name("Multiply by").click()
        self.driver.find_element_by_name("Nine").click()
        self.driver.find_element_by_name("Equals").click()
        self.assertEqual(self.getresults(),"81") 

    def test_subtraction(self):
        self.driver.find_element_by_name("Nine").click()
        self.driver.find_element_by_name("Minus").click()
        self.driver.find_element_by_name("One").click()
        self.driver.find_element_by_name("Equals").click()
        self.assertEqual(self.getresults(),"8")

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(SimpleCalculatorTests)
    unittest.TextTestRunner(verbosity=2).run(suite)

相關文章