Allure測試報告完整學習筆記

測試開發剛哥發表於2022-01-23

測試報告筆記

簡介

假如你想讓測試報告變得漂亮一點，那你一定會在搜尋引擎中找到Allure測試報告的。Allure官方給出了個線上網站，訪問以下連結就能一睹為快：

https://demo.qameta.io/allure/

可以在左下角切換為中文報告：

如此好看的測試報告，哪個老闆不喜歡呢？

安裝Allure

1.在GitHub releases：

https://github.com/allure-framework/allure2/releases

或者Maven Central：

https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/

下載安裝包並解壓。

2.找到bin目錄，Windows使用allure.bat，Unix使用allure：

3.為了後續方便使用，可以把bin完整路徑新增到系統環境變數PATH中，然後在命令列直接輸入allure命令。

官方也給出了通過Linux的ppa、Mac的brew、Windows的scoop的安裝方式，不過對於我們們老百姓來說，不如手動下載包來安裝實在。

安裝後就可以檢查是否安裝成功了：

$ allure --version
2.17.2

因為Allure是用Java寫的，所以如果無法執行，那麼要檢查下有沒有安裝Java。

Allure測試報告的結構

Overview

Suites

Test result page

Graphs

Timeline

Behaviors

Packages

Java TestNG整合Allure Report

整合

在Maven的pom.xml中新增：

<properties>
    <aspectj.version>1.8.10</aspectj.version>
</properties>
<dependencies>
    <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-testng</artifactId>
        <version>LAST_VERSION</version>
        <scope>test</scope>
    </dependency>
</dependencies>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.20</version>
            <configuration>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                </argLine>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

然後執行命令：

$ mvn clean test

在target/allure-results目錄就會生成allure測試報告，再執行命令：

$ allure serve target/allure-results

Allure會建立一個Jetty伺服器，拉起預設瀏覽器開啟測試報告。

@Description

新增測試描述：

package my.company.tests;
import org.junit.Test;
import io.qameta.allure.Description;
@Test
public class MyTests {
    @Test
    @Description("Some detailed test description")
    public void testSomething() throws Exception {
        ...
    }
}

@Step

新增測試步驟描述，除了文字，Allure2通過反射機制，能夠在@Step中獲取引數，比如：

public class User {
     private String name;
     private String password;
     ...
}

import io.qameta.allure.Step;
...
@Step("Type {user.name} / {user.password}.")
public void loginWith(User user) {
     ...
}

@Attachment

新增測試報告的附件，可以是String或byte[]型別，不是顯式的話Allure會呼叫toString()嘗試隱式轉換：

import io.qameta.allure.Attachment;
...
@Attachment
public String performedActions(ActionSequence actionSequence) {
    return actionSequence.toString();
}
@Attachment(value = "Page screenshot", type = "image/png")
public byte[] saveScreenshot(byte[] screenShot) {
    return screenShot;
}

type用於指定MIME型別，不是必須的，Allure會根據內容自動適配。

除了註解，也可以在程式碼中新增：

import io.qameta.allure.Allure;
...
Allure.addAttachment("My attachment", "My attachment content");
Path content = Paths.get("path-to-my-attachment-contnet");
try (InputStream is = Files.newInputStream(content)) {
    Allure.addAttachment("My attachment", is);
}

Links

新增超連結，有3個註解@Link、@Issue、@TmsLink（test management system）：

import io.qameta.allure.Link;
import io.qameta.allure.Issue;
import io.qameta.allure.TmsLink;
@Link("https://example.org")
@Link(name = "allure", type = "mylink")
public void testSomething() {
     ...
}
@Issue("123")
@Issue("432")
public void testSomething() {
     ...
}
@TmsLink("test-1")
@TmsLink("test-2")
public void testSomething() {
     ...
}

連結型別可以在properties中定義，Allure會用註解中的值替換{}：

allure.link.mylink.pattern=https://example.org/mylink/{}
allure.link.issue.pattern=https://example.org/issue/{}
allure.link.tms.pattern=https://example.org/tms/{}

@Severity

新增嚴重程度標識：

package org.example.tests;
import org.junit.Test;
import io.qameta.allure.Severity;
import io.qameta.allure.SeverityLevel;
public class MyTest {
    @Test
    @Severity(SeverityLevel.CRITICAL)
    public void testSomething() throws Exception {
        // ...
    }
}

敏捷標識

在敏捷開發中有Epic、Feature和Stories三個概念，用於對需求進行分層管理，同理可以運用到測試分層管理：

package org.example.tests;
import org.junit.Test;
import io.qameta.allure.Epic;
import io.qameta.allure.Feature;
import io.qameta.allure.Story;
@Epic("Allure examples")
@Feature("Junit 4 support")
public class MyTest {
    @Test
    @Story("Base support for bdd annotations")
    @Story("Advanced support for bdd annotations")
    public void testSomething() throws Exception {
        // ...
    }
}

Python Pytest整合Allure Report

整合

執行以下命令安裝：

$ pip install allure-pytest

它會同時安裝allure-pytest和allure-python-commons兩個包。

然後指定Allure測試報告目錄：

$ pytest --alluredir=/tmp/my_allure_results

最後執行命令開啟報告：

$ allure serve /tmp/my_allure_results

Allure會建立一個Jetty伺服器，拉起預設瀏覽器開啟測試報告。

pytest結果狀態

Allure會根據pytest的結果狀態進行標記：

import pytest
def test_success():
    """this test succeeds"""
    assert True
def test_failure():
    """this test fails"""
    assert False
def test_skip():
    """this test is skipped"""
    pytest.skip('for a reason!')
def test_broken():
    raise Exception('oops')

pytest特性

xfail

@pytest.mark.xfail(condition=lambda: True, reason='this test is expecting failure')
def test_xfail_expected_failure():
    """this test is an xfail that will be marked as expected failure"""
    assert False
@pytest.mark.xfail(condition=lambda: True, reason='this test is expecting failure')
def test_xfail_unexpected_pass():
    """this test is an xfail that will be marked as unexpected success"""
    assert True

會在Allure報告中展示：

Expected xpass failure

Unexpected xpass pass

skipif

@pytest.mark.skipif('2 + 2 != 5', reason='This test is skipped by a triggered condition in @pytest.mark.skipif')
def test_skip_by_triggered_condition():
    pass

會在Allure報告中展示：

Conditional skip triggered

fixtures

@pytest.fixture(params=[True, False], ids=['param_true', 'param_false'])
def function_scope_fixture_with_finalizer(request):
    if request.param:
        print('True')
    else:
        print('False')
    def function_scope_finalizer():
        function_scope_step()
    request.addfinalizer(function_scope_finalizer)
@pytest.fixture(scope='class')
def class_scope_fixture_with_finalizer(request):
    def class_finalizer_fixture():
        class_scope_step()
    request.addfinalizer(class_finalizer_fixture)
@pytest.fixture(scope='module')
def module_scope_fixture_with_finalizer(request):
    def module_finalizer_fixture():
        module_scope_step()
    request.addfinalizer(module_finalizer_fixture)
@pytest.fixture(scope='session')
def session_scope_fixture_with_finalizer(request):
    def session_finalizer_fixture():
        session_scope_step()
    request.addfinalizer(session_finalizer_fixture)
class TestClass(object):
    def test_with_scoped_finalizers(self,
                                    function_scope_fixture_with_finalizer,
                                    class_scope_fixture_with_finalizer,
                                    module_scope_fixture_with_finalizer,
                                    session_scope_fixture_with_finalizer):
        step_inside_test_body()

會在Allure報告中展示（fixtures會展示在Set up和Tear down中）：

Test with fixtures and finalizers executed within different scopes.

而對於fixture中的結果狀態，Allure也會進行標記：

import pytest
@pytest.fixture
def skip_fixture():
    pytest.skip()
@pytest.fixture
def fail_fixture():
    assert False
@pytest.fixture
def broken_fixture():
    raise Exception("Sorry, it's broken.")
def test_with_pytest_skip_in_the_fixture(skip_fixture):
    pass
def test_with_failure_in_the_fixture(fail_fixture):
    pass
def test_with_broken_fixture(broken_fixture):
    pass

Fixture execution outcome resulting in different statuses.

parametrize

import allure
import pytest
@allure.step
def simple_step(step_param1, step_param2 = None):
    pass
@pytest.mark.parametrize('param1', [True, False], ids=['id explaining value 1', 'id explaining value 2'])
def test_parameterize_with_id(param1):
    simple_step(param1)
@pytest.mark.parametrize('param1', [True, False])
@pytest.mark.parametrize('param2', ['value 1', 'value 2'])
def test_parametrize_with_two_parameters(param1, param2):
    simple_step(param1, param2)
@pytest.mark.parametrize('param1', [True], ids=['boolean parameter id'])
@pytest.mark.parametrize('param2', ['value 1', 'value 2'])
@pytest.mark.parametrize('param3', [1])
def test_parameterize_with_uneven_value_sets(param1, param2, param3):
    simple_step(param1, param3)
    simple_step(param2)

Allure會展示每個test和id：

Multiple invocations of tests with different parameters.

以及執行細節：

Multiple invocations of tests with different parameters.

Allure特性

@allure.step

import allure
import pytest
from .steps import imported_step
@allure.step
def passing_step():
    pass
@allure.step
def step_with_nested_steps():
    nested_step()
@allure.step
def nested_step():
    nested_step_with_arguments(1, 'abc')
@allure.step
def nested_step_with_arguments(arg1, arg2):
    pass
def test_with_imported_step():
    passing_step()
    imported_step()
def test_with_nested_steps():
    passing_step()
    step_with_nested_steps()

Nested steps and steps with arguments.

step能從入參中讀取值：

import allure
@allure.step('Step with placeholders in the title, positional: "{0}", keyword: "{key}"')
def step_with_title_placeholders(arg1, key=None):
    pass
def test_steps_with_placeholders():
    step_with_title_placeholders(1, key='something')
    step_with_title_placeholders(2)
    step_with_title_placeholders(3, 'anything')

Nested steps and steps with arguments.

step也能在conftest.py中用到fixtures上：

import allure
import pytest
@allure.step('step in conftest.py')
def conftest_step():
    pass
@pytest.fixture
def fixture_with_conftest_step():
    conftest_step()
import allure
from .steps import imported_step
@allure.step
def passing_step():
    pass
def test_with_step_in_fixture_from_conftest(fixture_with_conftest_step):
    passing_step()

Step in fixture resolved from conftest.py.

allure.attach

給Allure測試報告新增附件，allure.attach(body, name, attachment_type, extension)：

body 檔案內容（或者source指定檔案路徑）
name 檔名稱
attachment_type 附件型別（allure.attachment_type中的某個值）
extension 檔案擴充套件

import allure
import pytest
@pytest.fixture
def attach_file_in_module_scope_fixture_with_finalizer(request):
    allure.attach('A text attacment in module scope fixture', 'blah blah blah', allure.attachment_type.TEXT)
    def finalizer_module_scope_fixture():
        allure.attach('A text attacment in module scope finalizer', 'blah blah blah blah',
                      allure.attachment_type.TEXT)
    request.addfinalizer(finalizer_module_scope_fixture)
def test_with_attacments_in_fixture_and_finalizer(attach_file_in_module_scope_finalizer):
    pass
def test_multiple_attachments():
    allure.attach.file('./data/totally_open_source_kitten.png', attachment_type=allure.attachment_type.PNG)
    allure.attach('<head></head><body> a page </body>', 'Attach with HTML type', allure.attachment_type.HTML)

Attachments in the test body.

Descriptions

既可以使用裝飾器@allure.description或@allure.description_html新增描述：

import allure
@allure.description_html("""
<h1>Test with some complicated html description</h1>
<table style="width:100%">
  <tr>
    <th>Firstname</th>
    <th>Lastname</th>
    <th>Age</th>
  </tr>
  <tr align="center">
    <td>William</td>
    <td>Smith</td>
    <td>50</td>
  </tr>
  <tr align="center">
    <td>Vasya</td>
    <td>Jackson</td>
    <td>94</td>
  </tr>
</table>
""")
def test_html_description():
    assert True
@allure.description("""
Multiline test description.
That comes from the allure.description decorator.
Nothing special about it.
""")
def test_description_from_decorator():
    assert 42 == int(6 * 7)
def test_unicode_in_docstring_description():
    """Unicode in description.
    Этот тест проверяет юникод.
    你好夥計.
    """
    assert 42 == int(6 * 7)

Description from docstring.

也可以在程式碼中使用allure.dynamic.description動態新增描述：

import allure
@allure.description("""
This description will be replaced at the end of the test.
""")
def test_dynamic_description():
    assert 42 == int(6 * 7)
    allure.dynamic.description('A final description.')

Titles

既可以使用裝飾器@allure.title新增標題：

import allure
import pytest
@allure.title("This test has a custom title")
def test_with_a_title():
    assert 2 + 2 == 4
@allure.title("This test has a custom title with unicode: Привет!")
def test_with_unicode_title():
    assert 3 + 3 == 6

並且可以獲取引數值：

@allure.title("Parameterized test title: adding {param1} with {param2}")
@pytest.mark.parametrize('param1,param2,expected', [
    (2, 2, 4),
    (1, 2, 5)
])
def test_with_parameterized_title(param1, param2, expected):
    assert param1 + param2 == expected

也可以在程式碼中使用allure.dynamic.title動態新增：

@allure.title("This title will be replaced in a test body")
def test_with_dynamic_title():
    assert 2 + 2 == 4
    allure.dynamic.title('After a successful test finish, the title was replaced with this line.')

Description from docstring.

Links

@allure.link、 @allure.issue 和@allure.testcase：

import allure
TEST_CASE_LINK = 'https://github.com/qameta/allure-integrations/issues/8#issuecomment-268313637'
@allure.link('https://www.youtube.com/watch?v=4YYzUTYZRMU')
def test_with_link():
    pass
@allure.link('https://www.youtube.com/watch?v=Su5p2TqZxKU', name='Click me')
def test_with_named_link():
    pass
@allure.issue('140', 'Pytest-flaky test retries shows like test steps')
def test_with_issue_link():
    pass
@allure.testcase(TEST_CASE_LINK, 'Test case title')
def test_with_testcase_link():
    pass

Description from docstring.

@allure.link會提供可點選的超連結。

Test with a link of issue type.

@allure.issue會有個可供點選的圖示。

issure連結模板通過--allure-link-pattern定義：

$ pytest directory_with_tests/ --alluredir=/tmp/my_allure_report \
 --allure-link-pattern=issue:http://www.mytesttracker.com/issue/{}

Retries

Allure有個Retries標籤頁用來展示重試的測試：

import allure
import random
import time
@allure.step
def passing_step():
    pass
@allure.step
def flaky_broken_step():
    if random.randint(1, 5) != 1:
        raise Exception('Broken!')
def test_broken_with_randomized_time():
    passing_step()
    time.sleep(random.randint(1, 3))
    flaky_broken_step()

Retries tab for a test that was rerun.

敏捷標識

在敏捷開發中有Epic、Feature和Stories三個概念，用於對需求進行分層管理，同理可以運用到測試分層管理：

import allure
def test_without_any_annotations_that_wont_be_executed():
    pass
@allure.story('epic_1')
def test_with_epic_1():
    pass
@allure.story('story_1')
def test_with_story_1():
    pass
@allure.story('story_2')
def test_with_story_2():
    pass
@allure.feature('feature_2')
@allure.story('story_2')
def test_with_story_2_and_feature_2():
    pass

也可以使用--allure-epics、--allure-features、--allure-stories指定執行測試：

$ pytest tests.py --allure-stories story_1,story_2
collected 5 items
tests.py ...                                                                    [100%]
============================== 3 passed in 0.01 seconds ==============================
$ pytest tests.py --allure-features feature2 --allure-stories story2
collected 5 items
tests.py ...                                                                     [100%]
=============================== 2 passed in 0.01 seconds ==============================

嚴重程度

使用@allure.severity或allure.severity_level：

import allure
def test_with_no_severity_label():
    pass
@allure.severity(allure.severity_level.TRIVIAL)
def test_with_trivial_severity():
    pass
@allure.severity(allure.severity_level.NORMAL)
def test_with_normal_severity():
    pass
@allure.severity(allure.severity_level.NORMAL)
class TestClassWithNormalSeverity(object):
    def test_inside_the_normal_severity_test_class(self):
        pass
    @allure.severity(allure.severity_level.CRITICAL)
    def test_inside_the_normal_severity_test_class_with_overriding_critical_severity(self):
        pass

--allure-severities用來指定執行哪些嚴重程度的測試：

$ pytest tests.py --allure-severities normal,critical
collected 5 items
bdd_annotations_demo/test_severity_labels.py ...                                [100%]
================================ 3 passed in 0.01 seconds ============================

參考資料：

https://docs.qameta.io/allure-report/

Allure測試報告
2024-04-04
測試報告
Jmeter——結合Allure展示測試報告
2022-11-30
JMeter測試報告
allure 測試報告怎麼嵌入到測試平臺？
2020-10-14
測試報告
pytest(11)-Allure生成測試報告(一)
2022-02-21
測試報告
整合 extentreports、beautifulreport、allure 測試報告
2021-01-14
測試報告
如何利用Allure報告提升你的測試效率？
2024-03-04
記學習滲透測試之報告測試引數一
2021-12-24
記學習滲透測試之報告測試引數二
2022-01-21
記學習滲透測試之報告測試引數四
2022-02-12
記學習滲透測試之報告測試引數三
2022-01-28
學習筆記之測試
2019-03-27
筆記
Allure 測試報告：allure.title 如何去掉後方的引數化顯示
2020-07-22
測試報告
allure測試報告不出來，json類檔案不生成
2020-10-29
測試報告JSON
滲透測試學習之報告測試引數五
2022-02-12
JMM測試利器-JCStress學習筆記
2023-05-16
筆記
軟體測試學習筆記：測試點總結
2013-08-30
筆記
大資料測試學習筆記之測試工具集
2019-05-15
大資料筆記
Jest 測試框架使用的學習筆記
2021-10-02
框架筆記
Sqlite學習筆記(三)&&WAL效能測試
2015-08-24
SQLite筆記
Jenkins+Allure測試報告+飛書機器人傳送通知
2022-04-16
Jenkins測試報告機器人
h5學習筆記：sessionStorage 小測試
2019-03-11
H5筆記Session
kafka學習筆記1:測試環境搭建
2017-09-24
Kafka筆記
Pytest學習筆記10-生成html報告
2021-07-05
筆記HTML
Jenkins上實現Python + Jenkins + Allure Report 介面自動化測試持續整合，並生成allure-report測試報告
2021-01-15
JenkinsPython測試報告
TDD學習筆記【二】---單元測試簡介
2014-08-30
筆記
JPG學習筆記1（附完整程式碼）
2021-02-12
筆記
JPG學習筆記2（附完整程式碼）
2021-02-13
筆記
JPG學習筆記3（附完整程式碼）
2021-02-18
筆記
C++學習筆記，知識點+程式碼測試
2020-12-18
C++筆記
jenkins+pytest+allure 介面測試生成報告只顯示第一次的測試結果
2020-12-08
Jenkins
Jumper 測試報告
2020-10-18
測試報告
雲測試報告
2020-11-29
測試報告
Redis測試報告
2013-10-14
Redis測試報告
Bacula測試報告
2011-11-15
測試報告
測試計劃和測試報告
2021-05-22
測試報告
pytest+allure 生成測試報告，如果保留同一個測試用例的多次執行的日誌資訊。
2020-12-24
測試報告
升職加薪利器：Python+Pytest框架在Jenkins上生成Allure測試報告
2020-02-17
Python框架Jenkins測試報告
多 module 的 maven 專案，怎麼生成完整的測試報告
2020-12-04
Maven測試報告

Allure測試報告完整學習筆記

目錄

簡介

安裝Allure

Allure測試報告的結構

Overview

Categories

Suites

Graphs

Timeline

Behaviors

Packages

Java TestNG整合Allure Report

整合

@Description

@Step

@Attachment

Links

@Severity

敏捷標識

Python Pytest整合Allure Report

整合

pytest結果狀態

pytest特性

xfail

skipif

fixtures

parametrize

Allure特性

@allure.step

allure.attach

Descriptions

Titles

Links

Retries

敏捷標識

嚴重程度

相關文章