UI自动化测试框架搭建-标记性能较差用例

在拿到性能数据后需要将数据利用起来，下面对性能数据进行分析

实现：如果性能达到设定的阈值，那么这段时间执行的用例就是性能较差的用例

确定阈值

首先确定一个阈值来当做性能的告警值，暂定为以下算法

# threshold 阈值
# average 平均值
# max 最大值 
threshold = average + (max - average) * 0.8
1
2
3
4

计算各项值

在上一章已经拿到了性能数据，然后把他们插入到数据库中

由于平台是Java的所以用Java编写

构造一个HashMap来存放全部的数据

Map>> perfSummaryData = new HashMap<>();
1

最终返回的数据类型为：

{
"cpu":{"system":[363,25,121.43243243243244,314.6864864864865],"idle":[761,4,537.3716216216217,716.2743243243243],"pic_cpu":[373,3.2,66.42430555555556,311.6848611111111],"device_cpu_rate":[904,36,245.6418918918919,772.3283783783784],"user":[652,11,124.20945945945945,546.4418918918919]},
"mem":{"free_ram":[2707.73,1720.11,2044.2376923076922,2575.0315384615387],"pid_pss":[452.18,231.72,313.105,424.365],"total_ram":[5478.93,5478.93,5478.929999999999,5478.93]},

"other":null,

"power":{"current":[0,0,0,0],"tempreture":[36.8,29.9,33.69638554216868,36.17927710843373],"voltage":[4.34,4.13,4.284096385542169,4.328819277108433]},

"pss":{"pssList":[],"java_heap":[],"pss_system":[],"native_heap":[]},

"thread":{"threadList":[272,3,191.46308724832215,255.89261744966444]},

"traffic":{"device_transport":[6283.15,433.69,3777.436774193548,5782.00735483871],"device_receive":[167860.62,11597.64,106851.58806451612,155658.81361290324],"device_total":[174143.77,12031.33,110629.02451612902,161440.8209032258],"pid_tx":[6283.15,474.8,3779.061612903226,5782.332322580645],"pid_total":[174143.71,13349.69,110675.72096774195,161450.11219354838],"pid_rx":[167860.55,12874.89,106896.65838709677,155667.77167741934]},

"version":"微医用户版4.6.5-内部版本号:268-灰度-待发布-创建时间:2022-06-23 21:18:10-最近修改时间:2022-06-23 21:18:10"
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

依次解析每个版本每次执行的性能数据然后根据性能类型放到对应的List中

然后计算一下最大、最小、平均、阈值

floats.add(floats1.stream().mapToDouble(e -> e).filter(e -> e > 0.0).max().orElse(0.0));
floats.add(floats1.stream().mapToDouble(e -> e).filter(e -> e > 0.0).min().orElse(0.0));
floats.add(floats1.stream().mapToDouble(e -> e).filter(e -> e > 0.0).average().orElse(0.0));
floats.add(floats.get(2) + 0.8 * (floats.get(0) - floats.get(2))); // 阈值
1
2
3
4

完整代码

@Override
public List getPerfDataSummary(UiPerfPageQuery pageQuery) {
    List perfData = uiReportMapper.getPerfData(pageQuery);
    Map>> perfSummaryData = new HashMap<>();
    String[] cpuKeys = { "device_cpu_rate", "user", "system", "idle", "pic_cpu" };
    String[] memKeys = { "total_ram", "free_ram", "pid_pss" };
    String[] powerKeys = { "voltage", "tempreture", "current" };
    String[] threadKeys = { "threadList" };
    String[] pssKeys = { "pssList", "java_heap", "native_heap", "pss_system" };
    String[] trafficKeys = { "device_total", "device_receive", "device_transport", "pid_rx", "pid_tx",
        "pid_total" };
    String[][] allKeys = { cpuKeys, memKeys, powerKeys, threadKeys, pssKeys, trafficKeys };

    perfData.forEach(item -> {
        String version = item.getVersion();
        if (!perfSummaryData.containsKey(version)) {
            HashMap> perfVersion = new HashMap<>();
            for (String[] keys : allKeys) {
                for (String key : keys) {
                    perfVersion.put(key, new ArrayList<>());
                }
            }
            perfSummaryData.put(version, perfVersion);
        }
        Map> stringListMap = perfSummaryData.get(version);
        List cpus = JSONArray.parseArray(item.getCpu(), String.class);
        List mems = JSONArray.parseArray(item.getMem(), String.class);
        List powers = JSONArray.parseArray(item.getPower(), String.class);
        List threads = JSONArray.parseArray(item.getThread(), String.class);
        List psss = JSONArray.parseArray(item.getPss(), String.class);
        List traffics = JSONArray.parseArray(item.getTraffic(), String.class);
        cpus.forEach(cpu -> {
            List oneCpuInfo = JSONArray.parseArray(cpu, String.class);
            if (!oneCpuInfo.isEmpty()) {
                for (int i = 1; i < 6; i++) {
                    List datalist = stringListMap.get(cpuKeys[i - 1]);
                    datalist.add(Double.parseDouble(oneCpuInfo.get(i)));
                }
            }
        });
        mems.forEach(mem -> {
            List oneMemInfo = JSONArray.parseArray(mem, String.class);
            if (!oneMemInfo.isEmpty()) {
                for (int i = 1; i < 4; i++) {
                    List datalist = stringListMap.get(memKeys[i - 1]);
                    datalist.add(Double.parseDouble(oneMemInfo.get(i)));
                }
            }
        });
        powers.forEach(power -> {
            List onePowerInfo = JSONArray.parseArray(power, String.class);
            if (!onePowerInfo.isEmpty()) {
                for (int i = 1; i < 4; i++) {
                    List datalist = stringListMap.get(powerKeys[i - 1]);
                    datalist.add(Double.parseDouble(onePowerInfo.get(i)));
                }
            }
        });
        threads.forEach(thread -> {
            List oneThreadInfo = JSONArray.parseArray(thread, String.class);
            if (!oneThreadInfo.isEmpty()) {
                for (int i = 1; i < 2; i++) {
                    List datalist = stringListMap.get(threadKeys[i - 1]);
                    datalist.add(Double.parseDouble(oneThreadInfo.get(i)));
                }
            }
        });
        psss.forEach(pss -> {
            List onePssInfo = JSONArray.parseArray(pss, String.class);
            if (!onePssInfo.isEmpty()) {
                for (int i = 1; i < 5; i++) {
                    List datalist = stringListMap.get(pssKeys[i - 1]);
                    datalist.add(Double.parseDouble(onePssInfo.get(i)));
                }
            }
        });
        traffics.forEach(traffic -> {
            List oneTrafficInfo = JSONArray.parseArray(traffic, String.class);
            if (!oneTrafficInfo.isEmpty()) {
                for (int i = 1; i < 7; i++) {
                    List datalist = stringListMap.get(trafficKeys[i - 1]);
                    datalist.add(Double.parseDouble(oneTrafficInfo.get(i)));
                }
            }
        });
    });
    List uiPerfSummaryDos = new ArrayList<>();
    List floats;
    for (String key : perfSummaryData.keySet()) {
        UiPerfSummaryDo uiPerfSummaryDo = new UiPerfSummaryDo();
        Map> stringListMap = perfSummaryData.get(key);
        uiPerfSummaryDo.setVersion(key);
        for (String[] all_key : allKeys) {
            HashMap> dataDict = new HashMap<>();
            for (String key1 : all_key) {
                floats = new ArrayList<>();
                List floats1 = stringListMap.get(key1);
                if (!floats1.isEmpty()) {
                    floats.add(floats1.stream().mapToDouble(e -> e).filter(e -> e > 0.0).max().orElse(0.0));
                    floats.add(floats1.stream().mapToDouble(e -> e).filter(e -> e > 0.0).min().orElse(0.0));
                    floats.add(floats1.stream().mapToDouble(e -> e).filter(e -> e > 0.0).average().orElse(0.0));
                    floats.add(floats.get(2) + 0.8 * (floats.get(0) - floats.get(2))); // 阈值
                }
                dataDict.put(key1, floats);
            }
            if (Arrays.equals(all_key, cpuKeys)) {
                uiPerfSummaryDo.setCpu(dataDict);
            }
            if (Arrays.equals(all_key, memKeys)) {
                uiPerfSummaryDo.setMem(dataDict);
            }
            if (Arrays.equals(all_key, powerKeys)) {
                uiPerfSummaryDo.setPower(dataDict);
            }
            if (Arrays.equals(all_key, threadKeys)) {
                uiPerfSummaryDo.setThread(dataDict);
            }
            if (Arrays.equals(all_key, pssKeys)) {
                uiPerfSummaryDo.setPss(dataDict);
            }
            if (Arrays.equals(all_key, trafficKeys)) {
                uiPerfSummaryDo.setTraffic(dataDict);
            }
        }
        uiPerfSummaryDos.add(uiPerfSummaryDo);
    }
    return uiPerfSummaryDos;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128

解析Allure报告

拿到一个用例的开始时间和结束时间，方便确定用例执行的时间范围

在13.UI自动化测试框架搭建-处理Allure报告数据中有提到如何拿到allure的内容

data = {
    "fullName": full_name,
    "status": json_data.get("status"),
    "labels": labels,
    "start": json_data.get("start", 0),
    "stop": json_data.get("stop", 0),
    "duration": json_data.get("stop", 0) - json_data.get("start", 0),
    "parameters": parameters,
    "statusDetails": statusDetails1,
    "kano_url": ",".join(kano_url),
    "steps": ";".join(steps),
    "desc": ";"
}
1
2
3
4
5
6
7
8
9
10
11
12
13

添加了

"start": json_data.get("start", 0)
"stop": json_data.get("stop", 0)
"desc": ";"
1
2
3

现在将用例执行的情况写入到一份csv文件中，方便与其他的性能数据进行比对

# 拼接csv文件路径
casesfile = os.path.join(PERF_PATH, 'cases.csv')
# csv文件头部
title = ["datetime", "flag"] + list(allure_results[0].keys())
with open(casesfile, "a+", encoding="utf-8") as f:
    # 写入头部
    csv.writer(f, lineterminator='\n').writerow(title)
    for i in allure_results:
        allure_data1 = [timeoperator.get_localtime(i['start'], "%Y-%m-%d %H-%M-%S"), True] + list(
            i.values())
        allure_data2 = [timeoperator.get_localtime(i['stop'], "%Y-%m-%d %H-%M-%S"), False] + list(
            i.values())
        csv.writer(f, lineterminator='\n').writerow(allure_data1)
        csv.writer(f, lineterminator='\n').writerow(allure_data2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14

使得datetime等于start和stop，写入两遍，再使用flag来标记方便后面去除掉

这样就得到一份按照时间排序的用例执行结果了

datetime,flag,fullName,status,labels,start,stop,duration,parameters,statusDetails,kano_url,steps,desc
2022-08-01 14-04-44,True,src.cases_android.wy.test_health.TestMedicationReminder#test_close_medication_reminder,passed,微医APP_健康_用药提醒-关闭用药提醒(@钟鑫),1659333884508,1659333917256,32748,,"{'message': '', 'trace': ''}",,1-查看提醒状态;1.1-查看「用药提醒_已开启提醒」是否存在;2-关闭用药提醒;2.1-点击「用药提醒_提醒滑块1」;3-点击确认;3.1-点击「通用_确认关闭」;4-查看提醒状态;4.1-查看「用药提醒_已关闭提醒」是否存在;4.2-查看「用药提醒_已关闭提醒1」是否存在;5-打开用药提醒;5.1-点击「用药提醒_提醒滑块2」;5.2-点击「用药提醒_已关闭提醒1」;6-查看提醒状态;6.1-查看「用药提醒_已开启提醒」是否存在,;
2022-08-01 14-05-17,False,src.cases_android.wy.test_health.TestMedicationReminder#test_close_medication_reminder,passed,微医APP_健康_用药提醒-关闭用药提醒(@钟鑫),1659333884508,1659333917256,32748,,"{'message': '', 'trace': ''}",,1-查看提醒状态;1.1-查看「用药提醒_已开启提醒」是否存在;2-关闭用药提醒;2.1-点击「用药提醒_提醒滑块1」;3-点击确认;3.1-点击「通用_确认关闭」;4-查看提醒状态;4.1-查看「用药提醒_已关闭提醒」是否存在;4.2-查看「用药提醒_已关闭提醒1」是否存在;5-打开用药提醒;5.1-点击「用药提醒_提醒滑块2」;5.2-点击「用药提醒_已关闭提醒1」;6-查看提醒状态;6.1-查看「用药提醒_已开启提醒」是否存在,;
2022-08-01 13-24-32,True,src.cases_android.wy.test_me.TestSetting#test_entrance_text,passed,微医APP_个人中心_设置-子项入口文案检查(@钟鑫),1659331472953,1659331473076,123,'settings_message',"{'message': '', 'trace': ''}",,1-查看是否存在我_设置-消息通知;1.1-查看「我_设置-消息通知」是否存在,;
2022-08-01 13-24-33,False,src.cases_android.wy.test_me.TestSetting#test_entrance_text,passed,微医APP_个人中心_设置-子项入口文案检查(@钟鑫),1659331472953,1659331473076,123,'settings_message',"{'message': '', 'trace': ''}",,1-查看是否存在我_设置-消息通知;1.1-查看「我_设置-消息通知」是否存在,;
1
2
3
4
5

数据比对

读取用例执行结果

读取文件后将它的datetimes设置为datetime类型，方便后面的排序

def cases_handle(self, path=f"{PERF_PATH}/cases.csv"):
    df = self.read_csv(path)
    df['datetime'] = pd.to_datetime(df['datetime'], format="%Y-%m-%d %H-%M-%S")
    df = df.sort_values("datetime", ascending=True)
    return df
1
2
3
4
5

编写比较函数

def add_desc(self, **kwargs) -> [dict]:
    ...
1
2

**kwargs动态输入键值对来进行筛选

记录原先的列

base_title = list(cases_df.columns)
1

先进行cpu性能数据的处理

cpu = ['device_cpu_rate%', 'user%', 'system%', 'idle%', 'pid_cpu%']
1

遍历输入的键值对，如果输入的key值加上%在cpu这个列表里面，那就筛选出cpu性能数据中符合条件的数据

for k, v in kwargs.items():
    if f"{k}%" in cpu:
        name = f"{k}%"
        df1 = cpu_df[cpu_df[name] > v]
1
2
3
4

将两个表合并一下

cases_df = pd.concat([df1, cases_df]).sort_values("datetime", ascending=True)
1

拿到超过阈值的行号索引

error_index = list(cases_df[cases_df[name] > 0].index)
1

在超过阈值索引的附近几行添加描述

for i in error_index:
    cases_df.loc[i, 'desc'] += f"{k}超过阈值;"
    try:
        cases_df.loc[i - 1, 'desc'] += f"{k}超过阈值;"
    except Exception:
        pass
    try:
        cases_df.loc[i + 1, 'desc'] += f"{k}超过阈值;"
    except Exception:
        pass
1
2
3
4
5
6
7
8
9
10

剔除性能数据的列

cases_df = cases_df[pd.notnull(cases_df['fullName'])]
1

根据flag去除重复添加的用例信息

cases_df = cases_df[cases_df['flag'] == True]
1

根据之前保留的原始列信息将性能数据列去除

new_df = pd.DataFrame(cases_df, columns=base_title)
1

返回[dict]格式

return new_df.to_dict("records")
1

全部新增代码

def cases_handle(self, path=f"{PERF_PATH}/cases.csv"):
    df = self.read_csv(path)
    df['datetime'] = pd.to_datetime(df['datetime'], format="%Y-%m-%d %H-%M-%S")
    df = df.sort_values("datetime", ascending=True)
    return df

def add_desc(self, **kwargs) -> [dict]:
    cpu = ['device_cpu_rate%', 'user%', 'system%', 'idle%', 'pid_cpu%']
    mem = ['total_ram(MB)', 'free_ram(MB)', 'pid_pss(MB)']
    power = ['voltage(V)', 'tempreture(C)', 'current(mA)']
    traffic = ['device_total(KB)', 'device_receive(KB)',
               'device_transport(KB)',
               'pid_rx(KB)', 'pid_tx(KB)', 'pid_total(KB)']
    cpu_df = self.cpu_handle()
    mem_df = self.mem_handle()
    power_df = self.power_handle()
    thread_df = self.thread_num_handle()
    traffic_df = self.traffic_handle()
    cases_df = self.cases_handle()
    base_title = list(cases_df.columns)
    for k, v in kwargs.items():
        if f"{k}%" in cpu:
            name = f"{k}%"
            df1 = cpu_df[cpu_df[name] > v]
        if f"{k}(MB)" in mem:
            name = f"{k}(MB)"
            df1 = mem_df[mem_df[name] > v]
        if any([f"{k}({i})" in power for i in ['V', 'C', 'mA']]):
            if k == "voltage":
                name = f"{k}(V)"
                df1 = power_df[power_df[name] > v]
            if k == "tempreture":
                name = f"{k}(C)"
                df1 = power_df[power_df[name] > v]
            if k == "current":
                name = f"{k}(mA)"
                df1 = power_df[power_df[name] > v]
        if k == "threadList":
            name = "thread_num"
            df1 = thread_df[thread_df[name] > v]
        if f'{k}(KB)' in traffic:
            name = f'{k}(KB)'
            df1 = traffic_df[traffic_df[name] > v]
        cases_df = pd.concat([df1, cases_df]).sort_values("datetime", ascending=True)
        error_index = list(cases_df[cases_df[name] > 0].index)
        for i in error_index:
            cases_df.loc[i, 'desc'] += f"{k}超过阈值;"
            try:
                cases_df.loc[i - 1, 'desc'] += f"{k}超过阈值;"
            except Exception:
                pass
            try:
                cases_df.loc[i + 1, 'desc'] += f"{k}超过阈值;"
            except Exception:
                pass
        cases_df = cases_df[pd.notnull(cases_df['fullName'])]
    cases_df = cases_df[cases_df['flag'] == True]
    new_df = pd.DataFrame(cases_df, columns=base_title)
    return new_df.to_dict("records")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

测试

cases_data = d.add_desc(
        device_cpu_rate=700, user=125, system=300, idle=700, pid_cpu=300,
        free_ram=2575, pid_pss=424, total_ram=5478,
        tempreture=36, voltage=4,
        threadList=255,
        device_transport=5782, device_receive=155658, device_total=161440, 
        pid_tx=5782, pid_total=161450, pid_rx=155667,
    )
[
...
{'datetime': Timestamp('2022-08-01 14:05:17'), 'flag': True, 'fullName': 'src.cases_android.wy.test_health.TestMedicationReminder#test_edit_medication_reminder', 'status': 'passed', 'labels': '微医APP_健康_用药提醒-修改用药提醒时间(@钟鑫)', 'start': 1659333917269.0, 'stop': 1659333926307.0, 'duration': 9038.0, 'parameters': nan, 'statusDetails': "{'message': '', 'trace': ''}", 'kano_url': nan, 'steps': '1-点击一条用药提醒记录;1.1-点击「用药提醒_其中一条用药提醒」;2-选择必要时用药;2.1-获取元素的坐标;2.2-点击坐标「558.0」「640.5」所在的位置;2.3-点击坐标「558.0」「640.5」所在的位置;2.4-点击坐标「558.0」「640.5」所在的位置;2.5-从元素「用药提醒_用药时间-每隔几天用药」滑动到元素「用药提醒_用药时间-每隔几小时用药」的位置;2.6-点击「通用_完成」;3-点击保存;3.1-点击「通用_保存」;4-查看提醒时间为必要时;4.1-转换参数化元素;4.2-查看「用药提醒_用药时间」是否存在', 'desc': ';system超过阈值;device_cpu_rate超过阈值;voltage超过阈值;voltage超过阈值;voltage超过阈值;'}
...
]
1
2
3
4
5
6
7
8
9
10
11
12
13

最后： 可以在公众号：伤心的辣条 ！ 自行领取一份216页软件测试工程师面试宝典文档资料【免费的】。以及相对应的视频学习教程免费分享！，其中包括了有基础知识、Linux必备、Shell、互联网程序原理、Mysql数据库、抓包工具专题、接口测试工具、测试进阶-Python编程、Web自动化测试、APP自动化测试、接口自动化测试、测试高级持续集成、测试架构开发测试框架、性能测试、安全测试等。

现在我邀请你进入我们的软件测试学习交流群：【746506216】，备注“入群”， 大家可以一起探讨交流软件测试，共同学习软件测试技术、面试等软件测试方方面面，还会有免费直播课，收获更多测试技巧，我们一起进阶Python自动化测试/测试开发，走向高薪之路。

喜欢软件测试的小伙伴们，如果我的博客对你有帮助、如果你喜欢我的博客内容，请 “点赞” “评论” “收藏” 一 键三连哦！

软件测试工程师自学教程：

这才是2022最精细的自动化测试自学教程，我把它刷了无数遍才上岸字节跳动，做到涨薪20K【值得自学软件测试的人刷】

接口性能测试 — 软件测试人必会618实战场景分析

软件测试工程师月薪2W以上薪资必学技能 — Python接口自动化框架封装.

美团面试真题_高级测试25K岗位面试 — 软件测试人都应该看看

测试开发之全面剖析自动化测试平台 — 软件测试人的必经之路

软件测试必会_Jmeter大厂实战 — 仅6步可实现接口自动化测试

Jmeter实战讲解案例 — 软件测试人必会