100字范文,内容丰富有趣,生活中的好帮手!
100字范文 > python selenium 元素定位 网页爬虫 web自动化执行 通过yaml文件配置操作步骤

python selenium 元素定位 网页爬虫 web自动化执行 通过yaml文件配置操作步骤

时间:2024-04-25 12:17:29

相关推荐

python selenium 元素定位 网页爬虫 web自动化执行 通过yaml文件配置操作步骤

目录

说明环境安装引入包内容启动一个浏览器(我这里以谷歌为例)同时启动多个浏览器,多个任务执行(我这里以谷歌为例)selenium grid hub环境启动selenium grid中文文档hub_config.json启动hubnode_config.json (我这里只配置了谷歌)启动node(这里是windows 的批处理文件,如node.bat 注意文件格式ANSI)启动浏览器(用threading 去启动多个也行,启动多个浏览器打开同一个页面不会受到session的影响) 截图方法yaml文件读取操作步骤执行的封装类操作步骤配置,示例 后言

说明

解决问题

1.1、在写web自动化的过程中,有大量的操作步骤代码重复,操作步骤管理起来麻烦,特别是html页面容易出现变动、输入参数频繁变动的,则老是需要去修改源文件,故支持一下配置化,把操作步骤和代码分开,容易管理维护

1.2、同时大量相同步骤操作,一个一个执行耗费大量时间,支持多个并发解决方式

2.1、操作步骤支持配置化,支持yaml 文件、对应定位的多个元素for循环遍历、if 对遍历的元素进行判断等等,封装鼠标和键盘快捷键的操作,执行js脚本等

2.2、引入 selenium gird

环境安装

python 环境

下载python 3.9.10:/downloads/windows (其他版本应该也行)

执行命令安装依赖包:pip install -i /pypi/simple -r requirements.txt --default-timeout=1000

requirements.txt

selenium==4.10.0PyYAML==6.0

引入包

# _*_ coding:utf-8 _*_import osimport reimport threadingimport timeimport logging as logfrom selenium.webdriver.chrome.service import Servicefrom selenium import webdriverfrom mon.by import Byfrom mon.keys import Keysfrom selenium.webdriver.support import expected_conditions as ECfrom selenium.webdriver.support.wait import WebDriverWaitfrom mon.action_chains import ActionChains

内容

启动一个浏览器(我这里以谷歌为例)

def open_chrome(display_gui=True, max_window=True, hub_driver=False, command_executor="http://localhost:5555/wd/hub", chrome_version="107"):"""启动谷歌浏览器:param display_gui::param max_window::param hub_driver: 分布式测试:param command_executor: 仅启动hub时需要,node 节点地址:param chrome_version: 仅启动hub时需要,google 版本:return:"""# driver_path = '../Driver/chromedriver.exe' # TASKKILL /F /IM chromedriver.exedriver_path = PATH('../Driver/chromedriver.exe')log.info(f'chrome driver path: {driver_path}')options = webdriver.ChromeOptions()# 禁用日志--因为cmd运行的时候出现日志打印,且展示为乱码options.add_experimental_option('excludeSwitches', ['enable-logging'])# 隐藏GUIif not display_gui:options.add_argument('--headless')options.add_argument('--disable-gpu')options.add_argument('--no-sandbox')options.add_argument('--disable-dev-shm-usage')if hub_driver:# 分布式测试capabilities = {"browserName": "chrome", # firefox"seleniumProtocol": "WebDriver","maxInstances": "5","maxSession": "5","version": chrome_version,"javascriptEnabled": True# 是否启用js}driver = webdriver.Remote(command_executor=command_executor, desired_capabilities=capabilities)else:# 启动浏览器_service = Service(executable_path=driver_path, )driver = webdriver.Chrome(service=_service, options=options)if max_window:driver.maximize_window()driver.implicitly_wait(5)return driverif __name__ == '__main__':driver = open_chrome(max_window=False, hub_driver=True)

同时启动多个浏览器,多个任务执行(我这里以谷歌为例)

selenium grid hub环境启动

地址:http://selenium-release./index.html

推荐下载这个:selenium-server-standalone-3.9.0.jar

selenium grid中文文档

https://www.selenium.dev/zh-cn/documentation/grid/getting_started/

hub_config.json

{"port": 4444,"newSessionWaitTimeout": -1,"servlets" : [],"withoutServlets": [],"custom": {},"capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher","registry": "org.openqa.grid.internal.DefaultGridRegistry","throwOnCapabilityNotPresent": true,"cleanUpCycle": 5000,"role": "hub","debug": false,"browserTimeout": 0,"timeout": 1800}

启动hub

java -jar selenium-server-standalone-3.9.1.jar -role hub -hubConfig hub_config.json

node_config.json (我这里只配置了谷歌)

{"capabilities":[{"browserName": "chrome","maxInstances": 10,"seleniumProtocol": "WebDriver","platform": "WINDOWS","version": "10"}],"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy","maxSession": 10,"port": 5555,"register": true,"registerCycle": 5000,"hub": "http://localhost:4444","nodeStatusCheckTimeout": 5000,"nodePolling": 5000,"role": "node","unregisterIfStillDownAfter": 60000,"downPollingLimit": 2,"debug": false,"servlets" : [],"withoutServlets": [],"custom": {}}

启动node(这里是windows 的批处理文件,如node.bat 注意文件格式ANSI)

:: 谷歌驱动的路径set driverPath=D:\test\chromedriver.exeset serverFile=selenium-server-standalone-3.9.1.jar%driverPath% -versionjava -Dwebdriver.chrome.driver=%driverPath% -jar %serverFile% -role node -nodeConfig node_config.json

启动浏览器(用threading 去启动多个也行,启动多个浏览器打开同一个页面不会受到session的影响)

if __name__ == '__main__':chrome_params_01 = {"display_gui": True,# 是否显示浏览器"max_window": True,# 是否最大化浏览器"hub_driver": True,# 是否通过hub启动"command_executor": "http://localhost:5555/wd/hub",#hub地址+node端口"chrome_version": "107"# 浏览器版本}driver_01 = open_chrome(**chrome_params_01)driver_02 = open_chrome(**chrome_params_02)

截图方法

class ScreenShot():"""截图"""def __init__(self):self.img_dir = './'self.shot_img_path = Nonedef open_last_shot_img(self, open_img: bool = False):"""打开最近截图"""t = threading.Thread(target=lambda img_path: os.system(f'start {img_path}'), args=(self.shot_img_path, ))t.setDaemon(True)t.start()def shot_browser(self, driver: webdriver.Chrome):"""通过浏览器截图:param driver: 浏览器驱动:return:"""img_name = 'test.png'save_path = os.path.join(self.img_dir, img_name)# 截图driver.save_screenshot(save_path)if os.path.isfile(save_path):log.info(f'截图成功:{save_path}')self.shot_img_path = save_pathelse:log.info(f'截图未找到,本次截图可能失败:{save_path}')return save_path

yaml文件读取

def read_yaml_file(file_path):"""读取yaml配置文件"""with open(os.path.abspath(file_path), 'r', encoding='utf-8') as f:temp = yaml.load(f, Loader=yaml.FullLoader)return temp

操作步骤执行的封装类

class BasePage():"""页面操作"""loc_type = {"ID": By.ID,"CLASS_NAME": By.CLASS_NAME,"CLASS_NAMES": By.CLASS_NAME,"XPATH": By.XPATH,"XPATHS": By.XPATH,"CSS_SELECTOR": By.CSS_SELECTOR,"CSS_SELECTORS": By.CSS_SELECTOR,"LINK_TEXT": By.LINK_TEXT,"PARTIAL_LINK_TEXT": By.PARTIAL_LINK_TEXT,"TAG_NAME": By.TAG_NAME,"TAG_NAMES": By.TAG_NAME,"NAME": By.NAME,"NAMES": By.NAME}# 模拟键盘按键事件,除数字和字母外,如果需要其他按键则需要补充keyboard = {'ENTER': Keys.ENTER,# 回车'CONTROL': Keys.CONTROL, # ctrl 键'SHIFT': Keys.SHIFT,# shift 键'ADD': Keys.ADD,# + 键'ARROW_DOWN': Keys.ARROW_DOWN,# ↓键'ARROW_UP': Keys.ARROW_UP,# ↑键'ARROW_LEFT': Keys.ARROW_LEFT,# ←键'ARROW_RIGHT': Keys.ARROW_RIGHT, # →键'CANCEL': Keys.CANCEL, # ESC 键'DECIMAL': Keys.DECIMAL, # .键'DIVIDE': Keys.DIVIDE, # / 键'EQUALS': Keys.EQUALS, # = 键'MULTIPLY': Keys.MULTIPLY,# * 键'NULL': Keys.NULL, # '' 空键'PAUSE': Keys.PAUSE,# Pause 空键'SEMICOLON': Keys.SEMICOLON, # ; 键'SEPARATOR': Keys.SEPARATOR, # , 键'SUBTRACT': Keys.SUBTRACT,# - 键}element_single_name = 'element_for_single_061811322'# 主要用于记录for循环中的子步骤的continue、break操作action_for_status = Nonedef __init__(self, driver):self.driver: webdriver.Chrome = driver# 公共变量m_save = {}def js_action(self, step):"""js操作 执行器:param step: js 代码el is not None:arguments[0].click(); # 点击window.scrollTo(0,500);# 向下滚动500el is None:param el: 定位到的元素:return:支持输入变量,格式如:$变量名$"""if 'action' in step and step['action'] == 'js_action':if 'element' in step and step['element'] is not None:ret_actuator = self.driver.execute_script(step['loc_value'], m_save[step['element']])else:ret_actuator = self.driver.execute_script(step['loc_value'])m_save['js_action'] = ret_actuatordef varible_insert(self, str_cont: str):"""自定义变量插入到字符串中,插入的变量格式:$var$"""pattn = re.findall('(\$.*?\$)', str_cont)str_cont_ = str_contfor p in pattn:str_cont_ = str_cont_.replace(p, str(m_save[p.replace('$', '')]))return str_cont_@staticmethoddef switch_page(driver: webdriver.Chrome, switch_method: str, switch_page: int or str):"""切换浏览器窗口页面:param driver: obj:param switch_method: 切换页面的方式,当前支持有:switch_page_by_index 在当前浏览器根据索引切换到另一个页面,根据索引,页面索引说明:0, 3, 2, 1; 这里共4个页面,0 是打开浏览器第一个初始化的页面,1是最新(最近打开)的页面,页面索引从左到右也是这个switch_page_by_link根据链接切换switch_page_by_title 根据标题切换:param switch_page: 切换页面的值:return:"""win_handle = driver.window_handlesif switch_method == 'switch_page_by_index':if isinstance(switch_page, int) and 0 <= switch_page < len(win_handle):driver.switch_to.window(win_handle[switch_page])log.info(f'当前切换的页面为:{driver.title}')else:log.info(f'switch_page 的值必须填写数字且 0 <= switch_page < {len(win_handle)}')else:for page_key in win_handle:if switch_method == 'switch_page_by_link':driver.switch_to.window(page_key)if switch_page in driver.current_url:log.info(f'当前切换的页面为:{driver.title}')returnelif switch_method == 'switch_page_by_title':driver.switch_to.window(page_key)if switch_page in driver.title:log.info(f'当前切换的页面为:{driver.title}')returnelse:log.info(f'switch_method: {switch_method} 暂不支持')returnelse:log.info(f'{switch_method} 未找到')def __logical_method(self, step):"""逻辑操作方法,例如:if for"""if 'steps' in step and 'action' in step and step['action'] == 'for':if 'cycle_obj' in step and step['cycle_obj'] == 'els':self.action_for_status = Nonefor i, el in enumerate(m_save['els']):if 'skip' in step and step['skip'] == i:continue# 给for循环中的操作步骤集,每个步骤都加上一个元素,并且都操作被循环的元素elfor index, step_for in enumerate(step['steps']):step['steps'][index]['element'] = elself.exec_steps(step['steps'])if self.action_for_status is not None: # break、continue 如果子步骤存在该操作,则对应元素会执行该操作if self.action_for_status == 'break':breakelif self.action_for_status == 'continue':continueelse:passself.action_for_status = None# skip、cycle_obj、start、end、conditionelif 'cycle_obj' in step and step['cycle_obj'] != 'els':self.action_for_status = Nonefor i, cycle_obj_el in enumerate(step['cycle_obj']):if 'skip' in step and step['skip'] == i:continue# 给for循环中的操作步骤集,每个步骤都加上一个元素,并且都操作被循环的元素m_save['cycle_obj_el'] = cycle_obj_elself.exec_steps(step['steps'])if self.action_for_status is not None: # break、continue 如果子步骤存在该操作,则对应元素会执行该操作if self.action_for_status == 'break':breakelif self.action_for_status == 'continue':continueelse:passself.action_for_status = Noneelif 'start' in step and isinstance(step['start'], int) and 'end' in step and isinstance(step['end'], int) or 'end' in step and isinstance(step['end'], int):start = 0if 'start' in step:start = step['start']self.action_for_status = Nonefor i in range(start, step['end']):if 'skip' in step and step['skip'] == i:continuetry:m_save['cycle_obj_el'] = iself.exec_steps(step['steps'])if self.action_for_status is not None: # break、continue 如果子步骤存在该操作,则对应元素会执行该操作if self.action_for_status == 'break':breakelif self.action_for_status == 'continue':continueelse:passself.action_for_status = Noneexcept Exception as e:log.info(f'循环类型:cycle_time {e}')else:log.error('for 循环参数格式解析异常')# if 逻辑操作elif 'steps' in step and 'action' in step and step['action'] == 'if':if self.execute_condition(step['condition']):self.set_think_time(step)self.exec_steps(step['steps'])def __steps_init(self, step: dict):"""测试步骤变量扫描,格式$name$,如果定义了变量,则自动读取"""for key in ['loc_value', 'send_keys']:if key in step and isinstance(step[key], str):step[key] = self.varible_insert(step[key])return stepdef exec_steps(self, steps: list, comm_variable_import: dict = None):"""执行web自动化的操作步骤:param steps: 参数格式如下desc: 步骤描述信息,如:点击XXXsleep:执行完后的等待时间element: 填写已保存的元素名称,则loc_type、loc_value可不填,填定位的元素保存的变量名element_single_name 如果保存的元素是list类型,则子方法可以用这个值获取,并且action用for可以读取loc_type:ID 通过id定位CLASS_NAME通过class定位CLASS_NAMES通过class定位,且返回一个元素列表XPATH 通过xpath定位XPATHS通过xpath定位,且返回一个元素列表CSS_SELECTOR 通过CSS定位CSS_SELECTORS 通过CSS定位,且返回一个元素列表LINK_TEXT 通过文本链接定位PARTIAL_LINK_TEXT 通过部分内容文本链接定位TAG_NAME 通过标签名TAG_NAMES 通过标签名,且返回一个元素列表NAME 通过NAMENAMES 通过NAME,且返回一个元素列表JS_ACTION 执行js脚本 js_actionloc_index:当定位元素类型是列表时,则可以直接输入列表中的索引,得到单个元素,例如:-1,0,1,2,等等loc_method: 元素定位的方法method: 定位方法,当前支持的方法:如下wait_untilwait_not_untilwait_until_title_iswait_until_title_containswait_until_visibilitywait_until_visibility_by_locatorwait_until_presence_of_elwait_until_presence_of_elswait_until_presence_of_el_textwait_until_presence_of_el_valuewait_until_switch_framewait_until_alertwait_until_clickablewait_until_selectedwait_until_located_selectedwait_until_selection_statewait_until_located_selection_selectedwait_until_stalenesstimeout: 10 超时时间poll_frequency: 0.51次/0.5秒,查找元素频率title: method 为 wait_until_title_is、wait_until_title_contains 时填写text: method 为 wait_until_presence_of_el_text 时填写value: method 为 wait_until_presence_of_el_value 时填写select: method 为 wait_until_selection_state、wait_until_located_selection_selected 时填写loc_value: str元素定位的路径,默认传入定位元素list 多个定位元素,返回成功的那个dict 获取保存的变量传入定位元素,比如通过文本定位时:{‘key’: xxx}action:click 点击clear 清空输入框get_element_length 获取元素列表的元素个数get_text 获取文本get_attribute 获取属性值get_title 获取网页标题get_current_url 获取网址get_browser_name获取浏览器名称get_page_source 获取网页源码get_text 获取元素文本get_id获取元素id属性get_location 获取元素位置属性get_tag_name 获取元素标签名属性get_el_size获取元素大小属性for 循环操作步骤,支持 skip、cycle_obj、start、endcycle_obj_el for循环对象元素变量名break 到该步骤停止continue 跳过该步骤save_condition 保存condition的结果quit 退出浏览器,并结束该步骤close 关闭当前页面句柄,当前driver的操作页面,如果只有一个页面则关闭浏览器open_page 在当前页面打开页面open_new_page 在当前浏览器打开一个新的页面switch_page_by_index 在当前浏览器根据索引切换到另一个页面,根据索引,页面索引说明:0, 3, 2, 1; 这里共4个页面,0 是打开浏览器第一个初始化的页面,1是最新(最近打开)的页面,页面索引从左到右也是这个switch_page_by_link 在当前浏览器根据索引切换到另一个页面,根据部分、完整链接switch_page_by_title 在当前浏览器根据索引切换到另一个页面,根据部分、完整标题attribute_name: 需要获取属性名称,仅get_attributesend_keys:需要填写的值,仅send_keys, 如果是字符串则向输入框输入内容,如果是列表,则输入键盘事件或组合键; 支持输入变量:$name$- enter 回车键, 以下为特殊按键,字母和数字直接输入,如:A、B、1、2- control ctrl键- shift shift键- add + 键- arrow_down ↓ 键- arrow_left ← 键- arrow_right → 键- arrow_up↑ 键- cancel (ESC)键- decimal . 键- divide / 键- equals = 键- multiply* 键- null'' 空键- pause Pause 空键- semicolon; 键- separator, 键- subtract- 键mouse_action 鼠标操作# 字符串类型的操作click 鼠标左键click_on_element 鼠标左键点击元素double_click鼠标左键双击double_click_on_element鼠标左键双击元素context_click鼠标右键context_click_on_element 鼠标右键点击元素move_to_element 鼠标移动到某个元素# 字典类型的操作drag_and_drop: 从当前元素拖拽到某个元素然后松开,字段类型source: 原元素,变量名target: 目标元素,变量名move_by_offset: 鼠标重当前位置,移动到某个坐标x横坐标y纵坐标move_to_element_with_offset移动到距某个元素(左上角坐标)多少距离的位置element定位的元素x 横坐标y 纵坐标save: 保存key: 保存时的键名value:保存时的键值,如果是存在暂存,则重新赋值给一个新的keyreturn: 返回结果,只能返回已保存的,结构如下,yaml填写如:['arg1', 'arg2',...]steps:仅action为for时填写操作步骤格式同上一级格式支持内容一致,子集步骤不支持returncondition:条件判断和条件表达式,支持多个条件的使用,比如if分支中close_other_page 关闭其他标签页,不能关闭当前页,值:数字,标签页索引 关闭指定标签页非数字、空 关闭除当前页面的其他标签页skip 仅for循环支持使用,值:元素列表的索引,跳过指定元素不操作cycle_obj 仅for循环支持使用,循环对象start 仅for循环支持使用,循环开始索引end 仅for循环支持使用,循环结束索引:param comm_variable_import: 导入公共变量,后续操作步骤中可使用导入的值:return:"""# 导入变量if comm_variable_import is not None:m_save.update(comm_variable_import)log.info(f"m_save is: {m_save}")# 开始执行测试内容的步骤for step in steps:el, els = None, Nonestep = self.__steps_init(step)log.info(f'当前执行步骤:{step}')# 关闭其他标签页self.close_other_page(step)# 中途停止操作或中途跳过某操作if 'action' in step and step['action'] == 'break':if 'action' in step and 'condition' in step and step['condition'] is not None:if self.execute_condition(step['condition']):self.action_for_status = 'break'self.set_think_time(step)breakelse:self.action_for_status = 'break'self.set_think_time(step)breakelif 'action' in step and step['action'] == 'continue':if 'condition' in step and step['condition'] is not None:if self.execute_condition(step['condition']):log.info('跳过当前步骤成功')self.action_for_status = 'continue'self.set_think_time(step)continueelse:self.action_for_status = 'continue'self.set_think_time(step)continueelif 'action' in step and step['action'] == 'quit':if 'condition' in step and step['condition'] is not None:if self.execute_condition(step['condition']):self.driver.quit()log.info(f'关闭浏览器:{self.driver}')self.set_think_time(step)breakelse:self.driver.quit()log.info(f'关闭浏览器:{self.driver}')self.set_think_time(step)breakelif 'action' in step and step['action'] == 'close':if 'condition' in step and step['condition'] is not None:if self.execute_condition(step['condition']):log.info(f'关闭当前页面:{self.driver.title}')self.driver.close()self.set_think_time(step)continueelse:log.info(f'关闭当前页面:{self.driver.title}')self.driver.close()self.set_think_time(step)continueelif 'action' in step and step['action'] == 'open_page':if 'loc_value' in step and step['loc_value'] is not None:if 'condition' in step and step['condition'] is not None:if self.execute_condition(step['condition']):log.info(f"打开当前页面:{step['loc_value']}")self.driver.get(step['loc_value'])self.set_think_time(step)continueelse:log.info(f"打开当前页面:{step['loc_value']}")self.driver.get(step['loc_value'])self.set_think_time(step)continueelse:log.info('loc_value 值未定义,请输入需要打开页面的链接,例如:loc_value: ')elif 'action' in step and step['action'] == 'open_new_page':if 'loc_value' in step and step['loc_value'] is not None:if 'condition' in step and step['condition'] is not None:if self.execute_condition(step['condition']):log.info(f"打开当前页面:{step['loc_value']}")self.driver.execute_script(f'window.open("{step["loc_value"]}")')self.driver.switch_to.window(self.driver.window_handles[1])self.set_think_time(step)continueelse:log.info(f"打开当前页面:{step['loc_value']}")self.driver.execute_script(f'window.open("{step["loc_value"]}")')self.driver.switch_to.window(self.driver.window_handles[1])self.set_think_time(step)continueelse:log.info('loc_value 值未定义,请输入需要打开页面的链接,例如:loc_value: ')elif 'action' in step and step['action'] in ['switch_page_by_index', 'switch_page_by_link', 'switch_page_by_title']:if 'loc_value' in step and step['loc_value'] is not None:if 'condition' in step and step['condition'] is not None:if self.execute_condition(step['condition']):log.info(f"切换页面:{step['loc_value']}")self.switch_page(self.driver, step['action'], step['loc_value'])self.set_think_time(step)continueelse:log.info(f"切换页面:{step['loc_value']}")self.switch_page(self.driver, step['action'], step['loc_value'])self.set_think_time(step)continueelse:log.info('loc_value 值未定义,请输入需要切换页面的索引,例如:loc_value: 1')else:pass# 元素定位,或元素继承el, els = self.__get_element(step, el, els)self.js_action(step)# for循环,遍历定位到的元素 逻辑方法 和 if 方法if 'steps' in step and 'action' in step and step['action'] in ['if', 'for']:self.__logical_method(step)if step['action'] == 'if':continue# 获取元素的值,或页面的属性值,或对页面进行操作if 'mouse_action' in step:self.mouse_actions(step, el)else:self.__page_actions(step, el)# 获取条件执行结果self.order_condition(step)# 需要保存的值self.__save_page_values(step)# 返回结果if 'return' in step and step['return'] is not None:return self.__return_result(step)self.set_think_time(step)def mouse_actions(self, step, el: webdriver.Chrome = None):"""执行鼠标动作,如点击,移动到指定元素位置,鼠标悬停等等参考文档:/jiachuan/article/details/108099705:param step:mouse_action: 当前支持操作# 字符串类型click 鼠标左键click_on_element 鼠标左键点击元素double_click鼠标左键双击double_click_on_element鼠标左键双击元素context_click鼠标右键context_click_on_element 鼠标右键点击元素move_to_element 鼠标移动到某个元素:param el::return:"""ac = ActionChains(driver)if isinstance(step['mouse_action'], str):if step['mouse_action'] == 'click':ac.click()elif step['mouse_action'] == 'click_on_element':ac.move_to_element(el)ac.click(el)log.info(f'click 已执行')elif step['mouse_action'] == 'double_click':ac.double_click()elif step['mouse_action'] == 'double_click_on_element':ac.move_to_element(el)ac.double_click(el)log.info(f'鼠标左键双击 已执行')elif step['mouse_action'] == 'context_click':ac.context_click()elif step['mouse_action'] == 'context_click_on_element':ac.move_to_element(el)ac.context_click(el)log.info(f'点击鼠标右键 已执行')# 向 input 标签发送内容,类似向 input 标签输入内容elif step['mouse_action'] == 'send_keys_to_element' and 'send_keys' in step:ac.move_to_element(el)ac.click(el)ac.send_keys_to_element(el, step['send_keys'])elif step['mouse_action'] == 'move_to_element':ac.move_to_element(el)ac.perform()elif isinstance(step['mouse_action'], dict):"""drag_and_drop: 从当前元素拖拽到某个元素然后松开,字段类型source: 原元素,变量名target: 目标元素,变量名move_by_offset: 鼠标重当前位置,移动到某个坐标x横坐标y纵坐标move_to_element_with_offset移动到距某个元素(左上角坐标)多少距离的位置element定位的元素x 横坐标y 纵坐标"""if 'drag_and_drop' in step['mouse_action']:if 'source' in step['mouse_action']['drag_and_drop'] and 'target' in step['mouse_action']['drag_and_drop']:source = m_save[step['mouse_action']['drag_and_drop']['source']]target = m_save[step['mouse_action']['drag_and_drop']['target']]ac.drag_and_drop(source, target)else:log.error('drag_and_drop 未执行,source,target 未定义 ')elif 'move_by_offset' in step['mouse_action']:if 'x' in step['mouse_action']['drag_and_drop'] and 'y' in step['mouse_action']['drag_and_drop']:x = m_save[step['mouse_action']['drag_and_drop']['x']]y = m_save[step['mouse_action']['drag_and_drop']['y']]ac.move_by_offset(x, y)else:log.error('move_by_offset 未执行,x,y 未定义 ')elif 'move_to_element_with_offset' in step['mouse_action']:if 'x' in step['mouse_action']['move_to_element_with_offset'] and 'y' in step['mouse_action']['move_to_element_with_offset']:x = m_save[step['mouse_action']['drag_and_drop']['x']]y = m_save[step['mouse_action']['drag_and_drop']['y']]element = m_save[step['mouse_action']['drag_and_drop']['element']]ac.move_to_element_with_offset(element, x, y)else:log.error('move_to_element_with_offset 未执行,x,y 未定义 ')ac.perform()def close_other_page(self, step):"""关闭其他标签页"""if 'close_other_page' in step:curr_handle = self.driver.current_window_handleall_handles = self.driver.window_handlesif len(all_handles) <= 1:log.warning('当前只有一个标签页,跳过关闭')returnelse:returnif isinstance(step['close_other_page'], int):if all_handles.index(curr_handle) == step['close_other_page']:log.warning('不支持关闭当前标签页,只能关闭除当前标签外的标签页')returnelse:if 0 <= step['close_other_page'] < len(all_handles):self.driver.switch_to.window(all_handles[step['close_other_page']])self.driver.close()else:# 关闭除当前页面的所有标签页all_handles.remove(curr_handle)for handle in all_handles:self.driver.switch_to.window(handle)self.driver.close()# 回到当前打开的页面self.driver.switch_to.window(curr_handle)@staticmethoddef set_think_time(step):"""设置思考时间"""if 'sleep' in step and step['sleep'] is not None:time.sleep(step['sleep'])def __get_element_sec_loction(self, step, el=None, els=None):"""元素二次定位"""# 如果step中定义了 loc_type、loc_value 的值,则自动进行二次定位if step['loc_type'].upper().endswith('S'):# 如果该元素页面有多种类型,则可传入一个一个位置列表,定位方式需要都一样if isinstance(step['loc_value'], list):for loc_value in step['loc_value']:try:els = el.find_elements(self.loc_type[step['loc_type'].upper()], loc_value)m_save['els'] = elsbreakexcept Exception as e:passelse:log.info(f"元素未找到:{step['loc_value']}")else:els = el.find_elements(self.loc_type[step['loc_type'].upper()], step['loc_value'])m_save['els'] = els# 如果定义了索引,则自动获取if 'loc_index' in step and step['loc_index'] is not None:el = els[step['loc_index']]m_save['el'] = elelse:if isinstance(step['loc_value'], list):for loc_value in step['loc_value']:try:el = el.find_element(self.loc_type[step['loc_type'].upper()], loc_value)m_save['el'] = elbreakexcept Exception as e:passelse:log.info(f"元素未找到:{step['loc_value']}")else:el = el.find_element(self.loc_type[step['loc_type'].upper()], step['loc_value'])m_save['el'] = elreturn el, elsdef __get_element(self, step, el=None, els=None):"""获取元素,定位元素:param step::param el::param els::return:"""# 如果是存在变量则执行这边的方法if 'element' in step and step['element'] is not None:if step['element'] in m_save:if isinstance(m_save[step['element']], list):if 'loc_index' in step:el = m_save[step['element']][step['loc_index']]else:els = m_save[step['element']]else:el = m_save[step['element']]else:log.error(f"{step['element']} 不存在,请检查元素名,例如:el,els")# 如果step中定义了 loc_type、loc_value 的值,则自动进行二次定位if el is not None and 'loc_type' in step and 'loc_value' in step and step['loc_type'].upper() != 'JS_ACTION':el, els = self.__get_element_sec_loction(step, el)else:if 'loc_type' in step and 'loc_value' in step and step['loc_type'].upper() != 'JS_ACTION':if step['loc_type'].upper().endswith('S'):if isinstance(step['loc_value'], list):for loc_value in step['loc_value']:locator = (self.loc_type[step['loc_type'].upper()], loc_value)if 'loc_method' in step:# 显示等待执行,wait_until wait_not_untilel, els = self.expected_wait_find_element_method(step, locator)else:try:els = self.driver.find_elements(*locator)m_save['els'] = elsbreakexcept Exception as e:passelse:log.info(f"元素未找到:{step['loc_value']}")else:locator = (self.loc_type[step['loc_type'].upper()], step['loc_value'])if 'loc_method' in step:# 显示等待执行,wait_until wait_not_untilel, els = self.expected_wait_find_element_method(step, locator)else:els = self.driver.find_elements(*locator)m_save['els'] = els# 如果定义了索引,则自动获取if 'loc_index' in step and step['loc_index'] is not None:el = els[step['loc_index']]m_save['el'] = elelse:if isinstance(step['loc_value'], list):for loc_value in step['loc_value']:locator = (self.loc_type[step['loc_type'].upper()], loc_value)if 'loc_method' in step:# 显示等待执行,wait_until wait_not_untilel, els = self.expected_wait_find_element_method(step, locator)else:try:el = self.driver.find_element(*locator)m_save['el'] = elbreakexcept Exception as e:passelse:log.info(f"元素未找到:{step['loc_value']}")else:locator = (self.loc_type[step['loc_type'].upper()], step['loc_value'])if 'loc_method' in step:# 显示等待执行,wait_until wait_not_untilel, els = self.expected_wait_find_element_method(step, locator)else:el = self.driver.find_element(*locator)m_save['el'] = elcurr_element = '元素定位:'if el is not None:curr_element += f'el: {el} 'if els is not None:curr_element += f'els: {els} 'log.info(curr_element)if 'action' in step and step['action'] == 'get_element_length' and els is not None:m_save['get_element_length'] = len(els)log.info(f"get_element_length: {m_save['get_element_length']}")return el, elsdef __return_result(self, step):return_list = {}if isinstance(step['return'], list):for ret_key in step['return']:if ret_key in m_save:return_list[ret_key] = m_save[ret_key]else:log.warning(f'{ret_key} 该值未暂存,值未返回')return return_listdef __save_page_values(self, step):"""保存页面执行过程中产生的变量值"""# 保存获取的值if 'save' in step and step['save'] is not None:if 'key' not in step['save'] or 'value' not in step['save']:log.error('save 缺少参数 key, value')returnif step['save']['value'] in m_save:log.info(f"{step['save']['value']} 已作为键存在,已重新保存,键名:{step['save']['key']}")m_save[step['save']['key']] = m_save[step['save']['value']]else:m_save[step['save']['key']] = step['save']['value']def __page_actions(self, step, el: webdriver.Chrome = None):"""操作方法"""# 把非页面操作过滤掉if 'action' in step and step['action'] in ['break','continue','if','for','js_action','get_element_length']:returnweb_el_value = Noneif 'action' in step and step['action'] is not None:# 获取元素内容、网页页面内容、点击、输入内容if step['action'] == 'get_title':web_el_value = self.driver.titleelif step['action'] == 'get_current_url':web_el_value = self.driver.current_urlelif step['action'] == 'get_browser_name':web_el_value = self.driver.nameelif step['action'] == 'get_page_source':web_el_value = self.driver.page_sourceelif step['action'] == 'click' and el is not None:el.click()log.info(f'click 已执行')elif step['action'] == 'clear' and el is not None:el.clear()log.info(f'clear 已执行')elif step['action'] == 'get_text' and el is not None:web_el_value = el.textelif step['action'] == 'get_id' and el is not None:web_el_value = el.idelif step['action'] == 'get_location' and el is not None:web_el_value = el.locationelif step['action'] == 'get_tag_name' and el is not None:web_el_value = el.tag_nameelif step['action'] == 'get_el_size' and el is not None:web_el_value = el.sizeelif step['action'] == 'get_attribute' and 'attribute_name' in step and step['attribute_name'] is not None:web_el_value = el.get_attribute(step['attribute_name'])elif step['action'] == 'screen_shot':web_el_value = ScreenShot().shot_browser(self.driver)elif step['action'][:5] == 'wait_':try:self.wait_find_element_action(step)except Exception as e:log.info(f'expected_wait_method: {e}')else:log.info(f"action:{step['action']} 该方法暂不支持")if web_el_value is not None:log.info(f"当前获取页面值, {step['action']}: {web_el_value}")m_save[step['action']] = web_el_valueelif 'send_keys' in step and step['send_keys'] is not None:if isinstance(step['send_keys'], str):el.send_keys(step['send_keys'])m_save['send_keys'] = step['send_keys']log.info(f"send_keys value send is : {step['send_keys']}")elif isinstance(step['send_keys'], list):key_board_list = []key_board_list.extend(step['send_keys'])for i, k in enumerate(key_board_list):if k.upper() in self.keyboard:key_board_list[i] = self.keyboard[k.upper()]el.send_keys(*key_board_list)m_save['send_keys'] = step['send_keys']log.info(f"send_keys value send is : {key_board_list}")else:log.error(f"send_keys 格式错误:{step['send_keys']}")return web_el_valuedef order_condition(self, step):"""执行条件"""if 'condition' in step and step['condition'] is not None:ret = self.execute_condition(step['condition'])m_save['condition'] = retreturn retelse:return Nonedef execute_condition(self, condition_expr: str):"""条件表达式,或断言的条件表达式:param condition_expr: 输入的条件:return:"""if str(condition_expr).strip().upper() == 'TRUE':m_save['condition'] = Truereturn Trueif str(condition_expr).strip().upper() == 'FALSE':m_save['condition'] = Falsereturn Falselog.info(f'当前输入的表达式:{condition_expr}')patter = pile("[.!-_a-zA-Z0-9]*").findall(condition_expr)expr_key = ['in', 'not', 'and', 'or', '+', '-', '*', '/', '%', '**', '!=', '==', '>', '<', '>=', '<=', '(', ')']for i, p in enumerate(patter):if p != '' and not p.isdigit() and p not in expr_key:# 判断是否是小数if '.' in p:if p.split('.')[0].isdigit() and p.split('.')[1].isdigit():continueif p in m_save:patter[i] = f'm_save["{p}"]'else:patter[i] = f'"{p}"'condition_expr = " ".join(patter)condition_exec_result = eval(condition_expr)log.info(f'解析的表达式:{condition_expr},执行结果:{condition_exec_result}')return condition_exec_resultdef expected_wait_find_element_method(self, step, locator, el=None, els=None):"""支持所有元素显示等待定位method: 定位方法,当前支持的方法:如下wait_untilwait_not_untiltimeout: 10 超时时间poll_frequency: 0.51次/0.5秒,查找元素频率"""if 'method' not in step['loc_method'] or 'timeout' not in step['loc_method'] or 'poll_frequency' not in step['loc_method']:log.error('method, timeout, poll_frequency 值不能为空')return# 超时会直接抛出异常,需要捕获# 1、显示等待,wait = WebDriverWait(driver=self.driver, timeout=step['loc_method']['timeout'], poll_frequency=step['loc_method']['poll_frequency'])if step['loc_method']['method'] in ['wait_until','wait_not_until']:if step['loc_type'].upper().endswith('S'):if step['loc_method']['method'] == 'wait_until':els = wait.until(lambda driver: driver.find_elements(*locator))m_save['els'] = els# wait_not_untilelse:els = wait.until(lambda driver: driver.find_elements(*locator))m_save['els'] = elselse:if step['loc_method']['method'] == 'wait_until':el = wait.until(lambda driver: driver.find_element(*locator))m_save['el'] = el# wait_not_untilelse:el = wait.until(lambda driver: driver.find_element(*locator))m_save['el'] = ellog.info('wait loc_method run succeed.')return el, elselif step['loc_method']['method'] in ['wait_until_visibility',# 直到元素可见'wait_until_invisibility', # 直到元素不可见'wait_until_presence_of_el',# 直到元素加载出来'wait_until_presence_of_els',# 直到全部元素加载出来'wait_until_clickable', # 直到元素可点击'wait_until_alert']: # 直到alert弹窗出现# 3、判断元素是否可见(可见代表元素非隐藏,并且元素宽和高都不等于 0), 返回一个元素if step['loc_method']['method'] == 'wait_until_visibility':el = wait.until(EC.visibility_of_element_located(locator), message='判断元素是否可见,定位元素直到元素可见')m_save['el'] = el# 判断元素不可见,如果是可见元素会抛出超时异常elif step['loc_method']['method'] == 'wait_until_invisibility':el = wait.until(EC.invisibility_of_element_located(locator), message='判断元素不可见,定位元素直到元素不可见')m_save['el'] = el# 4、一个只要一个符合条件的元素加载出来就通过;另一个必须所有符合条件的元素都加载出来才行elif step['loc_method']['method'] == 'wait_until_presence_of_el':el = wait.until(EC.presence_of_element_located(locator), message='一个符合条件的元素加载出来就通过')m_save['el'] = elelif step['loc_method']['method'] == 'wait_until_presence_of_els':els = wait.until(EC.presence_of_all_elements_located(locator), message='所有符合条件的元素都加载出来则通过')m_save['els'] = els# 7、判断是否有alert出现,有则返回alert元素<class 'mon.alert.Alert'>,否则返回falseelif step['loc_method']['method'] == 'wait_until_alert':el = wait.until(EC.alert_is_present(), message='判断是否有alert出现,有则返回alert元素否则返回false')m_save['el'] = el# 8、以下条件判断元素是否可点击,传入locatorelif step['action'] == 'wait_until_clickable':el = wait.until(EC.element_to_be_clickable(locator), message='判断元素是否可点击,部分按钮需要加载出来后才能点击')m_save['el'] = elreturn el, elselse:passdef wait_find_element_action(self, step, el=None, els=None):"""timeout: 10 超时时间poll_frequency: 0.51次/0.5秒,查找元素频率"""if step['action'] in ['wait_until_title_is','wait_until_title_contains','wait_until_presence_of_el_text','wait_until_presence_of_el_value','wait_until_switch_frame','wait_until_selected','wait_until_located_selected','wait_until_selection_state','wait_until_located_selection_selected','wait_until_staleness']:locator = (self.loc_type[step['loc_type'].upper()], step['loc_value'])wait = WebDriverWait(driver=self.driver, timeout=step['loc_method']['timeout'], poll_frequency=step['loc_method']['poll_frequency'])# 2、判断当前页面的 title 是否完全等于(==)预期字符串,返回布尔值 <class 'bool'>if step['action'] == 'wait_until_title_is':el_bool = wait.until(EC.title_is(step['loc_method']['title']), message='断言当前页面标签标题')m_save['wait_until_title_is'] = el_bool# 判断当前页面的 title 是否包含预期字符串,返回布尔值 <class 'bool'>elif step['action'] == 'wait_until_title_contains':el_bool = wait.until(EC.title_contains(step['loc_method']['title']), message='断言当前页面标签标题')m_save['wait_until_title_contains'] = el_bool# 3、同上一方法,只是上一方法参数为locator,这个方法参数是 定位后的元素 返回一个元素,**这个方法似乎没用**# elif step['action'] == 'wait_until_visibility_by_locator':#if isinstance(m_save[step['element']], list):# els = wait.until(EC.visibility_of(m_save['els']), message='locator定位可见元素')# m_save['wait_until_visibility_by_locator'] = els#else:# el = wait.until(EC.visibility_of(m_save[step['element']]), message='locator定位可见元素')# m_save['wait_until_visibility_by_locator'] = el# 5、以下两个条件判断某段文本是否出现在某元素中,一个判断元素的text,一个判断元素的value, 返回布尔值 <class 'bool'>elif step['action'] == 'wait_until_presence_of_el_text':el_bool = wait.until(EC.text_to_be_present_in_element(locator, step['loc_method']['text']), message='判断文本是否出现在元素text中')m_save['wait_until_presence_of_el_text'] = el_boolelif step['action'] == 'wait_until_presence_of_el_value':el_bool = wait.until(EC.text_to_be_present_in_element_value(locator, step['loc_method']['value']), message='判断文本是否出现在元素value属性中')m_save['wait_until_presence_of_el_value'] = el_bool# 6、以下条件判断frame是否可切入,可传入locator元组或者直接传入定位方式:id、name、index或WebElementelif step['action'] == 'wait_until_switch_frame':el = wait.until(EC.frame_to_be_available_and_switch_to_it(locator), message='判断frame是否可切入,如果可以则自动切入')m_save['wait_until_switch_frame'] = el# 9、以下四个条件判断元素是否被选中,# 第一个条件传入WebElement对象# 第二个传入locator元组# 第三个传入WebElement对象以及状态,相等返回True,否则返回False# 第四个传入locator以及状态,相等返回True,否则返回Falseelif step['action'] == 'wait_until_selected':el = wait.until(EC.element_to_be_selected(m_save['el']), message='判断元素是否被选中-1')m_save['wait_until_selected'] = elelif step['action'] == 'wait_until_located_selected':el = wait.until(EC.element_located_to_be_selected(locator), message='判断元素是否被选中-2')m_save['wait_until_located_selected'] = elelif step['action'] == 'wait_until_selection_state':el_bool = wait.until(EC.element_selection_state_to_be(m_save['el'], step['loc_method']['select']), message='判断元素是否被选中-3')m_save['wait_until_selection_state'] = el_boolelif step['action'] == 'wait_until_located_selection_selected':el_bool = wait.until(EC.element_located_selection_state_to_be(locator, step['loc_method']['select']), message='判断元素是否被选中-4')m_save['wait_until_located_selection_selected'] = el_bool# 10、判断一个元素是否仍在DOM中,传入WebElement对象,可以判断页面是否刷新了,如果还是存在则会抛出超时异常elif step['action'] == 'wait_until_staleness':wait.until(EC.staleness_of(m_save['el']), message='判断一个元素是否仍在DOM中,传入WebElement对象,可以判断页面是否刷新了,如果还是存在则会抛出超时异常')if __name__ == '__main__':driver = open_chrome(max_window=False)bp = BasePage('driver')ryf = read_yaml_file('./case_test_basepagefunc.yaml')# 执行yaml文件中的步骤,提前导入一些参数# 提前导入一些内容,比如向输入框输入的 数字, 配置脚本要调用则使用,如:$phone_num$ret = bp.exec_steps(ryf['case_05']['steps'], {"phone_num": 123456789})print(ret)

操作步骤配置,示例

# 主要测试用例执行器,执行其使用参考这个文件内容编写# 测试定位方式,循环列表元素,自定义变量定义和获取页面内容并保存case_01:steps:-desc: 前往百度,向输入框中输入内容loc_type: idloc_value: kwaction: send_keyssend_keys: selenium grid-desc: 点百度一下,点击操作loc_type: idloc_value: suaction: click# 操作后休息5ssleep: 5-desc: 获取元素列表,定位元素并保存,默认会保存一个值:CSS_SELECTORSloc_type: CSS_SELECTORSloc_value: '#content_left > div'-desc: 保存用于判断结束循环的值send_keys: 博客园save_name: content_k2-desc: 遍历百度搜索的内容element: CSS_SELECTORSaction: forsteps:-desc: 获取元素列表的单个元素内容action: get_textsave_name: k2-desc: 如果【博客园】存在则停止循环condition: content_k2 in k2action: breaksleep: 5# 退出浏览器功能case_02:steps:-desc: 前往百度,向输入框中输入内容loc_type: idloc_value: kwaction: send_keyssend_keys: selenium grid-desc: 点百度一下,点击操作loc_type: idloc_value: suaction: clicksleep: 5-desc: 退出浏览器action: quit# 打开多个浏览器页面、关闭当前页面、切换页面case_03:steps:-desc: 打开百度loc_value: action: open_page-desc: 打开百度翻译loc_value: /?aldtype=16047#auto/zhaction: open_new_page-desc: 打开csdnloc_value: action: open_new_page-desc: 切换到百度翻译loc_value: 1action: switch_page_by_index-desc: 关闭百度翻译action: close-desc: 切换到百度翻译loc_value: 1action: switch_page_by_index-desc: 关闭浏览器action: quitcase_04:steps:-desc: 前往百度,向输入框中输入内容loc_type: idloc_value: kwaction: send_keyssend_keys: selenium grid-desc: 点百度一下,点击操作loc_type: idloc_value: suaction: clicksleep: 5-desc: 退出浏览器action: quitcase_05:steps:-desc: 打开页面action: open_pageloc_value: /-desc: 点登陆loc_type: xpathloc_value: //*[text()="登录"]action: click-desc: 判断是否打开登陆页面loc_type: class_nameloc_value: session-form__titleloc_method:method: wait_until_visibilitytimeout: 10poll_frequency: 0.5-desc: 清空输入框loc_type: idloc_value: user_loginaction: clear-desc: 向输入框中输入内容loc_type: idloc_value: user_loginaction: send_keyssend_keys: test test test-save:key: actionvalue: 77777-desc: 向输入框中输入内容loc_type: idloc_value: user_loginaction: send_keyssend_keys:- control- a-desc: 查看返回值return:- send_keys- actionsleep: 100# 这里主要是其他特殊操作的格式,配置示例case_XXX:steps:-save:key: bilibili_shouyevalue: B站首页-desc: 判断是打开B站首页成功loc_type: xpathloc_value: //span[@class="xxxxxxxx"]/span[2]loc_method:# 定位直到元素可见,页面60s内找不到就抛出异常method: wait_until_visibilitytimeout: 60poll_frequency: 0.5action: get_text-condition: get_text == bilibili_shouye# 只要执行return 就会停止后面的所有步骤,并返回指定内容return:- condition- bilibili_shouye# -------------------------------------------------------------------desc: 根据链接名称定位,60s内定位到后就点击链接loc_type: link_textloc_value: $phone_num$loc_method:method: wait_untiltimeout: 60poll_frequency: 0.5action: click# -------------------------------------------------------------------desc: xpath 定位单个元素loc_type: xpathloc_value: //input[@placeholder="xxxxxxxx"]/../span/span-desc: 通过js操作元素,这里的点击示例# element,继承前面定位的元素,所有定位的单个元素默认为 el,多个则 els,也可以使用save对定位到的元素重新指定 键 名称element: elaction: js_actionloc_value: |arguments[0].click();sleep: 3-desc: 给定位的元素重新指定键名称save:key: new_name_elvalue: el# ------------------------------------------------------------------# condition 可以执行 python 代码,也可以执行 bool运算和简单数学计算,例如:-desc: xpath + s 定位多个元素loc_type: xpathsloc_value: //tbody[@class="ant-table-tbody"]loc_index: -1save:# 重新给定位的元素命名一下key: zzzzzzzzvalue: el-desc: 获取label标签class属性值element: elloc_type: css_selectorloc_value: tr:nth-of-type(2) > td:nth-of-type(1) > label# 默认会暂存一个 键 为“get_attribute”值,值为class的属性值action: get_attributeattribute_name: class-desc: 判断“checked”字符串,是否在 get_attribute 中action: ifcondition: checked not in get_attributesteps:-desc: 对 zzzzzzzz 元素二次定位,找出 zzzzzzzz 下所有的 tr 标签element: zzzzzzzz# css_selector + s 定位多个元素loc_type: css_selectorsloc_value: tr-element: els# loc_index 元素索引,只获取索引为 1 的元素loc_index: 1action: click

后言

这里只汇总了大部分常见的一些操作方式,有些操作方式可能有遗漏,有问题、有好的建议的希望多留言,后面多更新完善一下。

本内容不代表本网观点和政治立场,如有侵犯你的权益请联系我们处理。
网友评论
网友评论仅供其表达个人看法,并不表明网站立场。