Compare commits

..
Author SHA1 Message Date
Christopher Hiller 922725b4f1 chore: publish
- appium@2.0.0-beta.70
 - @appium/docutils@0.3.13
2023-05-19 15:24:13 -07:00
Christopher Hiller f0ebe2e02f fix(docutils): fix broken executable 2023-05-19 15:22:46 -07:00
Christopher Hiller 2ca1581870 chore: publish
- appium@2.0.0-beta.69
 - @appium/docutils@0.3.12
2023-05-19 14:56:44 -07:00
Christopher Hiller 6aec50b074 chore(docutils): lint
- consolidate some eslint overrides
2023-05-19 13:42:08 -07:00
Christopher Hiller 426830d14a fix(docutils): exec mike/mkdocs via python -m
- also adds a "custom typedoc path" option
- consolidate the `whichPython*` functions into a single `findPython` fn
2023-05-19 13:42:08 -07:00
Christopher Hiller 9538c34131 fix(docutils): add @types/which as it is exported 2023-05-19 13:42:08 -07:00
Christopher Hiller 5d24ea1ad2 fix(docutils): run mkdocs via python -m 2023-05-19 13:42:08 -07:00
Christopher Hiller db73971018 fix(docutils): provide CJS executable for proper linking 2023-05-19 13:42:08 -07:00
Christopher Hiller 7b17754786 fix(docutils): try python3 if python not in PATH 2023-05-19 13:42:08 -07:00
Christopher Hiller ba07d2a854 docs(typedoc-plugin-appium): update README.md
- add option documentation
- add warning about using the module directly
- add instructions for updating snapshots
2023-05-19 12:49:48 -07:00
Jonathan Lipps ee365ad168 chore: publish
- appium@2.0.0-beta.68
 - @appium/base-driver@9.3.9
 - @appium/base-plugin@2.2.9
 - @appium/doctor@2.0.16
 - @appium/docutils@0.3.11
 - @appium/fake-plugin@3.1.13
 - @appium/images-plugin@2.1.1
 - @appium/support@4.0.1
 - @appium/test-support@3.0.15
2023-05-19 09:47:39 -07:00
renovate[bot] 5e351dc503 chore(deps): update definitelytyped 2023-05-19 14:04:30 +00:00
Mykola Mokhnach f512e596eb fix(images-plugin): Add the missing dependency to support (#18652) 2023-05-19 11:13:23 +02:00
Mykola Mokhnach 3eec0227a2 docs(support): Update readme (#18647) 2023-05-19 08:14:06 +02:00
Christopher Hiller ea42548442 fix(support): add missing @types/which 2023-05-18 13:09:03 -07:00
renovate[bot] e1fa6c2133 chore(deps): update dependency rimraf to v5 2023-05-18 11:30:26 -07:00
Jonathan Lipps 6b245534c2 chore: publish
- appium@2.0.0-beta.67
 - @appium/base-driver@9.3.8
 - @appium/base-plugin@2.2.8
 - @appium/doctor@2.0.15
 - @appium/docutils@0.3.10
 - @appium/driver-test-support@0.4.6
 - @appium/execute-driver-plugin@3.0.13
 - @appium/fake-plugin@3.1.12
 - @appium/images-plugin@2.1.0
 - @appium/opencv@2.1.0
 - @appium/plugin-test-support@0.3.14
 - @appium/support@4.0.0
 - @appium/test-support@3.0.14
 - @appium/typedoc-plugin-appium@0.6.4
 - @appium/types@0.11.1
2023-05-17 13:06:02 -07:00
Jonathan Lipps 6e4d992b8b test(appium): increase local test timeout to match CI; some computers are slow 2023-05-17 13:04:26 -07:00
Kazuaki Matsuo ac4d4f7262 docs: add users in GOVERNANCE.md (#18630) 2023-05-15 14:54:55 -07:00
Jonathan Lipps d891c9570a docs(appium): fix error in docs build script (#18639)
* docs(appium): fix error in docs build script
in the case where the gh-pages branch was out of sync with local
it would just fail. now we will attempt to rebase first.

* docs(appium): fix error in docs build argify

* test(typedoc-plugin-appium): fix failing test fixture
2023-05-14 23:43:26 -07:00
Jonathan Lipps 37e6376d10 docs(appium): add extension doc authoring guide (close #16802) (#18636) 2023-05-12 09:40:27 -07:00
Jonathan Lipps 5ceaa5b38a docs(appium): fix migration doc reference to chromedriver-executable (#18635) 2023-05-11 14:10:14 -07:00
Edgars Eglītis cfa7b2d442 docs: create aggregate changelog (#18629)
* docs: create aggregate changelog

* docs: mention plugins and clients
2023-05-10 00:58:26 -07:00
Kazuaki MatsuoandJonathan Lipps c8d707df90 docs(appium): add helper tools section in the ecosystem doc (#18621)
* docs(appium): add helper apps

* Update index.md

* Apply suggestions from code review

Co-authored-by: Jonathan Lipps <jlipps@gmail.com>

---------

Co-authored-by: Jonathan Lipps <jlipps@gmail.com>
2023-05-09 11:47:12 -07:00
Jonathan Lipps 40e5199caa fix(typedoc-plugin-appium): don't prefix execute methods with "Script: " (#18618)
When in the docs, it's clear that it's an execute method
This prefix adds a lot of extra noise to the commands list
I don't think we need it.
2023-05-08 13:42:52 -07:00
Jonathan Lipps cdfe7c3d82 fix(docutils): remove -t arg for mkdocs which prevents theme customizations (fix #18536) (#18617) 2023-05-08 10:31:09 -07:00
renovate[bot] 3c4d3acc09 fix(types): update dependency type-fest to v3.10.0 2023-05-07 08:49:43 +00:00
renovate[bot] 2f26c51dac chore(deps): update dependency eslint to v8.40.0 2023-05-07 07:54:32 +00:00
renovate[bot] 155f24bb4d chore(deps): update dependency lerna to v6.6.2 2023-05-06 14:08:09 +00:00
renovate[bot] 74f29cfd5e chore(deps): update dependency @types/node to v18.16.5 2023-05-06 11:44:47 +00:00
Kazuaki Matsuo ac85f051a7 docs: add notice in the changelog (#18608)
* docs: add notice in the changelog

* docs: update changelog in current master
2023-05-06 00:42:42 -07:00
Chaitya Shah e68f17b3cc refactor(support): #17494 Replace rimraf module with fs.rm (#18580)
* Replace rimraf module with fs.rm

* chore: Fix syntax and import rmSync

* chore: remove imports / dependency on rimraf module

* chore: add return type docstring for fs.rimraf
2023-05-05 12:23:35 -07:00
renovate[bot] 25fd473b98 chore(deps): update commitlint monorepo to v17.6.3 2023-05-05 12:41:48 +00:00
renovate[bot] 55060f8e8f chore(docutils): update dependency mkdocs to v1.4.3 2023-05-03 13:12:40 +00:00
Mykola Mokhnach 1da9de4225 refactor(support)!: Drop jimp and pngjs in favour of sharp (#18584)
BREAKING CHANGE: The following methods have been removed from imageUtil module: getJimpImage, base64ToImage, imageToBase64, cropImage
BREAKING CHANGE: The following constants have been removed from imageUtil module: MIME_JPEG, MIME_PNG, MIME_BMP
2023-05-02 22:02:07 +02:00
Mykola Mokhnach 6f10078591 test(images-plugin): Add tests for broken/empty template cases (#18589) 2023-05-02 21:36:22 +02:00
renovate[bot] c311c48c55 chore(docutils): update dependency mkdocs-material to v9.1.9 2023-05-02 10:04:36 +00:00
renovate[bot] 869b7aa4a2 fix(support): update dependency which to v3.0.1 2023-05-02 10:00:56 +00:00
renovate[bot] 82e2150c66 chore(deps): update eslint-related packages to v5.59.2 2023-05-02 07:20:23 +00:00
renovate[bot] 94a207fc97 fix(types): update dependency type-fest to v3.9.0 2023-05-01 14:18:41 +00:00
renovate[bot] c568523e01 fix(support): update dependency semver to v7.5.0 2023-05-01 10:50:17 +00:00
renovate[bot] 91a6bc5925 fix(support): update dependency axios to v1.4.0 2023-05-01 07:48:07 +00:00
renovate[bot] 6499eea0af fix(execute-driver-plugin): update dependency webdriverio to v7.31.1 2023-04-30 14:16:26 +00:00
renovate[bot] add0eaa05f fix(opencv): update dependency sharp to v0.32.1 2023-04-30 11:57:53 +00:00
renovate[bot] 3d1e4af858 fix(docutils): update dependency yargs to v17.7.2 2023-04-30 08:22:18 +00:00
renovate[bot] e7df6fd4f0 fix(doctor): update dependency appium-adb to v9.11.2 2023-04-29 14:13:53 +00:00
renovate[bot] c37072e6a9 chore(deps): update definitelytyped 2023-04-29 11:44:08 +00:00
renovate[bot] 864ab11b8a chore(deps): update dependency @types/node to v18.16.2 2023-04-28 12:07:07 +00:00
Mykola Mokhnach 4f564ef7ad fix(base-driver): Ignore unknown script arguments (#18575) 2023-04-27 22:40:20 +02:00
renovate[bot] b2bda2188b chore(deps): update dependency lint-staged to v13.2.2 2023-04-27 16:34:23 +00:00
renovate[bot] ddadd6977d fix(docutils): update dependency yaml to v2.2.2 2023-04-26 16:47:33 +00:00
renovate[bot] fa681f6355 chore(docutils): update dependency mkdocs-material to v9.1.8 2023-04-26 11:08:15 +00:00
renovate[bot] 522177f1d3 chore(deps): update dependency @types/node to v18.16.1 2023-04-26 11:05:08 +00:00
Mykola Mokhnach ab7958c8e8 refactor(images-plugin): Replace Jimp usages with Sharp (#18572) 2023-04-26 07:24:50 +02:00
renovate[bot] 1e4bc0a273 chore(deps): update eslint-related packages to v5.59.1 2023-04-25 14:48:45 +00:00
Mykola Mokhnach 9f8b9cef70 chore: Tune elements caching logic (#18570) 2023-04-25 10:09:48 +02:00
Mykola Mokhnach 2fb844c8ad fix(opencv): Make sure allocated CV objects are properly released afterwards (#18569) 2023-04-25 06:06:43 +02:00
Mykola Mokhnach 2796639b11 feat(image-plugin): Make it possible to find elements inside of other elements (#18555) 2023-04-24 22:22:53 +02:00
renovate[bot] 65f74d818f chore(deps): update eslint-related packages 2023-04-24 14:02:37 +00:00
renovate[bot] cf398bdfd6 chore(deps): update dependency @types/node to v18.16.0 2023-04-24 10:00:15 +00:00
renovate[bot] ddfca214bf chore(deps): update dependency prettier to v2.8.8 2023-04-23 12:30:05 +00:00
renovate[bot] 0f19902918 chore(docutils): update dependency mkdocs-material to v9.1.7 2023-04-23 08:57:51 +00:00
renovate[bot] 9ad7279fa7 fix(test-support): update dependency sinon to v15.0.4 2023-04-22 13:19:04 +00:00
renovate[bot] 66922279b7 fix(support): update dependency axios to v1.3.6 2023-04-22 08:26:47 +00:00
renovate[bot] 61ca898c47 fix(execute-driver-plugin): update dependency vm2 to v3.9.17 2023-04-21 14:04:24 +00:00
renovate[bot] d3c30e5079 chore(deps): update definitelytyped 2023-04-21 09:22:50 +00:00
Jonathan Lipps 35b5ac72ab docs(appium): remove client plugin from migration guide since it's not really a thing yet 2023-04-20 13:57:55 -07:00
76 changed files with 20531 additions and 7569 deletions
+9 -7
View File
@@ -10,12 +10,6 @@
"files": "packages/*/test/**/*",
"rules": {"func-names": "off"}
},
{
"files": ["packages/*/index.js", "packages/*/scripts/**/*.js", "test/*.js"],
"parserOptions": {
"sourceType": "script"
}
},
{
"files": [
"packages/appium/support.js",
@@ -27,9 +21,17 @@
}
},
{
"files": ["./test/setup.js", "./**/scripts/**/*.js", "./packages/*/index.js"],
"files": [
"./test/setup.js",
"./**/scripts/**/*.js",
"./packages/*/index.js",
"./packages/docutils/bin/appium-docs.js"
],
"rules": {
"@typescript-eslint/no-var-requires": "off"
},
"parserOptions": {
"sourceType": "script"
}
},
{
+1 -2
View File
@@ -7,6 +7,5 @@ module.exports = {
// forbids use of .only() in CI
forbidOnly: Boolean(process.env.CI),
color: true,
// increase default timeout for CI since it can be slow
timeout: process.env.CI ? '5s' : '2s',
timeout: '5s',
};
+36 -3917
View File
File diff suppressed because it is too large Load Diff
+2
View File
@@ -84,6 +84,8 @@ The current committers are (in addition to the TC members above):
* [@boneskull](https://github.com/boneskull)
* [@mwakizaka](https://github.com/mwakizaka)
* [@rerorero](https://github.com/rerorero)
* [@tomriddly](https://github.com/tomriddly)
* [@eglitise](https://github.com/eglitise)
### Contributors
+18778 -2544
View File
File diff suppressed because it is too large Load Diff
+14 -14
View File
@@ -85,29 +85,29 @@
},
"devDependencies": {
"@colors/colors": "1.5.0",
"@commitlint/cli": "17.6.1",
"@commitlint/config-conventional": "17.6.1",
"@commitlint/cli": "17.6.3",
"@commitlint/config-conventional": "17.6.3",
"@tsconfig/node14": "1.0.3",
"@types/chai": "4.3.4",
"@types/chai": "4.3.5",
"@types/chai-as-promised": "7.1.5",
"@types/diff": "5.0.3",
"@types/mocha": "10.0.1",
"@types/node": "18.15.11",
"@types/node": "18.16.13",
"@types/semver": "7.3.13",
"@types/sinon": "10.0.14",
"@types/sinon": "10.0.15",
"@types/sinon-chai": "3.2.9",
"@types/teen_process": "2.0.0",
"@types/ws": "8.5.4",
"@types/yargs": "17.0.24",
"@typescript-eslint/eslint-plugin": "5.58.0",
"@typescript-eslint/parser": "5.58.0",
"@typescript-eslint/eslint-plugin": "5.59.2",
"@typescript-eslint/parser": "5.59.2",
"asyncbox": "2.9.4",
"chai": "4.3.7",
"chai-as-promised": "7.1.1",
"chai-webdriverio-async": "2.7.0",
"conventional-changelog-conventionalcommits": "5.0.0",
"cpy-cli": "4.2.0",
"eslint": "8.38.0",
"eslint": "8.40.0",
"eslint-config-prettier": "8.8.0",
"eslint-find-rules": "4.1.0",
"eslint-import-resolver-typescript": "3.5.5",
@@ -120,8 +120,8 @@
"handlebars": "4.7.7",
"husky": "8.0.3",
"json-schema-to-typescript": "12.0.0",
"lerna": "6.6.1",
"lint-staged": "13.2.1",
"lerna": "6.6.2",
"lint-staged": "13.2.2",
"log-symbols": "4.1.0",
"midnight-smoker": "2.0.3",
"mjpeg-consumer": "2.0.0",
@@ -129,11 +129,11 @@
"mocha": "10.2.0",
"mocha-chai-jest-snapshot": "1.1.4",
"npm-run-all": "4.1.5",
"prettier": "2.8.7",
"prettier": "2.8.8",
"rewiremock": "3.14.5",
"rimraf": "3.0.2",
"rimraf": "5.0.1",
"serve-static": "1.15.0",
"sinon": "15.0.3",
"sinon": "15.0.4",
"sinon-chai": "3.7.0",
"sync-monorepo-packages": "1.0.2",
"through2": "4.0.2",
@@ -144,7 +144,7 @@
"typedoc-plugin-resolve-crossmodule-references": "0.3.3",
"typescript": "4.9.5",
"validate.js": "0.13.1",
"webdriverio": "7.30.2",
"webdriverio": "7.31.1",
"ws": "8.13.0",
"yaml-js": "0.3.1"
},
+39
View File
@@ -3,6 +3,45 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [2.0.0-beta.70](https://github.com/appium/appium/compare/appium@2.0.0-beta.69...appium@2.0.0-beta.70) (2023-05-19)
**Note:** Version bump only for package appium
## [2.0.0-beta.69](https://github.com/appium/appium/compare/appium@2.0.0-beta.68...appium@2.0.0-beta.69) (2023-05-19)
**Note:** Version bump only for package appium
## [2.0.0-beta.68](https://github.com/appium/appium/compare/appium@2.0.0-beta.67...appium@2.0.0-beta.68) (2023-05-19)
**Note:** Version bump only for package appium
## [2.0.0-beta.67](https://github.com/appium/appium/compare/appium@2.0.0-beta.66...appium@2.0.0-beta.67) (2023-05-17)
### Bug Fixes
* **docutils:** update dependency yaml to v2.2.2 ([ddadd69](https://github.com/appium/appium/commit/ddadd6977d76a8a5ce1559d2a4ff4a31b58fb42f))
* **execute-driver-plugin:** update dependency webdriverio to v7.31.1 ([6499eea](https://github.com/appium/appium/commit/6499eea0af6aea1bfb69b1e36c4c445e3ab05b82))
* **support:** update dependency axios to v1.3.6 ([6692227](https://github.com/appium/appium/commit/66922279b7742a08613f472585a4a1cb70f80683))
* **support:** update dependency axios to v1.4.0 ([91a6bc5](https://github.com/appium/appium/commit/91a6bc5925ab8ffc4ab6d05883900f7d186e49a9))
* **support:** update dependency semver to v7.5.0 ([c568523](https://github.com/appium/appium/commit/c568523e017a8b52bdf5132a48bbfde791de704f))
* **types:** update dependency type-fest to v3.10.0 ([3c4d3ac](https://github.com/appium/appium/commit/3c4d3acc09d2ca1ed74dc77c18c62482e4c70239))
* **types:** update dependency type-fest to v3.9.0 ([94a207f](https://github.com/appium/appium/commit/94a207fc9718068f3657c51cc8be0ef682f16b11))
## [2.0.0-beta.66](https://github.com/appium/appium/compare/appium@2.0.0-beta.65...appium@2.0.0-beta.66) (2023-04-20)
**Note:** Version bump only for package appium
@@ -2,4 +2,93 @@
title: Building Docs for Appium Extensions
---
TODO
Once you've [built a driver](./build-drivers.md) or [built a plugin](./build-plugins.md) for
Appium, you will hopefully want to document how that extension works for your users. The most basic
way of doing this is to write up a quick README and keep it in the root of your project's
repository. However, this can involve a lot of duplication of effort, especially when documenting
things like Appium commands.
Let's say your driver implements ~25 of the standard WebDriver protocol commands. You could write
up a description of these commands, how they map to the protocol, what parameters they take, and
what behaviour will result on your particular platform. But this information is already more or
less stored in your code, as the command implementation (and any docstrings or comments). Having
this information in two places creates an opportunity for the docs to get out of sync with the
reality of the code. Wouldn't it be nice to generate command reference documentation straight from
the code?
Another problem with the basic single file README approach is that many extensions might want
a whole set of documents including longer prose guides (like this one). It might be nice to have
code examples where you can toggle between different programming languages. It might be nice to be
able to add a project-specific logo. And so on.
Luckily, the Appium project has built tools to do all these things, and we've packaged up these
tools so our ecosystem developers building drivers and plugins can _also_ use them. The best way to
get going with these tools is probably to look at an existing Appium driver repo to see how it's
done, for example the [XCUITest driver repo](https://github.com/appium/appium-xcuitest-driver). But
this guide will outline the basic approach.
### Conceptual architecture
Appium settled on [MkDocs](https://www.mkdocs.org/) as a Markdown-based documentation site
generator. It uses a Python toolchain (and not Node.js), but it turned out to be the best option
for our purposes. You can adjust this, but by default Appium's utilities also assume that you'll be
using the [mkdocs-material](https://squidfunk.github.io/mkdocs-material/) theme/extension for
MkDocs.
From here, building a basic docs site is as easy as collecting your Markdown files together and
creating a sort of manifest file defining how you want them to be organized.
The other main piece is automatic documentation generation from your code files. Appium maintains
a plugin for [TypeDoc](https://typedoc.org/). This plugin is incorporated into our doc utility.
When you give it an entrypoint for you driver or plugin, it will scan and parse all your code files
looking for Appium command implementations. A set of Markdown reference files will be generated for these
commands, which can then be included in your docs site.
In order to make different versions of your docs available (one for each minor release of your
extension, typically), we also bundle [Mike](https://github.com/jimporter/mike).
### Setup
To take advantage of Appium's documentation utilities, you'll need to make sure of the following:
- Python 3 is available
- The appropriate Python dependencies are available (the versions used by Appium itself are listed
- in the doc utility's
[requirements.txt](https://github.com/appium/appium/blob/master/packages/docutils/requirements.txt)).
It's recommended you use the same versions.
- Include the `@appium/docutils` NPM package as a dev dependency of your project
- Annotate your commands using TypeDoc (this works for JS projects and not just TypeScript
projects).
- Put your markdown docs files in a directory
- Create a [`mkdocs.yml`](https://www.mkdocs.org/user-guide/configuration/) file that extends
Appium's base MkDocs configuration. (If your `mkdocs.yml` file is in the root of your repo, then
the line at the top of it should look like:
```yml
INHERIT: ./node_modules/@appium/docutils/base-mkdocs.yml
```
Basically, make sure you're inheriting the path to `@appium/docutils`'s `base-mkdocs.yml`.
### Usage
At this point, you can use the `appium-docs` CLI tool. Run this tool with no arguments to get the
full help output and see all the available subcommands and parameters. Here are a few usage
examples:
```bash
# Generate reference and build the mkdocs site into the site dir
appium-docs build
# Same as build, but host the docs on a local dev server
# and watch for changes and rebuild when files change
appium-docs build --serve
# Build the docs and deploy them with mike versioning to the docs-site branch
# using the included commit message and rebase strategy on the branch.
# This is particularly useful for pushing content to a GitHub pages branch!
appium-docs build \
--deploy \
-b docs-site \
-m 'docs: auto-build docs for appium-xcuitest-driver@%s' \
--rebase
```
@@ -97,3 +97,11 @@ are currently maintained by the Appium Team:
If you maintain an Appium plugin that you would like to be listed in the Appium docs, feel free
to make a PR to add it to this section with a link to the documentation for the plugin.
## Helper tools
The following tools might also be useful for Appium users:
|Name|Description|Supported By|
|---|---|---|
|[appium-installer](https://github.com/AppiumTestDistribution/appium-installer)|Help set up an Appium environment for Android and iOS|`@AppiumTestDistribution`|
@@ -80,7 +80,20 @@ APPIUM_SKIP_CHROMEDRIVER_INSTALL=1 appium driver install uiautomator2
With Appium 1.x, command-line options specific to particular drivers were all hosted on the main Appium server. So, for example, `--chromedriver-executable` was a CLI parameter you could use with Appium to set the location of a specific Chromedriver version for use with, say, the UiAutomator2 driver.
With Appium 2.x, all driver- and platform-specific CLI params have been moved to the drivers themselves. To access them, you'll now need to prepend the argument with the extension type (either `driver` or `plugin`) and the name of the extension. For example, `--chromedriver-executable` becomes `--driver-uiautomator2-chromedriver-executable`.
With Appium 2.x, all driver- and platform-specific CLI params have been moved to the drivers themselves. To access the corresponding functionality, you'll need to refer to the driver/plugin documentation. In some cases, the extension will continue to expose CLI parameters. For example, the XCUITest driver used to expose a parameter `--webdriveragent-port`. Now, to access this parameter, it should be prefixed with `driver-xcuitest`, to differentiate it from parameters other drivers might also expose. To use this parameter, you thus need to start Appium with something like:
```bash
appium --driver-xcuitest-webdriveragent-port=5000
```
Some drivers have done away with CLI args entirely in favour of default capabilities instead. With
the above-mentioned `--chromedriver-executable` param for example, you now need to take advantage
of the `appium:chromedriverExecutable` capability supported by the UiAutomator2 driver. To set this
capability from the command line, do the following:
```bash
appium --default-capabilities '{"appium:chromedriverExecutable": "/path/to/chromedriver"}'
```
### :bangbang: Driver-specific automation commands
@@ -231,10 +244,6 @@ any Appium command, or even adjust the way the underlying Appium HTTP server its
more about plugins, read the new [Appium Introduction](../intro/index.md). Interested in building
a plugin? Check out the [Building Plugins](../ecosystem/build-plugins.md) guide.
#### :tada: _Client Plugins_
TODO
### :tada: Install drivers and plugins from anywhere
You're no longer limited to the drivers that come with Appium, or that the Appium team even knows
+1 -2
View File
@@ -31,7 +31,6 @@ const remote = process.env.APPIUM_DOCS_REMOTE ?? DOCS_REMOTE;
const preview = Boolean(process.env.APPIUM_DOCS_PREVIEW);
const push = Boolean(process.env.APPIUM_DOCS_PUBLISH);
const rebase = push;
async function main() {
log.info(`Building Appium docs and committing to ${DOCS_BRANCH}`);
@@ -66,7 +65,7 @@ async function main() {
deployVersion: majMinVer,
push,
alias: LATEST_ALIAS,
rebase,
rebase: true, // we always want to rebase if branch is out of sync
message: `docs(appium): auto-build docs for appium@${majMinVer}, language ${lang}`,
});
}
+10 -10
View File
@@ -1,6 +1,6 @@
{
"name": "appium",
"version": "2.0.0-beta.66",
"version": "2.0.0-beta.70",
"description": "Automation for Apps.",
"keywords": [
"automation",
@@ -60,12 +60,12 @@
"test:unit": "mocha \"./test/unit/**/*.spec.js\""
},
"dependencies": {
"@appium/base-driver": "^9.3.7",
"@appium/base-plugin": "^2.2.7",
"@appium/docutils": "^0.3.9",
"@appium/base-driver": "^9.3.9",
"@appium/base-plugin": "^2.2.9",
"@appium/docutils": "^0.3.13",
"@appium/schema": "^0.2.6",
"@appium/support": "^3.1.11",
"@appium/types": "^0.11.0",
"@appium/support": "^4.0.1",
"@appium/types": "^0.11.1",
"@sidvind/better-ajv-errors": "2.1.0",
"@types/argparse": "2.0.10",
"@types/bluebird": "3.5.38",
@@ -78,7 +78,7 @@
"argparse": "2.0.1",
"async-lock": "1.4.0",
"asyncbox": "2.9.4",
"axios": "1.3.5",
"axios": "1.4.0",
"bluebird": "3.7.2",
"cross-env": "7.0.3",
"find-up": "5.0.0",
@@ -89,13 +89,13 @@
"ora": "5.4.1",
"package-changed": "3.0.0",
"resolve-from": "5.0.0",
"semver": "7.4.0",
"semver": "7.5.0",
"source-map-support": "0.5.21",
"teen_process": "2.0.2",
"type-fest": "3.8.0",
"type-fest": "3.10.0",
"winston": "3.8.2",
"wrap-ansi": "7.0.0",
"yaml": "2.2.1"
"yaml": "2.2.2"
},
"engines": {
"node": "^14.17.0 || ^16.13.0 || >=18.0.0",
@@ -1,5 +1,5 @@
{
"devDependencies": {
"webdriverio": "7.30.2"
"webdriverio": "7.31.1"
}
}
+21
View File
@@ -3,6 +3,27 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [9.3.9](https://github.com/appium/appium/compare/@appium/base-driver@9.3.8...@appium/base-driver@9.3.9) (2023-05-19)
**Note:** Version bump only for package @appium/base-driver
## [9.3.8](https://github.com/appium/appium/compare/@appium/base-driver@9.3.7...@appium/base-driver@9.3.8) (2023-05-17)
### Bug Fixes
* **base-driver:** Ignore unknown script arguments ([#18575](https://github.com/appium/appium/issues/18575)) ([4f564ef](https://github.com/appium/appium/commit/4f564ef7ad350831b90994ffd4e8b00de227577d))
* **support:** update dependency axios to v1.3.6 ([6692227](https://github.com/appium/appium/commit/66922279b7742a08613f472585a4a1cb70f80683))
* **support:** update dependency axios to v1.4.0 ([91a6bc5](https://github.com/appium/appium/commit/91a6bc5925ab8ffc4ab6d05883900f7d186e49a9))
* **types:** update dependency type-fest to v3.10.0 ([3c4d3ac](https://github.com/appium/appium/commit/3c4d3acc09d2ca1ed74dc77c18c62482e4c70239))
* **types:** update dependency type-fest to v3.9.0 ([94a207f](https://github.com/appium/appium/commit/94a207fc9718068f3657c51cc8be0ef682f16b11))
## [9.3.7](https://github.com/appium/appium/compare/@appium/base-driver@9.3.6...@appium/base-driver@9.3.7) (2023-04-14)
@@ -13,6 +13,7 @@ import B from 'bluebird';
import {formatResponseValue, formatStatus} from './helpers';
import {MAX_LOG_BODY_LENGTH, PROTOCOLS, DEFAULT_BASE_PATH} from '../constants';
import {isW3cCaps} from '../helpers/capabilities';
import log from '../basedriver/logger';
const CREATE_SESSION_COMMAND = 'createSession';
const DELETE_SESSION_COMMAND = 'deleteSession';
@@ -225,7 +226,7 @@ function validateExecuteMethodParams(params, paramSpec) {
`arguments to execute script and instead received: ${JSON.stringify(params)}`
);
}
const args = params[0] ?? {};
let args = params[0] ?? {};
if (!_.isPlainObject(args)) {
throw new errors.InvalidArgumentError(
`Did not receive an appropriate execute method parameters object. It needs to be ` +
@@ -237,6 +238,11 @@ function validateExecuteMethodParams(params, paramSpec) {
} else {
paramSpec.required ??= [];
paramSpec.optional ??= [];
const unknownNames = _.difference(_.keys(args), paramSpec.required, paramSpec.optional);
if (!_.isEmpty(unknownNames)) {
log.info(`The following script arguments are not known and will be ignored: ${unknownNames}`);
args = _.pickBy(args, (v, k) => !unknownNames.includes(k));
}
checkParams(paramSpec, args, null);
}
const argsToApply = makeArgs({}, args, paramSpec, null);
+6 -6
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/base-driver",
"version": "9.3.7",
"version": "9.3.9",
"description": "Base driver class for Appium drivers",
"keywords": [
"automation",
@@ -44,17 +44,17 @@
"test:types": "tsd"
},
"dependencies": {
"@appium/support": "^3.1.11",
"@appium/types": "^0.11.0",
"@appium/support": "^4.0.1",
"@appium/types": "^0.11.1",
"@colors/colors": "1.5.0",
"@types/async-lock": "1.4.0",
"@types/bluebird": "3.5.38",
"@types/express": "4.17.17",
"@types/method-override": "0.0.32",
"@types/serve-favicon": "2.5.3",
"@types/serve-favicon": "2.5.4",
"async-lock": "1.4.0",
"asyncbox": "2.9.4",
"axios": "1.3.5",
"axios": "1.4.0",
"bluebird": "3.7.2",
"body-parser": "1.20.2",
"es6-error": "4.1.1",
@@ -66,7 +66,7 @@
"morgan": "1.10.0",
"serve-favicon": "2.5.0",
"source-map-support": "0.5.21",
"type-fest": "3.8.0",
"type-fest": "3.10.0",
"validate.js": "0.13.1"
},
"engines": {
+16
View File
@@ -3,6 +3,22 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [2.2.9](https://github.com/appium/appium/compare/@appium/base-plugin@2.2.8...@appium/base-plugin@2.2.9) (2023-05-19)
**Note:** Version bump only for package @appium/base-plugin
## [2.2.8](https://github.com/appium/appium/compare/@appium/base-plugin@2.2.7...@appium/base-plugin@2.2.8) (2023-05-17)
**Note:** Version bump only for package @appium/base-plugin
## [2.2.7](https://github.com/appium/appium/compare/@appium/base-plugin@2.2.6...@appium/base-plugin@2.2.7) (2023-04-14)
**Note:** Version bump only for package @appium/base-plugin
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/base-plugin",
"version": "2.2.7",
"version": "2.2.9",
"description": "The base plugin used to create Appium 2.0 plugins",
"keywords": [
"automation",
@@ -40,8 +40,8 @@
"test:unit": "mocha \"./test/unit/**/*.spec.js\""
},
"dependencies": {
"@appium/base-driver": "^9.3.7",
"@appium/support": "^3.1.11"
"@appium/base-driver": "^9.3.9",
"@appium/support": "^4.0.1"
},
"engines": {
"node": "^14.17.0 || ^16.13.0 || >=18.0.0",
+18
View File
@@ -3,6 +3,24 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [2.0.16](https://github.com/appium/appium/compare/@appium/doctor@2.0.15...@appium/doctor@2.0.16) (2023-05-19)
**Note:** Version bump only for package @appium/doctor
## [2.0.15](https://github.com/appium/appium/compare/@appium/doctor@2.0.14...@appium/doctor@2.0.15) (2023-05-17)
### Bug Fixes
* **doctor:** update dependency appium-adb to v9.11.2 ([e7df6fd](https://github.com/appium/appium/commit/e7df6fd4f054d6ad6eb4125f77e7e0d58bc05ebf))
* **docutils:** update dependency yargs to v17.7.2 ([3d1e4af](https://github.com/appium/appium/commit/3d1e4af858ea21bcf3ab1e1015aa7bdf1776f3ce))
## [2.0.14](https://github.com/appium/appium/compare/@appium/doctor@2.0.13...@appium/doctor@2.0.14) (2023-04-20)
+4 -4
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/doctor",
"version": "2.0.14",
"version": "2.0.16",
"description": "Test environment for fitness to run Appium",
"keywords": [
"automation",
@@ -44,17 +44,17 @@
"test:unit": "mocha \"./test/unit/**/*.spec.js\""
},
"dependencies": {
"@appium/support": "^3.1.11",
"@appium/support": "^4.0.1",
"@colors/colors": "1.5.0",
"@types/bluebird": "3.5.38",
"@types/teen_process": "2.0.0",
"appium-adb": "9.11.1",
"appium-adb": "9.11.2",
"bluebird": "3.7.2",
"inquirer": "8.2.5",
"lodash": "4.17.21",
"source-map-support": "0.5.21",
"teen_process": "2.0.2",
"yargs": "17.7.1"
"yargs": "17.7.2"
},
"engines": {
"node": "^14.17.0 || ^16.13.0 || >=18.0.0",
+44
View File
@@ -3,6 +3,50 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.3.13](https://github.com/appium/appium/compare/@appium/docutils@0.3.12...@appium/docutils@0.3.13) (2023-05-19)
### Bug Fixes
* **docutils:** fix broken executable ([f0ebe2e](https://github.com/appium/appium/commit/f0ebe2e02f3e4164f07fd10ca36f460ff7a61682))
## [0.3.12](https://github.com/appium/appium/compare/@appium/docutils@0.3.11...@appium/docutils@0.3.12) (2023-05-19)
### Bug Fixes
* **docutils:** add @types/which as it is exported ([9538c34](https://github.com/appium/appium/commit/9538c34131cd2f8a3c45039efd65678bc0980497))
* **docutils:** exec mike/mkdocs via python -m ([426830d](https://github.com/appium/appium/commit/426830d14a348f163535a7f935a65b028008738d))
* **docutils:** provide CJS executable for proper linking ([db73971](https://github.com/appium/appium/commit/db73971018b560ca47527173094720eee3d20a55))
* **docutils:** run mkdocs via python -m ([5d24ea1](https://github.com/appium/appium/commit/5d24ea1ad294fa28f7ecce125e5342b081a84c83))
* **docutils:** try python3 if python not in PATH ([7b17754](https://github.com/appium/appium/commit/7b177547865a8343625cf089c97cebfb93fd6955))
## [0.3.11](https://github.com/appium/appium/compare/@appium/docutils@0.3.10...@appium/docutils@0.3.11) (2023-05-19)
**Note:** Version bump only for package @appium/docutils
## [0.3.10](https://github.com/appium/appium/compare/@appium/docutils@0.3.9...@appium/docutils@0.3.10) (2023-05-17)
### Bug Fixes
* **docutils:** remove -t arg for mkdocs which prevents theme customizations (fix [#18536](https://github.com/appium/appium/issues/18536)) ([#18617](https://github.com/appium/appium/issues/18617)) ([cdfe7c3](https://github.com/appium/appium/commit/cdfe7c3d82a61776a784236140dff94ff1f34540))
* **docutils:** update dependency yaml to v2.2.2 ([ddadd69](https://github.com/appium/appium/commit/ddadd6977d76a8a5ce1559d2a4ff4a31b58fb42f))
* **docutils:** update dependency yargs to v17.7.2 ([3d1e4af](https://github.com/appium/appium/commit/3d1e4af858ea21bcf3ab1e1015aa7bdf1776f3ce))
* **support:** update dependency semver to v7.5.0 ([c568523](https://github.com/appium/appium/commit/c568523e017a8b52bdf5132a48bbfde791de704f))
* **types:** update dependency type-fest to v3.10.0 ([3c4d3ac](https://github.com/appium/appium/commit/3c4d3acc09d2ca1ed74dc77c18c62482e4c70239))
* **types:** update dependency type-fest to v3.9.0 ([94a207f](https://github.com/appium/appium/commit/94a207fc9718068f3657c51cc8be0ef682f16b11))
## [0.3.9](https://github.com/appium/appium/compare/@appium/docutils@0.3.8...@appium/docutils@0.3.9) (2023-04-20)
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env node
'use strict';
const {main} = require('../build/lib/cli');
const {getLogger} = require('../build/lib/logger');
const log = getLogger('cli');
// eslint-disable-next-line promise/prefer-await-to-then, promise/prefer-await-to-callbacks
main().catch((err) => {
log.error('Caught otherwise-unhandled rejection (this is probably a bug):', err);
});
+28 -15
View File
@@ -13,10 +13,12 @@ import {
DEFAULT_SERVE_HOST,
DEFAULT_SERVE_PORT,
NAME_BIN,
NAME_MIKE,
NAME_MKDOCS_YML,
NAME_PYTHON,
} from '../constants';
import {DocutilsError} from '../error';
import {findMkDocsYml, readPackageJson, whichMike} from '../fs';
import {findMkDocsYml, findPython, readPackageJson} from '../fs';
import {getLogger} from '../logger';
import {argify, spawnBackgroundProcess, SpawnBackgroundProcessOpts, stopwatch} from '../util';
@@ -24,31 +26,33 @@ const log = getLogger('builder:deploy');
/**
* Runs `mike serve`
* @param pythonPath Path to Python 3 executable
* @param args Extra args to `mike build`
* @param opts Extra options for `teen_process.Subprocess.start`
* @param mikePath Path to `mike` executable
*/
async function doServe(
pythonPath: string,
args: string[] = [],
opts: SpawnBackgroundProcessOpts = {},
mikePath?: string
opts: SpawnBackgroundProcessOpts = {}
) {
mikePath = mikePath ?? (await whichMike());
const finalArgs = ['serve', ...args];
return spawnBackgroundProcess(mikePath, finalArgs, opts);
const finalArgs = ['-m,', NAME_MIKE, 'serve', ...args];
return spawnBackgroundProcess(pythonPath, finalArgs, opts);
}
/**
* Runs `mike build`
* @param pythonPath Path to Python 3 executable
* @param args Extra args to `mike build`
* @param opts Extra options to `teen_process.exec`
* @param mikePath Path to `mike` executable
*/
async function doDeploy(args: string[] = [], opts: TeenProcessExecOptions = {}, mikePath?: string) {
mikePath = mikePath ?? (await whichMike());
const finalArgs = ['deploy', ...args];
log.debug('Launching %s with args: %O', mikePath, finalArgs);
return await exec(mikePath, finalArgs, opts);
async function doDeploy(
pythonPath: string,
args: string[] = [],
opts: TeenProcessExecOptions = {}
) {
const finalArgs = ['-m', NAME_MIKE, 'deploy', ...args];
log.debug('Executing %s via: %s %O', NAME_MIKE, pythonPath, finalArgs);
return await exec(pythonPath, finalArgs, opts);
}
/**
@@ -90,6 +94,15 @@ export async function deploy({
execOpts,
}: DeployOpts = {}) {
const stop = stopwatch('deploy');
const pythonPath = await findPython();
if (!pythonPath) {
throw new DocutilsError(
`Could not find ${NAME_PYTHON}3/${NAME_PYTHON} executable in PATH; please install Python v3`
);
}
mkDocsYmlPath = mkDocsYmlPath ?? (await findMkDocsYml(cwd));
if (!mkDocsYmlPath) {
throw new DocutilsError(
@@ -122,7 +135,7 @@ export async function deploy({
}
stop(); // discard
// unsure about how SIGHUP is handled here
await doServe(mikeArgs, serveOpts);
await doServe(pythonPath, mikeArgs, serveOpts);
} else {
log.info('Deploying into branch %s', branch);
const mikeArgs = [
@@ -137,7 +150,7 @@ export async function deploy({
if (alias) {
mikeArgs.push(alias);
}
await doDeploy(mikeArgs, execOpts);
await doDeploy(pythonPath, mikeArgs, execOpts);
log.success('Finished deployment into branch %s (%dms)', branch, stop());
}
+2
View File
@@ -75,6 +75,7 @@ export function parseNav(nav: MkDocsYmlNav): ParsedNavData[] {
const queue: QueueItem[] = [{entries, keypath: ''}];
while (queue.length) {
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
const {entries, keypath} = queue.shift()!;
for (const [key, item] of entries) {
if (_.isString(item)) {
@@ -153,6 +154,7 @@ function getRootHeaderKeypath(headerItems: ParsedNavData[], header: string) {
// these are the parts of the keypath of the first item, which will contain the header string. it
// dosn't matter whether we pick the first one or any one; they will all contain the same root
// keypath by definition.
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
const rootHeaderKeypathParts = _.toPath(_.first(headerItems)!.keypath);
// this is the keypath up to the header string, inclusive.
+24 -23
View File
@@ -7,9 +7,9 @@
import path from 'node:path';
import {exec, TeenProcessExecOptions} from 'teen_process';
import {DEFAULT_SITE_DIR, NAME_BIN, NAME_MKDOCS_YML, NAME_THEME} from '../constants';
import {DEFAULT_SITE_DIR, NAME_BIN, NAME_MKDOCS, NAME_MKDOCS_YML, NAME_PYTHON} from '../constants';
import {DocutilsError} from '../error';
import {findMkDocsYml, readMkDocsYml, whichMkDocs} from '../fs';
import {findMkDocsYml, findPython, readMkDocsYml} from '../fs';
import {getLogger} from '../logger';
import {relative, spawnBackgroundProcess, SpawnBackgroundProcessOpts, stopwatch} from '../util';
@@ -17,37 +17,31 @@ const log = getLogger('mkdocs');
/**
* Runs `mkdocs serve`
* @param pythonPath Path to Python 3 executable
* @param args Extra args to `mkdocs build`
* @param opts Extra options for `teen_process.Subprocess.start`
* @param mkDocsPath Path to `mkdocs` executable
*/
async function doServe(
pythonPath: string,
args: string[] = [],
opts: SpawnBackgroundProcessOpts = {},
mkDocsPath?: string
opts: SpawnBackgroundProcessOpts = {}
) {
mkDocsPath = mkDocsPath ?? (await whichMkDocs());
const finalArgs = ['serve', ...args];
log.debug('Launching %s with args: %s', mkDocsPath, finalArgs);
return spawnBackgroundProcess(mkDocsPath, finalArgs, opts);
const finalArgs = ['-m', NAME_MKDOCS, 'serve', ...args];
log.debug('Executing %s via: %s, %O', NAME_MKDOCS, pythonPath, finalArgs);
return spawnBackgroundProcess(pythonPath, finalArgs, opts);
}
/**
* Runs `mkdocs build`
* @param pythonPath Path to Python 3 executable
* @param args Extra args to `mkdocs build`
* @param opts Extra options to `teen_process.exec`
* @param mkDocsPath Path to `mkdocs` executable
*/
async function doBuild(
args: string[] = [],
opts: TeenProcessExecOptions = {},
mkDocsPath?: string
) {
mkDocsPath = mkDocsPath ?? (await whichMkDocs());
const finalArgs = ['build', ...args];
log.debug('Launching %s with args: %O', mkDocsPath, finalArgs);
return await exec(mkDocsPath, finalArgs, opts);
async function doBuild(pythonPath: string, args: string[] = [], opts: TeenProcessExecOptions = {}) {
const finalArgs = ['-m', NAME_MKDOCS, 'build', ...args];
log.debug('Executing %s via: %s, %O', NAME_MKDOCS, pythonPath, finalArgs);
return await exec(pythonPath, finalArgs, opts);
}
/**
@@ -57,13 +51,20 @@ async function doBuild(
export async function buildSite({
mkdocsYml: mkDocsYmlPath,
siteDir,
theme = NAME_THEME,
cwd = process.cwd(),
serve = false,
serveOpts,
execOpts,
}: BuildMkDocsOpts = {}) {
const stop = stopwatch('build-mkdocs');
const pythonPath = await findPython();
if (!pythonPath) {
throw new DocutilsError(
`Could not find ${NAME_PYTHON}3/${NAME_PYTHON} executable in PATH; please install Python v3`
);
}
mkDocsYmlPath = mkDocsYmlPath
? path.resolve(process.cwd(), mkDocsYmlPath)
: await findMkDocsYml(cwd);
@@ -72,16 +73,16 @@ export async function buildSite({
`Could not find ${NAME_MKDOCS_YML} from ${cwd}; run "${NAME_BIN} init" to create it`
);
}
const mkdocsArgs = ['-f', mkDocsYmlPath, '-t', theme];
const mkdocsArgs = ['-f', mkDocsYmlPath];
if (siteDir) {
mkdocsArgs.push('-d', siteDir);
}
if (serve) {
// unsure about how SIGHUP is handled here
await doServe(mkdocsArgs, serveOpts);
await doServe(pythonPath, mkdocsArgs, serveOpts);
} else {
log.info('Building site...');
await doBuild(mkdocsArgs, execOpts);
await doBuild(pythonPath, mkdocsArgs, execOpts);
let relSiteDir;
if (siteDir) {
relSiteDir = relative(cwd, siteDir);
+10 -1
View File
@@ -50,7 +50,7 @@ const opts = {
},
'python-path': {
defaultDescription: '(derived from shell)',
description: 'Path to python 3 executable',
description: 'Path to python3 executable',
group: ValidateCommandGroup.Paths,
nargs: 1,
normalize: true,
@@ -81,6 +81,15 @@ const opts = {
requiresArg: true,
type: 'string',
},
'typedoc-path': {
defaultDescription: '(derived from shell)',
description: 'Path to typedoc executable',
group: ValidateCommandGroup.Paths,
nargs: 1,
normalize: true,
requiresArg: true,
type: 'string',
},
typescript: {
default: true,
description: 'Validate TypeScript environment',
+41 -20
View File
@@ -3,27 +3,26 @@
* @module
*/
import findUp from 'find-up';
import YAML from 'yaml';
import readPkg, {NormalizedPackageJson, PackageJson} from 'read-pkg';
import path from 'node:path';
import {JsonValue} from 'type-fest';
import {fs} from '@appium/support';
import findUp from 'find-up';
import * as JSON5 from 'json5';
import _ from 'lodash';
import path from 'node:path';
import _pkgDir from 'pkg-dir';
import {getLogger} from './logger';
import readPkg, {NormalizedPackageJson, PackageJson} from 'read-pkg';
import {JsonValue} from 'type-fest';
import {Application, TypeDocReader} from 'typedoc';
import YAML from 'yaml';
import {
NAME_TYPEDOC_JSON,
NAME_MKDOCS_YML,
NAME_PACKAGE_JSON,
NAME_MKDOCS,
NAME_NPM,
NAME_PACKAGE_JSON,
NAME_PYTHON,
NAME_MIKE,
NAME_TYPEDOC,
NAME_TYPEDOC_JSON,
} from './constants';
import {DocutilsError} from './error';
import {getLogger} from './logger';
import {MkDocsYml} from './model';
const log = getLogger('fs');
@@ -207,27 +206,49 @@ export function safeWriteFile(filepath: string, content: JsonValue, overwrite =
/**
* `which` with memoization
*/
export const cachedWhich = _.memoize(fs.which);
/**
* Finds `mkdocs` executable
*/
export const whichMkDocs = _.partial(cachedWhich, NAME_MKDOCS);
const cachedWhich = _.memoize(fs.which as typeof import('which'));
/**
* Finds `npm` executable
*/
export const whichNpm = _.partial(cachedWhich, NAME_NPM);
export const whichNpm = _.partial(cachedWhich, NAME_NPM, {nothrow: true});
/**
* Finds `python` executable
*/
export const whichPython = _.partial(cachedWhich, NAME_PYTHON);
const whichPython = _.partial(cachedWhich, NAME_PYTHON, {nothrow: true});
/**
* Finds `mike` executable
* Finds `python3` executable
*/
export const whichMike = _.partial(cachedWhich, NAME_MIKE);
const whichPython3 = _.partial(cachedWhich, `${NAME_PYTHON}3`, {nothrow: true});
/**
* Finds `typedoc` executable
*/
const whichTypeDoc = _.partial(cachedWhich, NAME_TYPEDOC, {nothrow: true});
/**
* Finds the `typedoc` executable.
*
* Looks in the `node_modules/.bin` dir from the current working directory, and if this fails, in the `node_modules` dir for the file which `.bin/typedoc` should be a symlink of, and if _that_ fails, try the `PATH`.
*/
export const findTypeDoc = _.memoize(async (cwd = process.cwd()): Promise<string | undefined> => {
// .cmd is for win32, of course. note that glob _always_ uses posix dir separators.
const globResult =
(await fs.glob('node_modules/.bin/typedoc?(.cmd)', {cwd, nodir: true})) ??
(await fs.glob('node_modules/**/typedoc/bin/typedoc', {cwd, nodir: true, follow: true}));
return _.first(globResult) ?? (await whichTypeDoc());
});
/**
* Finds the `python3` or `python` executable in the user's `PATH`.
*
* `python3` is preferred over `python`, since the latter could be Python 2.
*/
export const findPython = _.memoize(
async (): Promise<string | undefined> => (await whichPython3()) ?? (await whichPython())
);
/**
* Reads an `mkdocs.yml` file, merges inherited configs, and returns the result. The result is cached.
+4 -2
View File
@@ -20,7 +20,7 @@ import {createScaffoldTask, ScaffoldTaskOptions} from './scaffold';
import {getLogger} from './logger';
import {MkDocsYml, TsConfigJson, TypeDocJson} from './model';
import _ from 'lodash';
import {stringifyJson5, stringifyYaml} from './fs';
import {findPython, stringifyJson5, stringifyYaml} from './fs';
/**
* Data for the base `mkdocs.yml` file
@@ -158,10 +158,12 @@ export const initMkDocs = createScaffoldTask<InitMkDocsOptions, MkDocsYml>(
* @param opts Options
*/
export async function initPython({
pythonPath = NAME_PYTHON,
pythonPath,
dryRun = false,
upgrade = false,
}: InitPythonOptions = {}): Promise<void> {
pythonPath = pythonPath ?? (await findPython()) ?? NAME_PYTHON;
const args = ['-m', 'pip', 'install', '-r', REQUIREMENTS_TXT_PATH];
if (upgrade) {
args.push('--upgrade');
+9 -10
View File
@@ -6,20 +6,19 @@
* @module
*/
import figures from 'figures';
import logSymbols from 'log-symbols';
import chalk, {ForegroundColor, BackgroundColor} from 'chalk';
import chalk, {type BackgroundColor, type ForegroundColor} from 'chalk';
import consola, {
logType as LogType,
ConsolaReporterLogObject,
type Consola,
type ConsolaReporterLogObject,
FancyReporter,
FancyReporterOptions,
Consola,
ConsolaOptions,
LogLevel,
type FancyReporterOptions,
type LogLevel,
type logType as LogType,
} from 'consola';
import {DEFAULT_LOG_LEVEL, LogLevelMap} from './constants';
import figures from 'figures';
import _ from 'lodash';
import logSymbols from 'log-symbols';
import {DEFAULT_LOG_LEVEL, LogLevelMap} from './constants';
/**
* This is a reporter for `consola` which uses some extra/custom icons and colors.
+5 -7
View File
@@ -6,7 +6,6 @@
import _ from 'lodash';
import {SpawnOptions, spawn} from 'node:child_process';
import path from 'node:path';
import type {SubProcess} from 'teen_process';
/**
* Computes a relative path, prepending `./`
@@ -26,7 +25,7 @@ export function stopwatch(id: string) {
const start = Date.now();
stopwatch.cache.set(id, start);
return () => {
const result = Date.now() - stopwatch.cache.get(id)!;
const result = Date.now() - (stopwatch.cache.get(id) ?? 0);
stopwatch.cache.delete(id);
return result;
};
@@ -59,17 +58,16 @@ export const isStringArray = _.overEvery(_.isArray, _.partial(_.every, _, _.isSt
export const argify: (obj: Record<string, string | number | boolean | undefined>) => string[] =
_.flow(
_.entries,
_.flatten,
(list) =>
list.map((value, idx) => {
list.map(([key, value]) => {
if (value === true) {
return `--${value}`;
return [`--${key}`];
} else if (value === false || value === undefined) {
return;
}
return idx % 2 === 0 ? `--${value}` : value;
return [`--${key}`, value];
}),
_.compact
_.flatten
);
/**
+49 -30
View File
@@ -31,14 +31,14 @@ import {
} from './constants';
import {DocutilsError} from './error';
import {
findPkgDir,
findMkDocsYml,
findPkgDir,
findTypeDoc,
readJson5,
readTypedocJson,
whichMkDocs,
whichNpm,
whichPython,
readMkDocsYml,
readTypedocJson,
whichNpm,
findPython,
} from './fs';
import {getLogger} from './logger';
import {MkDocsYml, PipPackage, TypeDocJson} from './model';
@@ -62,7 +62,7 @@ const TYPEDOC_VERSION_REGEX = /TypeDoc\s(\d+\.\d+\..+)/;
/**
* Matches the MkDocs version string from `mkdocs --version`
*/
const MKDOCS_VERSION_REGEX = /mkdocs,\s+version\s+(\d+\.\d+\.\S+)/;
const MKDOCS_VERSION_REGEX = /\s+version\s+(\d+\.\d+\.\S+)/;
const log = getLogger('validate');
@@ -95,12 +95,12 @@ export class DocutilsValidator extends EventEmitter {
/**
* Path to `npm` executable.
*/
protected readonly npmPath: string | undefined;
protected readonly npmPath?: string;
/**
* Path to `python` executable.
*/
protected readonly pythonPath: string | undefined;
protected readonly pythonPath?: string;
/**
* List of validations to perform
@@ -120,27 +120,29 @@ export class DocutilsValidator extends EventEmitter {
/**
* Path to `mkdocs.yml`. If not provided, will be lazily resolved.
*/
protected mkDocsYmlPath: string | undefined;
protected mkDocsYmlPath?: string;
/**
* Path to `package.json`. If not provided, will be lazily resolved.
*/
protected packageJsonPath: string | undefined;
protected packageJsonPath?: string;
/**
* Path to the package directory. If not provided, will be lazily resolved.
*/
protected pkgDir: string | undefined;
protected pkgDir?: string;
/**
* Path to `tsconfig.json`. If not provided, will be lazily resolved.
*/
protected tsconfigJsonPath: string | undefined;
protected tsconfigJsonPath?: string;
/**
* Path to `typedoc.json`. If not provided, will be lazily resolved.
*/
protected typeDocJsonPath: string | undefined;
protected typeDocJsonPath?: string;
protected typeDocPath?: string;
/**
* Emitted when validation begins with a list of validation kinds to be performed
@@ -181,6 +183,7 @@ export class DocutilsValidator extends EventEmitter {
this.typeDocJsonPath = opts.typedocJson;
this.npmPath = opts.npm;
this.mkDocsYmlPath = opts.mkdocsYml;
this.typeDocPath = opts.typedocPath;
if (opts.python) {
this.validations.add(NAME_PYTHON);
@@ -314,21 +317,19 @@ export class DocutilsValidator extends EventEmitter {
* Validates that the correct version of `mkdocs` is installed
*/
protected async validateMkDocs() {
let mkDocsPath: string | undefined;
try {
mkDocsPath = await whichMkDocs();
} catch {
// _pretty sure_ the exception code is always ENOENT
const pythonPath = this.pythonPath ?? (await findPython());
if (!pythonPath) {
return this.fail(
`Could not find ${NAME_MKDOCS} executable in PATH. If it is installed, check your PATH environment variable.`
`Could not find ${NAME_PYTHON} executable in PATH. If it is installed, check your PATH environment variable.`
);
}
let rawMkDocsVersion: string | undefined;
try {
({stdout: rawMkDocsVersion} = await exec(mkDocsPath, ['--version']));
({stdout: rawMkDocsVersion} = await exec(pythonPath, ['-m', NAME_MKDOCS, '--version']));
} catch (err) {
return this.fail(`${mkDocsPath} --version failed: ${err}`);
return this.fail(`Failed to get MkDocs version: ${err}`);
}
const match = rawMkDocsVersion.match(MKDOCS_VERSION_REGEX);
if (match) {
@@ -343,12 +344,12 @@ export class DocutilsValidator extends EventEmitter {
const {version: mkDocsReqdVersion} = mkDocsPipPkg;
if (version !== mkDocsReqdVersion) {
return this.fail(
`${NAME_MKDOCS} at ${mkDocsPath} is v${version}, but ${REQUIREMENTS_TXT_PATH} requires v${mkDocsReqdVersion}`
`${NAME_MKDOCS} is v${version}, but ${REQUIREMENTS_TXT_PATH} requires v${mkDocsReqdVersion}`
);
}
} else {
throw new DocutilsError(
`Could not parse version from "${mkDocsPath} --version". This is a bug. Output was ${rawMkDocsVersion}`
`Could not parse version from MkDocs. This is a bug. Output was ${rawMkDocsVersion}`
);
}
@@ -423,7 +424,7 @@ export class DocutilsValidator extends EventEmitter {
*/
protected async validatePythonDeps() {
let pipListOutput: string;
const pythonPath = this.pythonPath ?? (await whichPython());
const pythonPath = this.pythonPath ?? (await findPython());
if (!pythonPath) {
return this.fail(`Could not find ${NAME_PYTHON} in PATH. Is it installed?`);
}
@@ -499,7 +500,7 @@ export class DocutilsValidator extends EventEmitter {
* Asserts that the Python version is 3.x
*/
protected async validatePythonVersion() {
const pythonPath = this.pythonPath ?? (await whichPython());
const pythonPath = this.pythonPath ?? (await findPython());
if (!pythonPath) {
return this.fail(`Could not find ${NAME_PYTHON} in PATH. Is it installed?`);
}
@@ -520,17 +521,30 @@ export class DocutilsValidator extends EventEmitter {
/**
* Asserts TypeDoc is installed, runnable, the correct version, and that the config file is readable
* and constaints required options
*
* @todo Another option would be to `npm exec typedoc@<version>` which delegates to `npx`.
*/
protected async validateTypeDoc() {
const pkgDir = await this.findPkgDir();
const typeDocPath = this.typeDocPath ?? (await findTypeDoc(pkgDir));
if (!typeDocPath) {
return this.fail(`Could not find ${NAME_TYPEDOC}; is it installed?`);
}
log.debug('Found %s at %s', NAME_TYPEDOC, typeDocPath);
let rawTypeDocVersion: string;
let typeDocVersion: string;
try {
({stdout: rawTypeDocVersion} = await exec('npm', ['exec', NAME_TYPEDOC, '--', '--version'], {
({stdout: rawTypeDocVersion} = await exec(process.execPath, [typeDocPath, '--version'], {
cwd: pkgDir,
}));
} catch {
return this.fail(`Could not find ${NAME_TYPEDOC} executable from ${pkgDir}`);
} catch (err) {
return this.fail(
`Could not execute ${process.execPath} ${typeDocPath} from ${pkgDir}. Reason: ${
(err as Error).message
}`
);
}
if (rawTypeDocVersion) {
@@ -543,7 +557,8 @@ export class DocutilsValidator extends EventEmitter {
);
}
const reqdTypeDocVersion = DOCUTILS_PKG.dependencies!.typedoc!;
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
const reqdTypeDocVersion = DOCUTILS_PKG.dependencies!.typedoc!; // this is our own package.json
if (!satisfies(typeDocVersion, reqdTypeDocVersion)) {
return this.fail(
`Found TypeDoc version ${typeDocVersion}, but ${reqdTypeDocVersion} is required`
@@ -596,7 +611,7 @@ export class DocutilsValidator extends EventEmitter {
if (!typeDocJson.out) {
return this.fail(
new DocutilsError(
`Missing "out" property in ${relTypeDocJsonPath}; path "${DEFAULT_REL_TYPEDOC_OUT_PATH} is recommended`
`Missing "out" property in ${relTypeDocJsonPath}; path "${DEFAULT_REL_TYPEDOC_OUT_PATH}" is recommended`
)
);
}
@@ -714,6 +729,10 @@ export interface DocutilsValidatorOpts {
* If `true`, run TypeDoc validation
*/
typedoc?: boolean;
/**
* Path to `typedoc` executable
*/
typedocPath?: string;
/**
* Path to `typedoc.json`
*/
+9 -8
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/docutils",
"version": "0.3.9",
"version": "0.3.13",
"description": "Documentation generation utilities for Appium and related projects",
"keywords": [
"automation",
@@ -28,7 +28,7 @@
"lib": "lib"
},
"bin": {
"appium-docs": "./build/lib/cli/index.js"
"appium-docs": "./bin/appium-docs.js"
},
"files": [
"index.js",
@@ -49,10 +49,11 @@
"start": "node ./build/lib/cli/index.js"
},
"dependencies": {
"@appium/support": "^3.1.11",
"@appium/support": "^4.0.1",
"@appium/tsconfig": "^0.3.0",
"@appium/typedoc-plugin-appium": "^0.6.3",
"@appium/typedoc-plugin-appium": "^0.6.4",
"@sliphua/lilconfig-ts-loader": "3.2.2",
"@types/which": "3.0.0",
"chalk": "4.1.2",
"consola": "2.15.3",
"diff": "5.1.0",
@@ -64,16 +65,16 @@
"log-symbols": "4.1.0",
"pkg-dir": "5.0.0",
"read-pkg": "5.2.0",
"semver": "7.4.0",
"semver": "7.5.0",
"source-map-support": "0.5.21",
"teen_process": "2.0.2",
"type-fest": "3.8.0",
"type-fest": "3.10.0",
"typedoc": "0.23.28",
"typedoc-plugin-markdown": "3.14.0",
"typedoc-plugin-resolve-crossmodule-references": "0.3.3",
"typescript": "4.9.5",
"yaml": "2.2.1",
"yargs": "17.7.1",
"yaml": "2.2.2",
"yargs": "17.7.2",
"yargs-parser": "21.1.1"
},
"engines": {
+2 -2
View File
@@ -1,4 +1,4 @@
mkdocs==1.4.2
mkdocs-material==9.1.6
mkdocs==1.4.3
mkdocs-material==9.1.9
mkdocs-material-extensions==1.1.1
mike==1.1.2
+44
View File
@@ -0,0 +1,44 @@
import {argify} from '../../lib/util';
import _ from 'lodash';
describe('argify', function () {
it('should create args from params', function () {
// deploy example
const version = '2.0';
const mikeOpts = {
'config-file': '/path/to/yml',
push: true,
remote: 'origin',
branch: 'gh-pages',
prefix: '2.0',
message: 'docs: a thing',
rebase: true,
port: 8100,
host: 'localhost',
};
const mikeArgs = [
...argify(
_.omitBy(
mikeOpts,
(value, key) => _.includes(['port', 'host'], key) || (!_.isNumber(value) && !value)
)
),
version,
];
mikeArgs.should.eql([
'--config-file',
'/path/to/yml',
'--push',
'--remote',
'origin',
'--branch',
'gh-pages',
'--prefix',
'2.0',
'--message',
'docs: a thing',
'--rebase',
'2.0',
]);
});
});
+13
View File
@@ -3,6 +3,19 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.4.6](https://github.com/appium/appium/compare/@appium/driver-test-support@0.4.5...@appium/driver-test-support@0.4.6) (2023-05-17)
### Bug Fixes
* **support:** update dependency axios to v1.3.6 ([6692227](https://github.com/appium/appium/commit/66922279b7742a08613f472585a4a1cb70f80683))
* **support:** update dependency axios to v1.4.0 ([91a6bc5](https://github.com/appium/appium/commit/91a6bc5925ab8ffc4ab6d05883900f7d186e49a9))
* **test-support:** update dependency sinon to v15.0.4 ([9ad7279](https://github.com/appium/appium/commit/9ad7279fa70d7198a8d02f0f206397cb4a830e1d))
* **types:** update dependency type-fest to v3.10.0 ([3c4d3ac](https://github.com/appium/appium/commit/3c4d3acc09d2ca1ed74dc77c18c62482e4c70239))
* **types:** update dependency type-fest to v3.9.0 ([94a207f](https://github.com/appium/appium/commit/94a207fc9718068f3657c51cc8be0ef682f16b11))
## [0.4.5](https://github.com/appium/appium/compare/@appium/driver-test-support@0.4.4...@appium/driver-test-support@0.4.5) (2023-04-14)
+5 -5
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/driver-test-support",
"version": "0.4.5",
"version": "0.4.6",
"description": "Test utilities for Appium drivers",
"keywords": [
"automation",
@@ -41,18 +41,18 @@
},
"types": "./build/lib/index.d.ts",
"dependencies": {
"@appium/types": "^0.11.0",
"@appium/types": "^0.11.1",
"@types/lodash": "4.14.194",
"@types/stoppable": "1.1.1",
"axios": "1.3.5",
"axios": "1.4.0",
"bluebird": "3.7.2",
"chai": "4.3.7",
"get-port": "5.1.1",
"lodash": "4.17.21",
"sinon": "15.0.3",
"sinon": "15.0.4",
"source-map-support": "0.5.21",
"stoppable": "1.1.0",
"type-fest": "3.8.0"
"type-fest": "3.10.0"
},
"peerDependencies": {
"appium": "^2.0.0-beta.43",
@@ -3,6 +3,16 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [3.0.13](https://github.com/appium/appium/compare/@appium/execute-driver-plugin@3.0.12...@appium/execute-driver-plugin@3.0.13) (2023-05-17)
### Bug Fixes
* **execute-driver-plugin:** update dependency vm2 to v3.9.17 ([61ca898](https://github.com/appium/appium/commit/61ca898c4730a821ea4b3b1e9c6a70a03d4787da))
* **execute-driver-plugin:** update dependency webdriverio to v7.31.1 ([6499eea](https://github.com/appium/appium/commit/6499eea0af6aea1bfb69b1e36c4c445e3ab05b82))
## [3.0.12](https://github.com/appium/appium/compare/@appium/execute-driver-plugin@3.0.11...@appium/execute-driver-plugin@3.0.12) (2023-04-12)
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/execute-driver-plugin",
"version": "3.0.12",
"version": "3.0.13",
"description": "Plugin for batching and executing driver commands with Appiums",
"keywords": [
"automation",
@@ -42,8 +42,8 @@
"bluebird": "3.7.2",
"lodash": "4.17.21",
"source-map-support": "0.5.21",
"vm2": "3.9.16",
"webdriverio": "7.30.2"
"vm2": "3.9.17",
"webdriverio": "7.31.1"
},
"peerDependencies": {
"appium": "^2.0.0-beta.35"
+16
View File
@@ -3,6 +3,22 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [3.1.13](https://github.com/appium/appium/compare/@appium/fake-plugin@3.1.12...@appium/fake-plugin@3.1.13) (2023-05-19)
**Note:** Version bump only for package @appium/fake-plugin
## [3.1.12](https://github.com/appium/appium/compare/@appium/fake-plugin@3.1.11...@appium/fake-plugin@3.1.12) (2023-05-17)
**Note:** Version bump only for package @appium/fake-plugin
## [3.1.11](https://github.com/appium/appium/compare/@appium/fake-plugin@3.1.10...@appium/fake-plugin@3.1.11) (2023-04-14)
**Note:** Version bump only for package @appium/fake-plugin
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/fake-plugin",
"version": "3.1.11",
"version": "3.1.13",
"description": "A fake Appium 2.0 plugin",
"keywords": [
"automation",
@@ -39,8 +39,8 @@
"test:unit": "mocha \"./test/unit/**/*.spec.js\""
},
"dependencies": {
"@appium/base-plugin": "^2.2.7",
"@appium/support": "^3.1.11",
"@appium/base-plugin": "^2.2.9",
"@appium/support": "^4.0.1",
"@types/bluebird": "3.5.38",
"bluebird": "3.7.2",
"lodash": "4.17.21",
+23
View File
@@ -3,6 +3,29 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [2.1.1](https://github.com/appium/appium/compare/@appium/images-plugin@2.1.0...@appium/images-plugin@2.1.1) (2023-05-19)
### Bug Fixes
* **images-plugin:** Add the missing dependency to support ([#18652](https://github.com/appium/appium/issues/18652)) ([f512e59](https://github.com/appium/appium/commit/f512e596eb54edb5ca8993b3451aa4a22184bc57))
## [2.1.0](https://github.com/appium/appium/compare/@appium/images-plugin@2.0.10...@appium/images-plugin@2.1.0) (2023-05-17)
### Features
* **image-plugin:** Make it possible to find elements inside of other elements ([#18555](https://github.com/appium/appium/issues/18555)) ([2796639](https://github.com/appium/appium/commit/2796639b1183f0efbae8d7adb6d24570d5d4fde4))
### Bug Fixes
* **opencv:** update dependency sharp to v0.32.1 ([add0eaa](https://github.com/appium/appium/commit/add0eaa05f476a5f2708925a43f2bc16cad75872))
## [2.0.10](https://github.com/appium/appium/compare/@appium/images-plugin@2.0.9...@appium/images-plugin@2.0.10) (2023-04-05)
**Note:** Version bump only for package @appium/images-plugin
+6 -17
View File
@@ -1,12 +1,7 @@
import _ from 'lodash';
import {errors} from 'appium/driver';
import {getImagesMatches, getImagesSimilarity, getImageOccurrence} from '@appium/opencv';
const MATCH_FEATURES_MODE = 'matchFeatures';
const GET_SIMILARITY_MODE = 'getSimilarity';
const MATCH_TEMPLATE_MODE = 'matchTemplate';
const DEFAULT_MATCH_THRESHOLD = 0.4;
import {MATCH_FEATURES_MODE, GET_SIMILARITY_MODE, MATCH_TEMPLATE_MODE} from './constants';
/**
* Performs images comparison using OpenCV framework features.
@@ -15,9 +10,9 @@ const DEFAULT_MATCH_THRESHOLD = 0.4;
*
* @param {string} mode - One of possible comparison modes:
* matchFeatures, getSimilarity, matchTemplate
* @param {string} firstImage - Base64-encoded image file.
* @param {string|Buffer} firstImage - Base64-encoded image file.
* All image formats, that OpenCV library itself accepts, are supported.
* @param {string} secondImage - Base64-encoded image file.
* @param {string|Buffer} secondImage - Base64-encoded image file.
* All image formats, that OpenCV library itself accepts, are supported.
* @param {object} [options] - The content of this dictionary depends
* on the actual `mode` value. See the documentation on `@appium/support`
@@ -34,8 +29,8 @@ const DEFAULT_MATCH_THRESHOLD = 0.4;
* @type {import('@appium/types').PluginCommand<[string, string, string, object|undefined], Promise<OccurrenceResult>>}
*/
async function compareImages(mode, firstImage, secondImage, options = {}) {
const img1 = Buffer.from(firstImage, 'base64');
const img2 = Buffer.from(secondImage, 'base64');
const img1 = Buffer.isBuffer(firstImage) ? firstImage : Buffer.from(firstImage, 'base64');
const img2 = Buffer.isBuffer(secondImage) ? secondImage : Buffer.from(secondImage, 'base64');
let result = null;
switch (_.toLower(mode)) {
case MATCH_FEATURES_MODE.toLowerCase():
@@ -84,13 +79,7 @@ function convertVisualizationToBase64(element) {
return element;
}
export {
compareImages,
DEFAULT_MATCH_THRESHOLD,
MATCH_TEMPLATE_MODE,
MATCH_FEATURES_MODE,
GET_SIMILARITY_MODE,
};
export {compareImages};
/**
* @typedef {import('@appium/opencv').OccurrenceResult} OccurrenceResult
+70
View File
@@ -0,0 +1,70 @@
import {node} from 'appium/support';
export const IMAGE_STRATEGY = '-image';
export const IMAGE_ELEMENT_PREFIX = 'appium-image-element-';
export const IMAGE_EL_TAP_STRATEGY_W3C = 'w3cActions';
export const IMAGE_EL_TAP_STRATEGY_MJSONWP = 'touchActions';
export const IMAGE_TAP_STRATEGIES = [IMAGE_EL_TAP_STRATEGY_MJSONWP, IMAGE_EL_TAP_STRATEGY_W3C];
export const DEFAULT_TEMPLATE_IMAGE_SCALE = 1.0;
export const MATCH_FEATURES_MODE = 'matchFeatures';
export const GET_SIMILARITY_MODE = 'getSimilarity';
export const MATCH_TEMPLATE_MODE = 'matchTemplate';
export const DEFAULT_MATCH_THRESHOLD = 0.4;
export const DEFAULT_FIX_IMAGE_TEMPLATE_SCALE = 1;
export const DEFAULT_SETTINGS = node.deepFreeze({
// value between 0 and 1 representing match strength, below which an image
// element will not be found
imageMatchThreshold: DEFAULT_MATCH_THRESHOLD,
// One of possible image matching methods.
// Read https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_template_matching/py_template_matching.html
// for more details.
// TM_CCOEFF_NORMED by default
imageMatchMethod: '',
// if the image returned by getScreenshot differs in size or aspect ratio
// from the screen, attempt to fix it automatically
fixImageFindScreenshotDims: true,
// whether Appium should ensure that an image template sent in during image
// element find should have its size adjusted so the match algorithm will not
// complain
fixImageTemplateSize: false,
// whether Appium should ensure that an image template sent in during image
// element find should have its scale adjusted to display size so the match
// algorithm will not complain.
// e.g. iOS has `width=375, height=667` window rect, but its screenshot is
// `width=750 × height=1334` pixels. This setting help to adjust the scale
// if a user use `width=750 × height=1334` pixels's base template image.
fixImageTemplateScale: false,
// Users might have scaled template image to reduce their storage size.
// This setting allows users to scale a template image they send to Appium server
// so that the Appium server compares the actual scale users originally had.
// e.g. If a user has an image of 270 x 32 pixels which was originally 1080 x 126 pixels,
// the user can set {defaultImageTemplateScale: 4.0} to scale the small image
// to the original one so that Appium can compare it as the original one.
defaultImageTemplateScale: DEFAULT_TEMPLATE_IMAGE_SCALE,
// whether Appium should re-check that an image element can be matched
// against the current screenshot before clicking it
checkForImageElementStaleness: true,
// whether before clicking on an image element Appium should re-determine the
// position of the element on screen
autoUpdateImageElementPosition: false,
// which method to use for tapping by coordinate for image elements. the
// options are 'w3c' or 'mjsonwp'
imageElementTapStrategy: IMAGE_EL_TAP_STRATEGY_W3C,
// which method to use to save the matched image area in ImageElement class.
// It is used for debugging purpose.
getMatchedImageResult: false,
});
+183 -192
View File
@@ -1,113 +1,72 @@
import _ from 'lodash';
import LRU from 'lru-cache';
import {errors} from 'appium/driver';
import {util, imageUtil} from 'appium/support';
import {
ImageElement,
DEFAULT_TEMPLATE_IMAGE_SCALE,
IMAGE_EL_TAP_STRATEGY_W3C,
} from './image-element';
import {MATCH_TEMPLATE_MODE, compareImages, DEFAULT_MATCH_THRESHOLD} from './compare';
import {ImageElement} from './image-element';
import {compareImages} from './compare';
import log from './logger';
import {
DEFAULT_SETTINGS, MATCH_TEMPLATE_MODE, DEFAULT_TEMPLATE_IMAGE_SCALE,
DEFAULT_FIX_IMAGE_TEMPLATE_SCALE,
} from './constants';
import sharp from 'sharp';
const MJSONWP_ELEMENT_KEY = 'ELEMENT';
const W3C_ELEMENT_KEY = util.W3C_WEB_ELEMENT_IDENTIFIER;
const DEFAULT_FIX_IMAGE_TEMPLATE_SCALE = 1;
// Used to compare ratio and screen width
// Pixel is basically under 1080 for example. 100K is probably enough fo a while.
const FLOAT_PRECISION = 100000;
const MAX_CACHE_ITEMS = 100;
const MAX_CACHE_SIZE_BYTES = 1024 * 1024 * 40; // 40mb
const MAX_CACHE_AGE_MS = 24 * 60 * 60 * 1000;
const DEFAULT_SETTINGS = {
// value between 0 and 1 representing match strength, below which an image
// element will not be found
imageMatchThreshold: DEFAULT_MATCH_THRESHOLD,
/**
* Checks if one rect fully contains another
*
* @param {import('@appium/types').Rect} templateRect The bounding rect
* @param {import('@appium/types').Rect} rect The rect to be checked for containment
* @returns {boolean} True if templateRect contains rect
*/
function containsRect(templateRect, rect) {
return templateRect.x <= rect.x && templateRect.y <= rect.y
&& rect.width <= templateRect.x + templateRect.width - rect.x
&& rect.height <= templateRect.y + templateRect.height - rect.y;
}
// One of possible image matching methods.
// Read https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_template_matching/py_template_matching.html
// for more details.
// TM_CCOEFF_NORMED by default
imageMatchMethod: '',
// if the image returned by getScreenshot differs in size or aspect ratio
// from the screen, attempt to fix it automatically
fixImageFindScreenshotDims: true,
// whether Appium should ensure that an image template sent in during image
// element find should have its size adjusted so the match algorithm will not
// complain
fixImageTemplateSize: false,
// whether Appium should ensure that an image template sent in during image
// element find should have its scale adjusted to display size so the match
// algorithm will not complain.
// e.g. iOS has `width=375, height=667` window rect, but its screenshot is
// `width=750 × height=1334` pixels. This setting help to adjust the scale
// if a user use `width=750 × height=1334` pixels's base template image.
fixImageTemplateScale: false,
// Users might have scaled template image to reduce their storage size.
// This setting allows users to scale a template image they send to Appium server
// so that the Appium server compares the actual scale users originally had.
// e.g. If a user has an image of 270 x 32 pixels which was originally 1080 x 126 pixels,
// the user can set {defaultImageTemplateScale: 4.0} to scale the small image
// to the original one so that Appium can compare it as the original one.
defaultImageTemplateScale: DEFAULT_TEMPLATE_IMAGE_SCALE,
// whether Appium should re-check that an image element can be matched
// against the current screenshot before clicking it
checkForImageElementStaleness: true,
// whether before clicking on an image element Appium should re-determine the
// position of the element on screen
autoUpdateImageElementPosition: false,
// which method to use for tapping by coordinate for image elements. the
// options are 'w3c' or 'mjsonwp'
imageElementTapStrategy: IMAGE_EL_TAP_STRATEGY_W3C,
// which method to use to save the matched image area in ImageElement class.
// It is used for debugging purpose.
getMatchedImageResult: false,
};
const NO_OCCURRENCES_PATTERN = /Cannot find any occurrences/;
const CONDITION_UNMET_PATTERN = /Condition unmet/;
export default class ImageElementFinder {
/** @type {ExternalDriver} */
driver;
/** @type {LRU<string,ImageElement>} */
imgElCache;
_imgElCache;
/**
*
* @param {ExternalDriver} driver
* @param {number} [maxSize]
* @param {number} max
*/
constructor(driver, maxSize = MAX_CACHE_SIZE_BYTES) {
this.driver = driver;
this.imgElCache = new LRU({
max: MAX_CACHE_ITEMS,
maxSize,
sizeCalculation: (el) => el.template.length,
constructor(max = MAX_CACHE_ITEMS) {
this._imgElCache = new LRU({
ttl: MAX_CACHE_AGE_MS,
updateAgeOnGet: true,
max,
});
}
setDriver(driver) {
this.driver = driver;
}
/**
* @param {ImageElement} imgEl
* @returns {Element}
*/
registerImageElement(imgEl) {
this.imgElCache.set(imgEl.id, imgEl);
const protoKey = this.driver.isW3CProtocol() ? W3C_ELEMENT_KEY : MJSONWP_ELEMENT_KEY;
return imgEl.asElement(protoKey);
this._imgElCache.set(imgEl.id, imgEl);
return imgEl.asElement();
}
/**
* @param {string} imgElId
* @returns {ImageElement|undefined}
*/
getImageElement(imgElId) {
return this._imgElCache.get(imgElId);
}
clearImageElements() {
this._imgElCache.clear();
}
/**
@@ -118,27 +77,28 @@ export default class ImageElementFinder {
* multiple
* @property {boolean} [ignoreDefaultImageTemplateScale=false] - Whether we
* ignore defaultImageTemplateScale. It can be used when you would like to
* scale b64Template with defaultImageTemplateScale setting.
* scale template with defaultImageTemplateScale setting.
* @property {import('@appium/types').Rect?} containerRect - The bounding
* rectangle to limit the search in
*/
/**
* Find a screen rect represented by an ImageElement corresponding to an image
* template sent in by the client
*
* @param {string} b64Template - base64-encoded image used as a template to be
* @param {Buffer} template - image used as a template to be
* matched in the screenshot
* @param {ExternalDriver} driver
* @param {FindByImageOptions} opts - additional options
*
* @returns {Promise<Element|Element[]|ImageElement>} - WebDriver element with a special id prefix
*/
async findByImage(
b64Template,
{shouldCheckStaleness = false, multiple = false, ignoreDefaultImageTemplateScale = false}
template,
driver,
{shouldCheckStaleness = false, multiple = false, ignoreDefaultImageTemplateScale = false, containerRect = null}
) {
if (!this.driver) {
throw new Error(`Can't find without a driver!`);
}
const settings = {...DEFAULT_SETTINGS, ...this.driver.settings.getSettings()};
const settings = {...DEFAULT_SETTINGS, ...driver.settings.getSettings()};
const {
imageMatchThreshold: threshold,
imageMatchMethod,
@@ -149,28 +109,38 @@ export default class ImageElementFinder {
} = settings;
log.info(`Finding image element with match threshold ${threshold}`);
if (!this.driver.getWindowSize) {
throw new Error("This driver does not support the required 'getWindowSize' command");
if (!driver.getWindowRect && !driver.getWindowSize) {
throw new Error("This driver does not support the required 'getWindowRect' command");
}
let screenSize;
if (driver.getWindowRect) {
const screenRect = await driver.getWindowRect();
screenSize = {
width: screenRect.width,
height: screenRect.height,
};
} else {
// TODO: Drop the deprecated endpoint
screenSize = await driver.getWindowSize();
}
const {width: screenWidth, height: screenHeight} = await this.driver.getWindowSize();
// someone might have sent in a template that's larger than the screen
// dimensions. If so let's check and cut it down to size since the algorithm
// will not work unless we do. But because it requires some potentially
// expensive commands, only do this if the user has requested it in settings.
if (fixImageTemplateSize) {
b64Template = await this.ensureTemplateSize(b64Template, screenWidth, screenHeight);
template = await this.ensureTemplateSize(template, {
width: containerRect ? containerRect.width : screenSize.width,
height: containerRect ? containerRect.height : screenSize.height,
});
}
const results = [];
const condition = async () => {
try {
const {b64Screenshot, scale} = await this.getScreenshotForImageFind(
screenWidth,
screenHeight
);
const {screenshot, scale} = await this.getScreenshotForImageFind(driver, screenSize);
b64Template = await this.fixImageTemplateScale(b64Template, {
template = await this.fixImageTemplateScale(template, {
defaultImageTemplateScale,
ignoreDefaultImageTemplateScale,
fixImageTemplateScale,
@@ -185,21 +155,21 @@ export default class ImageElementFinder {
if (imageMatchMethod) {
comparisonOpts.method = imageMatchMethod;
}
if (multiple) {
results.push(
...(await compareImages(
MATCH_TEMPLATE_MODE,
b64Screenshot,
b64Template,
comparisonOpts
))
);
} else {
results.push(
await compareImages(MATCH_TEMPLATE_MODE, b64Screenshot, b64Template, comparisonOpts)
);
}
return true;
const pushIfOk = (el) => {
if (containerRect && !containsRect(containerRect, el.rect)) {
log.debug(
`The matched element rectangle ${JSON.stringify(el.rect)} is not located ` +
`inside of the bounding rectangle ${JSON.stringify(containerRect)}, thus rejected`
);
return false;
}
results.push(el);
return true;
};
const elOrEls = await compareImages(MATCH_TEMPLATE_MODE, screenshot, template, comparisonOpts);
return _.some((_.isArray(elOrEls) ? elOrEls : [elOrEls]).map(pushIfOk));
} catch (err) {
// if compareImages fails, we'll get a specific error, but we should
// retry, so trap that and just return false to trigger the next round of
@@ -213,7 +183,7 @@ export default class ImageElementFinder {
};
try {
await this.driver.implicitWaitForCondition(condition);
await driver.implicitWaitForCondition(condition);
} catch (err) {
// this `implicitWaitForCondition` method will throw a 'Condition unmet'
// error if an element is not found eventually. In that case, we will
@@ -235,7 +205,14 @@ export default class ImageElementFinder {
const elements = results.map(({rect, score, visualization}) => {
log.info(`Image template matched: ${JSON.stringify(rect)}`);
return new ImageElement(b64Template, rect, score, visualization, this);
return new ImageElement({
template,
rect,
score,
match: visualization ? Buffer.from(visualization, 'base64') : null,
finder: this,
containerRect,
});
});
// if we're just checking staleness, return straightaway so we don't add
@@ -253,86 +230,90 @@ export default class ImageElementFinder {
/**
* Ensure that the image template sent in for a find is of a suitable size
*
* @param {string} b64Template - base64-encoded image
* @param {number} screenWidth - width of screen
* @param {number} screenHeight - height of screen
* @param {Buffer} template - template image
* @param {import('@appium/types').Size} maxSize - size of the bounding rectangle
*
* @returns {Promise<string>} base64-encoded image, potentially resized
* @returns {Promise<Buffer>} image, potentially resized
*/
async ensureTemplateSize(b64Template, screenWidth, screenHeight) {
let imgObj = await imageUtil.getJimpImage(b64Template);
let {width: tplWidth, height: tplHeight} = imgObj.bitmap;
async ensureTemplateSize(template, maxSize) {
const imgObj = sharp(template);
const {width: tplWidth, height: tplHeight} = await imgObj.metadata();
log.info(
`Template image is ${tplWidth}x${tplHeight}. Screen size is ${screenWidth}x${screenHeight}`
`Template image is ${tplWidth}x${tplHeight}. Bounding rectangle size is ${maxSize.width}x${maxSize.height}`
);
// if the template fits inside the screen dimensions, we're good
if (tplWidth <= screenWidth && tplHeight <= screenHeight) {
return b64Template;
if (tplWidth <= maxSize.width && tplHeight <= maxSize.height) {
return template;
}
log.info(
`Scaling template image from ${tplWidth}x${tplHeight} to match ` +
`screen at ${screenWidth}x${screenHeight}`
`the bounding rectangle at ${maxSize.width}x${maxSize.height}`
);
// otherwise, scale it to fit inside the screen dimensions
imgObj = imgObj.scaleToFit(screenWidth, screenHeight);
return (await imgObj.getBuffer(imageUtil.MIME_PNG)).toString('base64');
// otherwise, scale it to fit inside the bounding rectangle dimensions:
// https://sharp.pixelplumbing.com/api-resize
return await imgObj.resize({
width: parseInt(maxSize.width, 10),
height: parseInt(maxSize.height, 10),
fit: 'inside',
})
.toBuffer();
}
/**
* Get the screenshot image that will be used for find by element, potentially
* altering it in various ways based on user-requested settings
*
* @param {number} screenWidth - width of screen
* @param {number} screenHeight - height of screen
* @param {ExternalDriver} driver
* @param {import('@appium/types').Size} screenSize - The original size of the screen
*
* @returns {Promise<Screenshot & {scale?: ScreenshotScale}>} base64-encoded screenshot and ScreenshotScale
* @returns {Promise<Screenshot & {scale?: ScreenshotScale}>} PNG screenshot and ScreenshotScale
*/
async getScreenshotForImageFind(screenWidth, screenHeight) {
if (!this.driver.getScreenshot) {
async getScreenshotForImageFind(driver, screenSize) {
if (!driver.getScreenshot) {
throw new Error("This driver does not support the required 'getScreenshot' command");
}
const settings = Object.assign({}, DEFAULT_SETTINGS, this.driver.settings.getSettings());
const settings = Object.assign({}, DEFAULT_SETTINGS, driver.settings.getSettings());
const {fixImageFindScreenshotDims} = settings;
let b64Screenshot = await this.driver.getScreenshot();
const screenshot = Buffer.from(await driver.getScreenshot(), 'base64');
// if the user has requested not to correct for aspect or size differences
// between the screenshot and the screen, just return the screenshot now
if (!fixImageFindScreenshotDims) {
log.info(`Not verifying screenshot dimensions match screen`);
return {b64Screenshot};
return {screenshot};
}
if (screenWidth < 1 || screenHeight < 1) {
if (screenSize.width < 1 || screenSize.height < 1) {
log.warn(
`The retrieved screen size ${screenWidth}x${screenHeight} does ` +
`not seem to be valid. No changes will be applied to the screenshot`
`The retrieved screen size ${screenSize.width}x${screenSize.height} does ` +
`not seem to be valid. No changes will be applied to the screenshot`
);
return {b64Screenshot};
return {screenshot};
}
// otherwise, do some verification on the screenshot to make sure it matches
// the screen size and aspect ratio
log.info('Verifying screenshot size and aspect ratio');
let imgObj = await imageUtil.getJimpImage(b64Screenshot);
let {width: shotWidth, height: shotHeight} = imgObj.bitmap;
let imgObj = sharp(screenshot);
let {width: shotWidth, height: shotHeight} = await imgObj.metadata();
if (shotWidth < 1 || shotHeight < 1) {
log.warn(
`The retrieved screenshot size ${shotWidth}x${shotHeight} does ` +
`not seem to be valid. No changes will be applied to the screenshot`
`not seem to be valid. No changes will be applied to the screenshot`
);
return {b64Screenshot};
return {screenshot};
}
if (screenWidth === shotWidth && screenHeight === shotHeight) {
if (screenSize.width === shotWidth && screenSize.height === shotHeight) {
// the height and width of the screenshot and the device screen match, which
// means we should be safe when doing template matches
log.info('Screenshot size matched screen size');
return {b64Screenshot};
return {screenshot};
}
// otherwise, if they don't match, it could spell problems for the accuracy
@@ -343,19 +324,19 @@ export default class ImageElementFinder {
const scale = {xScale: 1.0, yScale: 1.0};
const screenAR = screenWidth / screenHeight;
const screenAR = screenSize.width / screenSize.height;
const shotAR = shotWidth / shotHeight;
if (Math.round(screenAR * FLOAT_PRECISION) === Math.round(shotAR * FLOAT_PRECISION)) {
log.info(
`Screenshot aspect ratio '${shotAR}' (${shotWidth}x${shotHeight}) matched ` +
`screen aspect ratio '${screenAR}' (${screenWidth}x${screenHeight})`
`screen aspect ratio '${screenAR}' (${screenSize.width}x${screenSize.height})`
);
} else {
log.warn(
`When trying to find an element, determined that the screen ` +
`aspect ratio and screenshot aspect ratio are different. Screen ` +
`is ${screenWidth}x${screenHeight} whereas screenshot is ` +
`${shotWidth}x${shotHeight}.`
`aspect ratio and screenshot aspect ratio are different. Screen ` +
`is ${screenSize.width}x${screenSize.height} whereas screenshot is ` +
`${shotWidth}x${shotHeight}.`
);
// In the case where the x-scale and y-scale are different, we need to decide
@@ -364,46 +345,56 @@ export default class ImageElementFinder {
// this.getScreenshot(shotWidth, shotHeight) is 540x397,
// this.getDeviceSize(screenWidth, screenHeight) is 1080x1920.
// The ratio would then be {xScale: 0.5, yScale: 0.2}.
// In this case, we must should `yScale: 0.2` as scaleFactor, because
// In this case, we must use `yScale: 0.2` as scaleFactor, because
// if we select the xScale, the height will be bigger than real screenshot size
// which is used to image comparison by OpenCV as a base image.
// All of this is primarily useful when the screenshot is a horizontal slice taken out of the
// screen (for example not including top/bottom nav bars)
const xScale = (1.0 * shotWidth) / screenWidth;
const yScale = (1.0 * shotHeight) / screenHeight;
const scaleFactor = xScale >= yScale ? yScale : xScale;
const xScale = (1.0 * shotWidth) / screenSize.width;
const yScale = (1.0 * shotHeight) / screenSize.height;
const scaleFactor = Math.min(xScale, yScale);
const [newWidth, newHeight] = [shotWidth * scaleFactor, shotHeight * scaleFactor]
.map((x) => parseInt(x, 10));
log.warn(
`Resizing screenshot to ${shotWidth * scaleFactor}x${shotHeight * scaleFactor} to match ` +
`screen aspect ratio so that image element coordinates have a ` +
`greater chance of being correct.`
`Resizing screenshot to ${newWidth}x${newHeight} to match ` +
`screen aspect ratio so that image element coordinates have a ` +
`greater chance of being correct.`
);
imgObj = imgObj.resize(shotWidth * scaleFactor, shotHeight * scaleFactor);
imgObj = imgObj.resize({
width: newWidth,
height: newHeight,
fit: 'fill',
});
scale.xScale *= scaleFactor;
scale.yScale *= scaleFactor;
shotWidth = imgObj.bitmap.width;
shotHeight = imgObj.bitmap.height;
[shotWidth, shotHeight] = [newWidth, newHeight];
}
// Resize based on the screen dimensions only if both width and height are mismatched
// since except for that, it might be a situation which is different window rect and
// screenshot size like `@driver.window_rect #=>x=0, y=0, width=1080, height=1794` and
// `"deviceScreenSize"=>"1080x1920"`
if (screenWidth !== shotWidth && screenHeight !== shotHeight) {
if (screenSize.width !== shotWidth && screenSize.height !== shotHeight) {
log.info(
`Scaling screenshot from ${shotWidth}x${shotHeight} to match ` +
`screen at ${screenWidth}x${screenHeight}`
`screen at ${screenSize.width}x${screenSize.height}`
);
imgObj = imgObj.resize(screenWidth, screenHeight);
imgObj = imgObj.resize({
width: parseInt(screenSize.width, 10),
height: parseInt(screenSize.height, 10),
fit: 'fill',
});
scale.xScale *= (1.0 * screenWidth) / shotWidth;
scale.yScale *= (1.0 * screenHeight) / shotHeight;
scale.xScale *= (1.0 * screenSize.width) / shotWidth;
scale.yScale *= (1.0 * screenSize.height) / shotHeight;
}
b64Screenshot = (await imgObj.getBuffer(imageUtil.MIME_PNG)).toString('base64');
return {b64Screenshot, scale};
return {
screenshot: await imgObj.toBuffer(),
scale,
};
}
/**
@@ -411,7 +402,7 @@ export default class ImageElementFinder {
* @property {boolean} fixImageTemplateScale - fixImageTemplateScale in device-settings
* @property {number} defaultImageTemplateScale - defaultImageTemplateScale in device-settings
* @property {boolean} ignoreDefaultImageTemplateScale - Ignore defaultImageTemplateScale if it has true.
* If b64Template has been scaled to defaultImageTemplateScale or should ignore the scale,
* If the template has been scaled to defaultImageTemplateScale or should ignore the scale,
* this parameter should be true. e.g. click in image-element module
* @property {number} xScale - Scale ratio for width
* @property {number} yScale - Scale ratio for height
@@ -421,20 +412,19 @@ export default class ImageElementFinder {
* Get a image that will be used for template maching.
* Returns scaled image if scale ratio is provided.
*
*
* @param {string} b64Template - base64-encoded image used as a template to be
* @param {Buffer} template - image used as a template to be
* matched in the screenshot
* @param {ImageTemplateSettings} opts - Image template scale related options
*
* @returns {Promise<string>} base64-encoded scaled template screenshot
* @returns {Promise<Buffer>} scaled template screenshot
*/
async fixImageTemplateScale(b64Template, opts) {
async fixImageTemplateScale(template, opts) {
if (!opts) {
return b64Template;
return template;
}
let {
fixImageTemplateScale = false,
fixImageTemplateScale: fixTplScale = false,
defaultImageTemplateScale = DEFAULT_TEMPLATE_IMAGE_SCALE,
ignoreDefaultImageTemplateScale = false,
xScale = DEFAULT_FIX_IMAGE_TEMPLATE_SCALE,
@@ -446,12 +436,12 @@ export default class ImageElementFinder {
}
// Default
if (defaultImageTemplateScale === DEFAULT_TEMPLATE_IMAGE_SCALE && !fixImageTemplateScale) {
return b64Template;
if (defaultImageTemplateScale === DEFAULT_TEMPLATE_IMAGE_SCALE && !fixTplScale) {
return template;
}
// Calculate xScale and yScale Appium should scale
if (fixImageTemplateScale) {
if (fixTplScale) {
xScale *= defaultImageTemplateScale;
yScale *= defaultImageTemplateScale;
} else {
@@ -460,7 +450,7 @@ export default class ImageElementFinder {
// xScale and yScale can be NaN if defaultImageTemplateScale is string, for example
if (!parseFloat(String(xScale)) || !parseFloat(String(yScale))) {
return b64Template;
return template;
}
// Return if the scale is default, 1, value
@@ -474,26 +464,27 @@ export default class ImageElementFinder {
)
)
) {
return b64Template;
return template;
}
let imgTempObj = await imageUtil.getJimpImage(b64Template);
let {width: baseTempWidth, height: baseTempHeigh} = imgTempObj.bitmap;
let imgObj = sharp(template);
const {width: baseTempWidth, height: baseTempHeigh} = await imgObj.metadata();
const scaledWidth = baseTempWidth * xScale;
const scaledHeight = baseTempHeigh * yScale;
log.info(
`Scaling template image from ${baseTempWidth}x${baseTempHeigh}` +
` to ${scaledWidth}x${scaledHeight}`
`Scaling template image from ${baseTempWidth}x${baseTempHeigh} to ${scaledWidth}x${scaledHeight}`
);
log.info(`The ratio is ${xScale} and ${yScale}`);
imgTempObj = await imgTempObj.resize(scaledWidth, scaledHeight);
return (await imgTempObj.getBuffer(imageUtil.MIME_PNG)).toString('base64');
imgObj = imgObj.resize({
width: parseInt(scaledWidth, 10),
height: parseInt(scaledHeight, 10),
fit: 'fill',
});
return await imgObj.toBuffer();
}
}
export {W3C_ELEMENT_KEY, MJSONWP_ELEMENT_KEY, DEFAULT_SETTINGS, DEFAULT_FIX_IMAGE_TEMPLATE_SCALE};
/**
* @typedef {import('@appium/types').ExternalDriver} ExternalDriver
* @typedef {import('@appium/types').Element} Element
@@ -501,7 +492,7 @@ export {W3C_ELEMENT_KEY, MJSONWP_ELEMENT_KEY, DEFAULT_SETTINGS, DEFAULT_FIX_IMAG
/**
* @typedef Screenshot
* @property {string} b64Screenshot - base64 based screenshot string
* @property {Buffer} screenshot - screenshot image as PNG
*/
/**
+64 -31
View File
@@ -2,14 +2,12 @@ import _ from 'lodash';
import {errors} from 'appium/driver';
import {util} from 'appium/support';
import log from './logger';
import {DEFAULT_SETTINGS} from './finder';
import {
IMAGE_STRATEGY, DEFAULT_SETTINGS, IMAGE_TAP_STRATEGIES,
IMAGE_ELEMENT_PREFIX, IMAGE_EL_TAP_STRATEGY_W3C,
} from './constants';
const IMAGE_ELEMENT_PREFIX = 'appium-image-element-';
const TAP_DURATION_MS = 125;
const IMAGE_EL_TAP_STRATEGY_W3C = 'w3cActions';
const IMAGE_EL_TAP_STRATEGY_MJSONWP = 'touchActions';
const IMAGE_TAP_STRATEGIES = [IMAGE_EL_TAP_STRATEGY_MJSONWP, IMAGE_EL_TAP_STRATEGY_W3C];
const DEFAULT_TEMPLATE_IMAGE_SCALE = 1.0;
/**
* @typedef Dimension
@@ -23,28 +21,41 @@ const DEFAULT_TEMPLATE_IMAGE_SCALE = 1.0;
* @property {number} y - y coordinate
*/
/**
* @typedef ImageElementOpts
* @property {Buffer} template - the image which was used to find this ImageElement
* @property {Rect} rect - bounds of matched image element
* @property {number} score The similarity score as a float number in range [0.0, 1.0].
* 1.0 is the highest score (means both images are totally equal).
* @property {Buffer?} match - the image which has matched marks. Defaults to null.
* @property {import('./finder').default?} finder - the finder we can use to re-check stale elements
* @property {import('@appium/types').Rect?} containerRect - The bounding
* rectangle to limit the search in
*/
/**
* Representation of an "image element", which is simply a set of coordinates
* and methods that can be used on that set of coordinates via the driver
*/
export default class ImageElement {
/**
* @param {string} b64Template - the base64-encoded image which was used to
* find this ImageElement
* @param {Rect} rect - bounds of matched image element
* @param {number} score The similarity score as a float number in range [0.0, 1.0].
* 1.0 is the highest score (means both images are totally equal).
* @param {string?} b64Result - the base64-encoded image which has matched marks.
* Defaults to null.
* @param {import('./finder').default?} finder - the finder we can use to re-check stale elements
* @param {ImageElementOpts}
*/
constructor(b64Template, rect, score, b64Result = null, finder = null) {
this.template = b64Template;
constructor({
template,
rect,
score,
match = null,
finder = null,
containerRect = null,
}) {
this.template = template;
this.rect = rect;
this.id = `${IMAGE_ELEMENT_PREFIX}${util.uuidV4()}`;
this.b64MatchedImage = b64Result;
this.match = match;
this.score = score;
this.finder = finder;
this.containerRect = containerRect;
}
/**
@@ -72,20 +83,25 @@ export default class ImageElement {
}
/**
* @returns {?string} - the base64-encoded image which has matched marks
* @returns {string} - the base64-encoded original image used for matching
*/
get originalImage() {
return this.template.toString('base64');
}
/**
* @returns {string|null} - the base64-encoded image which has matched marks
*/
get matchedImage() {
return this.b64MatchedImage;
return this.match?.toString('base64') ?? null;
}
/**
* @param {string} protocolKey - the protocol-specific JSON key for
* a WebElement
*
* @returns {Element} - this image element as a WebElement
*/
asElement(protocolKey) {
return {[protocolKey]: this.id};
asElement() {
return util.wrapElement(this.id);
}
/**
@@ -132,11 +148,12 @@ export default class ImageElement {
if (checkForImageElementStaleness || updatePos) {
log.info('Checking image element for staleness before clicking');
try {
newImgEl = await this.finder.findByImage(this.template, {
newImgEl = await this.finder.findByImage(this.template, driver, {
shouldCheckStaleness: true,
// Set ignoreDefaultImageTemplateScale because this.template is device screenshot based image
// managed inside Appium after finidng image by template which managed by a user
ignoreDefaultImageTemplateScale: true,
containerRect: this.containerRect,
});
} catch (err) {
throw new errors.StaleElementReferenceError();
@@ -209,6 +226,22 @@ export default class ImageElement {
);
}
/**
* Perform lookup of image element(s) inside of the current element
*
* @param {boolean} multiple - Whether to lookup multiple elements
* @param {import('appium/driver').BaseDriver} driver - The driver to use for commands
* @param {string[]} args = Rest of arguments for executeScripts
* @returns {Promise<Element|Element[]|ImageElement>} - WebDriver element with a special id prefix
*/
async find(multiple, driver, ...args) {
const [strategy, selector] = args;
if (strategy !== IMAGE_STRATEGY) {
throw new errors.InvalidSelectorError(`Lookup strategies other than '${IMAGE_STRATEGY}' are not supported`);
}
return await this.finder.findByImage(selector, driver, {multiple, containerRect: this.rect});
}
/**
* Handle various Appium commands that involve an image element
*
@@ -223,6 +256,10 @@ export default class ImageElement {
switch (cmd) {
case 'click':
return await imgEl.click(driver);
case 'findElementFromElement':
return await imgEl.find(false, driver, ...args);
case 'findElementsFromElement':
return await imgEl.find(true, driver, ...args);
case 'elementDisplayed':
return true;
case 'getSize':
@@ -232,6 +269,8 @@ export default class ImageElement {
return imgEl.location;
case 'getElementRect':
return imgEl.rect;
case 'getElementScreenshot':
return imgEl.originalImage;
case 'getAttribute':
// /session/:sessionId/element/:elementId/attribute/:name
// /session/:sessionId/element/:elementId/attribute/visual should retun the visual data
@@ -250,13 +289,7 @@ export default class ImageElement {
}
}
export {
ImageElement,
IMAGE_EL_TAP_STRATEGY_MJSONWP,
IMAGE_EL_TAP_STRATEGY_W3C,
DEFAULT_TEMPLATE_IMAGE_SCALE,
IMAGE_ELEMENT_PREFIX,
};
export {ImageElement};
/**
* @typedef {import('@appium/types').Rect} Rect
+10 -12
View File
@@ -5,16 +5,11 @@ import {errors} from 'appium/driver';
import BasePlugin from 'appium/plugin';
import {compareImages} from './compare';
import ImageElementFinder from './finder';
import {ImageElement, IMAGE_ELEMENT_PREFIX} from './image-element';
const IMAGE_STRATEGY = '-image';
import {ImageElement} from './image-element';
import {IMAGE_STRATEGY, IMAGE_ELEMENT_PREFIX} from './constants';
function getImgElFromArgs(args) {
for (let arg of args) {
if (_.isString(arg) && arg.startsWith(IMAGE_ELEMENT_PREFIX)) {
return arg;
}
}
return args.find((arg) => _.isString(arg) && arg.startsWith(IMAGE_ELEMENT_PREFIX));
}
export default class ImageElementPlugin extends BasePlugin {
@@ -57,8 +52,7 @@ export default class ImageElementPlugin extends BasePlugin {
return await next();
}
this.finder.setDriver(driver);
return await this.finder.findByImage(selector, {multiple});
return await this.finder.findByImage(selector, driver, {multiple});
}
async handle(next, driver, cmdName, ...args) {
@@ -66,13 +60,17 @@ export default class ImageElementPlugin extends BasePlugin {
// and execute the command on it
const imgElId = getImgElFromArgs(args);
if (imgElId) {
if (!this.finder.imgElCache.has(imgElId)) {
const imgEl = this.finder.getImageElement(imgElId);
if (!imgEl) {
throw new errors.NoSuchElementError();
}
const imgEl = this.finder.imgElCache.get(imgElId);
return await ImageElement.execute(driver, imgEl, cmdName, ...args);
}
if (cmdName === 'deleteSession') {
this.finder.clearImageElements();
}
// otherwise just do the normal thing
return await next();
}
+3 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/images-plugin",
"version": "2.0.10",
"version": "2.1.1",
"description": "Plugin for working with images and image elements in Appium",
"keywords": [
"automation",
@@ -39,9 +39,10 @@
"test:unit": "mocha \"./test/unit/**/*.spec.js\""
},
"dependencies": {
"@appium/opencv": "^2.0.10",
"@appium/opencv": "^2.1.0",
"lodash": "4.17.21",
"lru-cache": "7.18.3",
"sharp": "0.32.1",
"source-map-support": "0.5.21"
},
"peerDependencies": {
@@ -1,8 +1,11 @@
import _ from 'lodash';
import path from 'path';
import {remote as wdio} from 'webdriverio';
import {MATCH_FEATURES_MODE, GET_SIMILARITY_MODE} from '../../lib/compare';
import {MATCH_FEATURES_MODE, GET_SIMILARITY_MODE} from '../../lib/constants';
import {TEST_IMG_1_B64, TEST_IMG_2_B64, APPSTORE_IMG_PATH} from '../fixtures';
import {pluginE2EHarness} from '@appium/plugin-test-support';
import {tempDir, fs} from '@appium/support';
import sharp from 'sharp';
const THIS_PLUGIN_DIR = path.join(__dirname, '..', '..');
const APPIUM_HOME = path.join(THIS_PLUGIN_DIR, 'local_appium_home');
@@ -32,11 +35,14 @@ const WDIO_OPTS = {
};
describe('ImageElementPlugin', function () {
let server,
driver = null;
let server;
let driver;
// this hook is intended to be run before the hooks created by `e2eSetup`
after(async function () {
beforeEach(async function () {
driver = await wdio(WDIO_OPTS);
});
afterEach(async function () {
if (driver) {
await driver.deleteSession();
}
@@ -58,7 +64,6 @@ describe('ImageElementPlugin', function () {
});
it('should add the compareImages route', async function () {
driver = await wdio(WDIO_OPTS);
let comparison = await driver.compareImages(
MATCH_FEATURES_MODE,
TEST_IMG_1_B64,
@@ -85,4 +90,29 @@ describe('ImageElementPlugin', function () {
height.should.eql(91);
await imageEl.click();
});
it('should find subelements', async function () {
const imageEl = await driver.$(APPSTORE_IMG_PATH);
const {width, height} = await imageEl.getSize();
const tmpRoot = await tempDir.openDir();
try {
const screenshotPath = path.join(tmpRoot, 'element.png');
await imageEl.saveScreenshot(screenshotPath);
const tmpImgPath = path.join(tmpRoot, 'region.png');
await sharp(screenshotPath)
.extract(
{
left: parseInt(width / 4, 10),
top: parseInt(height / 4, 10),
width: parseInt(width / 2, 10),
height: parseInt(height / 2, 10),
}
)
.toFile(tmpImgPath);
const subEl = await imageEl.$(tmpImgPath);
_.isNil(subEl).should.be.false;
} finally {
await fs.rimraf(tmpRoot);
}
});
});
+5 -2
View File
@@ -2,6 +2,7 @@ import path from 'path';
import fs from 'fs';
const TINY_PNG = 'iVBORw0KGgoAAAANSUhEUgAAAAQAAAAECAIAAAAmkwkpAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAyhpVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuNi1jMTQwIDc5LjE2MDQ1MSwgMjAxNy8wNS8wNi0wMTowODoyMSAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvIiB4bWxuczp4bXBNTT0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLyIgeG1sbnM6c3RSZWY9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9zVHlwZS9SZXNvdXJjZVJlZiMiIHhtcDpDcmVhdG9yVG9vbD0iQWRvYmUgUGhvdG9zaG9wIENDIDIwMTggKE1hY2ludG9zaCkiIHhtcE1NOkluc3RhbmNlSUQ9InhtcC5paWQ6N0NDMDM4MDM4N0U2MTFFOEEzMzhGMTRFNUUwNzIwNUIiIHhtcE1NOkRvY3VtZW50SUQ9InhtcC5kaWQ6N0NDMDM4MDQ4N0U2MTFFOEEzMzhGMTRFNUUwNzIwNUIiPiA8eG1wTU06RGVyaXZlZEZyb20gc3RSZWY6aW5zdGFuY2VJRD0ieG1wLmlpZDo3Q0MwMzgwMTg3RTYxMUU4QTMzOEYxNEU1RTA3MjA1QiIgc3RSZWY6ZG9jdW1lbnRJRD0ieG1wLmRpZDo3Q0MwMzgwMjg3RTYxMUU4QTMzOEYxNEU1RTA3MjA1QiIvPiA8L3JkZjpEZXNjcmlwdGlvbj4gPC9yZGY6UkRGPiA8L3g6eG1wbWV0YT4gPD94cGFja2V0IGVuZD0iciI/PpdvJjQAAAAlSURBVHjaJInBEQAACIKw/Xe2Ul5wYBtwmJqkk4+zfvUQVoABAEg0EfrZwc0hAAAAAElFTkSuQmCC';
const TiNY_PNG_BUF = Buffer.from(TINY_PNG, 'base64');
const TINY_PNG_DIMS = [4, 4];
const TEST_IMG_1_PATH = path.resolve(__dirname, 'img1.png');
@@ -13,5 +14,7 @@ const TEST_IMG_2_B64 = fs.readFileSync(TEST_IMG_2_PATH).toString('base64');
const TEST_IMG_2_PART_B64 = fs.readFileSync(TEST_IMG_2_PART_PATH).toString('base64');
export { TINY_PNG, TINY_PNG_DIMS, TEST_IMG_1_B64, TEST_IMG_2_B64, TEST_IMG_2_PART_B64,
APPSTORE_IMG_PATH };
export {
TINY_PNG, TINY_PNG_DIMS, TEST_IMG_1_B64, TEST_IMG_2_B64, TEST_IMG_2_PART_B64,
APPSTORE_IMG_PATH, TiNY_PNG_BUF,
};
+117 -126
View File
@@ -1,18 +1,20 @@
import _ from 'lodash';
import {imageUtil} from 'appium/support';
import {util} from 'appium/support';
import {BaseDriver} from 'appium/driver';
import {ImageElementPlugin, IMAGE_STRATEGY} from '../../lib/plugin';
import {ImageElementPlugin} from '../../lib/plugin';
import {IMAGE_STRATEGY} from '../../lib/constants';
import ImageElementFinder from '../../lib/finder';
import ImageElement from '../../lib/image-element';
import sinon from 'sinon';
import {TINY_PNG, TINY_PNG_DIMS} from '../fixtures';
import {TINY_PNG, TiNY_PNG_BUF, TINY_PNG_DIMS} from '../fixtures';
import sharp from 'sharp';
const compareModule = require('../../lib/compare');
const plugin = new ImageElementPlugin();
class PluginDriver extends BaseDriver {
async getWindowSize() {}
async getWindowRect() {}
async getScreenshot() {}
findElement(strategy, selector) {
return plugin.findElement(_.noop, this, strategy, selector);
@@ -56,8 +58,8 @@ describe('finding elements by image', function () {
const rect = {x: 10, y: 20, width: 30, height: 40};
const score = 0.9;
const size = {width: 100, height: 200};
const screenshot = 'iVBORfoo';
const template = 'iVBORbar';
const screenshot = Buffer.from('iVBORfoo', 'base64');
const template = Buffer.from('iVBORbar', 'base64');
/** @type {PluginDriver} */
let d;
/** @type {ImageElementFinder} */
@@ -66,15 +68,18 @@ describe('finding elements by image', function () {
let compareStub;
function basicStub(driver, finder) {
const sizeStub = sandbox.stub(driver, 'getWindowSize').returns(size);
const rectStub = sandbox.stub(driver, 'getWindowRect').returns({
x: 0,
y: 0,
...size,
});
const screenStub = sandbox.stub(finder, 'getScreenshotForImageFind').returns(screenshot);
return {sizeStub, screenStub};
return {rectStub, screenStub};
}
function basicImgElVerify(imgElProto, finder) {
const imgElId = imgElProto.ELEMENT;
finder.imgElCache.has(imgElId).should.be.true;
const imgEl = finder.imgElCache.get(imgElId);
const imgElId = util.unwrapElement(imgElProto);
const imgEl = finder.getImageElement(imgElId);
(imgEl instanceof ImageElement).should.be.true;
imgEl.rect.should.eql(rect);
imgEl.score.should.eql(score);
@@ -83,7 +88,7 @@ describe('finding elements by image', function () {
beforeEach(function () {
d = new PluginDriver();
f = new ImageElementFinder(d);
f = new ImageElementFinder();
compareStub = sandbox;
compareStub = sandbox.stub(compareModule, 'compareImages');
compareStub.resolves({rect, score});
@@ -91,201 +96,194 @@ describe('finding elements by image', function () {
});
it('should find an image element happypath', async function () {
const imgElProto = await f.findByImage(template, {multiple: false});
const imgElProto = await f.findByImage(template, d, {multiple: false});
basicImgElVerify(imgElProto, f);
});
it('should find image elements happypath', async function () {
compareStub.resolves([{rect, score}]);
const els = await f.findByImage(template, {multiple: true});
const els = await f.findByImage(template, d, {multiple: true});
els.should.have.length(1);
basicImgElVerify(els[0], f);
});
it('should fail if driver does not support getWindowSize', async function () {
d.getWindowSize = null;
it('should fail if driver does not support getWindowRect', async function () {
d.getWindowRect = null;
await f
.findByImage(template, {multiple: false})
.findByImage(template, d, {multiple: false})
.should.eventually.be.rejectedWith(/driver does not support/);
});
it('should fix template size if requested', async function () {
const newTemplate = 'iVBORbaz';
const newTemplateBuf = Buffer.from(newTemplate, 'base64');
await d.settings.update({fixImageTemplateSize: true});
sandbox.stub(f, 'ensureTemplateSize').returns(newTemplate);
const imgElProto = await f.findByImage(template, {multiple: false});
sandbox.stub(f, 'ensureTemplateSize').returns(newTemplateBuf);
const imgElProto = await f.findByImage(template, d, {multiple: false});
const imgEl = basicImgElVerify(imgElProto, f);
imgEl.template.should.eql(newTemplate);
_.last(compareStub.args)[2].should.eql(newTemplate);
imgEl.originalImage.should.eql(newTemplate);
_.last(compareStub.args)[2].should.eql(newTemplateBuf);
});
it('should fix template size scale if requested', async function () {
const newTemplate = 'iVBORbaz';
const newTemplateBuf = Buffer.from(newTemplate, 'base64');
await d.settings.update({fixImageTemplateScale: true});
sandbox.stub(f, 'fixImageTemplateScale').returns(newTemplate);
const imgElProto = await f.findByImage(template, {multiple: false});
sandbox.stub(f, 'fixImageTemplateScale').returns(newTemplateBuf);
const imgElProto = await f.findByImage(template, d, {multiple: false});
const imgEl = basicImgElVerify(imgElProto, f);
imgEl.template.should.eql(newTemplate);
_.last(compareStub.args)[2].should.eql(newTemplate);
imgEl.originalImage.should.eql(newTemplate);
_.last(compareStub.args)[2].should.eql(newTemplateBuf);
});
it('should not fix template size scale if it is not requested', async function () {
const newTemplate = 'iVBORbaz';
const newTemplateBuf = Buffer.from(newTemplate, 'base64');
await d.settings.update({});
sandbox.stub(f, 'fixImageTemplateScale').returns(newTemplate);
sandbox.stub(f, 'fixImageTemplateScale').returns(newTemplateBuf);
f.fixImageTemplateScale.callCount.should.eql(0);
});
it('should throw an error if template match fails', async function () {
compareStub.rejects(new Error('Cannot find any occurrences'));
await f
.findByImage(template, {multiple: false})
.findByImage(template, d, {multiple: false})
.should.be.rejectedWith(/element could not be located/);
});
it('should return empty array for multiple elements if template match fails', async function () {
compareStub.rejects(new Error('Cannot find any occurrences'));
await f.findByImage(template, {multiple: true}).should.eventually.eql([]);
await f.findByImage(template, d, {multiple: true}).should.eventually.eql([]);
});
it('should respect implicit wait', async function () {
d.setImplicitWait(10);
compareStub.resetHistory();
compareStub.returns({rect, score});
compareStub.onFirstCall().throws(new Error('Cannot find any occurrences'));
const imgElProto = await f.findByImage(template, {multiple: false});
const imgElProto = await f.findByImage(template, d, {multiple: false});
basicImgElVerify(imgElProto, f);
compareStub.should.have.been.calledTwice;
});
it('should not add element to cache and return it directly when checking staleness', async function () {
const imgEl = await f.findByImage(template, {
const imgEl = await f.findByImage(template, d, {
multiple: false,
shouldCheckStaleness: true,
});
(imgEl instanceof ImageElement).should.be.true;
f.imgElCache.has(imgEl.id).should.be.false;
_.isNil(f.getImageElement(imgEl.id)).should.be.true;
imgEl.rect.should.eql(rect);
});
});
describe('fixImageTemplateScale', function () {
let d;
let f;
const basicTemplate = 'iVBORbaz';
const basicTemplateBuf = Buffer.from(basicTemplate, 'base64');
beforeEach(function () {
d = new PluginDriver();
f = new ImageElementFinder(d);
f = new ImageElementFinder();
});
it('should not fix template size scale if no scale value', async function () {
await f
.fixImageTemplateScale(basicTemplate, {fixImageTemplateScale: true})
.should.eventually.eql(basicTemplate);
.fixImageTemplateScale(basicTemplateBuf, {fixImageTemplateScale: true})
.should.eventually.eql(basicTemplateBuf);
});
it('should not fix template size scale if it is null', async function () {
await f.fixImageTemplateScale(basicTemplate, null).should.eventually.eql(basicTemplate);
await f.fixImageTemplateScale(basicTemplateBuf, null).should.eventually.eql(basicTemplateBuf);
});
it('should not fix template size scale if it is not number', async function () {
await f
.fixImageTemplateScale(basicTemplate, 'wrong-scale')
.should.eventually.eql(basicTemplate);
.fixImageTemplateScale(basicTemplateBuf, 'wrong-scale')
.should.eventually.eql(basicTemplateBuf);
});
it('should fix template size scale', async function () {
const actual =
'iVBORw0KGgoAAAANSUhEUgAAAAYAAAAGCAYAAADgzO9IAAAAWElEQVR4AU3BQRWAQAhAwa/PGBsEgrC16AFBKEIPXW7OXO+Rmey9iQjMjHFzrLUwM7qbqmLcHKpKRFBVuDvj4agq3B1VRUQYT2bS3QwRQVUZF/CaGRHB3wc1vSZbHO5+BgAAAABJRU5ErkJggg==';
await f
.fixImageTemplateScale(TINY_PNG, {
.fixImageTemplateScale(TiNY_PNG_BUF, {
fixImageTemplateScale: true,
xScale: 1.5,
yScale: 1.5,
})
.should.eventually.eql(actual);
.should.eventually.not.eql(TiNY_PNG_BUF);
});
it('should not fix template size scale because of fixImageTemplateScale being false', async function () {
await f
.fixImageTemplateScale(TINY_PNG, {
.fixImageTemplateScale(TiNY_PNG_BUF, {
fixImageTemplateScale: false,
xScale: 1.5,
yScale: 1.5,
})
.should.eventually.eql(TINY_PNG);
.should.eventually.eql(TiNY_PNG_BUF);
});
it('should fix template size scale with default scale', async function () {
const actual =
'iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAABwUlEQVR4AaXBPUsrQQCG0SeX+cBdkTjwTpG1NPgLpjY/fW1stt4UYmm2cJqwMCsaw70uJJ3CBc9Z/P3Cl+12S9u2tG1L27bEGLm/v2ez2bDZbJDEd/7wS4YT7z3X19fc3Nxwd3dHXdd47xnHkefnZ8ZxpKoq6rqmqiqMMcwMJ1VV0TQN0zThnOPj44O6rsk503UdkmiahqZpWK1WGGOYGU7quqZpGqy1SCLnTM6Z19dXcs5IYpomrLVI4uLigpnhpKoqVqsVkjgcDjw9PdF1HTlnuq5DEs45JHE4HDgznByPR97e3pimiVIK4zhyPB7x3hNCIITA5eUl3nsWiwVnhpNSCsMwsNvtGIaB/X5PKQVJpJSQxHq9RhLOOc4MJ9M0sdvt2G639H3PTBIxRiQhCUnEGLHWcmY4KaUwDAN93/P4+MhyuSSlhCRSSkjCOYe1FmstZ6bve2YvLy/s93tmy+USSUhCEpIIIfAd8/DwwOz9/Z1SCpJIKSGJ9XqNJJxz/MS0bcvs6uoKScQYkYQkJBFjxFrLT0zbtsxub29JKSGJlBKScM5hrcVay09MzplZjJHPz0+894QQCCHwP/7wS/8A4e6nAg+R8LwAAAAASUVORK5CYII=';
await f
.fixImageTemplateScale(TINY_PNG, {
.fixImageTemplateScale(TiNY_PNG_BUF, {
defaultImageTemplateScale: 4.0,
})
.should.eventually.eql(actual);
.should.eventually.not.eql(TiNY_PNG_BUF);
});
it('should fix template size scale with default scale and image scale', async function () {
const actual =
'iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAACaUlEQVR4AbXBMWvrWBSF0c9BsFPtW91UR1U6+///FKlKKt8qqnyqnMozggkI8xgMj6x1uv+L/6zryrIsrOvKsiys68qyLFwuF87nM5fLhfP5zOVy4Xw+84wXftkLv2ziQBK26b0TEVQVu4jANrvM5Hq9spOEJCQhCUlI4mjiQBK26b1TVewkYRvb7DKTMQaZiW1s01rDNraRxNHEgSRaa1QVO0m01jjKTDKTXe+d3jtVxU4SjyYOJGGbnSRs03snM8lMMpPb7UZmkplEBFXFThK2eTRxIAnbSMI2VcX39zdjDMYYZCaZyRiDMQZVxU4StqkqHk0cSEISf5KZ7DKTMQbLsrCTRGuN3jtVxaOJg6qiqqgqqoqqoqoYY5CZ7GwTEdzvd97f34kIeu/YRhKPJg6qiswkM7ndbmQmmUlmkpnsbBMR2CYimOeZ3ju2kcSjiYOqIjP5+vpi2za2bWPbNo5aa7TW2PXe6b3Te6e1hiQeTRxUFbfbjW3bGGNwvV4ZY2Ab27TWsI1tbGMb27TWsI0kHk0cVBWZybZtXK9XPj8/+fj4YJ5nIoLWGraJCOZ5RhKSkIQkJPFo4qCqyEy2bWOMwefnJ+u6cjqdsM3ONvM8cz6feca0ris/rtcrmcnONhHB/X7n/f2diKD3jm0k8axpWRZ+ZCaZyc42EYFtIoJ5num9YxtJPGta15U/sY1tdm9vb/Te6b1jG0k8a1qWhR+2sU1rjdYatrGNbWxjm9YaknjWtK4rPyKCiKC1hm0igojg9fUVSUhCEpJ41rQsC0e22dkmIrhcLvyNF/7H6XTib73wy174Zf8AJEsePtlPj10AAAAASUVORK5CYII=';
await f
.fixImageTemplateScale(TINY_PNG, {
.fixImageTemplateScale(TiNY_PNG_BUF, {
defaultImageTemplateScale: 4.0,
fixImageTemplateScale: true,
xScale: 1.5,
yScale: 1.5,
})
.should.eventually.eql(actual);
.should.eventually.not.eql(TiNY_PNG_BUF);
});
it('should not fix template size scale with default scale and image scale', async function () {
const actual =
'iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAABwUlEQVR4AaXBPUsrQQCG0SeX+cBdkTjwTpG1NPgLpjY/fW1stt4UYmm2cJqwMCsaw70uJJ3CBc9Z/P3Cl+12S9u2tG1L27bEGLm/v2ez2bDZbJDEd/7wS4YT7z3X19fc3Nxwd3dHXdd47xnHkefnZ8ZxpKoq6rqmqiqMMcwMJ1VV0TQN0zThnOPj44O6rsk503UdkmiahqZpWK1WGGOYGU7quqZpGqy1SCLnTM6Z19dXcs5IYpomrLVI4uLigpnhpKoqVqsVkjgcDjw9PdF1HTlnuq5DEs45JHE4HDgznByPR97e3pimiVIK4zhyPB7x3hNCIITA5eUl3nsWiwVnhpNSCsMwsNvtGIaB/X5PKQVJpJSQxHq9RhLOOc4MJ9M0sdvt2G639H3PTBIxRiQhCUnEGLHWcmY4KaUwDAN93/P4+MhyuSSlhCRSSkjCOYe1FmstZ6bve2YvLy/s93tmy+USSUhCEpIIIfAd8/DwwOz9/Z1SCpJIKSGJ9XqNJJxz/MS0bcvs6uoKScQYkYQkJBFjxFrLT0zbtsxub29JKSGJlBKScM5hrcVay09MzplZjJHPz0+894QQCCHwP/7wS/8A4e6nAg+R8LwAAAAASUVORK5CYII=';
await f
.fixImageTemplateScale(TINY_PNG, {
.fixImageTemplateScale(TiNY_PNG_BUF, {
defaultImageTemplateScale: 4.0,
fixImageTemplateScale: false,
xScale: 1.5,
yScale: 1.5,
})
.should.eventually.eql(actual);
.should.eventually.not.eql(TiNY_PNG_BUF);
});
it('should not fix template size scale because of ignoreDefaultImageTemplateScale', async function () {
await f
.fixImageTemplateScale(TINY_PNG, {
.fixImageTemplateScale(TiNY_PNG_BUF, {
defaultImageTemplateScale: 4.0,
ignoreDefaultImageTemplateScale: true,
})
.should.eventually.eql(TINY_PNG);
.should.eventually.eql(TiNY_PNG_BUF);
});
it('should ignore defaultImageTemplateScale to fix template size scale because of ignoreDefaultImageTemplateScale', async function () {
const actual =
'iVBORw0KGgoAAAANSUhEUgAAAAYAAAAGCAYAAADgzO9IAAAAWElEQVR4AU3BQRWAQAhAwa/PGBsEgrC16AFBKEIPXW7OXO+Rmey9iQjMjHFzrLUwM7qbqmLcHKpKRFBVuDvj4agq3B1VRUQYT2bS3QwRQVUZF/CaGRHB3wc1vSZbHO5+BgAAAABJRU5ErkJggg==';
await f
.fixImageTemplateScale(TINY_PNG, {
.fixImageTemplateScale(TiNY_PNG_BUF, {
defaultImageTemplateScale: 4.0,
ignoreDefaultImageTemplateScale: true,
fixImageTemplateScale: true,
xScale: 1.5,
yScale: 1.5,
})
.should.eventually.eql(actual);
.should.eventually.not.eql(TiNY_PNG_BUF);
});
});
describe('ensureTemplateSize', function () {
const d = new PluginDriver();
const f = new ImageElementFinder(d);
const f = new ImageElementFinder();
it('should not resize the template if it is smaller than the screen', async function () {
const screen = TINY_PNG_DIMS.map((n) => n * 2);
await f.ensureTemplateSize(TINY_PNG, ...screen).should.eventually.eql(TINY_PNG);
const [width, height] = TINY_PNG_DIMS.map((n) => n * 2);
await f.ensureTemplateSize(TiNY_PNG_BUF, {width, height}).should.eventually.eql(TiNY_PNG_BUF);
});
it('should not resize the template if it is the same size as the screen', async function () {
await f.ensureTemplateSize(TINY_PNG, ...TINY_PNG_DIMS).should.eventually.eql(TINY_PNG);
const [width, height] = TINY_PNG_DIMS;
await f.ensureTemplateSize(TiNY_PNG_BUF, {width, height}).should.eventually.eql(TiNY_PNG_BUF);
});
it('should resize the template if it is bigger than the screen', async function () {
const screen = TINY_PNG_DIMS.map((n) => n / 2);
const newTemplate = await f.ensureTemplateSize(TINY_PNG, ...screen);
newTemplate.should.not.eql(TINY_PNG);
newTemplate.length.should.be.below(TINY_PNG.length);
const [width, height] = TINY_PNG_DIMS.map((n) => n / 2);
const newTemplateBuf = await f.ensureTemplateSize(TiNY_PNG_BUF, {width, height});
newTemplateBuf.should.not.eql(TiNY_PNG_BUF);
newTemplateBuf.length.should.be.below(TiNY_PNG_BUF.length);
});
});
@@ -295,107 +293,100 @@ describe('finding elements by image', function () {
beforeEach(function () {
d = new PluginDriver();
f = new ImageElementFinder(d);
f = new ImageElementFinder();
sandbox.stub(d, 'getScreenshot').returns(TINY_PNG);
});
it('should fail if driver does not support getScreenshot', async function () {
const d = new BaseDriver();
const f = new ImageElementFinder(d);
await f
.getScreenshotForImageFind()
await new ImageElementFinder()
.getScreenshotForImageFind(new BaseDriver())
.should.eventually.be.rejectedWith(/driver does not support/);
});
it('should not adjust or verify screenshot if asked not to by settings', async function () {
await d.settings.update({fixImageFindScreenshotDims: false});
const screen = TINY_PNG_DIMS.map((n) => n + 1);
const {b64Screenshot, scale} = await f.getScreenshotForImageFind(...screen);
b64Screenshot.should.eql(TINY_PNG);
const [width, height] = TINY_PNG_DIMS.map((n) => n + 1);
const {screenshot, scale} = await f.getScreenshotForImageFind(d, {width, height});
screenshot.should.eql(TiNY_PNG_BUF);
should.equal(scale, undefined);
});
it('should return screenshot without adjustment if it matches screen size', async function () {
const {b64Screenshot, scale} = await f.getScreenshotForImageFind(...TINY_PNG_DIMS);
b64Screenshot.should.eql(TINY_PNG);
const [width, height] = TINY_PNG_DIMS;
const {screenshot, scale} = await f.getScreenshotForImageFind(d, {width, height});
screenshot.should.eql(TiNY_PNG_BUF);
should.equal(scale, undefined);
});
it('should return scaled screenshot with same aspect ratio if matching screen aspect ratio', async function () {
const screen = TINY_PNG_DIMS.map((n) => n * 1.5);
const {b64Screenshot, scale} = await f.getScreenshotForImageFind(...screen);
b64Screenshot.should.not.eql(TINY_PNG);
const screenshotObj = await imageUtil.getJimpImage(b64Screenshot);
screenshotObj.bitmap.width.should.eql(screen[0]);
screenshotObj.bitmap.height.should.eql(screen[1]);
const [width, height] = TINY_PNG_DIMS.map((n) => n * 1.5);
const {screenshot, scale} = await f.getScreenshotForImageFind(d, {width, height});
screenshot.should.not.eql(TiNY_PNG_BUF);
const screenshotObj = sharp(screenshot);
const {width: screenWidth, height: screenHeight} = await screenshotObj.metadata();
screenWidth.should.eql(width);
screenHeight.should.eql(height);
scale.should.eql({xScale: 1.5, yScale: 1.5});
});
it('should return scaled screenshot with different aspect ratio if not matching screen aspect ratio', async function () {
// try first with portrait screen, screen = 8 x 12
let screen = [TINY_PNG_DIMS[0] * 2, TINY_PNG_DIMS[1] * 3];
let [width, height] = [TINY_PNG_DIMS[0] * 2, TINY_PNG_DIMS[1] * 3];
let expectedScale = {xScale: 2.67, yScale: 4};
const {b64Screenshot, scale} = await f.getScreenshotForImageFind(...screen);
b64Screenshot.should.not.eql(TINY_PNG);
let screenshotObj = await imageUtil.getJimpImage(b64Screenshot);
screenshotObj.bitmap.width.should.eql(screen[0]);
screenshotObj.bitmap.height.should.eql(screen[1]);
const {screenshot, scale} = await f.getScreenshotForImageFind(d, {width, height});
screenshot.should.not.eql(TiNY_PNG_BUF);
let screenshotObj = sharp(screenshot);
let {width: screenWidth, height: screenHeight} = await screenshotObj.metadata();
screenWidth.should.eql(width);
screenHeight.should.eql(height);
scale.xScale.toFixed(2).should.eql(expectedScale.xScale.toString());
scale.yScale.should.eql(expectedScale.yScale);
// then with landscape screen, screen = 12 x 8
screen = [TINY_PNG_DIMS[0] * 3, TINY_PNG_DIMS[1] * 2];
[width, height] = [TINY_PNG_DIMS[0] * 3, TINY_PNG_DIMS[1] * 2];
expectedScale = {xScale: 4, yScale: 2.67};
const {b64Screenshot: newScreen, scale: newScale} = await f.getScreenshotForImageFind(
...screen
);
newScreen.should.not.eql(TINY_PNG);
screenshotObj = await imageUtil.getJimpImage(newScreen);
screenshotObj.bitmap.width.should.eql(screen[0]);
screenshotObj.bitmap.height.should.eql(screen[1]);
const {screenshot: newScreen, scale: newScale} = await f.getScreenshotForImageFind(d, {width, height});
newScreen.should.not.eql(TiNY_PNG_BUF);
screenshotObj = sharp(newScreen);
({width: screenWidth, height: screenHeight} = await screenshotObj.metadata());
screenWidth.should.eql(width);
screenHeight.should.eql(height);
newScale.xScale.should.eql(expectedScale.xScale);
newScale.yScale.toFixed(2).should.eql(expectedScale.yScale.toString());
});
it('should return scaled screenshot with different aspect ratio if not matching screen aspect ratio with fixImageTemplateScale', async function () {
// try first with portrait screen, screen = 8 x 12
let screen = [TINY_PNG_DIMS[0] * 2, TINY_PNG_DIMS[1] * 3];
let [width, height] = [TINY_PNG_DIMS[0] * 2, TINY_PNG_DIMS[1] * 3];
let expectedScale = {xScale: 2.67, yScale: 4};
const {b64Screenshot, scale} = await f.getScreenshotForImageFind(...screen);
b64Screenshot.should.not.eql(TINY_PNG);
let screenshotObj = await imageUtil.getJimpImage(b64Screenshot);
screenshotObj.bitmap.width.should.eql(screen[0]);
screenshotObj.bitmap.height.should.eql(screen[1]);
const {screenshot, scale} = await f.getScreenshotForImageFind(d, {width, height});
screenshot.should.not.eql(TiNY_PNG_BUF);
let screenshotObj = sharp(screenshot);
let {width: screenWidth, height: screenHeight} = await screenshotObj.metadata();
screenWidth.should.eql(width);
screenHeight.should.eql(height);
scale.xScale.toFixed(2).should.eql(expectedScale.xScale.toString());
scale.yScale.should.eql(expectedScale.yScale);
// 8 x 12 stretched TINY_PNG
await f
.fixImageTemplateScale(b64Screenshot, {
fixImageTemplateScale: true,
scale,
})
.should.eventually.eql(
'iVBORw0KGgoAAAANSUhEUgAAAAgAAAAMCAYAAABfnvydAAAAJ0lEQVR4AYXBAQEAIACDMKR/p0fTBrKdbZcPCRIkSJAgQYIECRIkPAzBA1TpeNwZAAAAAElFTkSuQmCC'
);
.fixImageTemplateScale(screenshot, {fixImageTemplateScale: true, scale})
.should.eventually.not.eql(TiNY_PNG_BUF);
// then with landscape screen, screen = 12 x 8
screen = [TINY_PNG_DIMS[0] * 3, TINY_PNG_DIMS[1] * 2];
[width, height] = [TINY_PNG_DIMS[0] * 3, TINY_PNG_DIMS[1] * 2];
expectedScale = {xScale: 4, yScale: 2.67};
const {b64Screenshot: newScreen, scale: newScale} = await f.getScreenshotForImageFind(
...screen
);
newScreen.should.not.eql(TINY_PNG);
screenshotObj = await imageUtil.getJimpImage(newScreen);
screenshotObj.bitmap.width.should.eql(screen[0]);
screenshotObj.bitmap.height.should.eql(screen[1]);
const {screenshot: newScreen, scale: newScale} = await f.getScreenshotForImageFind(d, {width, height});
newScreen.should.not.eql(TiNY_PNG_BUF);
screenshotObj = sharp(newScreen);
({width: screenWidth, height: screenHeight} = await screenshotObj.metadata());
screenWidth.should.eql(width);
screenHeight.should.eql(height);
newScale.xScale.should.eql(expectedScale.xScale);
newScale.yScale.toFixed(2).should.eql(expectedScale.yScale.toString());
// 12 x 8 stretched TINY_PNG
await f
.fixImageTemplateScale(newScreen, {fixImageTemplateScale: true, scale})
.should.eventually.eql(
'iVBORw0KGgoAAAANSUhEUgAAAAwAAAAICAYAAADN5B7xAAAAI0lEQVR4AZXBAQEAMAyDMI5/T5W2ayB5245AIokkkkgiiST6+W4DTLyo5PUAAAAASUVORK5CYII='
);
.should.eventually.not.eql(TiNY_PNG_BUF);
});
});
});
@@ -1,12 +1,14 @@
import _ from 'lodash';
import BaseDriver from 'appium/driver';
import {util} from 'appium/support';
import ImageElementFinder from '../../lib/finder';
import {getImgElFromArgs} from '../../lib/plugin';
import ImageElement, {IMAGE_ELEMENT_PREFIX} from '../../lib/image-element';
import ImageElement from '../../lib/image-element';
import sinon from 'sinon';
import {IMAGE_ELEMENT_PREFIX} from '../../lib/constants';
const defRect = {x: 100, y: 110, width: 50, height: 25};
const defTemplate = 'iVBORasdf';
const defTemplate = Buffer.from('iVBORasdf', 'base64');
describe('ImageElement', function () {
const driver = new BaseDriver();
@@ -23,21 +25,30 @@ describe('ImageElement', function () {
describe('.size', function () {
it('should return the width and height of the image el', function () {
const el = new ImageElement(defTemplate, defRect);
const el = new ImageElement({
template: defTemplate,
rect: defRect,
});
el.size.should.eql({width: defRect.width, height: defRect.height});
});
});
describe('.location', function () {
it('should return the location of the image el', function () {
const el = new ImageElement(defTemplate, defRect);
const el = new ImageElement({
template: defTemplate,
rect: defRect,
});
el.location.should.eql({x: defRect.x, y: defRect.y});
});
});
describe('.center', function () {
it('should return the center location of the image el', function () {
const el = new ImageElement(defTemplate, defRect);
const el = new ImageElement({
template: defTemplate,
rect: defRect,
});
el.center.should.eql({
x: defRect.x + defRect.width / 2,
y: defRect.y + defRect.height / 2,
@@ -47,21 +58,36 @@ describe('ImageElement', function () {
describe('.asElement', function () {
it('should get the webdriver object representation of the element', function () {
const el = new ImageElement(defTemplate, defRect);
el.asElement('ELEMENT').ELEMENT.should.match(/^appium-image-el/);
const el = new ImageElement({
template: defTemplate,
rect: defRect,
});
util.unwrapElement(el.asElement()).should.match(/^appium-image-el/);
});
});
describe('.equals', function () {
it('should say two image elements with same rect are equal', function () {
const el1 = new ImageElement('foo', defRect);
const el2 = new ImageElement('bar', defRect);
const el1 = new ImageElement({
template: Buffer.from('foo'),
rect: defRect
});
const el2 = new ImageElement({
template: Buffer.from('bar'),
rect: defRect
});
el1.equals(el2).should.be.true;
el2.equals(el1).should.be.true;
});
it('should say two image elements with different rect are not equal', function () {
const el1 = new ImageElement(defTemplate, {...defRect, x: 0});
const el2 = new ImageElement(defTemplate, defRect);
const el1 = new ImageElement({
template: defTemplate,
rect: {...defRect, x: 0}
});
const el2 = new ImageElement({
template: defTemplate,
rect: defRect,
});
el1.equals(el2).should.be.false;
el2.equals(el1).should.be.false;
});
@@ -70,15 +96,22 @@ describe('ImageElement', function () {
describe('.click', function () {
it('should reject an invalid tap strategy', async function () {
const d = new BaseDriver();
const el = new ImageElement(defTemplate, defRect);
const el = new ImageElement({
template: defTemplate,
rect: defRect,
});
await d.settings.update({imageElementTapStrategy: 'bad'});
await el.click(d).should.be.rejectedWith(/Incorrect imageElementTapStrategy/);
});
it('should try to check for image element staleness, and throw if stale', async function () {
const d = new BaseDriver();
const f = new ImageElementFinder(d);
const f = new ImageElementFinder();
sandbox.stub(f, 'findByImage').throws();
const el = new ImageElement(defTemplate, defRect, null, null, f);
const el = new ImageElement({
template: defTemplate,
rect: defRect,
finder: f,
});
// we need to check for staleness if explicitly requested to do so
await d.settings.update({
checkForImageElementStaleness: true,
@@ -97,10 +130,18 @@ describe('ImageElement', function () {
const d = new BaseDriver();
d.performActions = _.noop;
sandbox.stub(d, 'performActions');
const f = new ImageElementFinder(d);
const el = new ImageElement(defTemplate, defRect, null, null, f);
const f = new ImageElementFinder();
const el = new ImageElement({
template: defTemplate,
rect: defRect,
finder: f,
});
const newRect = {...defRect, x: defRect.x + 10, y: defRect.y + 5};
const elPos2 = new ImageElement(defTemplate, newRect, null, null, f);
const elPos2 = new ImageElement({
template: defTemplate,
rect: newRect,
finder: f,
});
sandbox.stub(f, 'findByImage').returns(elPos2);
await d.settings.update({
autoUpdateImageElementPosition: true,
@@ -113,7 +154,10 @@ describe('ImageElement', function () {
const d = new BaseDriver();
d.performActions = _.noop;
const actionStub = sandbox.stub(d, 'performActions');
const el = new ImageElement(defTemplate, defRect);
const el = new ImageElement({
template: defTemplate,
rect: defRect,
});
// skip the staleness check for this test
await d.settings.update({
checkForImageElementStaleness: false,
@@ -127,7 +171,10 @@ describe('ImageElement', function () {
const d = new BaseDriver();
d.performTouch = _.noop;
const actionStub = sandbox.stub(d, 'performTouch');
const el = new ImageElement(defTemplate, defRect);
const el = new ImageElement({
template: defTemplate,
rect: defRect,
});
// skip the staleness check for this test
await d.settings.update({
checkForImageElementStaleness: false,
@@ -143,7 +190,10 @@ describe('ImageElement', function () {
const w3cStub = sandbox.stub(d, 'performActions');
d.performTouch = _.noop;
const touchStub = sandbox.stub(d, 'performTouch');
const el = new ImageElement(defTemplate, defRect);
const el = new ImageElement({
template: defTemplate,
rect: defRect,
});
// skip the staleness check for this test
await d.settings.update({
checkForImageElementStaleness: false,
@@ -157,7 +207,10 @@ describe('ImageElement', function () {
});
it('should throw if driver does not implement any type of action', async function () {
const d = new BaseDriver();
const el = new ImageElement(defTemplate, defRect);
const el = new ImageElement({
template: defTemplate,
rect: defRect,
});
// skip the staleness check for this test
await d.settings.update({
checkForImageElementStaleness: false,
@@ -168,20 +221,22 @@ describe('ImageElement', function () {
describe('#execute', function () {
// aGFwcHkgdGVzdGluZw== is 'happy testing'
const f = new ImageElementFinder(driver);
const imgEl = new ImageElement(defTemplate, defRect, 0, 'aGFwcHkgdGVzdGluZw==', f);
const f = new ImageElementFinder();
const imgEl = new ImageElement({
template: defTemplate,
rect: defRect,
score: 0,
match: Buffer.from('aGFwcHkgdGVzdGluZw==', 'base64'),
finder: f,
});
let clickStub;
before(function () {
clickStub = sandbox.stub(imgEl, 'click');
f.imgElCache.set(imgEl.id, imgEl);
f.registerImageElement(imgEl);
clickStub.returns(true);
});
after(function () {
f.imgElCache.clear();
});
it('should reject executions for unsupported commands', async function () {
await ImageElement.execute(driver, imgEl, 'foobar').should.be.rejectedWith(
/not yet been implemented/
@@ -220,7 +275,10 @@ describe('ImageElement', function () {
);
});
it('should get null as visual of element by default', async function () {
const imgElement = new ImageElement(defTemplate, defRect);
const imgElement = new ImageElement({
template: defTemplate,
rect: defRect,
});
await ImageElement.execute(
driver,
imgElement,
@@ -244,24 +302,34 @@ describe('ImageElement', function () {
describe('image element LRU cache', function () {
it('should accept and cache image elements', function () {
const el1 = new ImageElement(defTemplate, defRect);
const el2 = new ImageElement(defTemplate, defRect);
const cache = new ImageElementFinder().imgElCache;
cache.set(el1.id, el1);
el1.equals(cache.get(el1.id)).should.be.true;
_.isUndefined(cache.get(el2.id)).should.be.true;
cache.has(el1.id).should.be.true;
cache.has(el2.id).should.be.false;
const el1 = new ImageElement({
template: defTemplate,
rect: defRect,
});
const el2 = new ImageElement({
template: defTemplate,
rect: defRect,
});
const finder = new ImageElementFinder();
finder.registerImageElement(el1);
el1.equals(finder.getImageElement(el1.id)).should.be.true;
_.isUndefined(finder.getImageElement(el2.id)).should.be.true;
});
it('once cache reaches max size, should eject image elements', function () {
const el1 = new ImageElement(defTemplate, defRect);
const el2 = new ImageElement(defTemplate, defRect);
const cache = new ImageElementFinder(null, defTemplate.length + 1).imgElCache;
cache.set(el1.id, el1);
cache.has(el1.id).should.be.true;
cache.set(el2.id, el2);
cache.has(el2.id).should.be.true;
cache.has(el1.id).should.be.false;
const el1 = new ImageElement({
template: defTemplate,
rect: defRect,
});
const el2 = new ImageElement({
template: defTemplate,
rect: defRect,
});
const finder = new ImageElementFinder(1);
finder.registerImageElement(el1);
_.isUndefined(finder.getImageElement(el1.id)).should.be.false;
finder.registerImageElement(el2);
_.isUndefined(finder.getImageElement(el1.id)).should.be.true;
_.isUndefined(finder.getImageElement(el2.id)).should.be.false;
});
});
+29 -11
View File
@@ -1,8 +1,13 @@
import {ImageElementPlugin, IMAGE_STRATEGY} from '../../lib/plugin';
import {MATCH_FEATURES_MODE, GET_SIMILARITY_MODE, MATCH_TEMPLATE_MODE} from '../../lib/compare';
import {ImageElementPlugin} from '../../lib/plugin';
import {
MATCH_FEATURES_MODE,
GET_SIMILARITY_MODE,
MATCH_TEMPLATE_MODE,
IMAGE_STRATEGY,
} from '../../lib/constants';
import BaseDriver from 'appium/driver';
import {W3C_ELEMENT_KEY} from '../../lib/finder';
import {TEST_IMG_1_B64, TEST_IMG_2_B64, TEST_IMG_2_PART_B64} from '../fixtures';
import {util} from '@appium/support';
describe('ImageElementPlugin#handle', function () {
const next = () => {};
@@ -26,8 +31,8 @@ describe('ImageElementPlugin#handle', function () {
next,
driver,
GET_SIMILARITY_MODE,
TEST_IMG_1_B64,
TEST_IMG_2_B64,
Buffer.from(TEST_IMG_1_B64, 'base64'),
Buffer.from(TEST_IMG_2_B64, 'base64'),
{}
);
res.score.should.be.above(0.2);
@@ -50,13 +55,26 @@ describe('ImageElementPlugin#handle', function () {
.compareImages(next, driver, 'some mode', '', '')
.should.eventually.be.rejectedWith(/comparison mode is unknown/);
});
it('should throw an error if image template is broken', async function () {
await p.compareImages(
next,
driver,
MATCH_TEMPLATE_MODE,
Buffer.from('d1423423424'),
Buffer.from('d1423423424')
).should.eventually.be.rejected;
});
it('should throw an error if image template is empty', async function () {
await p.compareImages(next, driver, MATCH_TEMPLATE_MODE, Buffer.from(''), Buffer.from(''))
.should.eventually.be.rejected;
});
});
describe('findElement(s)', function () {
driver.settings = {getSettings: () => ({})};
driver.isW3CProtocol = () => true;
driver.getScreenshot = () => TEST_IMG_2_B64;
driver.getWindowSize = () => ({width: 64, height: 64});
driver.getWindowRect = () => ({x: 0, y: 0, width: 64, height: 64});
it('should defer execution to regular command if not a find command', async function () {
const next = () => true;
await p.handle(next, driver, 'sendKeys').should.eventually.become(true);
@@ -68,12 +86,12 @@ describe('ImageElementPlugin#handle', function () {
});
it('should find an image element inside a screenshot', async function () {
const el = await p.findElement(next, driver, IMAGE_STRATEGY, TEST_IMG_2_PART_B64);
el[W3C_ELEMENT_KEY].should.include('appium-image-element');
util.unwrapElement(el).should.include('appium-image-element');
});
it('should find image elements inside a screenshot', async function () {
const els = await p.findElements(next, driver, IMAGE_STRATEGY, TEST_IMG_2_PART_B64);
els.should.have.length(1);
els[0][W3C_ELEMENT_KEY].should.include('appium-image-element');
util.unwrapElement(els[0]).should.include('appium-image-element');
});
});
@@ -83,9 +101,9 @@ describe('ImageElementPlugin#handle', function () {
driver.settings = {getSettings: () => ({})};
driver.isW3CProtocol = () => true;
driver.getScreenshot = () => TEST_IMG_2_B64;
driver.getWindowSize = () => ({width: 64, height: 64});
driver.getWindowRect = () => ({x: 0, y: 0, width: 64, height: 64});
const el = await p.findElement(next, driver, IMAGE_STRATEGY, TEST_IMG_2_PART_B64);
elId = el[W3C_ELEMENT_KEY];
elId = util.unwrapElement(el);
});
it('should click on the screen coords of the middle of the element', async function () {
let action = null;
@@ -140,7 +158,7 @@ describe('ImageElementPlugin#handle', function () {
}),
};
const el = await p.findElement(next, driver, IMAGE_STRATEGY, TEST_IMG_2_PART_B64);
elId = el[W3C_ELEMENT_KEY];
elId = util.unwrapElement(el);
await p
.handle(next, driver, 'getAttribute', 'visual', elId)
.should.eventually.include('iVBOR');
+15
View File
@@ -3,6 +3,21 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [2.1.0](https://github.com/appium/appium/compare/@appium/opencv@2.0.10...@appium/opencv@2.1.0) (2023-05-17)
### Features
* **image-plugin:** Make it possible to find elements inside of other elements ([#18555](https://github.com/appium/appium/issues/18555)) ([2796639](https://github.com/appium/appium/commit/2796639b1183f0efbae8d7adb6d24570d5d4fde4))
### Bug Fixes
* **opencv:** Make sure allocated CV objects are properly released afterwards ([#18569](https://github.com/appium/appium/issues/18569)) ([2fb844c](https://github.com/appium/appium/commit/2fb844c8ad5ea1e36c326ce5cf46869c0ff0d4f9))
* **opencv:** update dependency sharp to v0.32.1 ([add0eaa](https://github.com/appium/appium/commit/add0eaa05f476a5f2708925a43f2bc16cad75872))
## [2.0.10](https://github.com/appium/appium/compare/@appium/opencv@2.0.9...@appium/opencv@2.0.10) (2023-04-05)
+35
View File
@@ -0,0 +1,35 @@
import _ from 'lodash';
export class OpenCvAutoreleasePool {
/**
* @type {Set}
*/
_items;
constructor() {
this._items = new Set();
}
/**
* @template T
* @param {...T} items
* @return {T[]|T} the same items
*/
add(...items) {
for (const item of items) {
if (_.has(item, 'delete') && _.isFunction(item.delete)) {
this._items.add(item);
}
}
return items.length === 1 ? items[0] : items;
}
drain() {
for (const item of this._items) {
try {
item.delete();
} catch (ign) {}
}
this._items.clear();
}
}
+116 -123
View File
@@ -1,7 +1,8 @@
import _ from 'lodash';
import Jimp from 'jimp';
import {Buffer} from 'buffer';
import B from 'bluebird';
import sharp from 'sharp';
import { OpenCvAutoreleasePool } from './autorelease-pool';
/** @type {any} */
let cv;
@@ -104,6 +105,7 @@ async function initOpenCv() {
* for brute-force matching.
* Read https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature2d/py_matcher/py_matcher.html
* for more details.
* The caller is responsible for releasing the returned objects.
*
* @param {OpenCVBindings['Mat']} img Image data
* @param {OpenCVBindings['FeatureDetector']} detector OpenCV feature detector instance
@@ -124,7 +126,7 @@ function detectAndCompute(img, detector) {
/**
* Calculated the bounding rect coordinates for the array of matching points
*
* @param {Array<Point>} matchedPoints Array of matching points
* @param {Point[]} matchedPoints Array of matching points
* @returns {Rect} The matching bounding rect or a zero rect if no match
* can be found.
*/
@@ -173,28 +175,33 @@ function highlightRegion(mat, region) {
return;
}
// highlight in red
const color = new cv.Scalar(255, 0, 0, 255);
const thickness = 2;
const topLeft = new cv.Point(region.x, region.y);
const botRight = new cv.Point(region.x + region.width, region.y + region.height);
cv.rectangle(mat, topLeft, botRight, color, thickness, cv.LINE_8, 0);
return mat;
const pool = new OpenCvAutoreleasePool();
try {
// highlight in red
const color = pool.add(new cv.Scalar(255, 0, 0, 255));
const thickness = 2;
const topLeft = pool.add(new cv.Point(region.x, region.y));
const botRight = pool.add(new cv.Point(region.x + region.width, region.y + region.height));
cv.rectangle(mat, topLeft, botRight, color, thickness, cv.LINE_8, 0);
return mat;
} finally {
pool.drain();
}
}
/**
* @typedef MatchingOptions
* @property {?string} detectorName ['ORB'] One of possible OpenCV feature detector names
* @property {string} detectorName ['ORB'] One of possible OpenCV feature detector names
* from keys of the `AVAILABLE_DETECTORS` object.
* Some of these methods (FAST, AGAST, GFTT, FAST, SIFT and MSER) are not available
* in the default OpenCV installation and have to be enabled manually before
* library compilation.
* @property {?string} matchFunc ['BruteForce'] The name of the matching function.
* @property {string} matchFunc ['BruteForce'] The name of the matching function.
* Should be one of the keys of the `AVAILABLE_MATCHING_FUNCTIONS` object.
* @property {?number|Function} goodMatchesFactor The maximum count of "good" matches
* @property {number|Function?} goodMatchesFactor The maximum count of "good" matches
* (e. g. with minimal distances) or a function, which accepts 3 arguments: the current distance,
* minimal distance, maximum distance and returns true or false to include or exclude the match.
* @property {?boolean} visualize [false] Whether to return the resulting visalization
* @property {boolean?} visualize [false] Whether to return the resulting visalization
* as an image (useful for debugging purposes)
*/
@@ -206,13 +213,13 @@ function highlightRegion(mat, region) {
* It is equal to `count` if `goodMatchesFactor` does not limit the matches,
* otherwise it contains the total count of matches before `goodMatchesFactor` is
* applied.
* @property {?Buffer} visualization The visualization of the matching result
* @property {Buffer?} visualization The visualization of the matching result
* represented as PNG image buffer. This visualization looks like
* https://user-images.githubusercontent.com/31125521/29702731-c79e3142-8972-11e7-947e-db109d415469.jpg
* @property {Array<Point>} points1 The array of matching points on the first image
* @property {Point[]} points1 The array of matching points on the first image
* @property {Rect} rect1 The bounding rect for the `matchedPoints1` set or a zero rect
* if not enough matching points are found
* @property {Array<Point>} points2 The array of matching points on the second image
* @property {Point[]} points2 The array of matching points on the second image
* @property {Rect} rect2 The bounding rect for the `matchedPoints2` set or a zero rect
* if not enough matching points are found
*/
@@ -223,15 +230,15 @@ function highlightRegion(mat, region) {
*
* @param {Buffer} img1Data The data of the first image packed into a NodeJS buffer
* @param {Buffer} img2Data The data of the second image packed into a NodeJS buffer
* @param {?MatchingOptions} options [{}] Set of matching options
* @param {MatchingOptions} options [{}] Set of matching options
*
* @returns {MatchingResult} Maching result
* @returns {Promise<MatchingResult>} Maching result
* @throws {Error} If `detectorName` value is unknown.
*/
async function getImagesMatches(img1Data, img2Data, options = {}) {
await initOpenCv();
let img1, img2, detector, result1, result2, matcher, matchesVec;
const pool = new OpenCvAutoreleasePool();
try {
const {
detectorName = 'ORB',
@@ -254,13 +261,15 @@ async function getImagesMatches(img1Data, img2Data, options = {}) {
);
}
detector = new cv[AVAILABLE_DETECTORS[detectorName]]();
[img1, img2] = await B.all([cvMatFromImage(img1Data), cvMatFromImage(img2Data)]);
result1 = detectAndCompute(img1, detector);
result2 = detectAndCompute(img2, detector);
matcher = new cv.DescriptorMatcher(AVAILABLE_MATCHING_FUNCTIONS[matchFunc]);
matchesVec = new cv.DMatchVector();
let matches = [];
const detector = pool.add(new cv[AVAILABLE_DETECTORS[detectorName]]());
const [img1, img2] = (await B.all([cvMatFromImage(img1Data), cvMatFromImage(img2Data)]))
.map((x) => pool.add(x));
const result1 = detectAndCompute(img1, detector);
pool.add(result1.keyPoints, result1.descriptor);
const result2 = detectAndCompute(img2, detector);
pool.add(result2.keyPoints, result2.descriptor);
const matcher = pool.add(new cv.DescriptorMatcher(AVAILABLE_MATCHING_FUNCTIONS[matchFunc]));
const matchesVec = pool.add(new cv.DMatchVector());
matcher.match(result1.descriptor, result2.descriptor, matchesVec);
const totalCount = matchesVec.size();
if (totalCount < 1) {
@@ -269,6 +278,7 @@ async function getImagesMatches(img1Data, img2Data, options = {}) {
`resolution, or use another detector or matching function.`
);
}
let matches = [];
for (let i = 0; i < totalCount; i++) {
matches.push(matchesVec.get(i));
}
@@ -313,12 +323,12 @@ async function getImagesMatches(img1Data, img2Data, options = {}) {
count: matches.length,
};
if (visualize) {
const goodMatchesVec = new cv.DMatchVector();
const goodMatchesVec = pool.add(new cv.DMatchVector());
for (let i = 0; i < matches.length; i++) {
goodMatchesVec.push_back(matches[i]);
}
const visualization = new cv.Mat();
const color = new cv.Scalar(0, 255, 0, 255);
const visualization = pool.add(new cv.Mat());
const color = pool.add(new cv.Scalar(0, 255, 0, 255));
cv.drawMatches(
img1,
result1.keyPoints,
@@ -335,28 +345,18 @@ async function getImagesMatches(img1Data, img2Data, options = {}) {
width: rect2.width,
height: rect2.height,
});
result.visualization = await jimpImgFromCvMat(visualization).getBufferAsync(Jimp.MIME_PNG);
result.visualization = await cvMatToPng(visualization);
}
return result;
} finally {
try {
img1.delete();
img2.delete();
detector.delete();
result1.keyPoints.delete();
result1.descriptor.delete();
result2.keyPoints.delete();
result2.descriptor.delete();
matcher.delete();
matchesVec.delete();
} catch (ign) {}
pool.drain();
}
}
/**
* @typedef SimilarityOptions
* @property {?boolean} visualize [false] Whether to return the resulting visalization
* @property {boolean?} visualize [false] Whether to return the resulting visalization
* as an image (useful for debugging purposes)
* @property {string} method [TM_CCOEFF_NORMED] The name of the template matching method.
* Acceptable values are:
@@ -374,7 +374,7 @@ async function getImagesMatches(img1Data, img2Data, options = {}) {
* @typedef SimilarityResult
* @property {number} score The similarity score as a float number in range [0.0, 1.0].
* 1.0 is the highest score (means both images are totally equal).
* @property {?Buffer} visualization The visualization of the matching result
* @property {Buffer?} visualization The visualization of the matching result
* represented as PNG image buffer. This image includes both input pictures where
* difference regions are highlighted with rectangles.
*/
@@ -385,9 +385,9 @@ async function getImagesMatches(img1Data, img2Data, options = {}) {
*
* @param {Buffer} img1Data The data of the first image packed into a NodeJS buffer
* @param {Buffer} img2Data The data of the second image packed into a NodeJS buffer
* @param {?SimilarityOptions} options [{}] Set of similarity calculation options
* @param {SimilarityOptions} options [{}] Set of similarity calculation options
*
* @returns {SimilarityResult} The calculation result
* @returns {Promise<SimilarityResult>} The calculation result
* @throws {Error} If the given images have different resolution.
*/
async function getImagesSimilarity(img1Data, img2Data, options = {}) {
@@ -395,9 +395,10 @@ async function getImagesSimilarity(img1Data, img2Data, options = {}) {
const {method = DEFAULT_MATCHING_METHOD, visualize = false} = options;
let template, reference, matched;
const pool = new OpenCvAutoreleasePool();
try {
[template, reference] = await B.all([cvMatFromImage(img1Data), cvMatFromImage(img2Data)]);
const [template, reference] = (await B.all([cvMatFromImage(img1Data), cvMatFromImage(img2Data)]))
.map((x) => pool.add(x));
if (template.rows !== reference.rows || template.cols !== reference.cols) {
throw new Error(
'Both images are expected to have the same size in order to ' +
@@ -407,7 +408,7 @@ async function getImagesSimilarity(img1Data, img2Data, options = {}) {
template.convertTo(template, cv.CV_8UC3);
reference.convertTo(reference, cv.CV_8UC3);
matched = new cv.Mat();
const matched = pool.add(new cv.Mat());
cv.matchTemplate(reference, template, matched, toMatchingMethod(method));
const minMax = cv.minMaxLoc(matched);
const result = {
@@ -415,51 +416,36 @@ async function getImagesSimilarity(img1Data, img2Data, options = {}) {
};
if (visualize) {
let bothImages, resultMat, mask, contours, hierarchy;
try {
resultMat = new cv.Mat(template.rows, template.cols * 2, cv.CV_8UC3);
bothImages = new cv.MatVector();
bothImages.push_back(reference);
bothImages.push_back(template);
cv.hconcat(bothImages, resultMat);
const resultMat = pool.add(new cv.Mat(template.rows, template.cols * 2, cv.CV_8UC3));
const bothImages = pool.add(new cv.MatVector());
bothImages.push_back(reference);
bothImages.push_back(template);
cv.hconcat(bothImages, resultMat);
mask = new cv.Mat();
cv.absdiff(reference, template, mask);
cv.cvtColor(mask, mask, cv.COLOR_BGR2GRAY, 0);
const mask = pool.add(new cv.Mat());
cv.absdiff(reference, template, mask);
cv.cvtColor(mask, mask, cv.COLOR_BGR2GRAY, 0);
cv.threshold(mask, mask, 128, 255, cv.THRESH_BINARY | cv.THRESH_OTSU);
contours = new cv.MatVector();
hierarchy = new cv.Mat();
cv.findContours(mask, contours, hierarchy, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE);
cv.threshold(mask, mask, 128, 255, cv.THRESH_BINARY | cv.THRESH_OTSU);
const contours = pool.add(new cv.MatVector());
const hierarchy = pool.add(new cv.Mat());
cv.findContours(mask, contours, hierarchy, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE);
for (let i = 0; i < contours.size(); i++) {
const boundingRect = cv.boundingRect(contours.get(i));
highlightRegion(resultMat, boundingRect);
highlightRegion(resultMat, {
x: reference.cols + boundingRect.x,
y: boundingRect.y,
width: boundingRect.width,
height: boundingRect.height,
});
}
result.visualization = await jimpImgFromCvMat(resultMat).getBufferAsync(Jimp.MIME_PNG);
} finally {
try {
bothImages.delete();
resultMat.delete();
mask.delete();
contours.delete();
hierarchy.delete();
} catch (ign) {}
for (let i = 0; i < contours.size(); i++) {
const boundingRect = pool.add(cv.boundingRect(contours.get(i)));
highlightRegion(resultMat, boundingRect);
highlightRegion(resultMat, {
x: reference.cols + boundingRect.x,
y: boundingRect.y,
width: boundingRect.width,
height: boundingRect.height,
});
}
result.visualization = await cvMatToPng(resultMat);
}
return result;
} finally {
try {
template.delete();
reference.delete();
matched.delete();
} catch (ign) {}
pool.drain();
}
}
@@ -508,9 +494,9 @@ async function getImagesSimilarity(img1Data, img2Data, options = {}) {
*
* @param {Buffer} fullImgData The data of the full image packed into a NodeJS buffer
* @param {Buffer} partialImgData The data of the partial image packed into a NodeJS buffer
* @param {OccurrenceOptions?} [options] Set of occurrence calculation options
* @param {OccurrenceOptions} [options] Set of occurrence calculation options
*
* @returns {OccurrenceResult}
* @returns {Promise<OccurrenceResult>}
* @throws {Error} If no occurrences of the partial image can be found in the full image
*/
async function getImageOccurrence(fullImgData, partialImgData, options = {}) {
@@ -524,14 +510,11 @@ async function getImageOccurrence(fullImgData, partialImgData, options = {}) {
method = DEFAULT_MATCHING_METHOD,
} = options;
let fullImg, partialImg, matched;
const pool = new OpenCvAutoreleasePool();
try {
[fullImg, partialImg] = await B.all([
cvMatFromImage(fullImgData),
cvMatFromImage(partialImgData),
]);
matched = new cv.Mat();
const [fullImg, partialImg] = await B.all([cvMatFromImage(fullImgData), cvMatFromImage(partialImgData)])
.map((x) => pool.add(x));
const matched = pool.add(new cv.Mat());
const results = [];
let visualization = null;
@@ -579,7 +562,7 @@ async function getImageOccurrence(fullImgData, partialImgData, options = {}) {
if (_.isEmpty(results)) {
// Below error message, `Cannot find any occurrences` is referenced in find by image
throw new Error(
`Match threshold: ${threshold}. Highest match value ` + `found was ${minMax.maxVal}`
`Match threshold: ${threshold}. Highest match value found was ${minMax.maxVal}`
);
}
} catch (e) {
@@ -591,18 +574,23 @@ async function getImageOccurrence(fullImgData, partialImgData, options = {}) {
}
if (visualize) {
const fullHighlightedImage = fullImg.clone();
const fullHighlightedImage = pool.add(fullImg.clone());
const visualisePromises = [];
for (const result of results) {
const singleHighlightedImage = fullImg.clone();
const singleHighlightedImage = pool.add(fullImg.clone());
highlightRegion(singleHighlightedImage, result.rect);
highlightRegion(fullHighlightedImage, result.rect);
result.visualization = await jimpImgFromCvMat(singleHighlightedImage).getBufferAsync(
Jimp.MIME_PNG
);
visualisePromises.push(cvMatToPng(singleHighlightedImage));
}
let restPngBuffers = [];
[visualization, ...restPngBuffers] = await B.all(
[cvMatToPng(fullHighlightedImage), ...visualisePromises]
);
for (const [result, pngBuffer] of _.zip(results, restPngBuffers)) {
result.visualization = pngBuffer;
}
visualization = await jimpImgFromCvMat(fullHighlightedImage).getBufferAsync(Jimp.MIME_PNG);
}
return {
rect: results[0].rect,
@@ -611,46 +599,51 @@ async function getImageOccurrence(fullImgData, partialImgData, options = {}) {
multiple: results,
};
} finally {
try {
fullImg.delete();
partialImg.delete();
matched.delete();
} catch (ign) {}
pool.drain();
}
}
/**
* Convert an opencv image matrix into a Jimp image object
* Convert an opencv image matrix into a PNG buffer
*
* @param {cv.Mat} mat the image matrix
* @return {Jimp} the Jimp image
* @param {cv.Mat} mat OpenCV image matrix
* @return {Promise<Buffer>} PNG image data buffer
*/
function jimpImgFromCvMat(mat) {
return new Jimp({
width: mat.cols,
height: mat.rows,
data: Buffer.from(mat.data),
});
async function cvMatToPng(mat) {
return await sharp(Buffer.from(mat.data), {
raw: {
width: mat.cols,
height: mat.rows,
channels: 4,
}
})
.png()
.toBuffer();
}
/**
* Take a binary image buffer and return a cv.Mat
* Take an image buffer and return a cv.Mat
*
* @param {Buffer} img the image data buffer
* @return {cv.Mat} the opencv matrix
* @param {Buffer} img image data buffer. All image formats avilable for
* https://www.npmjs.com/package/sharp node library are supported.
* @return {Promise<cv.Mat>} OpenCV image matrix
*/
async function cvMatFromImage(img) {
const jimpImg = await Jimp.read(img);
return cv.matFromImageData(jimpImg.bitmap);
const {data, info} = await sharp(img)
.ensureAlpha()
.raw()
.toBuffer({resolveWithObject: true});
const {width, height} = info;
return cv.matFromImageData({data, width, height});
}
/**
* Filter out match results which have a matched neighbour
*
* @param {Array<Point>} nonZeroMatchResults matrix of image match results
* @param {Point[]} nonZeroMatchResults matrix of image match results
* @param {number} matchNeighbourThreshold The pixel distance within which we
* consider an element being a neighbour of an existing match
* @return {Array<Point>} the filtered array of matched points
* @return {Point[]} the filtered array of matched points
*/
function filterNearMatches(nonZeroMatchResults, matchNeighbourThreshold) {
return nonZeroMatchResults.reduce((acc, element) => {
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/opencv",
"version": "2.0.10",
"version": "2.1.0",
"description": "OpenCV-related helper library",
"keywords": [
"automation",
@@ -43,9 +43,9 @@
"dependencies": {
"@types/bluebird": "3.5.38",
"bluebird": "3.7.2",
"jimp": "0.22.7",
"lodash": "4.17.21",
"opencv-bindings": "4.5.5",
"sharp": "0.32.1",
"source-map-support": "0.5.21"
},
"engines": {
@@ -3,6 +3,14 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.3.14](https://github.com/appium/appium/compare/@appium/plugin-test-support@0.3.13...@appium/plugin-test-support@0.3.14) (2023-05-17)
**Note:** Version bump only for package @appium/plugin-test-support
## [0.3.13](https://github.com/appium/appium/compare/@appium/plugin-test-support@0.3.12...@appium/plugin-test-support@0.3.13) (2023-04-14)
**Note:** Version bump only for package @appium/plugin-test-support
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/plugin-test-support",
"version": "0.3.13",
"version": "0.3.14",
"description": "Testing utilities for Appium plugins",
"keywords": [
"automation",
@@ -41,7 +41,7 @@
"test:unit": "exit 0"
},
"dependencies": {
"@appium/types": "^0.11.0",
"@appium/types": "^0.11.1",
"get-port": "5.1.1",
"log-symbols": "4.1.0",
"source-map-support": "0.5.21",
+33
View File
@@ -3,6 +3,39 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [4.0.1](https://github.com/appium/appium/compare/@appium/support@4.0.0...@appium/support@4.0.1) (2023-05-19)
### Bug Fixes
* **support:** add missing @types/which ([ea42548](https://github.com/appium/appium/commit/ea42548442713f3fefa711719d4ad54dff212f04))
## [4.0.0](https://github.com/appium/appium/compare/@appium/support@3.1.11...@appium/support@4.0.0) (2023-05-17)
### ⚠ BREAKING CHANGES
* **support:** The following methods have been removed from imageUtil module: getJimpImage, base64ToImage, imageToBase64, cropImage
* **support:** The following constants have been removed from imageUtil module: MIME_JPEG, MIME_PNG, MIME_BMP
### Bug Fixes
* **support:** update dependency axios to v1.3.6 ([6692227](https://github.com/appium/appium/commit/66922279b7742a08613f472585a4a1cb70f80683))
* **support:** update dependency axios to v1.4.0 ([91a6bc5](https://github.com/appium/appium/commit/91a6bc5925ab8ffc4ab6d05883900f7d186e49a9))
* **support:** update dependency semver to v7.5.0 ([c568523](https://github.com/appium/appium/commit/c568523e017a8b52bdf5132a48bbfde791de704f))
* **support:** update dependency which to v3.0.1 ([869b7aa](https://github.com/appium/appium/commit/869b7aa4a21fba6612db6f0365cb081cbdf7fac2))
* **types:** update dependency type-fest to v3.10.0 ([3c4d3ac](https://github.com/appium/appium/commit/3c4d3acc09d2ca1ed74dc77c18c62482e4c70239))
* **types:** update dependency type-fest to v3.9.0 ([94a207f](https://github.com/appium/appium/commit/94a207fc9718068f3657c51cc8be0ef682f16b11))
### Code Refactoring
* **support:** Drop jimp and pngjs in favour of sharp ([#18584](https://github.com/appium/appium/issues/18584)) ([1da9de4](https://github.com/appium/appium/commit/1da9de4225ee177fa784bfaab4cf5fb91ec0f4f2))
## [3.1.11](https://github.com/appium/appium/compare/@appium/support@3.1.10...@appium/support@3.1.11) (2023-04-14)
+121 -72
View File
@@ -1,62 +1,112 @@
# appium-support
Utility functions used to support libs used across appium packages.
Utility functions used to support Appium drivers and plugins.
`npm install appium-support`
# Usage in drivers and plugins
Appium, as of version 1.5 is all based on promises, so this module provides promise wrappers for some common operations.
It is recommended to have Appium as a peer dependency in driver and plugin packages.
Add the following line to `peerDependencies` section of your module's `package.json`:
Most notably, we wrap `fs` for file system commands. Note the addition of `hasAccess`.
Also note that `fs.mkdir` doesn't throw an error if the directory already exists, it will just resolve.
```js
"peerDependencies": {
"appium": "^<minimum_server_version>"
}
```
### Methods
Afterwards import it in your code similarly to the below example:
- system.isWindows
- system.isMac
- system.isLinux
- system.isOSWin64
- system.arch
- system.macOsxVersion
```js
import {timing, util} from 'appium/support';
```
- util.hasContent - returns true if input string has content
- util.hasValue - returns true if input value is not undefined and no null
- util.escapeSpace
- util.escapeSpecialChars
- util.localIp
- util.cancellableDelay
- util.multiResolve - multiple path.resolve
- util.unwrapElement - parse an element ID from an element object: e.g.: `{ELEMENT: 123, "element-6066-11e4-a52e-4f735466cecf": 123}` returns `123`
- util.wrapElement - convert an element ID to an element object of the form: e.g.: `123` returns `{ELEMENT: 123, "element-6066-11e4-a52e-4f735466cecf": 123}`
# Usage in helper modules
- *fs.hasAccess* - use this over `fs.access`
- *fs.exists* - calls `fs.hasAccess`
- *fs.rimraf*
- *fs.mkdir* - doesn't throw an error if directory already exists
- *fs.copyFile*
- fs.open
- fs.close
- fs.access
- fs.readFile
- fs.writeFile
- fs.write
- fs.readlink
- fs.chmod
- fs.unlink
- fs.readdir
- fs.stat
- fs.rename
- *fs.md5*
If you want to use this module in a helper library, which is not a driver or a plugin,
then add the following line to `dependencies` section of your module's `package.json`:
- plist.parsePlistFile
- plist.updatePlistFile
```js
"dependencies": {
"@appium/support": "<module_version>"
}
```
- mkdirp
Afterwards import it in your code similarly to the below example:
- logger
```js
import {timing, util} from '@appium/support';
```
- zip.extractAllTo - Extracts contents of a zipfile to a directory
- zip.readEntries - Reads entries (files and directories) of a zipfile
- zip.toInMemoryZip - Converts a directory into a base64 zipfile
### Categories
All utility functions are split into a bunch of different categories. Each category has its own file under the `lib` folder. All utility functions in these files are documented.
#### fs
Most of the functions there are just thin wrappers over utility functions available in [Promises API](https://nodejs.org/api/fs.html#promises-api).
#### env
Several helpers needed by the server to cope with internal dependencies and manifests.
#### console
Wrappers for the command line interface abstraction used by the Appium server.
#### image-util
Utilities to work with images. Use [sharp](https://github.com/lovell/sharp) under the hood.
#### log-internal
Utilities needed for internal Appium log config assistance.
#### logging
See [below](#logger)
#### mjpeg
Helpers needed to implement [MJPEG streaming](https://en.wikipedia.org/wiki/Motion_JPEG#Video_streaming).
#### net
Helpers needed for network interactions, for example, upload and download of files.
#### node
Set of Node.js-specific utility functions needed, for example, to ensure objects immutability or to calculate their sizes.
#### npm
Set of [npm](https://en.wikipedia.org/wiki/Npm_(software))-related helpers.
#### plist
Set of utilities used to read and write data from [plist](https://en.wikipedia.org/wiki/Property_List) files in javascript.
#### process
Helpers for interactions with system processes. These APIs don't support Windows.
#### system
Set of helper functions needed to determine properties of the current operating system.
#### tempdir
Set of helpers that allow interactions with temporary folders.
#### timing
Helpers that allow to measure execution time.
#### util
Miscellaneous utilities.
#### zip
Helpers that allow to work with archives in .ZIP format.
## logger
@@ -85,10 +135,9 @@ Will produce
warn mymodule a warning
```
### Environment variables
There are two environment variable flags that affect the way `appium-base-driver` `logger` works.
There are two environment variable flags that affect the way `logger` works.
`_TESTING`
@@ -108,43 +157,43 @@ There are two environment variable flags that affect the way `appium-base-driver
`log[level](message)`
- logs to `level`
```js
import { logger } from 'appium-support';
let log = logger.getLogger('mymodule');
```js
import { logger } from 'appium-support';
let log = logger.getLogger('mymodule');
log.info('hi!');
// => info mymodule hi!
```
log.info('hi!');
// => info mymodule hi!
```
`log.unwrap()`
- retrieves the underlying [npmlog](https://github.com/npm/npmlog) object, in order to manage how logging is done at a low level (e.g., changing output streams, retrieving an array of messages, adding log levels, etc.).
```js
import { getLogger } from 'appium-base-driver';
let log = getLogger('mymodule');
```js
import { getLogger } from 'appium-base-driver';
let log = getLogger('mymodule');
log.info('hi!');
log.info('hi!');
let npmlogger = log.unwrap();
let npmlogger = log.unwrap();
// any `npmlog` methods
let logs = npmlogger.record;
// logs === [ { id: 0, level: 'info', prefix: 'mymodule', message: 'hi!', messageRaw: [ 'hi!' ] }]
```
// any `npmlog` methods
let logs = npmlogger.record;
// logs === [ { id: 0, level: 'info', prefix: 'mymodule', message: 'hi!', messageRaw: [ 'hi!' ] }]
```
`log.errorAndThrow(error)`
- logs the error passed in, at `error` level, and then throws the error. If the error passed in is not an instance of [Error](https://nodejs.org/api/errors.html#errors_class_error) (either directly, or a subclass of `Error`) it will be wrapped in a generic `Error` object.
```js
import { getLogger } from 'appium-base-driver';
let log = getLogger('mymodule');
```js
import { getLogger } from 'appium-base-driver';
let log = getLogger('mymodule');
// previously there would be two lines
log.error('This is an error');
throw new Error('This is an error');
// previously there would be two lines
log.error('This is an error');
throw new Error('This is an error');
// now is compacted
log.errorAndThrow('This is an error');
```
// now is compacted
log.errorAndThrow('This is an error');
```
+14 -8
View File
@@ -10,6 +10,7 @@ import {
promises as fsPromises,
read,
write,
rmSync,
open,
} from 'fs';
import glob from 'glob';
@@ -20,7 +21,6 @@ import ncp from 'ncp';
import path from 'path';
import pkgDir from 'pkg-dir';
import readPkg from 'read-pkg';
import rimrafIdx from 'rimraf';
import sanitize from 'sanitize-filename';
import which from 'which';
import log from './logger';
@@ -84,17 +84,23 @@ const fs = {
/**
* Remove a directory and all its contents, recursively
* @todo Replace with `rm()` from `fs.promises` when Node.js v12 support is dropped.
* @param {PathLike} filepath
* @returns Promise<void>
* @see https://nodejs.org/api/fs.html#fspromisesrmpath-options
*/
rimraf: /** @type {(dirpath: string, opts?: import('rimraf').Options) => Promise<void>} */ (
B.promisify(rimrafIdx)
),
async rimraf(filepath) {
return await fsPromises.rm(filepath, {recursive: true, force: true});
},
/**
* Alias of {@linkcode rimrafIdx.sync}
* @todo Replace with `rmSync()` from `fs` when Node.js v12 support is dropped.
* Remove a directory and all its contents, recursively in sync
* @param {PathLike} filepath
* @returns undefined
* @see https://nodejs.org/api/fs.html#fsrmsyncpath-options
*/
rimrafSync: rimrafIdx.sync,
rimrafSync(filepath) {
return rmSync(filepath, {recursive: true, force: true});
},
/**
* Like Node.js' `fsPromises.mkdir()`, but will _not_ reject if the directory already exists.
+7 -171
View File
@@ -1,178 +1,14 @@
import _ from 'lodash';
import Jimp from 'jimp';
import {Buffer} from 'buffer';
import {PNG} from 'pngjs';
import B from 'bluebird';
const BYTES_IN_PIXEL_BLOCK = 4;
const SCANLINE_FILTER_METHOD = 4;
const {MIME_JPEG, MIME_PNG, MIME_BMP} = Jimp;
/**
* Utility function to get a Jimp image object from buffer or base64 data. Jimp
* is a great library however it does IO in the constructor so it's not
* convenient for our async/await model.
*
* @param {Buffer|string} data - binary image buffer or base64-encoded image
* string
* @returns {Promise<AppiumJimp>} - the jimp image object
*/
async function getJimpImage(data) {
return await new B((resolve, reject) => {
if (!_.isString(data) && !Buffer.isBuffer(data)) {
return reject(new Error('Must initialize jimp object with string or buffer'));
}
const truncatedData = _.truncate(
Buffer.isBuffer(data) ? data.toString('utf8') : data,
{length: 50}
);
new Jimp(
// if data is a string, assume it is a base64-encoded image
Buffer.isBuffer(data) ? data : Buffer.from(String(data), 'base64'),
/**
* @param {Error?} err
* @param {AppiumJimp} imgObj
*/
(err, imgObj) => {
if (err) {
err.message = `The argument must be a valid base64-encoded image payload. ` +
`'${truncatedData}' was passed instead. Original error: ${err.message}`;
return reject(err);
}
if (!imgObj) {
return reject(new Error('Could not create jimp image from that data'));
}
imgObj.getBuffer = B.promisify(imgObj.getBuffer.bind(imgObj), {
context: imgObj,
});
resolve(imgObj);
}
);
});
}
import sharp from 'sharp';
/**
* Crop the image by given rectangle (use base64 string as input and output)
*
* @param {string} base64Image The string with base64 encoded image
* @param {Region} rect The selected region of image
* @param {string} base64Image The string with base64 encoded image.
* Supports all image formats natively supported by Sharp library.
* @param {sharp.Region} rect The selected region of image
* @return {Promise<string>} base64 encoded string of cropped image
*/
async function cropBase64Image(base64Image, rect) {
const image = await base64ToImage(base64Image);
cropImage(image, rect);
return await imageToBase64(image);
export async function cropBase64Image(base64Image, rect) {
const buf = await sharp(Buffer.from(base64Image, 'base64')).extract(rect).toBuffer();
return buf.toString('base64');
}
/**
* Create a pngjs image from given base64 image
*
* @param {string} base64Image The string with base64 encoded image
* @return {Promise<PNG>} The image object
*/
async function base64ToImage(base64Image) {
const imageBuffer = Buffer.from(base64Image, 'base64');
return await new B((resolve, reject) => {
const image = new PNG({filterType: SCANLINE_FILTER_METHOD});
image.parse(imageBuffer, (err, image) => {
// eslint-disable-line promise/prefer-await-to-callbacks
if (err) {
return reject(err);
}
resolve(image);
});
});
}
/**
* Create a base64 string for given image object
*
* @param {PNG} image The image object
* @return {Promise<string>} The string with base64 encoded image
*/
async function imageToBase64(image) {
return await new B((resolve, reject) => {
const chunks = [];
image
.pack()
.on('data', (chunk) => chunks.push(chunk))
.on('end', () => {
resolve(Buffer.concat(chunks).toString('base64'));
})
.on('error', (err) => {
// eslint-disable-line promise/prefer-await-to-callbacks
reject(err);
});
});
}
/**
* Crop the image by given rectangle
*
* @param {PNG} image The image to mutate by cropping
* @param {Region} rect The selected region of image
*/
function cropImage(image, rect) {
const imageRect = {width: image.width, height: image.height};
const interRect = getRectIntersection(rect, imageRect);
if (interRect.width < rect.width || interRect.height < rect.height) {
throw new Error(
`Cannot crop ${JSON.stringify(rect)} from ${JSON.stringify(
imageRect
)} because the intersection between them was not the size of the rect`
);
}
const firstVerticalPixel = interRect.top;
const lastVerticalPixel = interRect.top + interRect.height;
const firstHorizontalPixel = interRect.left;
const lastHorizontalPixel = interRect.left + interRect.width;
const croppedArray = [];
for (let y = firstVerticalPixel; y < lastVerticalPixel; y++) {
for (let x = firstHorizontalPixel; x < lastHorizontalPixel; x++) {
const firstByteIdxInPixelBlock = (imageRect.width * y + x) << 2;
for (let byteIdx = 0; byteIdx < BYTES_IN_PIXEL_BLOCK; byteIdx++) {
croppedArray.push(image.data[firstByteIdxInPixelBlock + byteIdx]);
}
}
}
image.data = Buffer.from(croppedArray);
image.width = interRect.width;
image.height = interRect.height;
return image;
}
function getRectIntersection(rect, imageSize) {
const left = rect.left >= imageSize.width ? imageSize.width : rect.left;
const top = rect.top >= imageSize.height ? imageSize.height : rect.top;
const width = imageSize.width >= left + rect.width ? rect.width : imageSize.width - left;
const height = imageSize.height >= top + rect.height ? rect.height : imageSize.height - top;
return {left, top, width, height};
}
export {
cropBase64Image,
base64ToImage,
imageToBase64,
cropImage,
getJimpImage,
MIME_JPEG,
MIME_PNG,
MIME_BMP,
};
/**
* @typedef {Omit<Jimp,'getBuffer'> & {getBuffer: Jimp['getBufferAsync']}} AppiumJimp
*/
/**
* @typedef Region
* @property {number} left
* @property {number} top
* @property {number} width
* @property {number} height
*/
+2 -3
View File
@@ -1,7 +1,7 @@
import _ from 'lodash';
import log from './logger';
import B from 'bluebird';
import {getJimpImage, MIME_PNG} from './image-util';
import sharp from 'sharp';
import {Writable} from 'stream';
import {requirePackage} from './node';
import axios from 'axios';
@@ -77,8 +77,7 @@ class MJpegStream extends Writable {
}
try {
const jpg = await getJimpImage(lastChunk);
return await jpg.getBuffer(MIME_PNG);
return await sharp(lastChunk).png().toBuffer();
} catch (e) {
return null;
}
+8 -11
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/support",
"version": "3.1.11",
"version": "4.0.1",
"description": "Support libs used across appium packages",
"keywords": [
"automation",
@@ -42,7 +42,7 @@
},
"dependencies": {
"@appium/tsconfig": "^0.3.0",
"@appium/types": "^0.11.0",
"@appium/types": "^0.11.1",
"@colors/colors": "1.5.0",
"@types/archiver": "5.3.2",
"@types/base64-stream": "1.0.2",
@@ -55,15 +55,14 @@
"@types/ncp": "2.0.5",
"@types/npmlog": "4.1.4",
"@types/pluralize": "0.0.29",
"@types/pngjs": "6.0.1",
"@types/rimraf": "3.0.2",
"@types/semver": "7.3.13",
"@types/shell-quote": "1.7.1",
"@types/supports-color": "8.1.1",
"@types/teen_process": "2.0.0",
"@types/uuid": "9.0.1",
"@types/which": "3.0.0",
"archiver": "5.3.1",
"axios": "1.3.5",
"axios": "1.4.0",
"base64-stream": "1.0.0",
"bluebird": "3.7.2",
"bplist-creator": "0.1.1",
@@ -71,7 +70,6 @@
"form-data": "4.0.0",
"get-stream": "6.0.1",
"glob": "8.1.0",
"jimp": "0.22.7",
"jsftp": "2.1.3",
"klaw": "4.1.0",
"lockfile": "1.0.4",
@@ -85,19 +83,18 @@
"pkg-dir": "5.0.0",
"plist": "3.0.6",
"pluralize": "8.0.0",
"pngjs": "7.0.0",
"read-pkg": "5.2.0",
"resolve-from": "5.0.0",
"rimraf": "3.0.2",
"sanitize-filename": "1.6.3",
"semver": "7.4.0",
"semver": "7.5.0",
"sharp": "0.32.1",
"shell-quote": "1.8.1",
"source-map-support": "0.5.21",
"supports-color": "8.1.1",
"teen_process": "2.0.2",
"type-fest": "3.8.0",
"type-fest": "3.10.0",
"uuid": "9.0.0",
"which": "3.0.0",
"which": "3.0.1",
"yauzl": "2.10.0"
},
"engines": {
File diff suppressed because one or more lines are too long
@@ -1,12 +1,6 @@
import {
base64ToImage,
imageToBase64,
cropImage,
getJimpImage,
MIME_PNG,
} from '../../lib/image-util';
import {cropBase64Image} from '../../lib/image-util';
import path from 'path';
import _ from 'lodash';
import sharp from 'sharp';
import {fs} from '../../lib';
const FIXTURES_ROOT = path.resolve(__dirname, 'fixture', 'images');
@@ -18,63 +12,25 @@ async function getImage(name) {
describe('image-util', function () {
describe('cropBase64Image', function () {
let originalImage = null;
let originalImageB64 = null;
before(async function () {
const originalImage64 = await getImage('full-image.b64');
originalImage = await base64ToImage(originalImage64);
// verify original image size, to be sure that original image is correct
originalImage.width.should.be.equal(640, 'unexpected width');
originalImage.height.should.be.equal(1136, 'unexpected height');
originalImageB64 = await getImage('full-image.b64');
});
it('should verify that an image is cropped correctly', async function () {
const croppedImage = await cropImage(originalImage, {
const croppedImageB64 = await cropBase64Image(originalImageB64, {
left: 35,
top: 107,
width: 323,
height: 485,
});
const croppedImage = sharp(Buffer.from(croppedImageB64, 'base64'));
const {width, height} = await croppedImage.metadata();
// verify cropped image size, it should be less than original image according to crop region
croppedImage.width.should.be.equal(323, 'unexpected width');
croppedImage.height.should.be.equal(485, 'unexpected height');
// verify that image cropped, compare base64 representation
const croppedImageShouldBe = await getImage('cropped-image.b64');
const croppedImage64 = await imageToBase64(croppedImage);
croppedImage64.should.be.equal(croppedImageShouldBe);
});
});
describe('Jimp helpers', function () {
it('should get a jimp object using image buffer', async function () {
const base64Image = await getImage('cropped-image.b64');
const imageBuffer = Buffer.from(base64Image, 'base64');
const jimpImg = await getJimpImage(imageBuffer);
jimpImg.hash().should.eql('80000000000');
jimpImg.bitmap.height.should.eql(485);
jimpImg.bitmap.width.should.eql(323);
});
it('should get a jimp object using b64 string', async function () {
const base64Image = await getImage('cropped-image.b64');
const jimpImg = await getJimpImage(base64Image);
jimpImg.hash().should.eql('80000000000');
jimpImg.bitmap.height.should.eql(485);
jimpImg.bitmap.width.should.eql(323);
});
it('should error with incorrect data type', async function () {
await getJimpImage(1234).should.eventually.be.rejectedWith(/string or buffer/);
});
it('should error with incorrect image data', async function () {
await getJimpImage('foo').should.eventually.be.rejectedWith(/Could not find MIME for Buffer/);
});
it('should get an image buffer via the overridden getBuffer method', async function () {
const base64Image = await getImage('cropped-image.b64');
const jimpImg = await getJimpImage(base64Image);
const buf = await jimpImg.getBuffer(MIME_PNG);
_.isBuffer(buf).should.be.true;
width.should.be.equal(323, 'unexpected width');
height.should.be.equal(485, 'unexpected height');
});
});
});
@@ -89,7 +89,6 @@ describe('MJpeg Stream (e2e)', function () {
const png = await stream.lastChunkPNGBase64();
png.should.be.a('string');
png.indexOf('iVBOR').should.eql(0);
png.length.should.be.above(400);
// now stop the stream and wait some more then assert no new data has come in
stream.stop();
+17
View File
@@ -3,6 +3,23 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [3.0.15](https://github.com/appium/appium/compare/@appium/test-support@3.0.14...@appium/test-support@3.0.15) (2023-05-19)
**Note:** Version bump only for package @appium/test-support
## [3.0.14](https://github.com/appium/appium/compare/@appium/test-support@3.0.13...@appium/test-support@3.0.14) (2023-05-17)
### Bug Fixes
* **test-support:** update dependency sinon to v15.0.4 ([9ad7279](https://github.com/appium/appium/commit/9ad7279fa70d7198a8d02f0f206397cb4a830e1d))
## [3.0.13](https://github.com/appium/appium/compare/@appium/test-support@3.0.12...@appium/test-support@3.0.13) (2023-04-14)
**Note:** Version bump only for package @appium/test-support
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/test-support",
"version": "3.0.13",
"version": "3.0.15",
"description": "A collection of test utilities used across Appium packages",
"keywords": [
"automation",
@@ -45,12 +45,12 @@
"test:unit": "mocha \"./test/unit/**/*.spec.js\""
},
"dependencies": {
"@appium/support": "^3.1.11",
"@appium/support": "^4.0.1",
"@colors/colors": "1.5.0",
"bluebird": "3.7.2",
"lodash": "4.17.21",
"loud-rejection": "2.2.0",
"sinon": "15.0.3",
"sinon": "15.0.4",
"source-map-support": "0.5.21"
},
"engines": {
@@ -3,6 +3,17 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.6.4](https://github.com/appium/appium/compare/@appium/typedoc-plugin-appium@0.6.3...@appium/typedoc-plugin-appium@0.6.4) (2023-05-17)
### Bug Fixes
* **typedoc-plugin-appium:** don't prefix execute methods with "Script: " ([#18618](https://github.com/appium/appium/issues/18618)) ([40e5199](https://github.com/appium/appium/commit/40e5199caa0bb6166af5af714f03d26fc5936b97))
* **types:** update dependency type-fest to v3.10.0 ([3c4d3ac](https://github.com/appium/appium/commit/3c4d3acc09d2ca1ed74dc77c18c62482e4c70239))
* **types:** update dependency type-fest to v3.9.0 ([94a207f](https://github.com/appium/appium/commit/94a207fc9718068f3657c51cc8be0ef682f16b11))
## [0.6.3](https://github.com/appium/appium/compare/@appium/typedoc-plugin-appium@0.6.2...@appium/typedoc-plugin-appium@0.6.3) (2023-04-18)
+38 -20
View File
@@ -6,32 +6,34 @@
This package leverages [TypeDoc](https://typedoc.org) to generate command documentation (HTTP endpoints, payload information, etc.) for Appium v2+ drivers and plugins.
## Important Note
_If you are an Appium extension author and just want to build HTML docs_, **don't install this directly**--instead, install [`@appium/docutils`](https://github.com/appium/appium/tree/master/packages/docutils), which is a higher-level package that configures everything for you.
If you _only_ want to build markdown docs for your extension, then you can use this package directly.
## Installation
`npm` v8+ is required to install this package.
```bash
npm install appium@next typedoc @appium/typedoc-plugin-appium --save-dev
npm install @appium/typedoc-plugin-appium --save-dev
```
`typedoc` and `appium` are peer dependencies of this package. Newer versions of `npm` will install these automatically (if possible).
The above command installs the plugin as well as all necessary peer dependencies. See [`package.json`](https://github.com/appium/appium/blob/master/packages/docutils/package.json) for the full list of dependencies.
## Usage
TypeDoc is configured via a `typedoc.json` or `typedoc.js` file ([read the docs](https://typedoc.org/guides/options/) for more information).
An Appium extension author wishing to generate documentation for their extension will need to create a `typedoc.json`. At minimum, it should contain:
> TODO: The plugin should be able to detect the entry points automatically, so this should not be necessary
An Appium extension author wishing to generate markdown documentation for their extension will need to create a `typedoc.json`. At minimum, it should contain:
```json
{
"entryPointStrategy": "packages",
"entryPoints": [
"./node_modules/appium",
"./node_modules/@appium/base-driver",
"./node_modules/@appium/types",
"."
],
"entryPoints": ["."],
"name": "<name of extension>",
"theme": "appium",
"out": "<path to output directory>"
}
```
@@ -42,21 +44,37 @@ Once this file is created, you can run `typedoc` to generate the documentation,
This plugin supports all of the options from [typedoc-plugin-markdown](https://npm.im/typedoc-plugin-markdown), as well as the following:
### `commandsDir`
### `outputModules`
> TODO
### `forceBreadcrumbs`
> TODO
`boolean` - Output module, class, interface, and other type information (the usual TypeDoc output) in addition to command documentation. This is needed for full documentation of types. _Default value: `true`_
### `outputBuiltinCommands`
> TODO
`boolean` - Outputs _all_ commands and types from Appium builtins--not just your extension. This is intended to be used by Appium itself. _Default value: `false`_
### `outputModules`
### `packageTitles`
> TODO
`Array<{name: string, title: string}>`: An array of objects containing module name `name` and display name `title`. By default, the module name is used for the title; use this to override that behavior. _Default value: `undefined`_
### `commandsDir`
`string` - The name of the "commands" directory relative to the TypeDoc output directory (`out`). _Default value: `commands`_
### `forceBreadcrumbs`
`boolean` - Forces breadcrumbs to be output; overrides `hideBreadcrumbs` from `typedoc-plugin-markdown`. _Default value: `false`_
## Development
This packages uses snapshot tests to assert the generated markdown is correct. If you have made changes which affect the plugin's output, you will need to update the snapshots.
To update the snapshots, execute:
```bash
UPDATE_SNAPSHOT=1 npm test
```
This will (likely) modify the snapshots in your working copy, so you will then need to commit them.
## License
@@ -1,6 +1,6 @@
{{#unless hasOwnDocument}}
### {{#ifNamedAnchors}} <a id="{{anchor}}" name="{{this.anchor}}"></a> {{/ifNamedAnchors}}Script: `{{name}}`
### {{#ifNamedAnchors}} <a id="{{anchor}}" name="{{this.anchor}}"></a> {{/ifNamedAnchors}}`{{name}}`
{{#if hasComment}}
{{> comment}}
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/typedoc-plugin-appium",
"version": "0.6.3",
"version": "0.6.4",
"description": "TypeDoc plugin for Appium & its extensions",
"homepage": "https://appium.io",
"license": "Apache-2.0",
@@ -63,7 +63,7 @@
"handlebars": "4.7.7",
"lodash": "4.17.21",
"pluralize": "8.0.0",
"type-fest": "3.8.0"
"type-fest": "3.10.0"
},
"gitHead": "8480a85ce2fa466360e0fb1a7f66628331907f02"
}
@@ -1113,7 +1113,7 @@ Use explicit session management commands instead
## Execute Methods
### Script: \`fake: addition\`
### \`fake: addition\`
#### Route
@@ -1131,7 +1131,7 @@ Use explicit session management commands instead
\`number\`
### Script: \`fake: getDeprecatedCommandsCalled\`
### \`fake: getDeprecatedCommandsCalled\`
This is a command that will return a list of deprecated command names called
@@ -1145,7 +1145,7 @@ This is a command that will return a list of deprecated command names called
\`string\`[]
### Script: \`fake: getThing\`
### \`fake: getThing\`
Gets a thing (a fake thing)
@@ -1159,7 +1159,7 @@ Gets a thing (a fake thing)
\`\`null\`\` \\\\| \`Thing\`
### Script: \`fake: setThing\`
### \`fake: setThing\`
Set the 'thing' value (so that it can be retrieved later)
+10
View File
@@ -3,6 +3,16 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.11.1](https://github.com/appium/appium/compare/@appium/types@0.11.0...@appium/types@0.11.1) (2023-05-17)
### Bug Fixes
* **types:** update dependency type-fest to v3.10.0 ([3c4d3ac](https://github.com/appium/appium/commit/3c4d3acc09d2ca1ed74dc77c18c62482e4c70239))
* **types:** update dependency type-fest to v3.9.0 ([94a207f](https://github.com/appium/appium/commit/94a207fc9718068f3657c51cc8be0ef682f16b11))
## [0.11.0](https://github.com/appium/appium/compare/@appium/types@0.10.4...@appium/types@0.11.0) (2023-04-14)
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@appium/types",
"version": "0.11.0",
"version": "0.11.1",
"description": "Various type declarations used across Appium",
"keywords": [
"automation",
@@ -43,7 +43,7 @@
"@types/express": "4.17.17",
"@types/npmlog": "4.1.4",
"@types/ws": "8.5.4",
"type-fest": "3.8.0"
"type-fest": "3.10.0"
},
"engines": {
"node": "^14.17.0 || ^16.13.0 || >=18.0.0",