- flip if/else conditions in various places
- add export `SERVER_SUBCOMMAND` constant to `parser.js` (which is the string `server`)
- add `SERVER_PROP` constant to `schema.js` (which is also the string `server` but means a difft thing)
- more constants in `ext-config-io.js`
- add some more types in `ArgParser` constructor; simplify
- add constant regexp in `schema-args.js`
- use more lodash in `grid-register.js`
- simplify `_.includes()` call in `main.js`
- simplify error-handling in `readExtensionSchema()` of `schema.js`; fix docstring; remove associated tests
- fix a sanity check in `schema-specs.js`
Summary: This PR adds support for configuration files (e.g., `.appiumrc.json`) and schema definitions. Command-line options _and_ configuration is now derived from the schema(s), for Appium _and_ its extensions (both plugins and drivers).
## Dependencies
### Development
- Added `through2` for a custom gulp task to convert `appium/lib/appium-config-schema.js` to a JSON file
- Added `json-schema-to-typescript` for converting aforementioned JSON file to a `.d.ts` file (which gets published)
### Production (applies to `appium` only)
- Added `libconfig` for multi-format config file loading, which is a smaller drop-in replacement for `cosmiconfig`
- Added `ajv` for schema support and validation
- Added `ajv-formats` to understand optional value formats (e.g., `ipv4`)
- Added `@sidvind/better-ajv-errors` which provides nice error output on the CLI if the config file is invalid
- Added `@oclif/errors` for better error output on the CLI if _arguments_ are invalid
## New Modules of Note
- `lib/schema.js` consumes `ajv` to create and modify a schema
- It exposes a handful of public API functions, so that consumers can query the schemas without actually running Appium.
- The flow is basically this: read the base schema (`appium-config-schema.js`). When extensions load and have schemas, add those to a pot. When `finalizeSchema()` is called, throw the base schema into the pot as well, stir, and add to singleton `ajv` instance. Now we can validate against the stew.
- The schema for an extension is read and registered via `readExtensionSchema()`
- Provides a way to get at default (for showing those "non-default" options) and `flattenSchema()` which squishes the finalized schema into a single `Record` key/value object, for use by the CLI. This may make sense to move into `schema-args.js`
- Provides access to ajv-and-JSON-schema-defined _formatters_ (which are basically just validation functions). For example, if we want any string, it's just a `string`. But if we want the string to match a regexp, that's a formatter. (e.g., the `hostname` type)
- lib/config-file.js` consumes `lilconfig` to read config files, validate them against the schema, and provide the result as CLI args
- Handles conversion between kebab-case and camelCase
- Contains `formatErrors()` which will be called after validation of a config file fails, which shows _exactly_ where the config file is wrong (usually).
- `lib/cli/schema-args.js` which serves as an adapter between the schema and `argparse`
- Translates between JSON schema and `argparse`, which...well, I did what I could. It's pretty close.
- Wraps the formatters as mentioned above, and provides custom ones.
- We cannot, unfortunately, wholly use schema validation against CLI args. The flag names are different, the structure is different, there's no actual _file_ to compare against, and `argparse` does not help. We _can_ use anything `argparse` natively provides, in addition to `ajv` formatters, and any custom ones (e.g., allowable number ranges).
- Will likely end up adding more formatters corresponding to various JSON schema features later. We _may_ be able to use _more_ of Ajv to do this (e.g., reuse its internal validators, if they are public APIs).
- `lib/appium-config-schema.js` which is the root schema. Extensions add to this before it is registered & compiled by `ajv`.
- The basic shape is that there are three top-level props, `server`, `driver`, and `plugin`. `server` contains all of the server options as you'd expect, and `driver` and `plugin` will be filled in by extensions. Each will have properties corresponding to the extension name. You can find examples in `sample-code` and the test fixtures
- There's a gulp task that takes this file, rewrites it as JSON, then pipes it into another tool which outputs a `.d.ts` file, nicely described using all of the metadata from the schema. This will be useful for mainly contributors, extension authors, and consumers of Appium (though not end-users).
- `lib/ext-config-io.js`
- A new module which handles reading/writing of YAML extension config files only.
- This class is a singleton.
- Some logic pulled out of `extension-config.js` and into here
- Previously, we would read/write this file multiple times (once per subclass of `ExtensionConfig`)
- This module uses a naive dirty-tracker to persist the extension data automatically.
- Subclasses of `ExtensionConfig` only have access to their "portion" of the config file; drivers can't see plugins and vice-versa.
- `DRIVER_TYPE` and `PLUGIN_TYPE` originate here, mainly to avoid a circular dependency
## Modifications
- The single source of truth for the CLI _and_ config file options is the schema, with the caveats that:
- Command-line arguments (for `appium server` are effectively a flat list, but the schema/config file is not.
- Extension-specific configuration lives in an extension-type-specific property (`driver` or `plugin`) in the config file
- Under each (`driver` or `plugin`), each extension may have a configuration section corresponding to the extension name
- The shape of the extension-specific options _can_ be defined by a schema (and referenced in the `appium.schema` prop of the extension's `package.json`)
- If an extension does not provide a schema, no validation will occur on any options found here -- TODO needs tests
- This does _not_ affect _CLI commands_ added by extensions; only options for `server`.
- The `--appium-home` flag is now removed, as we can't find extensions without `APPIUM_HOME`, which means we can't parse arguments (because extensions may have added some).
- `--plugin-args` and `--driver-args` are removed. These are replaced by individual flags for each argument, defined by the extension in its schema, e.g., `--driver-fake-host=127.0.0.1`
- Extensions can still define argument "constraints" using `argConstraints`, but this is considered deprecated
- `lib/main.js`
- Now exports a few public APIs
- split into functions `init` and `main`. `init` is intended to be run via a consumer, which does what the old `main` did up to starting the server. This bisection is not perfect, as `init` still does some CLI-related things; it needs to be further broken up.
- `driverConfig.read()` and `pluginConfig.read()` are now called in `lib/cli/parser.js`.
- Merges the CLI args, config file args, and defaults, in that order of precedence.
- `lib/grid-register.js`
- Supports embedded configuration instead of just a path to a JSON file.
- Not clear to me if `nodeconfig` is a thing we need to keep?
- `lib/extension-config.js`
- Given `APPIUM_HOME` is where we keep the extensions, this file now defines `APPIUM_HOME`. It is exported from the `appium` package.
- Add `getSchemaProblems()` method, which will read/validate/register the schema for an extension, if one exists.
- Added many type annotations
- `read()` and `write()` now delegate to the singleton `ExtConfigIO` instnace
- Re-exports `DRIVER_TYPE` and `PLUGIN_TYPE`
- `lib/driver-config.js`
- **Fix** the case where two drivers attempt to use the same `automationName`. It appears this was intended, but did not work correctly. This involved overloading `read()`.
- `lib/plugin-config.js`
- Literally nothing, which is neat.
- `lib/config.js`
- Remove unused validations (now occur via schema validation or are extension-specific)
- Rename `getNonDefaultArgs` to `getNonDefaultServerArgs` to be more specific
- Rewrote `getNonDefaultServerArgs`, which became much more complicated due to adding a config file and schema defaults to the mix. Flexed my lodash-fu, which is still kind of crappy because I'm better with `lodas/fp`.
- `lib/cli/parser.js`
- Created a wrapper for `argparse`: class `ArgParser`, which replaces the monkeypatching we were doing here.
- This class also stores extension-specific args in a namespace. For example, `--driver-foo-bar=baz` would normally be stored in prop `driverFooBar`, but this moves it into `driver: {foo: {bar: 'baz'}}`, which will be provided to its associated extension
- `addExtensionsToParser` renamed to `addExtensionCommandsToParser` which is more specific.
- Removed `getDefaultServerArgs` since those now come from the schema.
- Removed "shared args" stuff
- `getParser` is now async, and constructs an `ArgParser`. By necessity, the extension config YAML is read here (which is still wonky because we have to call `read()` twice; it only actually ends up reading once)
- `getParser` also _finalizes_ the schema, which means "katamari-up the base schema and all extension schemas, then tell ajv about the result"
- `lib/cli/parser-helpers.js`
- Allow security features to just be an array (as allowed in schema)
- TODO: needs test
- `lib/cli/extension.js`
- Use `APPIUM_HOME` exported from `lib/extension-config` instead of `args.appiumHome`
- Avoids re-instantiating `ExtensionConfig` subclasses if not needed
- `lib/cli/args.js`
- Remove all server argument definitions in lieu of schema; leave a few that do not make sense in a config file (like `--config`, which determines the location of the config file!)
- Server args are now pulled from the schema via a call to `toParserArgs()` with overrides for various validation functions (which are confusingly "types" in `argparse` parlance). Overrides can probably be removed or relocated to `schema-args`
- Instantiates the `DriverConfig` and `PluginConfig` instances, which are exported. These might want to move somewhere else
- Tests
- Added unit tests for `extension-config`, `grid-register` which were missing
- Added unit tests for new `schema` and `schema-args` and `config-file` modules
- Added e2e test for config file handling
- Other necessary modifications
- Fix path to `fake-driver` in `driver-e2e-specs`
- Removed stuff from `parser-specs` which no longer applies
## Other
- Added `scripts/generate-schema-declarations.js` which uses `json-schema-to-typescript` to create `appium-config.d.ts`
- Added config file samples in `.json` and `.yaml` formats
- Added custom gulp task to `appium/gulpfile.js` to generate schema in JSON format, since its "single source of truth" is actually just a big JS object
- Moved `appium` CLI-related tests into `test/cli`
- Added some declarations into `packages/appium/types`; the schema-related one is generated by the script mentioend above, but `types.d.ts` is just some stuff that was not expressible in a docstring
- `postTranspile` would only run from certain _other_ tasks (like `test`). This seemed incorrect, so now it runs after `transpile`, regardless of what task (or human) runs `transpile`.
- Added a `dev` task, which is like `watch`, except does not run tests (only transpiles)
- add `jsconfig.json` for the TS language server to understand the project
- update `.wallaby.js` to transpile fixtures if necessary; run on save to reduce lag
- update single-file Mocha debug launch config
- add `test` env in `babel.config.json` which is used by the former
- add some handy vscode tasks
- add a bunch of types
- remove unused `babel-eslint` (we use `@babel/eslint-parser`)
- bump `ecmaVersion` in `eslint-config-appium`
- remove cruft from babel configs (`shippedProposals: true` goes a long way)
- enable `sinon-chai` globally in tests
- add `rewiremock` for module-level mocking (sinon cannot do this itself, but they work in tandem)
* feat(base-driver): add type definition for optional commands
* update type definition
* reuse Rect in Size, Position type
* fix: actions is unwrapped
- `base-driver` gets a new `helpers` module with a `getTestPort` function; this is now used throughout the tests
- `fake-driver` can't use random ports because it depends on `appium-base-driver`'s test suite, and that is _not_ the current appium-2.0-compatible `@appium/base-driver`
Christopher Hiller