mirror of
https://github.com/SimoneAvogadro/android-reverse-engineering-skill.git
synced 2026-10-29 04:36:48 +08:00
Simone AvogadroandClaude Sonnet 5.5
b3fe549259
The per-vendor packer/RASP rules were selected from APKiD's rule set (GPL-3.0 or commercial); they are removed together with the [A:<rule>] source map and the vendor names. What stays: PairIP (seen in protected apps), the Play Licensing library and the Play Integrity / SafetyNet clients (public Google package names). New generic check (check_packed_code / Test-PackedCode, sh + ps1): components declared in AndroidManifest.xml (Application, activities, services, receivers, providers) that have no class in the decoded smali. >= 3 declared and >= 3 missing; >= 50% missing = medium, >= 80% = high; activity-alias and android.* are ignored. The protector is not named. Coverage trade-off: protections that hide nothing (e.g. a native anti-tamper SDK) are no longer reported, and the 'none' message now says so. Validated on the 21 decoded trees of the corpus: output identical to before (af1945 signature-vm, bulldozer license, the rest none, no false positive; a clean app misses 0-5 of 78-179 declared classes) and on a synthetic packed tree; sh == ps1 through Windows PowerShell 5.1. New tests/unit/t_protection.sh (26 checks); tests/run.sh quick passes. Docs updated (CLAUDE.md, SKILL.md, neutralize.md, neutralization-guide.md). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKSiYpKT5YKAMowGEgUwAw
tests/ — development test harness
Dev-only: nothing here ships with the plugin (the plugin lives under plugins/).
It guards the "registry as single source of truth" work: registry lint, golden
byte-identity of the neutralizer outputs, the consumed-registry snapshot, and (as the
features land) reader/audit unit tests, sh/ps1 parity and corpus ground truth.
tests/run.sh # quick: lint + unit (+ parity on fixtures when PowerShell is available)
tests/run.sh full # + golden, snapshot, corpus, bash 3.2, perf (~50 min with PowerShell + corpus)
tests/run.sh lint golden # selected suites; single tests by name: tests/run.sh t_lint
tests/run.sh --expect-pass lint golden snapshot # SKIP counts as FAIL for these
One line per test: PASS, FAIL, SKIP (tool or corpus missing) or SKIP(requires)
(the feature the test covers is not in the tree yet: see the # requires: header of each
test). Directives (# requires:, # awk-matrix) are read only from the leading comment
block of a test (up to the first line that does not start with #); t_runner fails
when one sits further down, where it would be ignored. A suite with no test script yet prints SKIP <suite> no tests found. Under
--expect-pass every SKIP, SKIP(requires) and empty suite of the named suites/tests is a
FAIL. Exit 0 = nothing failed.
Suite discovery: unit/t_*.sh, parity/t_*.sh; corpus = every corpus/*.sh (the
DESIGN names groundtruth.sh, exclusions-superset.sh, marker-verify.sh) except the
helper prepare-pristine.sh and perf*.sh (suite perf); bash32 = bash32/*.sh and
any <dir>/bash32.sh. A single script also runs by name (tests/run.sh marker-verify).
Before committing a registry change: tests/run.sh quick, tests/run.sh golden, and
bash tests/tools/registry-gaps.sh --write (then t_gaps_fresh stays green).
| Path | What |
|---|---|
run.sh, lib/common.sh |
runner and shared helpers (env: ARE_CORPUS, ARE_JADX, ARE_GOLD, ARE_PS, BASH32, ARE_PRISTINE, ...; defaults derive from the checkout's parent directory) |
lint/lint-registry.sh |
registry lint, rules L1-L15 of DESIGN 4.6 (--help lists them; L11 is a WARN, L14 an INFO report on the FROZEN detect-adwrapper tables; L5/L7 read the allowed keys and patterns from _schema.json), plus JSON-schema validation when python jsonschema is installed (--no-schema skips it) |
golden/capture.sh, golden/compare.sh, golden/t_golden.sh |
registry-scan (8 apps x depth 1-3 x ads/trackers/all + synth48) and detect-adwrapper (sh with/without --registry, ps1) must stay byte-identical to the frozen baseline $ARE_GOLD/before |
golden/synth48.before/ |
G-synth: registry-scan outputs on the synth48 fixture, captured with the product of 4c166fe |
golden/snapshot.sh, golden/snapshot.before.jsonl, golden/t_snapshot.sh |
the registry fields today's consumers read must not change for the 48 original entries |
fixtures/ |
generators of synthetic trees (synth48, adwrapper fixture, pitfalls) and expected outputs; no proprietary code |
shims/ |
nojq, jq15 (fake jq 1.5), nosudo, awk-{gawk-posix,mawk,busybox} for the portability paths (with_shim with nojq or jq15 also sets HOME to a missing dir, so a real ~/.local/bin/jq cannot beat the shim) |
parity/ps.sh |
runs a .ps1 with Windows PowerShell 5.1 from WSL (-File, Windows paths) |
parity/t_parity.sh |
sh/ps1 byte parity (--only <section>: registry-query, adwrapper, find, each incl. the in-process case; the 8-app corpus part of adwrapper, and the badpiggies + pitfall part of find, run when the section is named with --only or under run.sh full, which exports ARE_FULL=1; find compares find-*.ps1 with find-*.sh: --tsv/--summary byte-identical, text normalized, --json semantic, on the five layouts, a drive-letter copy (also as a subst drive root), trees with an unreadable directory or symlinks, . inside sources/ and HeartStar; PowerShell runs go in parallel, ARE_PS_JOBS, default 4) |
fixtures/registry-{mini,edge,bad}/, fixtures/expected/registry-query/ |
registry-query fixtures (clean; BOM + case-colliding keys; every integrity error) and their expected outputs |
unit/ |
unit tests (t_*.sh; t_runner.sh covers run.sh itself; t_reader, t_pkg_parity (T-PKG-PARITY, always per awk shim), t_categories (T-CAT), t_libroots (T-LIBROOTS), t_format (T-FORMAT) cover registry-query and _audit-shared.json; t_adwrapper (fixture, jq/nojq/jq15, malformed registries, --dump-exclusions, decode-apk.sh exit code; per awk shim) and t_network_roots (T-NETROOTS: the frozen NETWORK_ROOTS/COMMON_ROOTS tables, sh == ps1, covered by the registry) cover detect-adwrapper; t_deps covers jq as a dependency: install-dep.sh jq (package manager only with sudo, else the pinned static binary via the test-only ARE_JQ_URL/ARE_JQ_SHA256 hooks), the neutralize-all soft-fail, INSTALL_OPTIONAL:jq of the bash checkers with unchanged exit codes, every INSTALL_* name being an install-dep target, and the .ps1 checkers never asking for jq) |
corpus/exclusions-superset.sh |
I11: the .packages-based exclusions cover the legacy ones (legacy script of $ARE_BASE_COMMIT patched at test time to dump them), in the jq/nojq/jq15 modes; nojq/jq15 detect-adwrapper output on the 8 apps == G |
corpus/marker-verify.sh, corpus/marker-allowlist.tsv |
T-MARKER (DESIGN 3.1 R4): every detection.marker_classes item has its class file in the main smali dirs of at least one corpus app where the ground truth ($ARE_GROUND_TRUTH, default registry-unification/ground-truth.json) has that SDK, else a reviewed allowlist row with a reason; corpus tree per app = $ARE_PRISTINE/<app>, else $ARE_CORPUS/<app>-decoded, else the ground truth's decoded_dir |
unit/t_cli.sh, unit/t_layouts.sh, unit/t_pitfalls.sh, unit/t_tokens.sh |
the audit engine sdk-audit.sh behind find-trackers.sh / find-ads.sh: every historical flag, option normalization, exit codes 0/1/2 (t_cli); the five input layouts and their expected outputs fixtures/expected/layouts/ (t_layouts, fixture fixtures/make-layouts.sh; also the PowerShell twins find-*.ps1 on the same expected files, through \\wsl.localhost and a drive-letter copy, when PowerShell is reachable); the legacy false positives/negatives (t_pitfalls); T-TOKENS: every regex alternative of the legacy scripts has a resolvable home in fixtures/legacy-tokens.tsv (t_tokens) |
corpus/groundtruth.sh, corpus/expected-sdkids.json, corpus/known-deviations.tsv, corpus/ep-allowlist.tsv |
the corpus gates of the audit on the pristine apktool decodes and the jadx corpus: G-PREC (HIGH precision 1.0), G-REC (recall, jadx package aliases), G-HS (HeartStar), G-NEW, G-FP-KNOWN (known-deviations.tsv), G-EP (Entry Points only under the reviewed ep-allowlist.tsv prefixes); ARE_JADX_EXTRA names the HeartStar / Water Sort jadx dirs |
corpus/find-golden.sh, golden/find/ |
--tsv goldens of both tools on the 9 pristine decodes (ARE_UPDATE_EXPECTED=1 rewrites them) |
tools/registry-gaps.sh, unit/t_gaps_fresh.sh |
dev tool (jq) that generates the computed part of skills/sdk-neutralizer/references/registry-backlog.md (DO / P / A gaps of the registry entries, between the registry-gaps markers): no option = print, --check = rc 1 when stale, --write = refresh; t_gaps_fresh (gaps-fresh) runs --check and the tool's own checks |
bash32/bash32.sh (suite bash32) |
I18: bash -n under bash 3.2 of the scripts that promise it, then every unit test that runs product scripts through $BASH32 (and corpus/find-golden.sh) again with that bash 3.2; the binary is $BASH32, else registry-unification/scratch/bash32/bash32 next to the repo, else SKIP |
corpus/perf.sh (suite perf), corpus/prepare-pristine.sh |
engine time budgets, WARN only (the PowerShell twin is timed too: logged, Subway beyond 60 s is a WARN); the helper that decodes the corpus originals into $ARE_PRISTINE (not a test) |
Tests follow a tiny protocol (see lib/common.sh): exit 0 = PASS, 77 = SKIP, anything
else = FAIL. Each test can also be run on its own: bash tests/unit/t_lint.sh.
Intended neutralization changes (rebaseline)
The golden suite and T-SNAPSHOT freeze what the neutralizer consumes. A registry change
that is meant to alter neutralization (new targets, new manifest_components, a new
neutralizable entry) makes them fail by design. Only additive changes are accepted
without a separate review: never remove or rename an existing target, component or entry.
- Make the registry change;
tests/run.sh quick(lint) must pass. - Capture and compare:
tests/golden/capture.sh --no-ps after-x, thentests/golden/compare.sh before after-xlists the differing files. - Prove it is additive:
tests/golden/additive-check.sh before after-xmust printPURELY ADDITIVE(no line of anyregistry-targets.txt/registry-manifest.txtmissing) and list exactly the intended added lines. Review thestdoutdiffs too (only MATCHED counts and UNKNOWN_PACKAGE lines that became entries may change). - Dry-run
neutralize.shon the affected pristine decodes: the new targets must be applied (DRY_RUN:WOULD_*, noTARGET_SKIPPEDfor them). - Rebaseline, keeping the old one:
mv $ARE_GOLD/before $ARE_GOLD/before-<version>(and its.sha256),cp -a $ARE_GOLD/after-x/. $ARE_GOLD/before/ && rm -rf $ARE_GOLD/before/synth48, regenerate$ARE_GOLD/before.sha256;rm -rf tests/golden/synth48.before && cp -a $ARE_GOLD/after-x/synth48 tests/golden/synth48.before;tests/golden/snapshot.sh <registry> > tests/golden/snapshot.before.jsonl. tests/run.sh fullmust pass, then rebuild and device-test the affected apps.