mirror of
https://github.com/Tencent/flare.git
synced 2026-10-26 10:49:42 +08:00
master
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 CHEN FengandClaude Opus 4.8
|
92a08ae18b |
tracing: migrate opentracing-cpp → opentelemetry-cpp (API only) (#201)
* tracing: migrate opentracing-cpp -> opentelemetry-cpp (API only) opentracing-cpp is deprecated/superseded by OpenTelemetry. flare ships no real tracer -- it exposes a pluggable abstraction (TracingOpsProvider / QuickerSpan / TracingOps) over the tracing-API types, with providers plugged in by users. Re-base that abstraction on the opentelemetry-cpp **API** (header-only, v1.27.0) and drop opentracing-cpp from both build systems. Scope: API only -- no SDK/exporter, so no protobuf/grpc is pulled in and the deliberately-pinned (abseil-free) protobuf 3.21.12 stays intact. The abstraction and the opaque `bytes tracing_context` wire format are kept; only the types/bodies change. Source (flare/rpc/tracing/): - tracing_ops_provider.h: re-typed virtuals; new flare-owned SpanStartOptions (parent SpanContext + SpanKind + timestamps + start attrs) replacing opentracing::StartSpanOptions/ChildOf. Extract -> std::optional<SpanContext>. - tracing_ops.h/.cc: span_ is nostd::shared_ptr<trace::Span>; SetAttribute/ AddEvent/End; SpanContext is a value type; buffered ops keep flare's owning variant and convert to (non-owning) AttributeValue only at flush. - framework_tags.h -> constexpr std::string_view; string_view_interop.h -> ToStd/ToOtel (nostd<->std). Standard-tag table is a literal allowlist; span.kind moves from a tag to SpanStartOptions::kind. - Call sites rpc_channel.cc / normal_connection_handler.cc updated; rpc_meta.proto comment updated (wire field unchanged). - Tests/benchmark rewritten onto a recording fake otel::trace::Span (OTel's noop span can't be asserted on); integration test asserts span kind via the provider-recorded "span.kind" instead of an opentracing tag. Build: - blade: BLADE_ROOT vcpkg_config gains opentelemetry-cpp; thin wrapper //thirdparty/opentelemetry-cpp:api (refs opentelemetry_common -- the api component is header-only with no .a; that lib's include dir carries the api + absl headers). opentracing thirdparty dir removed. - bazel: deps.bzl http_archive for opentelemetry-cpp 1.27.0 + api-only bazel/opentelemetry-cpp.BUILD (header-only, bundled nostd -> no abseil); patch_cmds drop the archive's nested BUILDs so the api headers can be globbed. opentracing BUILD/patch + .bazelignore/.gitattributes entries removed. Verified: blade and bazel both build //flare/rpc/tracing/... and pass tracing_ops_test + integration_test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: bazelignore thirdparty/opentelemetry-cpp (blade BUILD) The repo-wide `bazel build ...` job parses every BUILD file; the blade thin wrapper thirdparty/opentelemetry-cpp/BUILD uses blade-only syntax (vcpkg#...:lib, visibility='PUBLIC'). Add it to .bazelignore (the opentracing entry was removed but its replacement wasn't added). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * tracing: clang-format (v15) the migrated files Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: drop accidentally-committed .claude/ tooling files Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * doc(tracing): note provider ABI contract for OpenTelemetry nostd types The TracingOpsProvider interface uses OpenTelemetry API types (nostd::shared_ptr<Span>, nostd::string_view, AttributeValue) directly. The nostd:: backing is a compile-time choice (bundled / std via OPENTELEMETRY_STL_VERSION / abseil) and the three are mutually incompatible types, so a provider must be built with the same OpenTelemetry configuration as flare. flare leaves the default nostd in place. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: gitignore .claude/ (Claude Code local tooling) Prevents accidentally re-committing .claude/settings*.json (cf. review comments on PR #201). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * tracing: address PR #201 review follow-ups (non-blocking) - forcibly_sampled now routes `flare.sampling_priority` through the provider's `SetFrameworkTag` translation (before `IsSampled`) instead of a raw `SetAttribute`, keeping the `flare.`-prefixed tag consistent with every other framework tag and letting providers honor the force-sample signal. Add `ext::kSamplingPriority`. - An empty log key (single-arg `AddTracingLog`) now materializes as the default event name "log" so backends that group/display by event name don't render a blank one; document the per-log-event mapping. - Guard `QuickerValue`'s reliance on P0608R3 (C++20) with a static_assert so a future standard downgrade can't silently bind `const char*` to bool. - Reset `DummySpan`'s inline-static tags/logs at the start of the recording test so assertions can't observe leftovers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
CHEN Feng
|
55d97852f7 |
thirdparty: migrate fmt/gflags/glog/zlib/lz4/zstd/snappy/xxhash to vcpkg (#180)
Migrate eight third-party libraries (fmt, gflags, glog, zlib, lz4, zstd, snappy, xxhash) from in-tree foreign builds to vcpkg, via blade native vcpkg support. Bundles the matching blade.zip (proto import-root + vcpkg protoc + transitive Requires linking + textual_hdrs) and the CI/Bazel adjustments (pinned vcpkg baseline; xxhash externalized for Bazel). |
||
|
CHEN FengandClaude Opus 4.7
|
362e00a180 |
docs(readme): update for C++20 + macOS support, retranslate zh/en (#172)
* docs(readme): update for C++20 + macOS support, retranslate zh/en Bring both READMEs in line with the project's current state: * C++ badge upgraded from a generic ``C++`` to ``C++20`` to match the actual standard the project compiles with (``-std=gnu++2a``). * Platform badge upgraded from ``linux`` to ``linux | macOS`` -- macOS port landed in #158, CI matrix in #167 onwards. * Features section: drop the historical ``C++11/14/17/2a`` mix in favor of just ``C++20`` (the standard is committed; the staged adoption is no longer informative for a new reader). * System Requirements section split into ``Linux`` and ``macOS`` subsections, each listing the supported kernel/OS and the minimum toolchain that ships ``std::atomic<T>::wait`` / ``notify_*``: - Linux: GCC 11+ (libstdc++ added the operations in 11.1, per P1135 -- exposed locally on flare/fiber/detail/scheduling_group after the futex path went away in #158). - macOS: 13 (Ventura) or later with Apple Clang from Xcode 14+ (its libc++ has the matching runtime symbols). Also calls out aarch64 / arm64 / x86_64 explicitly. * Retranslated the zh README to read more like native Chinese -- the previous version had several literal-translation-isms ("吸收先前服务 框架"... etc.). Cross-checked the en against the zh and rewrote several en sentences that drifted out of step over the years (badge alt text, "out of the box" phrasing, the third-party tarball blurb, etc.). Both files now describe the same project in the same shape. * Added a back-link from the en README to the zh one (was one-way before). * Updated the blade-build link from chen3feng/blade-build to blade-build/blade-build (org moved). * Fixed bazel/support/README.md link typo (was REAMD.md). No code changes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: fix two markdown-link-check failures surfaced by the README PR * Rename `bazel/support/REAMD.md` -> `bazel/support/README.md`. The file has carried the swapped-letter spelling since 2023; the previous README link pointed at the typo'd name, so it was working by accident. The README rewrite in this PR pointed at the canonical spelling and surfaced that the file on disk was wrong. * Add `.github/markdown-link-check-config.json` to suppress the spurious "Status: 0" failure for `https://bazel.build`. The site returns a valid 200 to a normal browser and to `curl`, but apparently UA-filters the markdown-link-check action -- the action reads Status 0 and counts it as broken. Listed it in `ignorePatterns`; wired the config into the existing `gaurav-nelson/github-action-markdown-link-check@v1` invocation via the documented `config-file` input. Also bumps `retryOn429` (small reliability win when the action hits rate limits) and pins the success set to 200/206. Both surfaced by CI on this PR; no other changes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
CHEN FengandClaude Opus 4.7
|
86f175c3c4 |
Add macOS (darwin/arm64) build support (#158)
* feat(macos): build flare on macOS (darwin/arm64) + thirdparty alignment & bug fixes
Make flare build with both blade and bazel on macOS (Apple Silicon,
darwin/arm64); previously Linux-only. Verified: `blade build flare/...` and
`bazel build //flare/...` (653 targets) both green on macOS arm64.
Third-party / build system:
- Align bazel deps with the verified blade build: glog 0.4.0->0.7.1,
zlib 1.2.13->1.3.2, openssl 1.0.2u->1.1.1w (1.1 required by HMAC_CTX usage).
- glog: extend the export-guard patch to every header gating <glog/export.h>
behind GLOG_USE_GLOG_EXPORT, and update CHECK_OP to use
google::logging::internal::{Make,}CheckOpString.
- foreign_cc macOS fixes: force a real `ar` (rules_foreign_cc defaults AR to
Apple's libtool, incompatible with autotools' ar-style calls); gate -lrt to
Linux; pick .dylib vs .so; build nghttp2 static-only (its sandbox dylib
install_name breaks curl's configure run-time check).
- Register the host preinstalled make toolchain (rules_foreign_cc's bundled
make segfaults building jemalloc on macOS; blade also uses the system make).
- Track patch files as unpack-rule inputs (foreign_build.bld).
- Silence third-party warnings (external/ and in-repo thirdparty/) via
per_file_copt.
flare portability:
- IO: new Poller abstraction with epoll (Linux) and kqueue (macOS) backends;
event_loop/descriptor rewired off raw epoll; eventfd -> pipe notifier.
- net/thread/cpu/rlimit platform guards; OpenSSL 1.1 HMAC_CTX API.
- Add missing `gflags/gflags.h` / `<unistd.h>` includes (were pulled in
transitively by the old glog).
- gperftools/tcmalloc is Linux-only: excluded on darwin (mirrors blade);
macOS uses jemalloc.
Bug fixes (also affect Linux):
- base/expected.h: Unexpected::swap referenced a nonexistent member
(other.unex -> other.unex_); never compiled once instantiated.
- io/detail/writing_buffer_list.cc: FlushTo crashed (CHECK) on a benign drain
race (head_ stale while tail_ == nullptr); now treats the list as empty.
- base/chrono.{h,cc}: clocks delegated to a hand-rolled clock_gettime that
mixed clock epochs (broke TimerWorker::wait_until); now use
std::chrono::*::now(); ReadCoarseSystemClock duration_casts to the clock's
duration.
- fiber/detail/scheduling_group.cc: spin-wait bound was a hardcoded TSC-tick
count assuming a GHz-scale TSC; now derived from the measured TSC frequency.
- base/internal/dpc.cc: "Run N DPCs" logged cbs.size() (always 0 at loop exit)
instead of the accumulated dpcs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fiber: defeat Clang's TLS-base CSE across returns_twice on AArch64
On AArch64, Clang caches the resolved thread_local slot address in a
callee-saved register and reuses it across `[[gnu::returns_twice]]`
calls. `jump_context` may resume the same C++ frame on a different OS
thread, so the cached slot points to the previous worker's TLS region
on resume, and accesses go to the wrong memory.
GCC handles `returns_twice` more conservatively (it re-issues
`mrs tpidr_el0` after the call), which is why Linux flare worked
inlined for years -- Linux + GCC, not "Linux".
Centralize the canonical `jump_context` declaration in a new
`context.h` (so every TU sees the `returns_twice` attribute) and
force the three TLS accessors out-of-line: `[[gnu::noinline]]` makes
each access an ABI call, and ABI rules force the caller to redo the
TLS resolution -- Clang can't CSE across a real function boundary.
Discovered while porting flare to macOS arm64 (Apple Clang), but the
underlying hazard exists on any Clang aarch64 build; GCC users were
just lucky.
* io/kqueue: fix Modify to disable removed filters; stop fanning EV_EOF to opposite direction
Two macOS-port correctness fixes in the kqueue poller, both surfaced
by a flare RPC client that died with STATUS_IO_ERROR on the first
write to a freshly-opened connection.
1. `Modify` now explicitly EV_DISABLEs filters that aren't in the
new event mask. Unlike Linux epoll (EPOLL_CTL_MOD replaces the
whole event bitmask in one shot), kqueue registers EVFILT_READ
and EVFILT_WRITE independently -- a bare EV_ADD on one filter
does NOT affect the other. Without explicit disable, the kernel
keeps firing the "removed" filter under level-triggered mode,
and `OnWritable()` runs on an empty writing buffer, which the
connection layer mistakes for "remote closed" and kills the
connection. EV_DISABLE (not EV_DELETE) is what we want -- it
pauses delivery while preserving the filter's internal readiness
state, mirroring epoll's behaviour. (EV_DELETE + later EV_ADD on
a socket that is already readable produces no edge with EV_CLEAR,
and the reader stalls forever.)
2. `Wait` no longer fans an EV_EOF flag out into both kPollerRead
and kPollerWrite. EV_EOF on EVFILT_READ just means the peer
half-closed for reading; the write direction is still usable
until the next write returns EPIPE. The old fan-out manufactured
a fake write event whenever the read side EOF'd, kicking
`OnWritable()` on an empty writing buffer -- same false-disconnect
chain as #1.
* io/util/socket: SetCloseOnExec must use F_SETFD; rename helper to SetStatusFlags
FD_CLOEXEC is a *file-descriptor* flag (F_GETFD / F_SETFD), not a
file-status flag (F_GETFL / F_SETFL). The existing `SetCloseOnExec`
routed through the generic `SetFlags` helper, which goes through
F_SETFL, and so passed FD_CLOEXEC = 0x01 into the status-flag space.
On Linux glibc, the unsupported bit is silently discarded by F_SETFL,
so this was invisible. On Darwin the kernel actually interprets the
0x01 bit inside the F_SETFL flag space: O_NONBLOCK gets wiped off the
listening socket and O_APPEND gets set. The downstream effect is that
`accept(2)` on the listening socket reverts to blocking and the
server's accept loop hangs forever after the first connection.
Fix `SetCloseOnExec` to use F_GETFD / F_SETFD directly, and rename
the generic helper from `SetFlags` to `SetStatusFlags` so the
descriptor-flag-vs-status-flag distinction is hard to miss next time.
The only remaining caller of the helper is `SetNonBlocking`, which
is correct (O_NONBLOCK *is* a status flag).
* io/native: treat ENOTCONN as transient in FlushWritingBuffer
When a non-blocking TCP socket is in the middle of its `connect(2)`,
Linux returns `EAGAIN` on `write(2)`/`sendmsg(2)`, which flare maps
to `FlushStatus::SystemBufferSaturated` and retries via the write
event. Darwin instead returns `ENOTCONN` ("Socket is not connected")
in the same situation -- the kernel distinguishes "would block on a
connected socket" (EAGAIN) from "not yet connected" (ENOTCONN).
The existing code treated ENOTCONN as a fatal write error, returned
`NothingWritten`, and `Kill(CleanupReason::Disconnect)` immediately
fired. The first RPC on every channel died with STATUS_IO_ERROR.
Fold ENOTCONN into the EAGAIN / EWOULDBLOCK branch so the write is
deferred until the kernel signals writability (which happens when
the connect completes), matching Linux semantics.
* base/write_mostly: order members so mutex outlives TLS wrappers
`WriteMostly` had `tls_buffer_` declared before `mutex`, which means
`mutex` was destroyed *first* (members are destroyed in reverse
declaration order). When `tls_buffer_` was then torn down, each
per-thread `WriteBufferWrapper` ran its destructor and tried to
`std::unique_lock(parent_->mutex)` -- locking an already-destroyed
mutex.
libstdc++ on Linux silently allowed this (the underlying pthread
mutex didn't actually reject), so the bug was invisible.
libc++ on Darwin returns EINVAL from `pthread_mutex_lock`, std::mutex
throws `system_error("mutex lock failed: Invalid argument")`, and
since destructors are noexcept by default the unhandled exception
aborts via std::terminate.
Reorder the members so `tls_buffer_` is the last declared field
(thus the first destroyed): both `mutex` and `exited_thread_combined_`
are still alive while the wrappers merge into them.
* base/deferred_test: initialise bool to avoid UB read
`Deferred.All` declared `bool f4;` and then `ASSERT_FALSE(f4)`
before any store. Reading an uninitialised bool is undefined
behaviour: the value is "indeterminate" and a load of an
indeterminate value is itself UB regardless of which bit pattern
happens to be in the stack slot.
Linux happened to observe zero on the stack and slipped through.
macOS Clang traps on the indeterminate read (SIGTRAP, exit 133),
exposing the bug.
Initialise `bool f4 = false`.
* macOS port: platform-specific test adjustments
A bag of test-only changes that fall out of the macOS port.
* attribute_test.cc -- SetCurrentAffinity skipped on Darwin.
Darwin has no pthread_setaffinity_np equivalent and the legacy
thread_policy_set THREAD_AFFINITY_POLICY is hint-only; the
`expected == GetCurrentProcessorId()` assertion cannot hold.
* descriptor_test.cc -- ConcurrentRestartRead skipped on Darwin.
The test attaches a PipeDesc with Event::Write to the read end
of a closed-write pipe. Linux epoll never fires EPOLLOUT on
that fd; macOS kqueue fires EVFILT_WRITE (no write buffer ==
"has space"). The fire trips a pre-existing race in
`Descriptor::QueueCleanupCallbackCheck`: FireXxxEvent does not
consult `cleanup_pending_` before fetch_add, so an event between
cleanup-task queueing and execution makes the strict CHECK at
descriptor.cc abort. Fixing that race needs Dekker-style
ordering between FireXxxEvent and Kill -- bigger scope than
this batch.
* writing_datagram_list_test.cc -- shrink the test datagram from
16384 to 4096 bytes. Darwin's `net.inet.udp.maxdgram` defaults
to 9216, and sendmsg() rejects larger UDP payloads with
EMSGSIZE; Linux is happy up to ~64KB.
* datagram_transceiver_test.cc -- cap the fragment count from
60'000 to 8192 on Darwin. Still well above IOV_MAX (1024 on
both libc's) so the "flatten on iovec overflow" path is still
exercised; total payload now fits under net.inet.udp.maxdgram.
* uri_matcher_test.cc -- regex pattern `"\\abc.*efg"` used the
`\a` escape, which is not a valid ECMAScript escape. libstdc++
accepted it silently (treating it as literal `a`); libc++ on
Darwin throws regex_error("invalid escaped character") and the
test aborts. The intended pattern was `abc.*efg`.
* dirty_hook_test.cc, hooking_mock_test.cc -- skipped on Darwin.
Both rely on `mprotect`-ing the text segment to writable to
overwrite a function prologue. macOS arm64 enforces W^X on
signed code pages: mprotect returns EPERM, the CHECK aborts.
Toggling JIT-style write-execute would require
`pthread_jit_write_protect_np`, the
`com.apple.security.cs.allow-jit` entitlement, and code placed
in a JIT region -- none of which apply to the regular text
segment that DirtyHook patches. (The single non-hook subtest
in hooking_mock_test, `NotEnabled`, still runs.)
* fixup: satisfy clang-format-15 (LLVM style) on the macOS port commits
CI's clang-format-15 (LLVM style) flagged two spots from the macOS
port batch:
* flare/fiber/detail/context.h: `} // extern "C"` -> `} // extern "C"`
(LLVM trailing-comment style wants two spaces before //).
* flare/io/detail/kqueue_poller.cc: the `read_flags`/`write_flags`
ternaries were hand-aligned to a column width that clang-format
didn't pick. Drop the explicit alignment and pull the disabled
case into a named `disable_flags` local so each assignment fits
on a single line that clang-format leaves alone.
No behaviour change.
* thirdparty/blade: bump blade.zip for target_arch cache fix
Picks up the toolchain.target_arch caching from
https://github.com/blade-build/blade-build/pull/1224.
BUILD-file load on this workspace drops from 13.2s to 0.25s.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kang jinci
|
f0bf9a4868 | feat(bazel): add flare/rpc/protocol/http/builtin resource_library (#129) | ||
|
kang jinci
|
218257ab00 |
feat(bazel): add example (#123)
* feat(bazel): add example * feat(bazel): add example |
||
|
kang jinci
|
1d60bd710b |
feat: support bazel build (#121)
* feat: suppport bazel build * feat(bazel): add rpc plugin * code change * fix comment |