Testing and Validation

Local performance report

UDP report and buffer diagnosis · Buffer configuration

Functional tests establish correctness; sustained stress checks repeated work, restarts and cleanup. Performance measurements run locally after those checks pass and use a Release build without sanitizers. CI checks regressions and smoke execution; its shared-runner timing is not the published baseline.

Download the validation logs and source mapping with the SHA256 checksum. The archive includes bilingual instructions and an independent verifier.

Build and Run

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DARKNET_ENABLE_SSL=ON -DARKNET_BUILD_TESTS=ON \
  -DARKNET_BUILD_BENCHMARKS=ON -DARKNET_USE_BOOST_ASIO=ON
cmake --build build --parallel 3
cmake --build build --target arknet_check
build/tests/arknet_stress_test --protocol all --seconds 60 \
  --clients 16 --rounds 2 --window 4 --certs tests/certs

For Standalone Asio, use a separate build directory with -DARKNET_USE_BOOST_ASIO=OFF -DARKNET_ASIO_INCLUDE_DIR=/path/to/asio/include. HTTP and WebSocket checks require Boost.Asio and Boost.Beast; the two configurations therefore contain different checks. Windows executables have the .exe suffix. The repository’s TLS certificates are for testing, not production identities.

Tests use doctest 2.5.3, stored under tests/vendor/doctest; the framework is not installed or linked by library consumers. arknet_check runs the test executables, install-consumer check and enabled benchmark smoke cases directly, with timeouts and logs in build/tests/logs/. It does not use CTest.

build/tests/arknet_tcp_test --list-test-cases
build/tests/arknet_tcp_test --test-case="*framing*"
build/tests/arknet_tls_test --reporters=junit --out=build/tls.xml

The functional suites and benchmarks are independent C++ programs with their own main; compile and run them directly. Python, Ruby, jq, GNU timeout and shasum are not prerequisites for these C++ tests or single benchmarks.

Correctness Coverage

Sustained stress validates content/sequence, completion counts, connection and server restarts, pending writes and queue cleanup. Each protocol splits --seconds across the built-in scheduler, one context/one thread, one context/four threads and four contexts/one thread each, then across the requested restart rounds. Seven protocols with --seconds 60 therefore request seven minutes of traffic in total, plus setup and teardown. Slow receivers test backpressure rather than throughput.

Local Verification

The complete-suite source snapshot is 6c8418e; its networking changes are included in measured revision 2aa5bc0. The separate test-only snapshot 25bb9f8 covers allocation-failure portability and native-connect ordering below. Pool-shutdown coverage identifies its separate source snapshot 550fb58. PR #16 records the candidate review and cross-platform CI; local results do not substitute for its checks.

Validation dates: 2026-10-15–16. The machine is a Mac mini Mac16,11, Apple M4 Pro, 14 physical/logical cores (10 performance and 4 efficiency cores), 48 GiB RAM, macOS 27.0.1 (26A434). Dependencies are Standalone Asio 1.38.2, Boost 1.90.0 and OpenSSL 3.6.4. Ordinary builds use Apple Clang 21.0.0; sanitizer builds use LLVM 22.1.7.

Complete Checks

All four builds use Debug, TLS and tests ON, examples and benchmarks OFF. Compilation finishes before checks run serially with real database services.

Checks count programs invoked by arknet_check; cases count doctest executions. Assertion totals are observed counts, including duration-based stress cases. The enabled default suites have zero failures or environment skips. Archived Zephyr interoperability is not enabled in these runs; its independent evidence is recorded below. Reference ARQ/FEC interoperability is included.

Each configuration includes external-context stop (15 cases / 375 assertions), queued-connect cancellation (4 / 44), protocol and runtime-exit suites, stress, and installed-package consumption. The installed-consumer child build does not inherit sanitizer flags; its binaries are not sanitizer-instrumented.

Database and Shutdown Coverage

All four configurations execute the optional real-service cases against isolated MySQL Community 8.4.11 and Redis 7.4.2, without database-case skips. MySQL verifies TLS; Redis covers TCP, TLS and mTLS. Coverage includes connection/client and pool operations, permanent close_and_wait(), late connects, callback/capture retirement, publication failures and recovery, cancellation, and reset isolation. Without the documented service environment variables, real-service cases skip rather than substitute simulated services. Performance uses Valkey 8.0.11 for the Redis protocol; it is separate from this Redis 7.4.2 interoperability evidence.

TCP-based endpoint checks cover native completion and capture retirement. Complete concurrent owner-stop serialization is verified for TCP servers and HTTP/WebSocket/TLS-derived servers. It is not a new guarantee for every protocol. External-context owners keep their runners alive through drain and stop external producers first; worker callbacks must not block waiting for themselves. See lifecycle and the formal shutdown and fault tests. These results do not establish dynamic-library unloading safety.

Additional Focused Checks

The NoTLS selection covers TCP, external-stop, manual-runtime, queued-connect cancellation and four Redis shutdown/publication-failure targets; it is not a complete NoTLS matrix or a MySQL-without-OpenSSL result. The 72 ASan executions comprise 32 queued-connect runs, 32 selected HTTP cases and eight external-stop runs. Cases excluded by the explicit HTTP filter are not environment skips. The complete unfiltered HTTP suite passes in the complete checks above.

The test-only allocation and connect-ordering snapshot has separate coverage:

All focused cases pass without sanitizer diagnostics. The 128 selected native-error executions each check nine assertions; other cases excluded by the filter are not environment skips. These runs are additional to the complete snapshot, not a complete-suite run of the test-only changes. The MSVC-specific CRT allocation path is covered by the Windows CI jobs; this macOS validation does not execute it.

Pool Shutdown Checks

Snapshot 550fb58 retires queued acquire requests in place in both pools, avoiding allocation-dependent unordered-map move construction. Its formal regressions cover two queued waiters, reentrant close/cancel/acquire, once-only completion and capture retirement. Owner-side guards keep test state alive through permanent closure on fatal assertion paths.

All eight target runs pass with zero failures, skips or sanitizer reports. The four configurations also pass unchanged MySQL return-failure tests (5 cases / 222 assertions) and Redis return-failure tests (4 / 107). Those return-failure runs preceded the guard-only test edit; their tested source files and both pool headers match this snapshot. These are focused checks, not a complete-suite run of this snapshot.

Redis diagnostic tests retain inspected exception snapshots until their concurrent observer joins. The original Linux TSan and Windows fault-injection failures, source mapping and focused logs remain in the validation archive. The pool correction removes one allocation boundary; it does not guarantee that every child-close path can recover from heap exhaustion. Performance measurements remain tied to 2aa5bc0, before this shutdown-only correction.

ASan leak checking is disabled on this macOS host. The complete-suite ASan runs also disable online symbolization to avoid a diagnosed symbolizer hang; memory instrumentation remains enabled. No sanitizer suppressions or forced-include workarounds were added. Only the existing reference-KCP test translation unit excludes null,object-size instrumentation.

RUDP Interoperability Evidence

The following targeted runs were recorded on 2026-10-13. Their source revisions and scope remain in the linked record; their earlier complete-check counts do not describe the current project.

The RUDP FEC endpoint suite passed 11 cases and 2,564 assertions in both Release dependency configurations and the separate ASan/UBSan and TSan builds. Real zephyr/arknet process pairs passed all 12 scenarios per dependency configuration, including reconnect, ARK-style binary headers, 4096-byte bodies, ARQ packet loss and reordering. The relay preserves SYN/ACK/FIN; this does not establish reliable shutdown when FIN is lost.

Logs and reproduction commands record the source revision and exact scope. The test-only canonical KCP source disables null,object-size instrumentation; arknet’s own code remains fully instrumented. RUDP performance has a separate report.

Sanitizers run separately from performance measurements. LeakSanitizer support differs by platform; a passing Apple ASan run is not a Linux leak-check result.

Sanitizer Checks

Sanitizers instrument a development build and report errors observed while tests run.

ASan/UBSan and TSan use separate builds. They add execution overhead and are disabled for published performance measurements. A passing run covers the paths and thread schedules exercised; it does not prove that all errors are absent.

CI and Remaining Coverage

CI checks Linux/macOS/Windows with both dependency configurations; Linux also runs ASan/UBSan and TSan. Job names include OS, dependency, TLS and check mode. The full 14 checks run for pull requests, merge-queue candidates and manual dispatches. A successful PR head does not validate the queue candidate containing the latest main and preceding queued changes. CI does not repeat the same matrix after the candidate is merged into main. The README badge shows merge-queue runs.

Runs keep the existing per-PR or ref concurrency group. A new run does not cancel the active run in that group; it waits for the group. GitHub may replace an older pending run in the same group, so this setting is not a FIFO history of every push (GitHub concurrency rules). Required checks on the candidate still need to succeed. Documentation publishing keeps its separate main-push workflow. CI uploads test logs and runs benchmark smoke checks; performance report matrices come from local measurements.

Local RUDP measurements cover random/burst loss, delay, duplication and reordering, with six 120-second sustained runs. They cannot predict cross-host capacity or WAN latency. Descriptor exhaustion, production-duration endurance and ARK business acceptance remain to be validated.

All 14 independent client/server usage programs were also built with both dependency configurations. The seven protocol pairs per dependency configuration verified echo contents and all 28 processes exited normally. Build/run logs and reproduction scripts retain this check separately from timed performance measurements.