[Crashlytics] App crashes with OutOfMemoryError due to unbounded full-body HTTP logging (749 events / 78.9% of all crashes)

XMLWordPrintable

    • STOP TB Sprint 12

      Summary

      Fatal OutOfMemoryError in logging and OkHttp interceptor frames, caused by full HTTP request/response bodies being logged and retained in memory/file with no size cap.

      Background / Context

      Source: StopTB Android Crashlytics RCA (project stoptb-8e404, app org.piramalswasthya.stoptb, 7-day window Oct 15–21 2026, latest release 2.2.3 (22)). RCA ref RCA-01, status: Probable Root Cause (supported pending reproduction; no heap dump). Code inspected: PSMRI/STOP-TB-App, branch release-2.2.3, commit cd02b155. Same code is present in release-2.0 to release-2.3.

      Overall impact: 949 crash events, 153 users, crash-free users 76.01%, crash-free sessions 75.34%. OOM = 749 events (78.9% of all crashes). Priority: P0.

      Problem Statement

      Network calls with large bodies (15–19 MB allocations seen, mainly sync payloads) cause allocation failures. Sample events on Lenovo TB336ZU, Android 16; heap near 512 MB limit with ~8–10 MB free.

      Current Behaviour (Actual)

      • Release builds run HttpLoggingInterceptor at Level.BODY (the BuildConfig.DEBUG guard is commented out in AppModule.provideHttpLoggingInterceptor).
      • SyncLogTree is planted in all builds (SakhiApplication.kt) and routes those bodies into the in-memory and file log with no size cap. Every HTTP body is logged in full and copied 3–4 times (e.g. message.lowercase() in SyncLogTree).
      • SyncLogManager keeps up to 500 full-message entries (cap by count, not bytes).
      • SyncLogFileWriter Handler queue is unbounded; StringBuilder.append fails at SyncLogFileWriter.kt:54.
      • AccountDeactivationInterceptor does peekBody(1 MB) on every response (extra heap pressure).
      • largeHeap=true hides growth until 512 MB.

      Crashlytics issues in this group (events / users):

      1. SyncLogTree.promoteIfError (SyncLogTree.kt:71) — 422 / 33 — repetitive, 2.0.3→2.2.3
      2. SyncLogFileWriter.writeLog (:54) — 147 / 28 — repetitive, 2.2.0→2.2.3
      3. CampModeUrlInterceptor.intercept (:40) — 144 / 26 — repetitive, 2.1.0→2.2.3 (interceptor itself only rewrites URL; hypothesis H3 rejected)
      4. OOM, no app frame — 28 / 15 (inferred link)
      5. FrameIdentify.doFrameCheck (mediatek) — 3 / 3 (inferred)
      6. CancellableContinuationImpl.resumedState — 2 / 2 (inferred)
      7. AccountDeactivationInterceptor.intercept (:19) — 1 / 1
      8. ScrollPolicy.releaseFPSGOControl (mediatek) — 1 / 1 (inferred)
      9. SyncLogFileWriter.writeLog$lambda$1 (:60) — 1 / 1

      Expected Behaviour / Requirement

      Release builds must not log HTTP bodies, and any logging that remains must be size-bounded so that large sync payloads cannot exhaust the heap.

      User Story

      As a field user syncing data on a tablet, I want the app to complete sync without crashing, so that my work is not interrupted or lost.

      Acceptance Criteria

      • Given a release build, When a full sync runs, Then no request/response bodies appear in logcat or sync-log files (CAPA-01).
      • Given a 20 MB log message, When passed to SyncLogTree, Then it is skipped/truncated before any copy and heap stays flat (CAPA-02).
      • Given a sync in progress, When logs accumulate, Then SyncLogManager retained size is bounded by bytes and SyncLogEntry.message is truncated (CAPA-03).
      • Given large-payload sync on Lenovo TB336ZU, When it runs, Then it completes without OOM; SyncLogFileWriter caps line length and uses a bounded queue (CAPA-04).
      • Given a release build with HttpLoggingInterceptor set to BODY, When CI runs, Then the build fails (CAPA-05).
      • Given a release, When tested, Then heap profile of a full sync is attached to the release checklist and a Crashlytics OOM-count alert exists (CAPA-06).
      • After release, Crashlytics shows no new OOM events from the above issues.

      Business Rules / Validations

      • Where body logging is retained for debug builds, auth headers and personal/beneficiary data must be redacted; exported sync-logs reviewed for sensitive content (CAPA-07).

      Dependencies

      • None identified in the RCA. Sync-log export feature behaviour [TBD/CLARIFICATION REQUIRED].

      Impact / Affected Areas

      AppModule.kt (provideHttpLoggingInterceptor), LoggingInterceptor.kt, SakhiApplication.kt, SyncLogTree.kt, SyncLogManager.kt, SyncLogFileWriter.kt, AccountDeactivationInterceptor.kt; all sync flows.

      Out of Scope

      Separate memory-leak investigation (H2 unresolved) unless OOM persists after this fix; MediaTek/vendor frames beyond verifying post-fix.

      QA / Testing Considerations

      Full sync on Lenovo TB336ZU (Android 16) before and after; heap profile during sync; verify logcat and exported sync-logs contain no bodies/tokens; CI deliberate-BODY test.

      Open Questions / Clarifications

      • Is the sync-log UI/export relied on operationally, and what message length/byte cap is acceptable? [TBD]
      • RCA used branch tip, not shipped tag; line numbers differ by one at SyncLogTree (71 vs 72) — confirm against shipped 2.2.3.
      • Body sizes per endpoint not measured; issues #7, #14, #15, #21 not inspected (inferred).

      Source: StopTB_Crashlytics_RCA.xlsx (RCA-01, CAPA-01 to CAPA-07). Status: Draft RCA pending human confirmation.

              Assignee:
              Puneet Sharma
              Reporter:
              Shivani Garg
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: