Skip to content

K2GO-376 fix(content-exchange): longer import read timeout for slow Kolibri - #724

Merged
luisguzman-adfa merged 8 commits into
mainfrom
fix/K2GO-376-import-read-timeout
Nov 3, 2026
Merged

luisguzman-adfa merged 8 commits into
mainfrom
fix/K2GO-376-import-read-timeout

Conversation

@luisguzman-adfa

Copy link
Copy Markdown
Collaborator

What changes

  • ConexCoreDataSource.postImportItem uses a dedicated 15 min read timeout (IMPORT_READ_TIMEOUT_MS), passed through postJson. Other calls keep the 20 s default.

Why

  • A Kolibri channel import on the box takes 35-141 s. At 20 s the app reported those items as failed while the box finished them ("Imported 18 of 21 items").

Verifications

  • Reproduced on the OnePlus 7T through the app: the 3 Kolibri channels false-failed at 20 s; Calibre and Kiwix items imported.
  • Backend battery OnePlus -> Samsung: 22/22 items imported; the largest Kolibri import took 141 s.
  • APK builds and is installed on the OnePlus. A UI re-run with the fix is still pending.
    • Deferred: make import a prepare/poll long operation like export (K2GO-507). A fixed timeout has a ceiling for very large channels.

Notes

  • Delete already uses a dedicated 120 s timeout for the same reason.

…olibri

A Kolibri channel import on the box (importchannel + importcontent copying the whole
channel) takes 35-141s, but postImportItem used the generic 20s read timeout, so every
sizable Kolibri import false-failed while the box kept importing ("Imported 18 of 21
items, some failed"). Give import a dedicated 15-min read timeout, threaded through
postJson, matching the existing delete-timeout pattern. Robust follow-up: make import a
prepare/poll long operation like export (K2GO-507).
The default 60 s read timeout cut a Kolibri import through the conex-core relay with a 504. dash-node 1.4.8.
@luisguzman-adfa

Copy link
Copy Markdown
Collaborator Author

Update (d61893b): adds the box half of the fix. The app reaches conex-core through nginx /k2go-api/ and the dash-node relay. That location had no read timeout, so nginx returned 504 after 60 s on a Kolibri import. It now waits up to 3600 s. dash-node 1.4.8 (+CHANGELOG), so deployed boxes get it on self-update.

Device (OnePlus 7T, nginx live-patched with the same lines): import from disk 21 of 21, two runs.

… watchdog

Import becomes a start/poll/cancel long operation like export (POST starts it
and returns an opId; GET /v1/import/status; DELETE cancels), so a 35-141 s
Kolibri import no longer times out at the dash-node/nginx relay. A shared
watchdog supervises the Kolibri import and export CLIs by byte and CPU progress
(no fixed timeout; window relative to the task) and names a signal-9 exit as a
likely OOM kill. conex-core 0.3.3.
…equest

ContentImportService starts the import op and polls its status (like the export
service), so a 35-141 s Kolibri import no longer hits the dash-node/nginx relay
read timeout (the 504). Each poll is a short request; the box watchdog owns the
liveness decision, so the app keeps short per-request timeouts, not a long one.
…sweep

Drop the box op on DONE (an idempotent delete), so import has an explicit
normal-completion cleanup like export's fetch-close, instead of leaving a
consumed op to the 600 s abandonment sweep.
0.3.3 is the observability release in its own PR.
@luisguzman-adfa

Copy link
Copy Markdown
Collaborator Author

Update (613386b..8054cdc): import is now a long operation, so no fixed timeout sits on its path.

  • conex-core 0.3.4: POST /v1/import/item starts the import and returns an opId. GET /v1/import/status/{op} polls it. DELETE /v1/import/{op} cancels it. Import and export share one op registry and one abandonment sweep.
    • A liveness watchdog supervises the Kolibri import and export CLIs. Moving bytes = alive, with no limit. CPU only = a longer window. No progress = stalled: the op is cancelled and reported. The window is relative to the task. A signal-9 exit is reported as a likely out-of-memory kill. The rule is in ADR-long-running-operation-contract.
    • App: ContentImportService starts the op and polls it, like the export. The 15-min read timeout from 4befb14 is removed.
    • 213 conex-core tests pass. Two review passes done.
      Device (OnePlus 7T, real relay path nginx -> dash-node -> conex-core): the POST returns in 0.02-0.04 s. A 3.3 GB Kolibri channel completes in 55 s. It also completes with nginx at its default 60 s.

…brary

A fully successful import counts down 3 s to the Library; Cancel stays on the screen. Reuses the Clone countdown and setup Cancel strings.
@luisguzman-adfa

Copy link
Copy Markdown
Collaborator Author

Update (d70e2f5): a fully successful import now confirms and returns to the Library.

UI changes:

  • The settled message "Imported N items." shows in green (k2go_leaf) on full success. It stays ink on a partial result or an error.
    • Below it: "Opening your library in 3 s..." counts down, then the app opens the Library tab. A "Cancel" link stops the countdown and keeps the screen (Finish stays).
    • No new strings: reuses k2go_clone_exit_redirecting and k2go_setup_cancel (already in all locales).
    • Same screen for import from a drive and for receive over Wi-Fi.

…ead-timeout

# Conflicts:
#	static/conex-core/CHANGELOG.md
#	static/conex-core/conex_core/__init__.py
#	static/conex-core/conex_core/server.py
@luisguzman-adfa
luisguzman-adfa merged commit c7a8814 into main Nov 3, 2026
4 checks passed
@luisguzman-adfa
luisguzman-adfa deleted the fix/K2GO-376-import-read-timeout branch November 3, 2026 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant