Fix CI index abort: bound open-buffer indexing, pre-size the symbol map

[metif12]

Fixes Windows CI abort Too many retries in GC_alloc_large in
test_large_vlang_v_workspace_from_env: indexing the whole vlang/v
tree inserts thousands of rich entries one by one, and a late doubling
of the map backing store fails under a fragmented heap (backtrace:
map__set → DenseArray__expand via reindex_uri).

• Pre-size the symbol index once from the walk's file count
  (map.reserve), so bulk walks allocate up front instead of
  doubling repeatedly.
• Apply the 2MB index size gate to open buffers too; previously
  only disk files were capped, so one huge buffer could force an
  unbounded index build.

Both changes are behavior-preserving for normal sizes; oversized
entries are dropped and marked skipped exactly like oversized disk
files. Verified with targeted tests (one proves teeth by failing
with the gate disabled) plus the existing large-project suites.

[JalonSolov]

Local AI Finding

P2: Pre-sizing is skipped for incomplete walks. When collect_v_files reaches its 20,000-file or directory-entry limit, index.v:638 skips the reservation branch, and entries are then inserted one by one until the global cap at index.v:672. Large workspaces therefore retain the repeated-growth and allocation-failure behavior this PR intends to fix. Reserve a bounded target for partial walks as well.

[metif12]

Good catch — agreed, and fixed in d94fbc9.

The reservation no longer depends on the walk being complete. Both branches now share one bounded target:

if !walk_complete {
    app.index_incomplete_scopes[scope] = true
}
target := index_reserve_target(app.symbol_index.len, files.len)
if target > 0 {
    app.symbol_index.reserve(u32(target))
}

index_reserve_target returns 0 when the index is already at or past the global cap, so the bound is unchanged there; otherwise a partial walk reserves what it found, and the growth toward the cap is bounded from the start instead of doubling repeatedly. The target is capped at index_max_files, so files.len from a partial walk (which is already limited) cannot push it over.

Tests: test_index_reserve_target_covers_partial_walks pins the target for empty, small, mid and at-cap cases, and test_partial_walk_still_reserves_what_it_found walks a real directory with an incomplete scope marked by an earlier walk and asserts all entries land.

[medvednikov]

Reviewed the latest head (d94fbc9), focusing on code behavior and ignoring CI as requested. The earlier incomplete-walk reservation concern has been addressed, but I found three remaining edge cases:

1. [P2] Clamp the reservation instead of dropping it when the projected count exceeds the cap (index.v, index_reserve_target). For example, index_reserve_target(12_000, 10_000) returns 0, although ensure_dirs_indexed may still insert another 8,000 entries before reaching index_max_files. In multi-root/partially indexed workspaces, that brings back the incremental map growth this change is meant to prevent. Consider returning min(indexed + found, index_max_files) when indexed < index_max_files, and test the crossing-cap case. Also, test_partial_walk_still_reserves_what_it_found currently performs a complete 20-file walk, so it doesn't actually exercise the partial-walk branch.

2. [P2] Mark oversized unsaved open buffers as incomplete for rename (reindex_uri / index_is_complete_for_scope). The new open-buffer size check sets index_skipped_uris[uri], but index_is_complete_for_scope only rejects a skipped URI if os.is_file(uri_to_path(uri)) is true. An unsaved new file (>2 MB) in a project has no on-disk file, so rename can proceed as if indexing were complete while silently omitting its references. The completeness check should also recognize skipped URIs that are still in app.open_files.

3. [P2] Apply the oversized-buffer guard to the occurrence cache too (occurrences_for). reindex_uri drops ref_occurrences for large buffers, but occurrences_for reads the same open buffer without any size gate and rebuilds the full token-occurrence map. The normal local-binding hover path calls it, so hovering an identifier in a large file can still trigger the unbounded index allocation that this change is trying to avoid. Short-circuit occurrence indexing for oversized sources and cover that call path with a test.

These are independent of CI status.

[JalonSolov]

Can you fix the 3 items from Alex's review?

[metif12]

Can you fix the 3 items from Alex's review?

On it

[metif12]

Can you fix the 3 items from Alex's review?

All three addressed