Skip to content

Add guidellm profile tooltip to ISL/OSL dropdown - #97

Open
aas008 wants to merge 24 commits into
openshift-psap:stagingfrom
aas008:feat/profile-tooltip-isl-osl-staging
Open

aas008 wants to merge 24 commits into
openshift-psap:stagingfrom
aas008:feat/profile-tooltip-isl-osl-staging

Conversation

@aas008

@aas008 aas008 commented Sep 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Add hover tooltips to ISL/OSL profile selectbox options across RHAIIS and LLM-D dashboards. When users open the dropdown and hover over a profile option, a tooltip shows the guidellm workload specification (token counts, samples, turns, etc.).

New files

  • custom_dropdown.py — Streamlit's st.selectbox does not support per-option hover tooltips natively — it only allows a single help string for the entire widget. To work around this, custom_dropdown.py injects JavaScript into the parent document via st.components.v1.html(). A MutationObserver watches for [role="option"] elements as the dropdown opens and attaches mouseenter/mouseleave event listeners that show a positioned tooltip with the profile's guidellm specification. Also contains PROFILE_DETAILS (the profile definitions) and tags ISL/OSL help icons with a CSS class for the ℹ️ icon styling.

Changes to existing files

  • dashboard.py — Calls inject_profile_tooltips() on page load. Adds help="Hover over dropdown options to see guidellm profile details" to ISL/OSL selectboxes in Performance Plots, Performance Trends, and Compare Versions sections.

  • dashboard_styles.py — CSS for the ℹ️ info icon on ISL/OSL selectboxes (scoped via .profile-info-icon class, does not affect other help icons).

  • llmd_dashboard.py — Same tooltip integration for LLM-D dashboard: calls inject_profile_tooltips(), restores native st.selectbox (was briefly using a custom HTML dropdown that couldn't return values), adds help= parameter.

Adding tooltips for new profiles

No tooltip appears in the dropdown until an entry is added to PROFILE_DETAILS in custom_dropdown.py:

"4000/512": {
    "name": "Your Profile Name",
    "prompt_tokens": "4000",
    "output_tokens": "512",
},

The k-notation variant (e.g. (4k/512)) is auto-generated. For non-standard display names, add "aliases": ["Custom Name"] to the entry.

Profiles shown in tooltips

Dropdown option Tooltip
(1k/1k) Balanced Profile — Input: 1000, Output: 1000
(512/2k) Heterogeneous — Input: 512 (stdev=128), Output: 2048 (stdev=512)
(2k/128) Short Prefill-Heavy — Input: 2048, Output: 128
(8k/1k) Prefill-Heavy — Input: 8000, Output: 1000, Samples: 50
(100k/1k) Long Context Prefill-Heavy — Input: 100000, Output: 1000, Samples: 10
(8k/800) Heavy Heterogeneous — Input: 8000 (stdev=8500), Output: 800 (stdev=1500)
Multi-turn Multi-turn Profile — Input: 128, Output: 128, Turns: 5, Prefix: 512
Screenshot 2026-09-11 at 10 55 52 AM Screenshot 2026-09-11 at 10 56 05 AM

🤖 Generated with Claude Code

aas008 and others added 4 commits September 9, 2026 15:37
Display detailed profile information (input tokens, output tokens, and description)
when users select an ISL/OSL profile. This helps users understand what each profile
tests and why certain token combinations are chosen.

- Added _PROFILE_DETAILS metadata for all guidellm profiles (0/0, 1000/1000, 8000/800, 128/128)
- Added get_profile_tooltip() to format profile details as markdown
- Updated ISL/OSL selectbox with help text and info box showing selected profile details

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Link profile definitions to the official workloads.yaml from the RHAIIS orchestration config.
Include actual token distributions, request rates, and accurate descriptions for each profile:

- Profile 1 (1000/1000): Balanced baseline with rates [1, 50, 100, 200, 300]
- Profile 6 (128/128): Multi-turn conversations with 5 turns and 512-token prefix
- Profile 7 (8000/800): Long context with variable token distributions

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Include hover/selection details for all profiles from workloads.yaml:
- Profile 1 (1000/1000): Balanced baseline
- Profile 2 (512/2048): Variable distributions
- Profile 3 (2048/128): Prompt-heavy context
- Profile 4 (8000/1000): Long context with samples
- Profile 5 (100000/1000): Extreme context (100K+ tokens)
- Profile 6 (128/128 display): Multi-turn conversations with prefix
- Profile 7 (8000/800): Heterogeneous with variable distributions
- Real Dataset (0/0): Real conversation data
- Custom: User-configurable stub

Display profile-specific details (samples, turns, prefix tokens) when available.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Change statistical notation from σ to 'stdev' for clarity in profile tooltips.
Improves readability for users unfamiliar with mathematical notation.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 7a782318-2590-4671-b274-d80a5b545826

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

aas008 and others added 7 commits September 9, 2026 15:46
These profiles are not actual guidellm workloads from workloads.yaml.
Keep tooltips for the 7 official profiles (1-7) only.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Remove redundant text about testing baseline performance across varied concurrency.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Add help parameter to selectbox to show tooltip hint when hovering over the help icon.
This provides guidance to users that selecting a profile will show detailed specifications.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Move profile details and tooltip function to profile_config.py for reuse across all dashboards.
This allows consistent profile hover information across RHAIIS, LLM-D, CPU, and other views.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Add help text and detailed profile info boxes to:
- Main dashboard (RHAIIS view) - Performance Trends section
- CPU dashboard - Workload filter

Allows users to see guidellm specifications for all profiles across all dashboard views.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Add token extraction from profile display names to match with PROFILE_DETAILS.
Handles formats like:
- "Profile B: Variable Workload (512/2k)" -> "512/2048"
- "1k/1k" -> "1000/1000"
- "8k/800" -> "8000/800"

This enables profile info boxes to appear for all dashboard views with correct mapping.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
CPU dashboard has different profiles not in the guidellm workload set.
Keep profile tooltips only for RHAIIS and LLM-D dashboards.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
@aas008
aas008 marked this pull request as draft September 9, 2026 19:54
Comment thread profile_config.py Outdated
aas008 and others added 11 commits September 10, 2026 11:55
Rates are test parameters, not profile specifications. Keep only
the core profile attributes: token counts, samples, turns, and prefix.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Show only token counts and relevant parameters.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Keep profile name for clarity.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Add visible caption below profile selectbox instead of relying on help icon.
Makes it clearer that selecting a profile displays specifications.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Fix indentation and use the shared get_profile_details() function
instead of non-existent _PROFILE_DETAILS dict.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Create custom HTML dropdown component that displays rich tooltips on hover
for each profile option, showing token counts and descriptions.

Features:
- Hover tooltips with profile details (name, tokens, samples, turns)
- Smooth animations and styling
- Keyboard accessible
- Clean integration with Streamlit

Implemented in llmd_dashboard.py with custom_dropdown.py component.
…o icon

- Replace broken custom HTML dropdown with JS MutationObserver that injects
  hover tooltips onto native Streamlit selectbox [role="option"] elements
- Use st.components.v1.html() to inject JS (st.markdown strips <script> tags)
- Add blue ℹ info icon (CSS scoped via .profile-info-icon class) on ISL/OSL
  selectboxes only, other help icons remain as default ?
- Update profile text per reviewer feedback: shorter names, remove verbose
  descriptions, add prefix_count field for multi-turn
- Add help="Hover over dropdown options..." to all ISL/OSL selectboxes
- Works across RHAIIS (Performance Plots, Trends, Compare) and LLM-D views

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@aas008
aas008 marked this pull request as ready for review September 11, 2026 15:23
…ooltip

- Move PROFILE_DETAILS and get_profile_details() from profile_config.py
  into custom_dropdown.py to eliminate a separate file
- Remove unused imports (get_profile_tooltip, PROFILE_DETAILS) from
  dashboard.py and llmd_dashboard.py
- Replace hardcoded profile_names dict with aliases from PROFILE_DETAILS
- Fix tooltip not showing for (2k/128): add rounded k-decoded variants
  so JS can match 2048→2k→2000 lookups
- Delete profile_config.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@aas008 aas008 added the kind/design Categorizes issue or PR as related to design. label Sep 14, 2026
@aas008 aas008 self-assigned this Sep 16, 2026
…rror guard

- Add type annotation for PROFILE_DETAILS (dict[str, dict[str, Any]])
- Add variant fallback for rounded k-notation in get_profile_details()
- Guard details['description'] with .get() in dashboard.py, llmd_dashboard.py
- Add keyboard accessibility (aria-selected tracking, aria-live region)
- Add 22 tests for profile details lookup and display variants
- Fix ruff import formatting in test file

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/design Categorizes issue or PR as related to design.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants