Museums increasingly rely on digital technologies to improve visitor engagement, accessibility, and educational experiences. Audio guide systems are commonly used to provide additional exhibit information and assist visitors with visual impairments, reading difficulties, or language barriers. Many existing museum guide solutions require visitors to manually interact with exhibits by entering exhibit numbers on dedicated devices, scanning QR codes, or using smartphone applications to access content. More advanced commercial systems, such as Guide-ID, provide interactive and location-aware audio experiences but often require specialized infrastructure and proprietary technology. These approaches can create additional barriers for users seeking a simple and seamless museum experience. The Assistive Museum Headphone System addresses this need by automatically detecting exhibit markers and playing the corresponding audio description without requiring manual exhibit selection, mobile applications, or additional user interaction. By combining computer vision, embedded processing, and real-time audio playback, the system aims to provide a more accessible, intuitive, and cost-effective museum guidance solution.
The Assistive Museum Headphone System is a wearable embedded device that automatically detects museum exhibits and provides corresponding audio descriptions to users in real time. The system reduces the need for manual interaction while improving accessibility and user engagement. After a marker is detected, the system temporarily pauses additional scanning until the user manually resets the detection state for the next exhibit.
The final system integrates a Raspberry Pi 4, STM32F4 Discovery board, USB camera, four control buttons, two bone conduction speakesr, and a custom 3D-printed enclosure. The camera, buttons, and bone conduction speakers are wired to the glasses using a custom wiring harness, and are held in place with 3D-printed clasps.
The Raspberry Pi executes a Python-based controller responsible for visual marker detection, GUI management, audio-file mapping, and playback control. The STM32 manages real-time button-control and speaker tasks using FreeRTOS.
A GUI system allows users to add and modify marker-audio pairs without changing source code. To prevent repeated detections, marker scanning is paused after a marker is detected and resumes only after a manual reset.
The user wears or carries the device while walking through the museum. A camera connected to the Raspberry Pi continuously scans the environment for visual markers. When a marker is detected:
- The visual detection software identifies the marker ID.
- The marker ID is mapped to a corresponding audio file.
- Audio playback begins automatically.
- The user can control playback and volume using hardware buttons.
- Audio is delivered through headphones.
- The user manually resets the system before scanning resumes for the next marker.
The intended users of the Assistive Museum Headphone System include general museum visitors, students, elderly users, visually impaired individuals, and international visitors who may benefit from automated audio guidance and simplified exhibit interaction.
The system is designed for indoor museum environments where exhibits are associated with visual markers such as fiducial markers or QR-code-style patterns.
The Assistive Museum Headphone System provides a low-cost and scalable accessibility solution that improves user engagement and educational accessibility without requiring complex user interaction.
The Assistive Museum Headphone project was completed by a four-member engineering team that collaborated on the design, implementation, testing, and integration of the embedded system. The team followed an iterative development process where each subsystem was first implemented independently and later integrated into the complete system. While each team member focused primarily on specific subsystems based on their technical strengths and interests, all members participated in subsystem integration, troubleshooting, testing, and final system validation. Weekly team meetings were held to review progress, discuss technical challenges, coordinate integration efforts, and ensure the project remained on schedule. This collaborative structure allowed the team to efficiently combine hardware, software, and mechanical components into a unified embedded assistive technology platform.
| Team Member | Responsibilities |
|---|---|
| Grace Zhou | Project manager, visual detection system implementation, data mapping/storage system implementation, documentation, system integration, debugging |
| Violet Monserate | Speaker system implementation, wiring design, smart glasses development, wiring harnesses, system integration, debugging |
| Kevin Huang | 3D-printed enclosure development, system integration, debugging |
| Hanjing Fang | Audio controller system implementation, documentation, system integration, debugging |
The project demonstrated the feasibility of an assistive museum audio system that combines visual marker detection, local audio-file mapping, embedded user controls, speaker output, and portable mechanical packaging. The enclosure portion of the project addressed the practical challenge of integrating several development boards and cables into one usable system. By using Fusion 360 and additive manufacturing, a custom case could be designed around the specific dimensions and access requirements of the Raspberry Pi 4, STM32F4 Discovery board, power components, and control wiring.
The enclosure design satisfied the primary design objective of organizing the major components while keeping them removable for debugging. The decision to use anti-slide rails instead of screws was appropriate for the prototype stage because it simplified assembly and reduced the risk of damaging or obstructing development-board hardware. The next mechanical priority is to complete or verify all port cutouts and ensure that power, programming, communication, and audio cables can be attached without opening the enclosure.
Several sources of error or uncertainty should be considered. Measurements made with a ruler rather than a caliper may result in inaccurate board or connector clearances. 3D printing tolerances may cause rails or cutouts to fit more tightly than expected. Wiring positions may also change during final integration, potentially requiring additional cable-routing space. These issues can be reduced by carefully measuring the final hardware configuration, providing approximately 1–2 mm clearance around removable parts and cable openings, test-printing critical connector regions, and revising the CAD model after the first physical fit test.
Future improvements should include a more compact enclosure, improved ventilation near the Raspberry Pi, a better removable lid mechanism, labeled external ports, integrated cable strain relief, and a secure but removable battery holder. A later product version could also integrate the electronics into a smaller custom PCB or wearable housing suitable for regular museum use.
As a team, this project reinforced our skills in a variety of areas, including soldering, 3D modeling, high-level Python programming, microcontroller development, embedded systems integration, and hardware prototyping. Throughout the design and implementation process, we gained valuable experience in troubleshooting both software and hardware issues, coordinating subsystem integration, and managing design trade-offs between functionality, portability, and ease of use. The project also highlighted the importance of iterative design, testing, and communication within a multidisciplinary engineering team. Overall, the successful development of the assistive museum audio system demonstrated not only the technical feasibility of the concept but also the practical engineering skills required to transform an idea into a functional prototype.
Repository can be found at https://github.com/ctrl-vi/CSE475-STM32/