Healthcare providers often spend hours transcribing, documenting, and categorizing patient interactions—time that could be spent on patient care. The Medical Transcription Analysis (MTA) tool was designed to streamline this process by leveraging AWS services to automate the transcription, comprehension, and translation of clinical conversations.
MTA integrates:
•Amazon Transcribe Medical – for speech-to-text transcription
•Amazon Comprehend Medical – for entity detection and classification
•Amazon Translate – for multilingual support
This open-source solution demonstrates how machine learning can enhance clinical workflows, reduce manual entry, and surface meaningful health insights.
The Problem
Medical transcription is time-consuming, error-prone, and often requires manual tagging and note generation. Existing tools lack automation, standardization, and multilingual support.
We needed to build an experience that would:
•Automatically transcribe clinical conversations in real time
•Identify key medical entities and associate them with clinical codes
•Support editing and redaction of protected health information (PHI)
•Generate pre-formatted clinical notes and summaries
•Enable translations to support diverse patient populations
My Contribution
As the sole UX designer on the MTA project, I led the end-to-end experience strategy, from concept to final high-fidelity implementation. My work included:
•Conducting foundational and evaluative UX research
•Creating task flows and defining information architecture
•Designing prototypes and high-fidelity UI
•Collaborating closely with engineering, PMs, and domain stakeholders
•Delivering all design assets for implementation, while balancing feasibility within a rapid build cycle
Design Process
Step 1: Transcription Input Options
The homepage offers multiple ways to start transcription:
•Record via microphone
•Upload an audio file
•Use a pre-loaded sample
Users can begin transcription via live recording, upload, or sample files.
Design Process
Step 1: Transcription Input Options
The homepage offers multiple ways to start transcription:
•Record via microphone
•Upload an audio file
•Use a pre-loaded sample
Step 2: Real-Time Transcription & Entity Detection
As users interact, a WebSocket connection displays real-time transcription from Amazon Transcribe Medical. Detected entities are automatically highlighted and categorized.
Real-time transcription with medical entities highlighted and color-coded by category.
Users can:
•Manually edit the transcript
•Redact PHI
•Correct misidentified terms
Step 3: Entity Editing and ICD-10 Selection
Amazon Comprehend Medical detects entities and proposes options with confidence scores. Users can:
•Review detected entities
•Select preferred ICD-10 matches from dropdowns
•Adjust confidence thresholds for entity filtering
Step 4: Clinical Note Generation
Once finalized, the transcript populates a structured clinical note with editable fields:
•Assessment
•Diagnosis
•Subjective / Objective observations
Automatically populated note fields provide a jumpstart on required documentation.
Step 5: Summary and Translation
The system generates a summary that includes:
•Full transcript
•All identified entities
•Structured notes
•One-click translation to 71 supported languages
Final summary view with multilingual support via Amazon Translate.
UX Challenges & Unimplemented Recommendations
Although the MVP was delivered, I conducted ~20 usability sessions to identify areas for improvement. Due to scope limitations, several key enhancements were not implemented but are documented below as future opportunities.
Homepage Clarity
Problem: Users needed more guidance on how the app works.
Recommendation:
•Add overview text to the homepage
•Clarify available input methods
MTA homepage (original version)
Recommended homepage redesign to improve onboarding and task clarity.
System Mental Model
Problem: Users didn’t fully understand the workflow or process steps.
Recommendation:
•Introduce an onboarding tour with contextual tips
Example of guided tour UI to build understanding of the app’s purpose and workflow.
Entity Visualization
Problem: Relationship between transcript and extracted entities was unclear.
Recommendation:
Explore different layouts for entity display Improve visual link between transcript and extracted data
Explore different layouts for entity display Improve visual link between transcript and extracted data
Proposed improvements for visualizing extracted entities and their relationships.
Confidence Threshold Controls
Problem: Users wanted more control over entity suggestions
Recommendation:
•Add setting to adjust confidence threshold
•Tailor interface for users preferring more or fewer suggestions
Impact
Delivered a fully functional demo showcasing AWS’s ML capabilities in healthcare
•Provided a reference implementation for customers to integrate into EHR systems
•MTA is now an open-source project on GitHub and used by AWS customers across healthcare workflows
•Identified multiple future UX improvements to enhance usability and adoption
Closing Thoughts
The Medical Transcription Analysis tool demonstrates how cloud-based AI services can automate and accelerate clinical documentation. With deeper investment in UX, this tool—and others like it—can empower healthcare providers to focus less on paperwork and more on patient care.