LLM-Enhanced Meeting Minutes Generator with Open Source LLM Model

1. User Research and Prototyping Journey

Every day in our business, we faced the challenge of manually transcribing and summarizing meeting recordings—a process that was not only time-consuming but also prone to errors. Driven by our curiosity about AI and the desire to improve our workflow, we embarked on user research to explore solutions that could simplify this task.

  • Interviews and Observations:
    We spoke with meeting organizers, administrative staff, and transcriptionists to understand their pain points. Their feedback confirmed that manual note-taking was inefficient and that there was a real need for an automated solution.
  • Surveys:
    Our surveys revealed that users value speed, accuracy, and simplicity. They needed a tool that could quickly convert audio into well-structured meeting minutes without requiring technical expertise.
  • Prototyping with Gradio:
    We built an early prototype using Gradio, allowing users to upload MP3 files, trigger transcription, and view the generated meeting minutes. The real-time feedback we received helped us refine the interface, improve error handling, and enhance overall usability. This iterative process reassured us that our solution could truly address a common challenge in our day-to-day work.

2. Project Specification

Project Title

LLM-Enhanced Meeting Minutes Generator

Overview

Our application transforms MP3 audio recordings of meetings into detailed, structured meeting minutes in markdown format. By combining advanced audio transcription with powerful language processing, we’ve developed a solution that automates the creation of clear, actionable meeting notes.

Objectives

  • Automate Documentation:
    Eliminate the tedious manual process of taking meeting minutes.
  • Boost Efficiency:
    Generate high-quality meeting summaries quickly after a meeting.
  • Enhance Clarity:
    Deliver outputs that are easy to read, including a summary with key details, discussion points, takeaways, and actionable items with designated owners.

Functional Requirements

  1. Audio Upload & Management:
    • Integrate with Google Drive for secure file storage.
    • Enable users to select and upload MP3 files through the interface.
  2. Audio Transcription:
    • Use OpenAI’s Whisper API to convert audio into accurate text.
    • Support various audio qualities and recording lengths.
  3. Text Summarization & Minutes Generation:
    • Process the transcription using a large language model (Meta Llama) via Hugging Face Transformers.
    • Optimize the model using 4-bit quantization for efficient inference on limited hardware.
    • Generate structured meeting minutes that include a summary (with attendees, location, and date), key discussion points, takeaways, and actionable items.
  4. User Interface:
    • Develop a user-friendly Gradio interface for uploading files and viewing results.
    • Render the final output in markdown format with clear sectioning.

Non-Functional Requirements

  • Performance:
    The system should generate meeting minutes promptly for typical meeting durations.
  • Scalability:
    The architecture should support larger files and multiple users concurrently.
  • Usability:
    The interface must be intuitive, even for non-technical users.
  • Reliability:
    Incorporate robust error handling throughout the process—from file uploads to transcription and summarization.

Technical Architecture

  • Frontend:
    • Gradio for creating an interactive, web-based interface.
  • Backend:
    • Google Colab for development and GPU-enabled model inference.
    • Google Drive integration for file management.
    • APIs:
      • OpenAI Whisper API: For audio transcription.
      • Hugging Face Transformers (with BitsAndBytes 4-bit quantization): For efficient language model inference.
  • Security:
    • Secure API key management and adherence to data privacy guidelines.

Testing and Future Enhancements

  • Testing:
    • Conduct unit tests for individual modules (transcription, summarization) and end-to-end testing with sample MP3 files.
    • Gather continuous user feedback to refine and improve the system.
  • Future Enhancements:
    • Extend support to additional audio formats.
    • Explore real-time processing for live meetings.
    • Adapt the model to support multiple languages, ensuring smooth communication across different linguistic groups.

3. Project Overview and Impact

Driven by our daily challenges and a deep curiosity about AI, we developed the LLM-Enhanced Meeting Minutes Generator to simplify how we capture and process meeting content. Our solution leverages:

  • Advanced Transcription:
    By using OpenAI’s Whisper, we ensure that the audio is accurately converted to text, forming a reliable basis for further processing.
  • Efficient Summarization:
    We utilize Meta Llama via Hugging Face Transformers, optimized with 4-bit quantization, to generate concise and structured meeting minutes even on limited hardware.
  • User-Friendly Design:
    Our interactive Gradio interface makes it easy for anyone to upload recordings and instantly receive clear, actionable notes.

Expanding Possibilities

This framework opens up many exciting applications. For example, you could upload MP3 files extracted from YouTube videos and have our app highlight the key points. Lecture recordings can be transformed into summaries that help you review content more effectively. Additionally, by adapting the model to support multiple languages, we can facilitate smoother communication across diverse linguistic groups. And that’s just the start—we are committed to ongoing AI research and sharing our insights. Stay tuned to know more and experience more!

Back to blog