"See" - Empowering Independence Through AI

Welcome to "See", an AI-driven assistive tool designed to transform how visually impaired users interact with the world. Built at Meta’s first Llama Hackathon in Toronto, this project leverages Meta's open-source Llama models to provide a highly intuitive and personalized experience.

Overview

Blind users often face significant challenges with current assistive technologies, from dependence on volunteers to complex, impersonal AI systems. See addresses these issues with a streamlined, voice-first interface that adapts to individual needs, enabling greater independence and usability.

Key Features

• Personalized AI Companion: Customize speech style and persona for a natural, human-like interaction.
• Voice-First Interaction: No need for typing—ask questions directly through voice commands.
• Context-Aware Assistance: Learns user preferences to provide tailored responses.
• Device Compatibility: Works seamlessly across computers and mobile phones.
• Offline Mode: Ensures functionality without constant internet access.
• Enhanced Accessibility: Simplifies interaction with a focus on auditory feedback.

Llama Model Integration

• Models Used:
  • Llama 3.2 90B Vision
  • Llama 70B
• Hardware: Optimized for Nebius AI using NVIDIA® H100 Tensor Core GPUs.
• Purpose: Powering real-time, context-sensitive, and user-friendly AI interactions.

How It Works

1. Activation: A single tap launches the assistant.
2. Voice Input: Speak commands like “What’s around me?” or “Read the menu.”
3. Response: AI provides auditory feedback tailored to your surroundings and queries.
4. Adaptation: The system learns your preferences over time to enhance usability.

Benefits

• Reduces frustration with current tools.
• Empowers blind users to navigate their environments independently.
• Provides quick, intuitive access to information in one step.

Demo

1. Identify Surroundings

   • User: "What’s nearby?"
   • See: "You are near a park. To your right is a bench and a water fountain."

2. Read Printed Material

   • User: "Read this menu."
   • See: "The menu includes coffee, tea, and pastries."

3. Navigate

   • User: "Help me find the exit."
   • See: "The exit is 15 feet ahead, slightly to your left."
     See more https://youtu.be/wZFQBMKpn3w

---

Rediscover independence with "See".