Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Web aois alphalab #687

Merged
merged 2 commits into from
May 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions alpha-lab/.vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ let theme_config_additions = {
{ text: "Map Gaze Onto Body Parts", link: "/dense-pose/" },
{ text: "Map Gaze Onto a 3D Model of an Environment", link: "/nerfs/" },
{ text: "Map Gaze Onto Facial Landmarks", link: "/gaze-on-face/" },
{ text: "Map Gaze Onto Website AOIs", link: "/web-aois/" },
],
},
{
Expand Down
10 changes: 10 additions & 0 deletions alpha-lab/cards.json
Original file line number Diff line number Diff line change
Expand Up @@ -128,5 +128,15 @@
},
"image": "/alpha-lab/map-gaze-nadia.webp",
"category": "Gaze Mapping"
},
{
"title": "Map Gaze Onto Website AOIs",
"details": "Define areas of interest on a website and map gaze onto them using our Web-AOI tool.",
"link": {
"text": "View",
"href": "/alpha-lab/web-aois/"
},
"image": "/alpha-lab/web-aoi.webp",
"category": "Gaze Mapping"
}
]
Binary file added alpha-lab/public/web-aoi.webp
Binary file not shown.
Binary file added alpha-lab/web-aois/heatmap-gazes-overlaid.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added alpha-lab/web-aois/heatmap_output.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
115 changes: 115 additions & 0 deletions alpha-lab/web-aois/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
---
title: Map Gaze Onto Website AOIs

Check warning on line 2 in alpha-lab/web-aois/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (AOIs)
description: "Define areas of interest on a website and map gaze onto them using our Web-AOI tool. "
permalink: /alpha-lab/web-aois

Check warning on line 4 in alpha-lab/web-aois/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (aois)
meta:
- name: twitter:card
content: player
- name: twitter:image
content: "https://i.ytimg.com/vi/1yJfhtdJoMA/maxresdefault.jpg"
- name: twitter:player
content: "https://www.youtube.com/embed/DzK055NbRPM"
- name: twitter:width
content: "1280"
- name: twitter:height
content: "720"
- property: og:image
content: "https://i.ytimg.com/vi/1yJfhtdJoMA/maxresdefault.jpg"
tags: [Neon]
---

<script setup>
import TagLinks from '@components/TagLinks.vue'
</script>

# Map Gaze Onto Website AOIs

<TagLinks :tags="$frontmatter.tags" />

Check warning on line 27 in alpha-lab/web-aois/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (frontmatter)

<Youtube src="DzK055NbRPM"/>

::: tip
Want to see your website through your users' eyes? Discover what really captures their attention as they scroll using our website AOI Tool + Neon eye tracking!
:::

## Understanding Gaze Patterns in Web Interaction

Understanding where people focus their attention on websites is key to optimizing user interfaces and improving user experiences, whether individuals are making decisions about online shopping or simply browsing content. This knowledge is also useful for educational technology, commonly referred to as EdTech. Consider interactive displays in classrooms or online learning platforms—grasping how users interact with these tools is fundamental for enhancing learning outcomes.

By gathering data on gaze patterns during these interactions, designs can be tailored to prioritize user needs. This results in websites that are easier to navigate, applications with better user interfaces, online stores that offer more intuitive shopping experiences, and EdTech applications that foster more effective learning environments.

In this guide, we will introduce a desktop application featuring an integrated browser that incorporates AprilTag markers onto webpages. This tool integrates with Neon, facilitating the recording and visualization of gaze data mapped onto areas of interest on a website.

## Introducing Web Gaze Mapping

Pupil Cloud offers powerful tools like the [Marker Mapper](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/marker-mapper/) and [Reference Image Mapper](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/reference-image-mapper/) enrichments, which enable users to map gaze onto areas of interest. However, they do not provide a turnkey solution for defining AOIs on a website and importantly, maintaining gaze mapping even during scrolling—a behavior typical of regular website usage.

By following this guide, you can easily define AOIs on websites of your choice and record Neon data. Importantly, with this tool, the AOIs are not lost as you scroll. Afterward, you'll receive gaze mapping for each AOI, including individual AOI heatmaps and a full-page heatmap.

Check warning on line 47 in alpha-lab/web-aois/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (heatmaps)

### How Does This Tool Work?

We leverage [Playwright](https://playwright.dev/), an open-source automation library for browser testing and web scraping, alongside AprilTags automatically placed on the webpage within the browser interface. Through Playwright, we generate AOIs using selectable web elements, while the AprilTags facilitate the real-time transformation of gaze data from *scene-camera* to *screen-based* coordinates. For a deeper understanding of this transformation, refer to [the documentation](https://docs.pupil-labs.com/alpha-lab/gaze-contingency-assistive/#how-to-use-a-head-mounted-eye-tracker-for-screen-based-interaction).

## Steps To Recreate

Explore the [GitHub repository](https://github.com/pupil-labs/web-aois) and follow these simple steps:

1. Define Areas of Interest (AOIs) on your selected website.
2. Put Neon and start collecting data.
3. Process your recording to generate CSV files with gaze mapped onto AOI coordinates.
4. Visualize the data with customized heatmaps.

Check warning on line 60 in alpha-lab/web-aois/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (heatmaps)

## Visualize Mapped Gaze Data

After running the code, new files will be generated.

- For every AOI, you will get `png` files with transparent and overlaid heatmaps and csv files with the gaze data mapped on the AOI coordinates.

Check warning on line 66 in alpha-lab/web-aois/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (heatmaps)
- A transparent and overlaid heatmap will also be provided for the full page (e.g., see below), along with a `gaze.csv` file that will include the mapped gaze data to the webpage in full.

<div style="margin-top: 20px;"></div>

<div class="grid grid-cols-2 gap-4">
<div class="image-column">
<img src="./heatmap_output.png" alt="Heatmap generated for one AOI" style="height: 300px; object-fit: cover;">
</div>
<div class="image-column">
<img src="./heatmap-gazes-overlaid.png" alt="Heatmap generated over the page" style="height: 300px; object-fit: cover;">
</div>
</div>
<font size=2><b></b> Heatmaps generated for one AOI (left) and for the entire page (right).</font>

Check warning on line 79 in alpha-lab/web-aois/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (Heatmaps)

You can find a detailed list of the outputs [in this section of our Github repository](https://github.com/pupil-labs/web-aois?tab=readme-ov-file#output-files).

<!-- ![Heatmap generated from Web AOI tool](./heatmap_output.png) -->

This data can be used to generate outcome metrics like time to first gaze (e.g., how long it took for the user to gaze at each AOI for the first time) or dwell time/total gaze duration (e.g., sum of the gaze sample durations, defined as the period between the timestamps of consecutive gaze samples).

::: tip
Need guidance in calculating even more metrics for your website AOIs? Reach out to us [by email](mailto:[email protected]), on our [Discord server](https://pupil-labs.com/chat/), or visit our [Support Page](https://pupil-labs.com/products/support/) for dedicated support options.
:::

<style scoped>
.mcontainer{

Check warning on line 92 in alpha-lab/web-aois/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (mcontainer)
display: flex;
flex-wrap: wrap;
}
.col-mcontainer{

Check warning on line 96 in alpha-lab/web-aois/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (mcontainer)
flex: 50%;
padding: 0 4px;
}
@media screen and (min-width: 1025px) and (max-width: 1200px) {
.col-mcontainer{

Check warning on line 101 in alpha-lab/web-aois/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (mcontainer)
flex: 100%;
}
}
@media screen and (max-width: 800px) {
.col-mcontainer{
flex: 50%;
}
}
@media screen and (max-width: 400px) {
.col-mcontainer{
flex: 100%;
}
}
</style>
Loading