Skip to content

Commit

Permalink
remove further mentions of discover and generate
Browse files Browse the repository at this point in the history
  • Loading branch information
fabiomaienschein committed Oct 10, 2024
1 parent 2ba300a commit 6f02a0f
Show file tree
Hide file tree
Showing 11 changed files with 26 additions and 26 deletions.
2 changes: 1 addition & 1 deletion edit-test-case.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Octomind test steps can be flexibly manipulated. You can **drag and drop** steps

You can edit the prompt and have the AI agent `regenerate` test steps based on the new prompt. We will ask you to select **the last step** in the sequence you want to keep. The AI agent will regenerate test steps from this step onwards.

Or you have simply hit the `regenerate` button to rediscover the steps based on the same prompt.
Or you have simply hit the `regenerate` button to regenerate the steps based on the same prompt.

<Frame caption="Edit prompt and restart AI auto-generation, 7/2024">
<img
Expand Down
2 changes: 1 addition & 1 deletion first-steps.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ You should have your **first active test cases** generated. The AI agent has fou

Functioning tests are turned `on` which means they will run once you trigger a **test report**.

<Frame caption="First active test cases discovered and auto-generated - screenshot 07/2024">
<Frame caption="First active test cases auto-generated - screenshot 07/2024">
<img
src="/images/setup/setup-8-first-tests.png"
alt="First active test cases AI auto-generated - screenshot 07/2024 "
Expand Down
4 changes: 2 additions & 2 deletions more-info/faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ We will add building blocks which allow for more demanding scenarios over time,

## 2. How are my tests generated?

We are using our AI agents for test case discovery and test step generation. We'll discover the interaction chain of the test case in an intermediate representation. We'll generate a corresponding Playwright code on the fly and execute on a manual trigger, a schedule or against your pull request.
We are using our AI agents for finding test cases and for generating test steps. We'll generate an interaction chain of the test case in an intermediate representation. We'll then generate a corresponding Playwright code on the fly and execute on a manual trigger, a schedule or against your pull request.

We generate tests on [sign-up](/first-steps#4-open-the-octomind-app-for-the-first-time), when [we auto-generate 3 initial test cases](/first-steps#5-we-are-auto-generating-test-cases-for-you) and when you ask our AI agents to [generate more](/generate-more-test-cases) tests.

Expand All @@ -27,7 +27,7 @@ End-to-end tests are notoriously flaky. Some of our strategies to fight flakines
- Smart learning based retries
- active interaction timing (sleeps)
- AI based analysis of unexpected circumstances
- Rediscovery in case of user flow changes
- Regeneration in case of user flow changes

## 5. How can I run your tests locally?

Expand Down
2 changes: 1 addition & 1 deletion more-info/octo-dictionary.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: "Essential terminology used within the Octomind app"
icon: "book"
---

### AI discovery
### AI auto-generation

Happens when our AI agents traverse the publicly accessible code in the DOM and use the vision capability of the underlying multimodal LLM to add visual context.
They are looking for meaningful user flows to be tested.
Expand Down
4 changes: 2 additions & 2 deletions more-info/under-the-hood.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: "If you are interested in how we make the tests work, this is your
icon: "engine"
---

## AI agents discovering and generating test steps
## AI agents finding test cases and generating test steps

A web app is typically composed of user flows. A user flow lets a user accomplish a certain goal. "Sign-in with email & password" is an example.
To perform a sign-in with email & password, a certain sequence of steps - `interactions` - is required.
Expand Down Expand Up @@ -63,5 +63,5 @@ Test flakiness is the biggest problem of browser tests. Fighting flakiness is an
| Active interaction timing to handle varying response times | work in progress |
| Smart learning based retries to understand flakiness level | open |
| AI based analysis of unexpected circumstances to handle temporary pop-ups, toasts and similar stuff which would otherwise break the test | open |
| AI based rediscovery in case of permanent user flow changes | open |
| AI based regeneration in case of permanent user flow changes | open |
| Coding rules and best practices of how to write tests in a way to avoid common pitfalls. | work in progress |
2 changes: 1 addition & 1 deletion new-test-case.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ If the AI agent generated wrong steps or signals a failed step with a yellow ale

### Pre-prompted login test

If your app has a login functionality that wasn't AI discovered at the set-up, we have a pre-prompted login test case for you. You can trigger it after giving us test user credentials.
If your app has a login functionality that wasn't AI auto-generated during set-up, we have a pre-prompted login test case for you. You can trigger it after giving us test user credentials.

Go to `settings` a.k.a. the `gear icon` in the upper right corner and select `test account`. Add test user credentials we can use to run the login test.

Expand Down
8 changes: 4 additions & 4 deletions private-location.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ It requires access permission to specific public IPs: `35.192.162.70` or `34.159

The embedded Squid proxy server allows requests from the private network, mimicking public proxy requests from within the customer’s network.

### Test execution and discovery
### Test execution and generation

After registering the worker with Octomind, users can configure the proxy location for executing discovery or test runs.
After registering the worker with Octomind, users can configure the proxy location for executing generation or test runs.

## Usage

Expand All @@ -50,8 +50,8 @@ be unique for your organization.

<Info>
{" "}
Currently, only one instance of the worker is supported. We will support
multiple instances of the worker with the same name as we scale.
Currently, only one instance of the worker is supported. We will support multiple

Check warning on line 53 in private-location.mdx

View workflow job for this annotation

GitHub Actions / eslint

Replace `·multiple⏎·` with `⏎··multiple`
instances of the worker with the same name as we scale.
</Info>

## Run
Expand Down
8 changes: 4 additions & 4 deletions proxy/private-location.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ It requires access permission to specific public IPs: `35.192.162.70` or `34.159

The embedded Squid proxy server allows requests from the private network, mimicking public proxy requests from within the customer’s network.

### Test execution and discovery
### Test execution and generation

After registering the worker with Octomind, users can configure the proxy location for executing discovery or test runs.
After registering the worker with Octomind, users can configure the proxy location for executing generation or test runs.

## Usage

Expand All @@ -50,8 +50,8 @@ be unique for your organization.

<Info>
{" "}
Currently, only one instance of the worker is supported. We will support
multiple instances of the worker with the same name as we scale.
Currently, only one instance of the worker is supported. We will support multiple

Check warning on line 53 in proxy/private-location.mdx

View workflow job for this annotation

GitHub Actions / eslint

Replace `·multiple⏎·` with `⏎··multiple`
instances of the worker with the same name as we scale.
</Info>

## Run
Expand Down
4 changes: 2 additions & 2 deletions proxy/proxy-settings.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ icon: "server"

## Overview

In project settings you can control from which location your website is being discovered and also
from where your test will be runned.
In the project settings you can control from which location your website is being accessed and also
from where your test will be ran.

<Frame caption="proxy settings">
<img src="/images/proxy/proxy-settings.png" alt="proxy settings" />
Expand Down
12 changes: 6 additions & 6 deletions rate-limiting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ description: "What is rate limiting and how to deal with it"
icon: "gauge-max"
---

Whenever the AI agent discovers user flows on your page, or when the test runner executes the tests - we are sending requests to your application.
Whenever the AI agent generates tests for user flows on your page, or when the test runner executes the tests - we are sending requests to your application.

To ensure a well defined behavior all requests are sent through **two fixed IP addresses** - `35.192.162.70` and `34.159.153.198` (europe proxy).

## AI agent discovery
## AI generation

AI agent discovers user flows on your site in a similar way a human would use your web application. However, most of the time the agent interacts with the website slightly faster than a human would.
Also, more discoveries might run in parallel.
AI agent generates test cases based on user flows on your site in a similar way a human would use your web application. However, most of the time the agent interacts with the website slightly faster than a human would.
Also, test case generation can be run in parallel.

## Test runner

Expand All @@ -20,7 +20,7 @@ This can lead to either an overload of your application or a rate limiter gettin

## Login / Authentication

If your application requires a login, the configured test user is used for the AI agent discovery as well as for the test runs. This could also lead to **hitting a limit of allowed logins**, depending on your login system.
If your application requires a login, the configured test user is used for the AI test case auto-generation as well as for the test runs. This could also lead to **hitting a limit of allowed logins**, depending on your login system.
If you are using a 3rd party login service, there are most likely security configurations in place that prevent users from logging in too frequently.

### OTP / 2FA
Expand All @@ -29,5 +29,5 @@ While Octomind supports a login with a **one time password (OTP)**, it might run

## Rate limiter

If your application is protected against [DoS Attacks](https://en.wikipedia.org/wiki/Denial-of-service_attack) with a [rate limiter](https://en.wikipedia.org/wiki/Rate_limiting) it is very likely that Octomind test runs and/or AI discoveries will fail.
If your application is protected against [DoS Attacks](https://en.wikipedia.org/wiki/Denial-of-service_attack) with a [rate limiter](https://en.wikipedia.org/wiki/Rate_limiting) it is very likely that Octomind test runs and/or AI generation will fail.
The easiest solutions is to configure and exception for our IP address: `35.192.162.70` and `34.159.153.198` (europe proxy) for white listing.
4 changes: 2 additions & 2 deletions test-case-creation-strategy.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ You can have the AI agent generate more test cases. Go into the test case and ho

The AI agent will auto-generate up to 3 new tests based on the one you started it from. The original test will be added as **dependency**. We will auto-generate steps for every found test case and validate them.

<Frame caption="Have the AI agent suggest more test cases, 10/2024">
<Frame caption="Have the AI agent generate more test cases, 10/2024">
<img
src="/images/expand/generate-more.png"
alt="have AI suggest more test cases"
alt="have AI generate more test cases"
/>
</Frame>

Expand Down

0 comments on commit 6f02a0f

Please sign in to comment.