Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ui): file upload data shows 1970 #941

Merged
merged 83 commits into from
Aug 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
f63ef7f
init
andrewrisse Aug 14, 2024
442ffcb
remove ./uds
andrewrisse Aug 14, 2024
2ae86d2
add insecure keycloak admin user
andrewrisse Aug 14, 2024
89ef006
add wait for keycloak secret
andrewrisse Aug 14, 2024
c6ca7df
trigger
andrewrisse Aug 14, 2024
72c079e
Test trigger
andrewrisse Aug 14, 2024
4f8323d
Test trigger 2
andrewrisse Aug 14, 2024
7d002c1
Add debug statements
andrewrisse Aug 14, 2024
f3483cc
try workflow var
andrewrisse Aug 14, 2024
8a68a62
try hardcoding command
andrewrisse Aug 14, 2024
aa67235
Merge branch 'main' into workflow-keycloak
andrewrisse Aug 15, 2024
5b9e9f0
change nltk version
andrewrisse Aug 15, 2024
621ca63
fix csp
andrewrisse Aug 15, 2024
de4433a
revert to older setup
andrewrisse Aug 15, 2024
884edf8
Remove wait
andrewrisse Aug 15, 2024
22a7de3
add uds-config
andrewrisse Aug 15, 2024
d47d629
Remove line
andrewrisse Aug 15, 2024
5e0387f
Revert playwright e2e
andrewrisse Aug 15, 2024
1bdcbe8
revert action
andrewrisse Aug 15, 2024
83cd7c7
merge main and delete uds config
andrewrisse Aug 15, 2024
7b54202
add uds-config 2
andrewrisse Aug 15, 2024
5308f5e
add config back in
andrewrisse Aug 15, 2024
4c5ee86
Add debugs and single quote
andrewrisse Aug 15, 2024
492fdc5
Back to double quote
andrewrisse Aug 15, 2024
e43545c
Trigger test 1
andrewrisse Aug 15, 2024
fc72123
Trigger test 2
andrewrisse Aug 15, 2024
e2d8338
merge main
andrewrisse Aug 15, 2024
17b90dd
remove duplicate header
andrewrisse Aug 15, 2024
c68ce6b
use createUser script
andrewrisse Aug 15, 2024
eedff06
use make command and cleanup
andrewrisse Aug 15, 2024
87abf37
change order of dep install
andrewrisse Aug 15, 2024
ade6ba9
Explicitly enable keycloak
andrewrisse Aug 15, 2024
b893039
disable keycloak in build step
andrewrisse Aug 15, 2024
3a51045
fix var name
andrewrisse Aug 15, 2024
e88d4ea
double fill user pass
andrewrisse Aug 16, 2024
c7d8213
Remove --http flag
andrewrisse Aug 16, 2024
42a018a
Debug statements
andrewrisse Aug 16, 2024
680b126
Debug statements 2
andrewrisse Aug 16, 2024
d3e8206
Merge branch 'main' into workflow-keycloak
andrewrisse Aug 16, 2024
aab3691
more debugging
andrewrisse Aug 16, 2024
608c3c7
Merge branch 'main' into workflow-keycloak
justinthelaw Aug 16, 2024
90b9c00
more debugging
andrewrisse Aug 16, 2024
bb26910
add statement to print out created user
andrewrisse Aug 16, 2024
b567243
add http1.1 flag
andrewrisse Aug 20, 2024
f121cea
Debug statements
andrewrisse Aug 20, 2024
60c7397
--data
andrewrisse Aug 20, 2024
754fb41
debug 2
andrewrisse Aug 20, 2024
1d94b68
tac tac
andrewrisse Aug 20, 2024
5fd43c3
echo into new var
andrewrisse Aug 20, 2024
2730e2e
echo curl result
andrewrisse Aug 20, 2024
cb5028d
bump uds cli to v14
andrewrisse Aug 20, 2024
c0313d3
cleanup
andrewrisse Aug 20, 2024
ced7cca
Merge branch 'main' into workflow-keycloak
andrewrisse Aug 20, 2024
8538132
use env var and remove extra echo step
andrewrisse Aug 20, 2024
50553a0
Test password mask
andrewrisse Aug 20, 2024
eec0b1c
add echo tests for createUser.sh
andrewrisse Aug 20, 2024
6481be3
Test echo without mask
andrewrisse Aug 21, 2024
fdabb8d
use var directly
andrewrisse Aug 21, 2024
0e410c9
add missing quote
andrewrisse Aug 22, 2024
b870514
merge main and adjust password so it always has special characters
andrewrisse Aug 22, 2024
ff9deb6
update password generator
andrewrisse Aug 22, 2024
adec79a
Delete test file
andrewrisse Aug 22, 2024
699c230
Merge branch 'main' into workflow-keycloak
andrewrisse Aug 22, 2024
8b32164
init
andrewrisse Aug 22, 2024
d38b689
new debug 1
andrewrisse Aug 22, 2024
81aac3d
wip
andrewrisse Aug 22, 2024
6269ecb
fix cleanup auth
andrewrisse Aug 22, 2024
e82f592
add ts-expect-error
andrewrisse Aug 22, 2024
e427e22
Remove comment
andrewrisse Aug 22, 2024
0d251f6
use github env
andrewrisse Aug 22, 2024
78fe2e8
add and test password mask
andrewrisse Aug 22, 2024
4830c90
Fix import
andrewrisse Aug 22, 2024
5550d2a
Remove echos
andrewrisse Aug 22, 2024
05b240f
Remove test files
andrewrisse Aug 22, 2024
1965375
Merge branch 'workflow-keycloak' into fix-date-bug
andrewrisse Aug 22, 2024
f8bd283
debug service role key
andrewrisse Aug 22, 2024
774ddf1
move debug statement up
andrewrisse Aug 23, 2024
2503d3b
debug
andrewrisse Aug 23, 2024
383fcdb
debug 2
andrewrisse Aug 23, 2024
1c39db3
debug 3
andrewrisse Aug 23, 2024
62da4c6
remove openai client import
andrewrisse Aug 23, 2024
2cde239
debug 4
andrewrisse Aug 23, 2024
c065bb8
lint and add comment
andrewrisse Aug 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/e2e-playwright.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ jobs:
- name: Generate Fake Playwright User Password
id: generate-password
run: |
PASSWORD=$(cat <(openssl rand -base64 32 | tr -dc 'a-zA-Z0-9!@#$%^&*()_+-=[]{}|;:,.<>?' | head -c 20 | sed 's/\(.\{5\}\)/\1!@/g' | head -c 22))
PASSWORD=$(cat <(openssl rand -base64 32 | tr -dc 'a-zA-Z0-9!@#$%^&*()_+-=[]{}|;:,.<>?' | head -c 20) <(echo '!@1Aa') | fold -w1 | shuf | tr -d '\n')
echo "::add-mask::$PASSWORD"
echo "FAKE_E2E_USER_PASSWORD=$PASSWORD" >> $GITHUB_ENV

Expand Down Expand Up @@ -120,7 +120,9 @@ jobs:
cp src/leapfrogai_ui/.env.example src/leapfrogai_ui/.env
mkdir -p playwright/auth
touch playwright/auth.user.json
SERVICE_ROLE_KEY=$(uds zarf tools kubectl get secret -n leapfrogai supabase-bootstrap-jwt -o jsonpath={.data.service-key} | base64 -d) TEST_ENV=CI USERNAME=doug PASSWORD=$FAKE_E2E_USER_PASSWORD PUBLIC_SUPABASE_ANON_KEY=$ANON_KEY npm --prefix src/leapfrogai_ui run test:integration:ci
SERVICE_ROLE_KEY=$(uds zarf tools kubectl get secret -n leapfrogai supabase-bootstrap-jwt -o jsonpath={.data.service-key} | base64 -d)
echo "::add-mask::$SERVICE_ROLE_KEY"
SERVICE_ROLE_KEY=$SERVICE_ROLE_KEY TEST_ENV=CI USERNAME=doug PASSWORD=$FAKE_E2E_USER_PASSWORD PUBLIC_SUPABASE_ANON_KEY=$ANON_KEY npm --prefix src/leapfrogai_ui run test:integration:ci

# Upload the Playwright report as an artifact
- name: Archive Playwright Report
Expand Down
4 changes: 2 additions & 2 deletions src/leapfrogai_ui/src/lib/components/ImportExport.svelte
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<script lang="ts">
import { Button, Spinner } from 'flowbite-svelte';
import { DownloadOutline } from 'flowbite-svelte-icons';
import { DownloadOutline, UploadOutline } from 'flowbite-svelte-icons';
import { threadsStore, toastStore } from '$stores';
import { threadsSchema } from '$schemas/threadSchema';
import type { LFThread } from '$lib/types/threads';
Expand Down Expand Up @@ -88,7 +88,7 @@
size="sm"
on:change={(e) => onUpload(e.detail)}
accept={['application/json']}
disabled={importing}>Import chat history</LFFileUploadBtn
disabled={importing}>Import chat history <UploadOutline /></LFFileUploadBtn
>
{/if}

Expand Down
3 changes: 3 additions & 0 deletions src/leapfrogai_ui/src/lib/helpers/dates.ts
Original file line number Diff line number Diff line change
Expand Up @@ -237,3 +237,6 @@ export const calculateDays = (beginDate: number, endDate: number) => {

return Math.round(differenceInDays);
};

export const convertToMilliseconds = (dateValue: number) =>
dateValue > 10000000000 ? dateValue : dateValue * 1000;
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { superValidate, withFiles, fail } from 'sveltekit-superforms';
import { fail, superValidate, withFiles } from 'sveltekit-superforms';
import type { Actions } from './$types';
import { yup } from 'sveltekit-superforms/adapters';
import type { FileObject } from 'openai/resources/files';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
import { fade } from 'svelte/transition';
import { yup } from 'sveltekit-superforms/adapters';
import { superForm } from 'sveltekit-superforms';
import { formatDate } from '$helpers/dates';
import { convertToMilliseconds, formatDate } from '$helpers/dates';
import { filesSchema } from '$schemas/files';
import { filesStore, toastStore } from '$stores';
import { ACCEPTED_FILE_TYPES } from '$constants';
Expand Down Expand Up @@ -350,9 +350,11 @@
{:else}
<TableBodyCell tdClass="px-4 py-3">{item.filename}</TableBodyCell>
{/if}
<TableBodyCell tdClass="px-4 py-3"
>{formatDate(new Date(item.created_at))}</TableBodyCell
>
{#if item.created_at}
<TableBodyCell tdClass="px-4 py-3"
>{formatDate(new Date(convertToMilliseconds(item.created_at)))}</TableBodyCell
>
{/if}
</TableBodyRow>
{/each}
</TableBody>
Expand Down
37 changes: 34 additions & 3 deletions src/leapfrogai_ui/tests/fixtures.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,39 @@ type MyFixtures = {
openAIClient: OpenAI;
};

export const getOpenAIClient = () => {
export async function getAccessToken() {
const supabaseUrl = process.env.PUBLIC_SUPABASE_URL;
const serviceRoleKey = process.env.SERVICE_ROLE_KEY;

const response = await fetch(`${supabaseUrl}/auth/v1/token?grant_type=password`, {
method: 'POST',
// @ts-expect-error: apikey is a required header for this request
headers: {
'Content-Type': 'application/json',
apikey: serviceRoleKey,
Authorization: `Bearer ${serviceRoleKey}`
},
body: JSON.stringify({
email: process.env.USERNAME,
password: process.env.PASSWORD
})
});

const data = await response.json();

if (response.ok) {
return data.access_token;
} else {
console.error('Error fetching access token:', data);
throw new Error(data.error_description || 'Failed to fetch access token');
}
}

export const getOpenAIClient = async () => {
const token = await getAccessToken();

return new OpenAI({
apiKey: process.env.OPENAI_API_KEY || process.env.SERVICE_ROLE_KEY,
apiKey: process.env.OPENAI_API_KEY || token,
baseURL: process.env.OPENAI_API_KEY
? `${process.env.LEAPFROGAI_API_BASE_URL}/v1`
: `${process.env.LEAPFROGAI_API_BASE_URL}/openai/v1`
Expand All @@ -17,7 +47,8 @@ export const getOpenAIClient = () => {
export const test = base.extend<MyFixtures>({
// eslint-disable-next-line no-empty-pattern
openAIClient: async ({}, use) => {
const client = getOpenAIClient();
const client = await getOpenAIClient();
console.log('client', client);
await use(client);
}
});
Expand Down
10 changes: 4 additions & 6 deletions src/leapfrogai_ui/tests/global.setup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ import { expect, test as setup } from './fixtures';
import * as OTPAuth from 'otpauth';
import { delay } from 'msw';
import type { Page } from '@playwright/test';
import { cleanup } from './helpers/cleanup';

const authFile = 'playwright/.auth/user.json';

Expand Down Expand Up @@ -86,7 +85,10 @@ const logout = async (page: Page) => {
}
};

setup('authenticate', async ({ page, openAIClient }) => {
// NOTE - do not try to use openAIClient from the fixtures here. The user that it attempts to get a token for
// exists in Keycloak because the workflow creates it, but the user has not yet logged in and does not exist in
// Supabase, so the tests will fail.
setup('authenticate', async ({ page }) => {
page.on('pageerror', (err) => {
console.log(err.message);
});
Expand Down Expand Up @@ -117,8 +119,4 @@ setup('authenticate', async ({ page, openAIClient }) => {
// End of authentication steps.

await page.context().storageState({ path: authFile });

if (process.env.TEST_ENV !== 'CI') {
await cleanup(openAIClient);
}
});
5 changes: 4 additions & 1 deletion src/leapfrogai_ui/tests/helpers/apiHelpers.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
import { getAccessToken } from '../fixtures';

export const deleteAllTestAPIKeys = async () => {
const token = await getAccessToken();
const res = await fetch(`${process.env.LEAPFROGAI_API_BASE_URL}/leapfrogai/v1/auth/api-keys`, {
headers: {
Authorization: `Bearer ${process.env.SERVICE_ROLE_KEY}`
Authorization: `Bearer ${token}`
}
});
const keys = await res.json();
Expand Down
5 changes: 2 additions & 3 deletions src/leapfrogai_ui/tests/helpers/cleanup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,13 @@ import { deleteAllGeneratedFixtureFiles, deleteAllTestFilesWithApi } from './fil
import { deleteAllAssistants, deleteAssistantAvatars } from './assistantHelpers';
import { deleteAllTestThreadsWithApi } from './threadHelpers';
import type OpenAI from 'openai';
import { deleteAllTestAPIKeys } from './apiHelpers';

export const cleanup = async (openAIClient: OpenAI) => {
deleteAllGeneratedFixtureFiles();
await deleteAllTestFilesWithApi(openAIClient);
await deleteAllAssistants(openAIClient);
await deleteAllTestThreadsWithApi(openAIClient);
await deleteAssistantAvatars();
// TODO - the deleteAllTestAPIKeys helper uses a leapfrog endpoint that is not authorizing the SERVICE_ROLE_KEY
// https://github.com/defenseunicorns/leapfrogai/issues/936
// await deleteAllTestAPIKeys();
await deleteAllTestAPIKeys();
};
2 changes: 2 additions & 0 deletions src/leapfrogai_ui/tests/helpers/fileHelpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import pptxgen from 'pptxgenjs';
import { expect } from '../fixtures';
import type { FileObject } from 'openai/resources/files';
import { getTableRow } from './helpers';
import { formatDate } from '../../src/lib/helpers/dates';

export const uploadFileWithApi = async (
filename = 'test.pdf',
Expand Down Expand Up @@ -219,6 +220,7 @@ export const testFileUpload = async (filename: string, page: Page, openAIClient:
expect(rowCheckboxesBefore.length).toEqual(0);
await expect(fileUploadedIcon).toBeVisible();
await expect(uploadingFileIcon).not.toBeVisible();
await expect(row.getByText(formatDate(new Date()))).toBeVisible();

// Checkbox should now be present
const rowCheckboxesAfter = await row!.getByRole('checkbox').all();
Expand Down
Loading