Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TM-1531] add job connection #745

Open
wants to merge 25 commits into
base: staging
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
aa5b7e6
[TM-1531] add job service url and script
egrojMonroy Dec 11, 2024
153050a
[TM-1531] apiSlice add job resource
egrojMonroy Dec 12, 2024
fa4d2c4
[TM-1531] updates for job service
egrojMonroy Dec 12, 2024
9fbece2
[TM-1531] sort alphabetical
egrojMonroy Dec 12, 2024
8303877
[TM-1531] add connection for list the delayed jobs
egrojMonroy Dec 13, 2024
06783b3
[TM-1531] remove unnecesary code and consistency in selector and API …
egrojMonroy Dec 16, 2024
4972958
[TM-1531] Merge branch 'staging' into feat/TM-1531-add-job-connection
egrojMonroy Dec 16, 2024
e8cef37
[TM-1531] add bulk update call on clear
egrojMonroy Dec 17, 2024
ded0b85
[TM-1531] add entityname and display data in notification
egrojMonroy Dec 17, 2024
c8d7c71
[TM-1531] use store and display only not acknowledged
egrojMonroy Dec 17, 2024
07e2d56
[TM-1531] remove from apifetcher call and add continous call for dela…
egrojMonroy Dec 18, 2024
9d45753
[TM-1531] Merge branch 'staging' into feat/TM-1531-add-job-service
egrojMonroy Dec 18, 2024
05fdae6
[TM-1531] Merge branch 'feat/TM-1531-add-job-service' into feat/TM-15…
egrojMonroy Dec 18, 2024
4637c98
[TM-1531] rollback apifetcher functionality
egrojMonroy Dec 18, 2024
e58c0a1
[TM-1531] adding message to notification
egrojMonroy Dec 19, 2024
996d5a7
[TM-1531] display error messages in notifications
egrojMonroy Dec 19, 2024
b6e0102
[TM-1531] add error messages for failed jobs
egrojMonroy Dec 19, 2024
fc1a52a
[TM-1531] improve connection and remove awaits and asyncs
egrojMonroy Dec 19, 2024
69d08d7
[TM-1531] changes and remove log
egrojMonroy Dec 19, 2024
d5ba353
[TM-1531] Merge branch 'release/utltimate-ulmus' into feat/TM-1531-ad…
egrojMonroy Dec 19, 2024
042ecbc
[TM-1531] yarn generate:services
egrojMonroy Dec 19, 2024
06a7958
[TM-1531] divide names loading for connections
egrojMonroy Dec 20, 2024
6cad9a9
[TM-1531] running with build
egrojMonroy Dec 20, 2024
af9b8da
[TM-1531] only send not pending
egrojMonroy Dec 20, 2024
1b73259
[TM-1531] fix loaded delayedjobs and filter
egrojMonroy Dec 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 1 addition & 7 deletions src/admin/components/ResourceTabs/PolygonReviewTab/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ import { IconNames } from "@/components/extensive/Icon/Icon";
import ModalAdd from "@/components/extensive/Modal/ModalAdd";
import ModalConfirm from "@/components/extensive/Modal/ModalConfirm";
import { ModalId } from "@/components/extensive/Modal/ModalConst";
import { useLoading } from "@/context/loaderAdmin.provider";
import { useMapAreaContext } from "@/context/mapArea.provider";
import { useModalContext } from "@/context/modal.provider";
import { useMonitoredDataContext } from "@/context/monitoredData.provider";
Expand Down Expand Up @@ -152,7 +151,6 @@ const PolygonReviewTab: FC<IProps> = props => {
const [saveFlags, setSaveFlags] = useState<boolean>(false);
const [polygonFromMap, setPolygonFromMap] = useState<IpolygonFromMap>({ isOpen: false, uuid: "" });
const [errorMessage, setErrorMessage] = useState<string | null>(null);
const { showLoader, hideLoader } = useLoading();
const {
setSelectedPolygonsInCheckbox,
setPolygonCriteriaMap,
Expand Down Expand Up @@ -314,8 +312,7 @@ const PolygonReviewTab: FC<IProps> = props => {
}, [shouldRefetchValidation]);
const uploadFiles = async () => {
const uploadPromises = [];
showLoader();

closeModal(ModalId.ADD_POLYGON);
for (const file of files) {
const fileToUpload = file.rawFile as File;
const site_uuid = record.uuid;
Expand Down Expand Up @@ -350,10 +347,8 @@ const PolygonReviewTab: FC<IProps> = props => {
}
refetch();
refetchSiteBbox();
closeModal(ModalId.ADD_POLYGON);
setPolygonLoaded(false);
setSubmitPolygonLoaded(false);
hideLoader();
} catch (error) {
let errorMessage;

Expand All @@ -379,7 +374,6 @@ const PolygonReviewTab: FC<IProps> = props => {
errorMessage = t("An unknown error occurred");
}
openNotification("error", t("Error uploading file"), errorMessage || t("An unknown error occurred"));
hideLoader();
}
};

Expand Down
19 changes: 2 additions & 17 deletions src/admin/modules/sites/components/SiteShow.tsx
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
import { FC, useState } from "react";
import { FC } from "react";
import { Show, TabbedShowLayout } from "react-admin";

import ShowActions from "@/admin/components/Actions/ShowActions";
import DelayedJobsProgressAlert from "@/admin/components/Alerts/DelayedJobsProgressAlert";
import AuditLogTab from "@/admin/components/ResourceTabs/AuditLogTab/AuditLogTab";
import { AuditLogButtonStates } from "@/admin/components/ResourceTabs/AuditLogTab/constants/enum";
import ChangeRequestsTab from "@/admin/components/ResourceTabs/ChangeRequestsTab/ChangeRequestsTab";
Expand All @@ -16,9 +15,6 @@ import { RecordFrameworkProvider } from "@/context/framework.provider";
import { MapAreaProvider } from "@/context/mapArea.provider";

const SiteShow: FC = () => {
const [isLoadingDelayedJob, setIsLoadingDelayedJob] = useState(false);
const [alertTitle, setAlertTitle] = useState("");

return (
<Show
title={<ShowTitle moduleName="Site" getTitle={record => record?.name} />}
Expand All @@ -30,13 +26,7 @@ const SiteShow: FC = () => {
<InformationTab type="sites" />
<TabbedShowLayout.Tab label="Polygon Review">
<MapAreaProvider>
<PolygonReviewTab
label=""
type={"sites"}
setIsLoadingDelayedJob={setIsLoadingDelayedJob!}
isLoadingDelayedJob={isLoadingDelayedJob!}
setAlertTitle={setAlertTitle!}
/>
<PolygonReviewTab label="" type={"sites"} />
</MapAreaProvider>
</TabbedShowLayout.Tab>
<GalleryTab label="Site Gallery" entity="sites" />
Expand All @@ -46,11 +36,6 @@ const SiteShow: FC = () => {
<AuditLogTab entity={AuditLogButtonStates.SITE} />
</TabbedShowLayout>
</RecordFrameworkProvider>
<DelayedJobsProgressAlert
show={isLoadingDelayedJob}
title={alertTitle}
setIsLoadingDelayedJob={setIsLoadingDelayedJob!}
/>
</Show>
);
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,7 @@ const CheckPolygonControl = (props: CheckSitePolygonProps) => {

const runFixPolygonOverlaps = () => {
if (siteUuid) {
closeModal(ModalId.FIX_POLYGONS);
setIsLoadingDelayedJob?.(true);
setAlertTitle?.("Fix Polygons");
clipPolygons({ pathParams: { uuid: siteUuid } });
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ const ProcessBulkPolygonsControl = ({
className: "px-8 py-3",
variant: "primary",
onClick: () => {
closeModal(ModalId.FIX_POLYGONS);
setIsLoadingDelayedJob?.(true);
setAlertTitle?.("Fix Polygons");
fixPolygons(
Expand All @@ -124,7 +125,7 @@ const ProcessBulkPolygonsControl = ({
{
onSuccess: response => {
const processedNames = response?.processed?.map(item => item.poly_name).join(", ");
closeModal(ModalId.FIX_POLYGONS);

setIsLoadingDelayedJob?.(false);
ApiSlice.addTotalContent(0);
ApiSlice.addProgressContent(0);
Expand Down
138 changes: 114 additions & 24 deletions src/components/elements/Notification/FloatNotification.tsx
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
import { LinearProgress } from "@mui/material";
import classNames from "classnames";
import { useState } from "react";
import { useEffect, useState } from "react";
import { When } from "react-if";

import Icon, { IconNames } from "@/components/extensive/Icon/Icon";
import { triggerBulkUpdate, useDelayedJobs } from "@/connections/DelayedJob";
import { DelayedJobData, DelayedJobDto } from "@/generated/v3/jobService/jobServiceSchemas";
import { getErrorMessageFromPayload } from "@/utils/errors";

import LinearProgressBar from "../ProgressBar/LinearProgressBar/LinearProgressBar";
import Text from "../Text/Text";
Expand All @@ -17,8 +21,49 @@ export interface FloatNotificationProps {
data: FloatNotificationDataProps[];
}

const FloatNotification = ({ data }: FloatNotificationProps) => {
const FloatNotification = () => {
const [openModalNotification, setOpenModalNotification] = useState(false);
const [isLoaded, { delayedJobs }] = useDelayedJobs();
const [notAcknowledgedJobs, setNotAcknowledgedJobs] = useState<DelayedJobDto[]>([]);
const clearJobs = () => {
if (delayedJobs === undefined) return;
const newJobsData: DelayedJobData[] = delayedJobs
.filter((job: DelayedJobDto) => job.status !== "pending")
.map((job: DelayedJobDto) => {
return {
uuid: job.uuid,
type: "delayedJobs",
attributes: {
isAcknowledged: true
}
};
});
triggerBulkUpdate(newJobsData);
};
useEffect(() => {
if (delayedJobs === undefined) return;
const notAcknowledgedJobs = delayedJobs.filter((job: DelayedJobDto) => !job.isAcknowledged);
setNotAcknowledgedJobs(notAcknowledgedJobs);
}, [delayedJobs]);
useEffect(() => {
if (!notAcknowledgedJobs.length) {
setOpenModalNotification(false);
}
}, [notAcknowledgedJobs]);
const listOfPolygonsFixed = (data: Record<string, any> | null) => {
if (data?.updated_polygons) {
const updatedPolygonNames = data.updated_polygons
?.map((p: any) => p.poly_name)
.filter(Boolean)
.join(", ");
if (updatedPolygonNames) {
return "Success! The following polygons have been fixed: " + updatedPolygonNames;
} else {
return "No polygons were fixed";
}
}
return null;
};

return (
<div className="fixed bottom-10 right-10 z-50">
Expand All @@ -38,36 +83,81 @@ const FloatNotification = ({ data }: FloatNotificationProps) => {
<Text variant="text-14-light" className="text-neutral-400">
Actions Taken
</Text>
<Text variant="text-12-semibold" className="text-primary">
<Text variant="text-12-semibold" className="text-primary" onClick={clearJobs}>
Clear completed
</Text>
</div>
<div className="-mr-2 flex flex-1 flex-col gap-3 overflow-auto pr-2">
{data.map((item, index) => (
<div key={index} className="rounded-lg border-2 border-grey-350 bg-white p-4 hover:border-primary">
<div className="mb-2 flex items-center gap-1">
<div className="h-2 w-2 rounded-full bg-primary" />
<Text variant="text-14-light" className="leading-[normal] text-darkCustom " as={"span"}>
{item.label}
</Text>
</div>
<Text variant="text-14-light" className="text-darkCustom">
Site: <b>{item.site}</b>
</Text>
<div className="mt-2 flex items-center gap-2">
<LinearProgressBar value={parseInt(item.value)} className="h-2 bg-success-40" color="success-600" />
<Text variant="text-12-semibold" className="text-black">
{item.value}
{isLoaded &&
notAcknowledgedJobs &&
notAcknowledgedJobs.map((item, index) => (
<div key={index} className="rounded-lg border-2 border-grey-350 bg-white p-4 hover:border-primary">
<div className="mb-2 flex items-center gap-1">
<div className="h-2 w-2 rounded-full bg-primary" />
<Text variant="text-14-light" className="leading-[normal] text-darkCustom " as={"span"}>
{item.name}
</Text>
</div>
<Text variant="text-14-light" className="text-darkCustom">
Site: <b>{item.entityName}</b>
</Text>
<div className="mt-2">
{item.status === "failed" ? (
<Text variant="text-12-semibold" className="text-error-600">
{item.payload ? getErrorMessageFromPayload(item.payload) : "Failed to complete"}
</Text>
) : (
<div className="flex items-center gap-2">
{item.name === "Polygon Upload" &&
(item.processedContent === null || item.totalContent === null) &&
item.status === "pending" ? (
<div style={{ width: "100%" }}>
<LinearProgress
sx={{
height: 9,
borderRadius: 99,
backgroundColor: "#a9e7d6",
"& .MuiLinearProgress-bar": { backgroundColor: "#29c499" }
}}
/>
</div>
) : (
<LinearProgressBar
value={
item.status === "succeeded"
? 100
: ((item.processedContent ?? 0) / (item.totalContent ?? 1)) * 100
}
className="h-2 bg-success-40"
color="success-600"
/>
)}
<Text variant="text-12-semibold" className="text-black">
{item.name === "Polygon Upload"
? item.status === "succeeded"
? "Done!"
: ""
: item.status === "succeeded"
? "Done!"
: `${Math.round(((item.processedContent ?? 0) / (item.totalContent ?? 1)) * 100)}%`}
</Text>
</div>
)}

{item.status === "succeeded" && (
<Text variant="text-12-light" className="mt-2 text-neutral-500">
{listOfPolygonsFixed(item.payload)}
</Text>
)}
</div>
</div>
</div>
))}
))}
</div>
</div>
</div>
<When condition={data.length > 0}>
<When condition={isLoaded && (notAcknowledgedJobs ?? []).length > 0}>
<div className="text-14-bold absolute right-[-5px] top-[-5px] z-20 flex min-h-[24px] min-w-[24px] items-center justify-center rounded-full bg-red-300 leading-[normal] text-white">
{data.length}
{notAcknowledgedJobs?.length}
</div>
</When>
<button
Expand All @@ -77,8 +167,8 @@ const FloatNotification = ({ data }: FloatNotificationProps) => {
className={classNames(
"z-10 flex h-15 w-15 items-center justify-center rounded-full border border-grey-950 bg-primary duration-300 hover:scale-105",
{
hidden: data.length < 1,
visible: data.length > 0
hidden: (notAcknowledgedJobs?.length ?? 0) === 0,
visible: (notAcknowledgedJobs?.length ?? 0) > 0
}
)}
>
Expand Down
100 changes: 100 additions & 0 deletions src/connections/DelayedJob.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
import { useEffect } from "react";
import { createSelector } from "reselect";

import { bulkUpdateJobs, listDelayedJobs } from "@/generated/v3/jobService/jobServiceComponents";
import {
bulkUpdateJobsFetchFailed,
bulkUpdateJobsIsFetching,
listDelayedJobsFetchFailed
} from "@/generated/v3/jobService/jobServicePredicates";
import { DelayedJobData, DelayedJobDto } from "@/generated/v3/jobService/jobServiceSchemas";
import { useConnection } from "@/hooks/useConnection";
import { ApiDataStore } from "@/store/apiSlice";
import { Connection } from "@/types/connection";
import { connectionLoader } from "@/utils/connectionShortcuts";

type DelayedJobsConnection = {
delayedJobs?: DelayedJobDto[];
delayedJobsIsLoading: boolean;
delayedJobsHasFailed: boolean;
};

const delayedJobsSelector = (store: ApiDataStore) =>
Object.values(store.delayedJobs ?? {}).map(resource => resource.attributes);

const delayedJobsLoadFailedSelector = (store: ApiDataStore) => listDelayedJobsFetchFailed(store) != null;

const delayedJobsIsLoaded = ({ delayedJobs, delayedJobsHasFailed, delayedJobsIsLoading }: DelayedJobsConnection) =>
delayedJobs != null || delayedJobsHasFailed;

const delayedJobsConnection: Connection<DelayedJobsConnection> = {
load: connection => {
if (!delayedJobsIsLoaded(connection)) {
console.log("list call");
listDelayedJobs();
}
},
isLoaded: delayedJobsIsLoaded,
selector: createSelector(
[delayedJobsSelector, delayedJobsLoadFailedSelector],
(delayedJobs, delayedJobsHasFailed) => ({
delayedJobs,
delayedJobsIsLoading: delayedJobs == null && !delayedJobsHasFailed,
delayedJobsHasFailed
})
)
};

export const useDelayedJobs = () => {
const connection = useConnection(delayedJobsConnection);

useEffect(() => {
const intervalId = setInterval(() => {
console.log("This call");
listDelayedJobs();
}, 1500);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will cause this interval to keep refreshing the delayed jobs for as long as the surrounding component is mounted. Is that intentional? It seems like clearing the interval when all the delayed jobs have completed would be good to keep from continuing to hit the jobs endpoint. I'm OK with letting this live to prod if that's complicated to wire up; it's just not ideal and should be cleaned up in a future ticket.

Please either decide to implement an eventual stop like that in this PR, or see that a tech debt ticket is created to clean up in a future release.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes indeed, it is supposed to be attentive for any job created on any session.
Nonetheless I think it might not be good.
So for now the debt ticket will be needed, because we would have to think on the user cases.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the component using this hook expected to be mounted a lot of the time, or does the user have to go to a specific view for it? If this is up all the time, then I think we need to consider a better approach. If it's a view that's only mounted when the user is actively watching for something to finish, then I think this OK for now.


return () => {
clearInterval(intervalId);
};
roguenet marked this conversation as resolved.
Show resolved Hide resolved
}, []);

return connection;
};

type BulkUpdateJobsConnection = {
bulkUpdateJobsIsLoading: boolean;
bulkUpdateJobsHasFailed: boolean;
updatedJobsResponse?: DelayedJobDto[];
};

const bulkUpdateJobsSelector = (store: ApiDataStore) => ({
bulkUpdateJobsIsLoading: bulkUpdateJobsIsFetching(store),
bulkUpdateJobsHasFailed: bulkUpdateJobsFetchFailed(store) != null,
updatedJobsResponse: Object.values(store.delayedJobs ?? {}).map(resource => resource.attributes as DelayedJobDto)
});

const bulkUpdateJobsIsLoaded = ({ bulkUpdateJobsIsLoading, bulkUpdateJobsHasFailed }: BulkUpdateJobsConnection) =>
!bulkUpdateJobsIsLoading && !bulkUpdateJobsHasFailed;

const bulkUpdateJobsConnection: Connection<BulkUpdateJobsConnection, { jobs: DelayedJobData[] }> = {
load: (connection, { jobs }) => {
if (!bulkUpdateJobsIsLoaded(connection)) {
bulkUpdateJobs({ body: { data: jobs } });
}
},
isLoaded: bulkUpdateJobsIsLoaded,
selector: createSelector(
bulkUpdateJobsSelector,
({ bulkUpdateJobsIsLoading, bulkUpdateJobsHasFailed, updatedJobsResponse }) => ({
bulkUpdateJobsIsLoading,
bulkUpdateJobsHasFailed,
updatedJobsResponse
})
)
};
roguenet marked this conversation as resolved.
Show resolved Hide resolved

export const useBulkUpdateJobs = connectionLoader(bulkUpdateJobsConnection);

// Function to trigger bulk update
export const triggerBulkUpdate = (jobs: DelayedJobData[]) => bulkUpdateJobs({ body: { data: jobs } });
Loading
Loading