Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TM-1531] add job connection #745

Open
wants to merge 25 commits into
base: staging
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
aa5b7e6
[TM-1531] add job service url and script
egrojMonroy Dec 11, 2024
153050a
[TM-1531] apiSlice add job resource
egrojMonroy Dec 12, 2024
fa4d2c4
[TM-1531] updates for job service
egrojMonroy Dec 12, 2024
9fbece2
[TM-1531] sort alphabetical
egrojMonroy Dec 12, 2024
8303877
[TM-1531] add connection for list the delayed jobs
egrojMonroy Dec 13, 2024
06783b3
[TM-1531] remove unnecesary code and consistency in selector and API …
egrojMonroy Dec 16, 2024
4972958
[TM-1531] Merge branch 'staging' into feat/TM-1531-add-job-connection
egrojMonroy Dec 16, 2024
e8cef37
[TM-1531] add bulk update call on clear
egrojMonroy Dec 17, 2024
ded0b85
[TM-1531] add entityname and display data in notification
egrojMonroy Dec 17, 2024
c8d7c71
[TM-1531] use store and display only not acknowledged
egrojMonroy Dec 17, 2024
07e2d56
[TM-1531] remove from apifetcher call and add continous call for dela…
egrojMonroy Dec 18, 2024
9d45753
[TM-1531] Merge branch 'staging' into feat/TM-1531-add-job-service
egrojMonroy Dec 18, 2024
05fdae6
[TM-1531] Merge branch 'feat/TM-1531-add-job-service' into feat/TM-15…
egrojMonroy Dec 18, 2024
4637c98
[TM-1531] rollback apifetcher functionality
egrojMonroy Dec 18, 2024
e58c0a1
[TM-1531] adding message to notification
egrojMonroy Dec 19, 2024
996d5a7
[TM-1531] display error messages in notifications
egrojMonroy Dec 19, 2024
b6e0102
[TM-1531] add error messages for failed jobs
egrojMonroy Dec 19, 2024
fc1a52a
[TM-1531] improve connection and remove awaits and asyncs
egrojMonroy Dec 19, 2024
69d08d7
[TM-1531] changes and remove log
egrojMonroy Dec 19, 2024
d5ba353
[TM-1531] Merge branch 'release/utltimate-ulmus' into feat/TM-1531-ad…
egrojMonroy Dec 19, 2024
042ecbc
[TM-1531] yarn generate:services
egrojMonroy Dec 19, 2024
06a7958
[TM-1531] divide names loading for connections
egrojMonroy Dec 20, 2024
6cad9a9
[TM-1531] running with build
egrojMonroy Dec 20, 2024
af9b8da
[TM-1531] only send not pending
egrojMonroy Dec 20, 2024
1b73259
[TM-1531] fix loaded delayedjobs and filter
egrojMonroy Dec 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 1 addition & 7 deletions src/admin/components/ResourceTabs/PolygonReviewTab/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ import { IconNames } from "@/components/extensive/Icon/Icon";
import ModalAdd from "@/components/extensive/Modal/ModalAdd";
import ModalConfirm from "@/components/extensive/Modal/ModalConfirm";
import { ModalId } from "@/components/extensive/Modal/ModalConst";
import { useLoading } from "@/context/loaderAdmin.provider";
import { useMapAreaContext } from "@/context/mapArea.provider";
import { useModalContext } from "@/context/modal.provider";
import { useMonitoredDataContext } from "@/context/monitoredData.provider";
Expand Down Expand Up @@ -152,7 +151,6 @@ const PolygonReviewTab: FC<IProps> = props => {
const [saveFlags, setSaveFlags] = useState<boolean>(false);
const [polygonFromMap, setPolygonFromMap] = useState<IpolygonFromMap>({ isOpen: false, uuid: "" });
const [errorMessage, setErrorMessage] = useState<string | null>(null);
const { showLoader, hideLoader } = useLoading();
const {
setSelectedPolygonsInCheckbox,
setPolygonCriteriaMap,
Expand Down Expand Up @@ -314,8 +312,7 @@ const PolygonReviewTab: FC<IProps> = props => {
}, [shouldRefetchValidation]);
const uploadFiles = async () => {
const uploadPromises = [];
showLoader();

closeModal(ModalId.ADD_POLYGON);
for (const file of files) {
const fileToUpload = file.rawFile as File;
const site_uuid = record.uuid;
Expand Down Expand Up @@ -350,10 +347,8 @@ const PolygonReviewTab: FC<IProps> = props => {
}
refetch();
refetchSiteBbox();
closeModal(ModalId.ADD_POLYGON);
setPolygonLoaded(false);
setSubmitPolygonLoaded(false);
hideLoader();
} catch (error) {
let errorMessage;

Expand All @@ -379,7 +374,6 @@ const PolygonReviewTab: FC<IProps> = props => {
errorMessage = t("An unknown error occurred");
}
openNotification("error", t("Error uploading file"), errorMessage || t("An unknown error occurred"));
hideLoader();
}
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,7 @@ const CheckPolygonControl = (props: CheckSitePolygonProps) => {

const runFixPolygonOverlaps = () => {
if (siteUuid) {
closeModal(ModalId.FIX_POLYGONS);
setIsLoadingDelayedJob?.(true);
setAlertTitle?.("Fix Polygons");
clipPolygons({ pathParams: { uuid: siteUuid } });
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ const ProcessBulkPolygonsControl = ({
className: "px-8 py-3",
variant: "primary",
onClick: () => {
closeModal(ModalId.FIX_POLYGONS);
setIsLoadingDelayedJob?.(true);
setAlertTitle?.("Fix Polygons");
fixPolygons(
Expand All @@ -124,7 +125,7 @@ const ProcessBulkPolygonsControl = ({
{
onSuccess: response => {
const processedNames = response?.processed?.map(item => item.poly_name).join(", ");
closeModal(ModalId.FIX_POLYGONS);

setIsLoadingDelayedJob?.(false);
ApiSlice.addTotalContent(0);
ApiSlice.addProgressContent(0);
Expand Down
14 changes: 11 additions & 3 deletions src/components/elements/Notification/FloatNotification.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ const FloatNotification = () => {
}
};
});
console.log("Should trigger bulk update");
triggerBulkUpdate(newJobsData);
};
useEffect(() => {
Expand All @@ -54,7 +55,11 @@ const FloatNotification = () => {
?.map((p: any) => p.poly_name)
.filter(Boolean)
.join(", ");
return "Success! The following polygons have been fixed: " + updatedPolygonNames;
if (updatedPolygonNames) {
return "Success! The following polygons have been fixed: " + updatedPolygonNames;
} else {
return "No polygons were fixed";
}
}
return null;
};
Expand Down Expand Up @@ -126,9 +131,12 @@ const FloatNotification = () => {
color="success-600"
/>
)}

<Text variant="text-12-semibold" className="text-black">
{item.status === "succeeded"
{item.name === "Polygon Upload"
? item.status === "succeeded"
? "Done!"
: ""
: item.status === "succeeded"
? "Done!"
: `${Math.round(((item.processedContent ?? 0) / (item.totalContent ?? 1)) * 100)}%`}
</Text>
Expand Down
61 changes: 24 additions & 37 deletions src/connections/DelayedJob.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@ import {
listDelayedJobsFetchFailed
} from "@/generated/v3/jobService/jobServicePredicates";
import { DelayedJobData, DelayedJobDto } from "@/generated/v3/jobService/jobServiceSchemas";
import { useConnection } from "@/hooks/useConnection";
import { ApiDataStore } from "@/store/apiSlice";
import { Connection } from "@/types/connection";
import { connectionHook, connectionLoader } from "@/utils/connectionShortcuts";
import { connectionLoader } from "@/utils/connectionShortcuts";

// --- Delayed Jobs Connection ---
type DelayedJobsConnection = {
Expand All @@ -19,18 +20,13 @@ type DelayedJobsConnection = {
hasLoadFailed: boolean;
};

const delayedJobsSelector = (store: ApiDataStore) => {
const delayedJobsMap = store.delayedJobs || {};
return Object.values(delayedJobsMap).map(resource => resource.attributes);
};
const delayedJobsSelector = (store: ApiDataStore) =>
Object.values(store.delayedJobs ?? {}).map(resource => resource.attributes);

const delayedJobsLoadFailedSelector = (store: ApiDataStore) => {
return listDelayedJobsFetchFailed(store) != null;
};
const delayedJobsLoadFailedSelector = (store: ApiDataStore) => listDelayedJobsFetchFailed(store) != null;

const connectionIsLoaded = ({ delayedJobs, hasLoadFailed, isLoading }: DelayedJobsConnection) => {
return (delayedJobs != null && delayedJobs.length > 0) || hasLoadFailed || isLoading;
};
const connectionIsLoaded = ({ delayedJobs, hasLoadFailed, isLoading }: DelayedJobsConnection) =>
(delayedJobs != null && delayedJobs.length > 0) || hasLoadFailed || isLoading;

const delayedJobsConnection: Connection<DelayedJobsConnection> = {
load: connection => {
Expand All @@ -40,18 +36,15 @@ const delayedJobsConnection: Connection<DelayedJobsConnection> = {
}
},
isLoaded: connectionIsLoaded,
selector: createSelector(
[delayedJobsSelector, store => delayedJobsLoadFailedSelector(store)],
(delayedJobs, hasLoadFailed): DelayedJobsConnection => ({
delayedJobs,
isLoading: delayedJobs == null && !hasLoadFailed,
hasLoadFailed
})
)
selector: createSelector([delayedJobsSelector, delayedJobsLoadFailedSelector], (delayedJobs, hasLoadFailed) => ({
delayedJobs,
isLoading: delayedJobs == null && !hasLoadFailed,
hasLoadFailed
}))
};

export const useDelayedJobs = () => {
const connection = connectionHook(delayedJobsConnection)();
const connection = useConnection(delayedJobsConnection);

useEffect(() => {
const intervalId = setInterval(() => {
Expand All @@ -66,38 +59,32 @@ export const useDelayedJobs = () => {
return connection;
};

export const triggerBulkUpdate = (jobs: DelayedJobData[]) => {
return bulkUpdateJobs({ body: { data: jobs } });
};
export const triggerBulkUpdate = (jobs: DelayedJobData[]) => bulkUpdateJobs({ body: { data: jobs } });

const bulkUpdateJobsSelector = (store: ApiDataStore) => {
const bulkUpdateState = store.delayedJobs || {};
return {
isLoading: bulkUpdateJobsIsFetching(store),
hasLoadFailed: bulkUpdateJobsFetchFailed(store) != null,
response: bulkUpdateState.response
};
};
const bulkUpdateJobsSelector = (store: ApiDataStore) => ({
isLoading: bulkUpdateJobsIsFetching(store),
hasLoadFailed: bulkUpdateJobsFetchFailed(store) != null,
response: store.delayedJobs
});

const bulkUpdateJobsConnection: Connection<DelayedJobsConnection, { jobs: DelayedJobData[] }> = {
load: async (connection, { jobs }) => {
load: (connection, { jobs }) => {
const isLoaded = connectionBulkUpdateIsLoaded(connection);
if (!isLoaded) {
await bulkUpdateJobs({ body: { data: jobs } });
bulkUpdateJobs({ body: { data: jobs } });
}
},

isLoaded: state => !state.isLoading,

selector: createSelector([store => bulkUpdateJobsSelector(store)], ({ isLoading, hasLoadFailed, response }) => ({
selector: createSelector(bulkUpdateJobsSelector, ({ isLoading, hasLoadFailed, response }) => ({
isLoading,
hasLoadFailed,
response
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like it should be throwing an error in the TS compiler... Maybe I'm missing something, but this shape does not seem to match DelayedJobsConnection.

A couple of thoughts here:

  • Typically, each connection will have its own shape. That's not required, but it's normal
  • I've found in the past that properties like isLoading, hasLoadFailed and response, while seeming convenient when looking at a single connection, are actually a disaster down the road. The reason being that if we follow such generic conventions for individual connections, then when a component ends up using 2 or more connections, it gets really awkward to juggle all the isLoadings that each connection returns. Instead, I think it's better to be super specific with the properties of the connection shape. So something more like bulkUpdateJobsIsLoading and bulkUpdateJobsHasFailed.

As for response - this appears to be just returning all the delayed jobs like the other connection does. Are these used in the same component? Perhaps a better approach would be to combine them into a single connection. I think of connections as being responsible for providing a given set of data from the redux cache, and a set of functions for operating on that data. It would be totally reasonable for this bulkUpdateJobs call to be triggered from a function that's included on the other connection. Or, the bulkUpdate function could be simply exposed by this file, and then we simply rely on the original connection to provide the delayedJobs from the store. One of the primary wins of this system is that a connection returns data that got cached in the store regardless of where it came from. So if one call does the initial fetch, and another call modifies that data, as long as the data ends up back in the store under the same ID, the component that relies on it will rerender when it gets updated.

}))
};

const connectionBulkUpdateIsLoaded = ({ isLoading, hasLoadFailed }: { isLoading: boolean; hasLoadFailed: boolean }) => {
return !isLoading && !hasLoadFailed;
};
const connectionBulkUpdateIsLoaded = ({ isLoading, hasLoadFailed }: { isLoading: boolean; hasLoadFailed: boolean }) =>
!isLoading && !hasLoadFailed;

export const useBulkUpdateJobs = connectionLoader(bulkUpdateJobsConnection);
6 changes: 5 additions & 1 deletion src/generated/v3/jobService/jobServiceSchemas.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,13 @@ export type DelayedJobDto = {
*/
name: string | null;
/**
* The name of the related entity (e.g., site, project).
* The name of the related entity (e.g., Kerrawarra, New Site, etc).
*/
entityName?: string | null;
/**
* The type of the entity (e.g., site, project).
*/
entityType?: string | null;
};

export type DelayedJobAttributes = {
Expand Down
1 change: 0 additions & 1 deletion src/store/apiSlice.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ export type Relationships = {
};

export type StoreResource<AttributeType> = {
id?: string;
attributes: AttributeType;
// We do a bit of munging on the shape from the API, removing the intermediate "data" member, and
// ensuring there's always an array, to make consuming the data clientside a little smoother.
Expand Down
Loading