Skip to content

Commit

Permalink
Memolanes archive export v2 (#110)
Browse files Browse the repository at this point in the history
```
This feature will read all snapshots stored in your Fog Machine, then convert and optimize them one by one. After that, they will be merged into an archive file that can be read by MemoLanes (mldx).
Specifically:
1. All snapshots will be processed in the order of snapshot time and converted into journeys in MemoLanes.
2. Each journey's date is based on the snapshot time and your current time zone.
3. We subtract the previous snapshot from each snapshot, so each journey will only contain the diff.
4. Area that exists in a snapshot but not in the final snapshot will be removed. If you've used the eraser, ensure the final snapshot doesn't contain areas you don't want.
5. Snapshots that are too small will be ignored.
```

with the current optimization, it rought takes ~10s (instead of 60s)
with my full data in debug (2s in release). We might need more memory on
the server tho.
  • Loading branch information
CaviarChen authored Nov 30, 2024
1 parent d4f26d3 commit 553eb49
Show file tree
Hide file tree
Showing 9 changed files with 948 additions and 338 deletions.
6 changes: 6 additions & 0 deletions frontend/src/I18n.ts
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,9 @@ i18n
"snapshot-list-source-upload": "Upload",
"snapshot-list-download": "Download",
"snapshot-list-export-mldx": "Export MemoLanes Archive",
"snapshot-list-export-mldx-prompt":
"This feature will read all snapshots stored in your Fog Machine, then convert and optimize them one by one. After that, they will be merged into an archive file that can be read by MemoLanes (mldx).\nSpecifically:\n1. All snapshots will be processed in the order of snapshot time and converted into journeys in MemoLanes.\n2. Each journey's date is based on the snapshot time and your current time zone.\n3. We subtract the previous snapshot from each snapshot, so each journey will only contain the diff.\n4. Area that exists in a snapshot but not in the final snapshot will be removed. If you've used the eraser, ensure the final snapshot doesn't contain areas you don't want.\n5. Snapshots that are too small will be ignored.",
"snapshot-list-export-mldx-confirm": "Export",
"snapshot-list-view": "View",
"snapshot-list-note-edit": "Edit Note",
"snapshot-list-note-edit-err-tolong":
Expand Down Expand Up @@ -156,6 +159,9 @@ i18n
"snapshot-list-source-upload": "上传",
"snapshot-list-download": "下载",
"snapshot-list-export-mldx": "导出迹忆归档",
"snapshot-list-export-mldx-prompt":
"此功能将读取你在 迷雾机器 中的所有快照并逐一转换、优化并合并成为可供 迹忆 导入的归档文件(mldx)。\n其中:\n1. 所有快照会按时间顺序逐一处理,转换成 迹忆 的旅程。\n2. 每个旅程的日期基于快照时间以及你当前的时区。\n3. 每个快照会减去上一个快照中的数据,使得每个旅程之包含差异数据。\n4. 每个快照中不存在于最后一个快照的数据将被去除。如果你使用过橡皮擦,只需要保证最后的快照中不包含你不需要的数据即可。\n5. 过小的快照将会被忽略。",
"snapshot-list-export-mldx-confirm": "导出",
"snapshot-list-view": "查看",
"snapshot-list-note-edit": "编辑备注",
"snapshot-list-note-edit-err-tolong": "备注过长,请修改",
Expand Down
4 changes: 3 additions & 1 deletion frontend/src/time-machine/Api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -345,8 +345,10 @@ export default class Api {
public static async getMemoleanesArchiveDownloadToken(): Promise<
Result<string>
> {
const timezone = Intl.DateTimeFormat().resolvedOptions().timeZone;
const result = await this.requestApi(
"memolanes_archive/download_token",
"memolanes_archive/download_token?timezone=" +
encodeURIComponent(timezone),
"get",
true
);
Expand Down
4 changes: 4 additions & 0 deletions frontend/src/time-machine/DashboardSnapshot.css
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,7 @@
text-align: left;
height: 18px;
}

.rs-notification-content {
max-width: 600px !important;
}
59 changes: 47 additions & 12 deletions frontend/src/time-machine/DashboardSnapshot.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -432,6 +432,50 @@ function DashboardSnapshot() {
});
};

const openMemolanesExportConfirmation = () => {
// TODO: It is a bit wierd to use `Notifaction` as a modal. Maybe we shoudln't do this, but it works okish for now.
// e.g. when `Notifaction` is open, user can still click other things on the page.
// no easy way to close the current one other than using `clear` which close all things.
// We *SHOULDN'T* allow user to open multiple confirmation, that's really confusing.

const message = (
<Notification
type="info"
header={t("snapshot-list-export-mldx")}
closable
>
<pre style={{ whiteSpace: "pre-wrap", wordWrap: "break-word" }}>
{t("snapshot-list-export-mldx-prompt")}
</pre>
<hr />
<ButtonToolbar style={{ padding: 10 }}>
<Button
appearance="primary"
onClick={async () => {
notificationToaster.clear();

const token = await Api.getMemoleanesArchiveDownloadToken();
if (token.ok) {
window.open(
Api.backendUrl + "misc/download?token=" + token.ok,
"_blank"
);
} else {
//TODO: error handling
}
}}
>
{t("snapshot-list-export-mldx-confirm")}
</Button>
</ButtonToolbar>
</Notification>
);
notificationToaster.push(message, {
placement: "topCenter",
duration: 0,
});
};

const fileUploadUrl = Api.backendUrl + "misc/upload";
const headers = Api.tokenHeaders();

Expand Down Expand Up @@ -462,24 +506,15 @@ function DashboardSnapshot() {
>
{t("snapshot-list-upload")}
</IconButton>
{/* TODO: This feature is not ready. */}
{/* <IconButton
<IconButton
style={{ marginLeft: "10px" }}
icon={<FileDownloadIcon />}
onClick={async () => {
const token = await Api.getMemoleanesArchiveDownloadToken();
if (token.ok) {
window.open(
Api.backendUrl + "misc/download?token=" + token.ok,
"_blank"
);
} else {
//TODO: error handling
}
openMemolanesExportConfirmation();
}}
>
{t("snapshot-list-export-mldx")}
</IconButton> */}
</IconButton>
</Stack>
</Stack>
}
Expand Down
Loading

0 comments on commit 553eb49

Please sign in to comment.