Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLDR-14913 Add Download XML button to Survey Tool #4010

Merged
merged 1 commit into from
Sep 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions tools/cldr-apps/js/src/esm/cldrAnnounce.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,6 @@ import * as cldrAjax from "./cldrAjax.mjs";
import * as cldrSchedule from "./cldrSchedule.mjs";
import * as cldrStatus from "./cldrStatus.mjs";

/**
* This should be false for production. It can be made true during debugging, which
* may be useful for performance testing.
*/
const DISABLE_ANNOUNCEMENTS = false;

const CLDR_ANNOUNCE_DEBUG = false;

const ANNOUNCE_REFRESH_SECONDS = 60; // one minute
Expand All @@ -38,6 +32,16 @@ const MOST_RECENT_ID_UNKNOWN = -1; // must be less than zero
*/
let alreadyGotId = MOST_RECENT_ID_UNKNOWN;

/**
* Ordinarily announcements are enabled. They may be temporarily disabled during
* critical operations such as VXML generation, or for debugging.
*/
let announcementsEnabled = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does it make a difference?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thrown exceptions due to uncommitted non-auto-commit db connections were preventing VXML generation from running to completion. While debugging that, the Announce requests were running in parallel with the VXML requests, making the problem harder to track down. In the past there have also been concurrency bugs when generation was implemented as an auto-scheduled background task, and it's not clear whether those were ever solved or just avoided by not auto-scheduling. Even when all is going well, I like to keep an eye on the various things that are happening during VXML generation. Temporarily disabling announcements reduces complexity and server load and makes for one less thing to worry about.


function enableAnnouncements(enable) {
announcementsEnabled = Boolean(enable);
}

/**
* Get the number of unread announcements, to display in the main menu
*
Expand All @@ -60,7 +64,7 @@ async function getUnreadCount(setUnreadCount) {
* we're only getting the number of unread announcements to display in the main header
*/
async function refresh(viewCallbackSetData, viewCallbackSetCounts) {
if (DISABLE_ANNOUNCEMENTS) {
if (!announcementsEnabled) {
return;
}
if (viewCallbackSetData) {
Expand Down Expand Up @@ -185,6 +189,7 @@ export {
canAnnounce,
canChooseAllOrgs,
compose,
enableAnnouncements,
getUnreadCount,
refresh,
resetSchedule,
Expand Down
88 changes: 88 additions & 0 deletions tools/cldr-apps/js/src/esm/cldrGenerateVxml.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
/*
* cldrGenerateVxml: for Survey Tool feature "Generate VXML". The display logic is in GenerateVxml.vue.
*/
import * as cldrAjax from "./cldrAjax.mjs";
import * as cldrAnnounce from "./cldrAnnounce.mjs";
import * as cldrNotify from "./cldrNotify.mjs";
import * as cldrStatus from "./cldrStatus.mjs";

const SECONDS_IN_MS = 1000;

const NORMAL_RETRY = 10 * SECONDS_IN_MS; // "Normal" retry: starting or about to start

const VXML_URL = "api/vxml";

// These must match the back end; used in requests
class LoadingPolicy {
static START = "START"; // start generating vxml
static CONTINUE = "CONTINUE"; // continue generating vxml
static STOP = "STOP"; // stop (cancel) generating vxml
}

// These must match the back end; used in responses
class Status {
static INIT = "INIT"; // before making a request (back end does not have INIT)
static WAITING = "WAITING"; // waiting on other users/tasks
static PROCESSING = "PROCESSING"; // in progress
static READY = "READY"; // finished successfully
static STOPPED = "STOPPED"; // due to error or cancellation (LoadingPolicy.STOP)
}

let canGenerate = false;

let callbackToSetData = null;

function canGenerateVxml() {
return canGenerate;
}

function viewMounted(setData) {
callbackToSetData = setData;
const perm = cldrStatus.getPermissions();
canGenerate = Boolean(perm?.userIsAdmin);
}

function start() {
// Disable announcements during VXML generation to reduce risk of interference
cldrAnnounce.enableAnnouncements(false);
requestVxml(LoadingPolicy.START);
}

function fetchStatus() {
if (!canGenerate || "generate_vxml" !== cldrStatus.getCurrentSpecial()) {
canGenerate = false;
} else if (canGenerate) {
requestVxml(LoadingPolicy.CONTINUE);
}
}

function stop() {
requestVxml(LoadingPolicy.STOP);
}

function requestVxml(loadingPolicy) {
const args = { loadingPolicy: loadingPolicy };
const init = cldrAjax.makePostData(args);
cldrAjax
.doFetch(VXML_URL, init)
.then(cldrAjax.handleFetchErrors)
.then((r) => r.json())
.then(setVxmlData)
.catch((e) => {
cldrNotify.exception(e, "generating VXML");
});
Comment on lines +66 to +73
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could consider using cldrClient - provides error checking of input parameters. at least for future

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea for future enhancement. I'll study it. I think there may still be problem with cldrClient when running locally with nginx, confusion between ports 8888 and 9080. It can be bypassed by skipping nginx and going directly to 9080, but I'd like to get it working right.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it works via nginx in production. And anyway, I think the baseline url issue may have been fixed when I fixed a similar issue with Chrome.

Or, maybe it's not fixed in main, it's waiting in the "CLA PR" which is awaiting review.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"CLA PR"?

Copy link
Member

@srl295 srl295 Sep 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"CLA PR"?

#3653 - i'll rebase and merge it.

}

function setVxmlData(data) {
if (!callbackToSetData) {
return;
}
callbackToSetData(data);
if (data.status === Status.WAITING || data.status === Status.PROCESSING) {
window.setTimeout(fetchStatus.bind(this), NORMAL_RETRY);
} else if (data.status === Status.READY || data.status === Status.STOPPED) {
cldrAnnounce.enableAnnouncements(true); // restore
}
}

export { Status, canGenerateVxml, start, stop, viewMounted };
1 change: 1 addition & 0 deletions tools/cldr-apps/js/src/esm/cldrText.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -497,6 +497,7 @@ const strings = {
special_forum: "Forum Posts",
special_forum_participation: "Forum Participation",
special_general: "General Info",
special_generate_vxml: "Generate VXML",
special_list_emails: "List Email Addresses",
special_list_users: "List Users",
special_locales: "Locale List",
Expand Down
4 changes: 2 additions & 2 deletions tools/cldr-apps/js/src/esm/cldrVueMap.mjs
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
import * as cldrLoad from "./cldrLoad.mjs";

import AboutPanel from "../views/AboutPanel.vue";
import AnnouncePanel from "../views/AnnouncePanel.vue";
import AddUser from "../views/AddUser.vue";
import AutoImport from "../views/AutoImport.vue";
import DowngradedVotes from "../views/DowngradedVotes.vue";
import GeneralInfo from "../views/GeneralInfo.vue";
import GenerateVxml from "../views/GenerateVxml.vue";
import LockAccount from "../views/LockAccount.vue";
import LookUp from "../views/LookUp.vue";
import MainMenu from "../views/MainMenu.vue";
Expand All @@ -27,6 +26,7 @@ const specialToComponentMap = {
auto_import: AutoImport,
downgraded: DowngradedVotes,
general: GeneralInfo, // see cldrLoad.GENERAL_SPECIAL
generate_vxml: GenerateVxml,
lock_account: LockAccount,
lookup: LookUp,
menu: MainMenu,
Expand Down
75 changes: 75 additions & 0 deletions tools/cldr-apps/js/src/views/GenerateVxml.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
<script setup>
import { onMounted, ref } from "vue";

import * as cldrGenerateVxml from "../esm/cldrGenerateVxml.mjs";

const STATUS = cldrGenerateVxml.Status;

let hasPermission = ref(false);
let message = ref("");
let output = ref("");
let percent = ref(0);
let status = ref(STATUS.INIT);

function mounted() {
cldrGenerateVxml.viewMounted(setData);
hasPermission.value = Boolean(cldrGenerateVxml.canGenerateVxml());
}

onMounted(mounted);

function start() {
if (hasPermission) {
cldrGenerateVxml.start();
status.value = STATUS.WAITING;
}
}

function stop() {
cldrGenerateVxml.stop();
status.value = STATUS.STOPPED;
}

function canStop() {
return status.value === STATUS.WAITING || status.value === STATUS.PROCESSING;
}

function setData(data) {
message.value = data.message;
percent.value = data.percent;
status.value = data.status;
output.value = data.output;
}

defineExpose({
setData,
});
</script>

<template>
<div v-if="!hasPermission">Please log in as Admin to use this feature.</div>
<div v-else>
<p v-if="status != STATUS.INIT">Current Status: {{ status }}</p>
<p v-if="message">
<span v-html="message"></span>
</p>
<p class="buttons">
<button v-if="canStop()" @click="stop()">Stop</button>
<button v-else @click="start()">Generate VXML Now</button>
</p>
<p class="progressBar">
<a-progress :percent="percent" />
</p>
<p v-html="output"></p>
</div>
</template>

<style scoped>
.buttons {
margin: 1em;
}

.progressBar {
width: 50%;
}
</style>
1 change: 1 addition & 0 deletions tools/cldr-apps/js/src/views/MainMenu.vue
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
<ul>
<template v-if="loggedIn">
<li v-if="isAdmin"><a href="#admin///">Admin Panel</a></li>
<li v-if="isAdmin"><a href="#generate_vxml///">Generate VXML</a></li>
<!-- My Account only has border-top (section-header) if Admin Panel is shown -->
<li v-if="isAdmin" class="section-header">My Account</li>
<li v-else>My Account</li>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,9 @@ public String getDBInfo() {
public static final String DB_SQL_BINCOLLATE = " COLLATE latin1_bin ";
public static final String DB_SQL_ENGINE_INNO = "ENGINE=InnoDB";
public static final String DB_SQL_MB4 = " CHARACTER SET utf8mb4 COLLATE utf8mb4_bin";

// This could be useful for debugging but is not accurate, since it is not decremented
// when a connection is auto-closed as with "try (Connection conn = ...)"
public static int db_number_open = 0;
public static int db_number_used = 0;
private static final int db_UnicodeType = java.sql.Types.BLOB;
Expand Down Expand Up @@ -635,10 +638,11 @@ public final Connection getAConnection() {
c.setAutoCommit(true);
return c;
}
db_number_open++;
return datasource.getConnection();
} catch (SQLException se) {
se.printStackTrace();
SurveyMain.busted("Fatal in getConnection()", se);
SurveyMain.busted("Fatal in getAConnection()", se);
return null;
}
}
Expand All @@ -651,6 +655,7 @@ public final Connection getAConnection() {
*/
private static Connection getDBConnectionFor(final String connectionUrl) {
try {
db_number_open++;
return DriverManager.getConnection(connectionUrl);
} catch (SQLException e) {
throw new RuntimeException("getConnection() failed for url", e);
Expand Down
Loading
Loading