Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CB-5369. Added API for transaction log for commits #3183

Merged
merged 27 commits into from
Jan 14, 2025
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
02baaf3
CB-5369. Added API for transaction log for commits
DenisSinelnikov Jan 9, 2025
d352e50
Merge branch 'devel' into CB-5369-transaction-log-for-commits
DenisSinelnikov Jan 9, 2025
8d47710
CB-5369. Refactor after review
DenisSinelnikov Jan 9, 2025
1f04b43
Merge remote-tracking branch 'origin/CB-5369-transaction-log-for-comm…
DenisSinelnikov Jan 9, 2025
89e0ff4
CB-5369 add transaction logs table
devnaumov Jan 9, 2025
d0c621b
Merge branches 'CB-5369-transaction-log-for-commits' and 'CB-5369-tra…
devnaumov Jan 9, 2025
854f830
CB-5369. Added id for data
DenisSinelnikov Jan 10, 2025
0848dbf
Merge remote-tracking branch 'origin/CB-5369-transaction-log-for-comm…
DenisSinelnikov Jan 10, 2025
5f44a3b
CB-5369 fix typo
devnaumov Jan 10, 2025
b9fd5fb
CB-5369. Added event for get count
DenisSinelnikov Jan 10, 2025
a171da8
Merge remote-tracking branch 'origin/CB-5369-transaction-log-for-comm…
DenisSinelnikov Jan 10, 2025
024c671
CB-5369. Refactor event
DenisSinelnikov Jan 12, 2025
6659111
CB-5369. Refactor event
DenisSinelnikov Jan 12, 2025
bc54897
CB-5369. Refactor after review
DenisSinelnikov Jan 12, 2025
5622605
CB-5369. Refactor after review
DenisSinelnikov Jan 12, 2025
4e7b62f
CB-5369. Added connectionId to event
DenisSinelnikov Jan 12, 2025
3314f75
CB-5369 support count event
devnaumov Jan 12, 2025
c1f5834
CB-5369. Rename eventId
DenisSinelnikov Jan 13, 2025
96954ff
Merge remote-tracking branch 'origin/CB-5369-transaction-log-for-comm…
DenisSinelnikov Jan 13, 2025
0647177
CB-5369 adjust count styles
devnaumov Jan 13, 2025
6bea6b0
CB-5369 format time
devnaumov Jan 13, 2025
41c60fa
CB-5369 add missing deps
devnaumov Jan 13, 2025
abad3a0
CB-5369 remove extra important flags
devnaumov Jan 13, 2025
3c5332a
CB-5369 review fixes
devnaumov Jan 13, 2025
8f6e63e
CB-5369 remove extra import
devnaumov Jan 13, 2025
e0e5f82
Merge branch 'devel' into CB-5369-transaction-log-for-commits
dariamarutkina Jan 14, 2025
691ae67
Merge branch 'devel' into CB-5369-transaction-log-for-commits
dariamarutkina Jan 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ enum CBServerEventId {

cb_database_output_log_updated,

cb_transactional_count @since(version: "24.3.3")
Wroud marked this conversation as resolved.
Show resolved Hide resolved

cb_session_task_info_updated @since(version: "24.3.1")
}

Expand Down Expand Up @@ -62,7 +64,9 @@ enum CBEventTopic {
cb_session_task, @since(version: "24.3.1")

cb_datasource_connection,
cb_delete_temp_folder
cb_delete_temp_folder,

cb_transaction @since(version: "24.3.3")
}

# Base server event interface
Expand Down Expand Up @@ -212,6 +216,16 @@ type WSDataSourceConnectEvent implements CBServerEvent {
timestamp: Int!
}

# Datasource count event in transactional mode
type WSTransactionalCountEvent implements CBServerEvent {
id: CBServerEventId!
topicId: CBEventTopic
contextId: String!
projectId: String!
connectionId: String!
transactionalCount: Int!
}

extend type Query {
emptyEvent: Boolean
}
Expand Down
24 changes: 24 additions & 0 deletions server/bundles/io.cloudbeaver.server/schema/service.sql.graphqls
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,24 @@ type DynamicTraceProperty {
description: String
}

####################################################
# Transactional info
####################################################
type TransactionLogInfoItem {
id: Int!
time: DateTime!
type: String!
queryString: String!
durationMs: Int!
rows: Int!
result: String!
}
type TransactionLogInfos {
count: Int!
transactionLogInfos: [TransactionLogInfoItem!]!
}


####################################################
# Query and Mutation
####################################################
Expand Down Expand Up @@ -331,6 +349,12 @@ extend type Mutation {
dataFormat: ResultDataFormat
): AsyncTaskInfo!

getTransactionLogInfo(
projectId: ID!,
connectionId: ID!,
contextId: ID!
): TransactionLogInfos!

# Close results (free resources)
sqlResultClose(projectId: ID, connectionId: ID!, contextId: ID!, resultId: ID!): Boolean!

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/*
* DBeaver - Universal Database Manager
* Copyright (C) 2010-2025 DBeaver Corp and others
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.cloudbeaver.model;

import org.jkiss.code.NotNull;

import java.util.List;

public record WebTransactionLogInfo(@NotNull List<WebTransactionLogItemInfo> transactionLogInfos, int count) {
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/*
* DBeaver - Universal Database Manager
* Copyright (C) 2010-2025 DBeaver Corp and others
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.cloudbeaver.model;

import org.jkiss.code.NotNull;
import org.jkiss.code.Nullable;

public record WebTransactionLogItemInfo(
@Nullable Integer id,
@NotNull String time,
@NotNull String type,
@NotNull String queryString,
long durationMs,
long rows,
String result
) {
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/*
* DBeaver - Universal Database Manager
* Copyright (C) 2010-2024 DBeaver Corp and others
* Copyright (C) 2010-2025 DBeaver Corp and others
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand All @@ -20,6 +20,7 @@
import io.cloudbeaver.WebAction;
import io.cloudbeaver.model.WebAsyncTaskInfo;
import io.cloudbeaver.model.WebConnectionInfo;
import io.cloudbeaver.model.WebTransactionLogInfo;
import io.cloudbeaver.model.session.WebSession;
import io.cloudbeaver.service.DBWService;
import org.jkiss.code.NotNull;
Expand Down Expand Up @@ -202,4 +203,10 @@
WebAsyncTaskInfo asyncSqlCommitTransaction(
@NotNull WebSession webSession,
@NotNull WebSQLContextInfo sqlContext);

@WebAction

Check warning on line 207 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/DBWServiceSQL.java

View workflow job for this annotation

GitHub Actions / Server / Lint

[checkstyle] reported by reviewdog 🐶 Missing a Javadoc comment. Raw Output: /github/workspace/./server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/DBWServiceSQL.java:207:5: warning: Missing a Javadoc comment. (com.puppycrawl.tools.checkstyle.checks.javadoc.MissingJavadocMethodCheck)

Check warning on line 207 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/DBWServiceSQL.java

View check run for this annotation

Jenkins-CI-integration / CheckStyle Java Report

server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/DBWServiceSQL.java#L207

Missing a Javadoc comment.
WebTransactionLogInfo getTransactionLogInfo(
@NotNull WebSession webSession,
@NotNull WebSQLContextInfo sqlContext
);
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/*
* DBeaver - Universal Database Manager
* Copyright (C) 2010-2024 DBeaver Corp and others
* Copyright (C) 2010-2025 DBeaver Corp and others
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand All @@ -26,22 +26,36 @@
import org.jkiss.code.NotNull;
import org.jkiss.dbeaver.DBException;
import org.jkiss.dbeaver.Log;
import org.jkiss.dbeaver.model.DBConstants;
import org.jkiss.dbeaver.model.DBUtils;
import org.jkiss.dbeaver.model.data.DBDAttributeBinding;
import org.jkiss.dbeaver.model.exec.*;
import org.jkiss.dbeaver.model.exec.trace.DBCTrace;
import org.jkiss.dbeaver.model.messages.ModelMessages;
import org.jkiss.dbeaver.model.meta.Property;
import io.cloudbeaver.model.WebTransactionLogInfo;

Check warning on line 36 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java

View workflow job for this annotation

GitHub Actions / Server / Lint

[checkstyle] reported by reviewdog 🐶 Wrong lexicographical order for 'io.cloudbeaver.model.WebTransactionLogInfo' import. Should be before 'org.jkiss.dbeaver.model.meta.Property'. Raw Output: /github/workspace/./server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java:36:1: warning: Wrong lexicographical order for 'io.cloudbeaver.model.WebTransactionLogInfo' import. Should be before 'org.jkiss.dbeaver.model.meta.Property'. (com.puppycrawl.tools.checkstyle.checks.imports.CustomImportOrderCheck)

Check warning on line 36 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java

View check run for this annotation

Jenkins-CI-integration / CheckStyle Java Report

server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java#L36

Wrong lexicographical order for io.cloudbeaver.model.WebTransactionLogInfo import. Should be before org.jkiss.dbeaver.model.meta.Property.
import io.cloudbeaver.model.WebTransactionLogItemInfo;

Check warning on line 37 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java

View workflow job for this annotation

GitHub Actions / Server / Lint

[checkstyle] reported by reviewdog 🐶 Wrong lexicographical order for 'io.cloudbeaver.model.WebTransactionLogItemInfo' import. Should be before 'org.jkiss.dbeaver.model.meta.Property'. Raw Output: /github/workspace/./server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java:37:1: warning: Wrong lexicographical order for 'io.cloudbeaver.model.WebTransactionLogItemInfo' import. Should be before 'org.jkiss.dbeaver.model.meta.Property'. (com.puppycrawl.tools.checkstyle.checks.imports.CustomImportOrderCheck)

Check warning on line 37 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java

View check run for this annotation

Jenkins-CI-integration / CheckStyle Java Report

server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java#L37

Wrong lexicographical order for io.cloudbeaver.model.WebTransactionLogItemInfo import. Should be before org.jkiss.dbeaver.model.meta.Property.
import org.jkiss.dbeaver.model.qm.QMTransactionState;
import org.jkiss.dbeaver.model.qm.QMUtils;
import org.jkiss.dbeaver.model.qm.meta.QMMConnectionInfo;
import org.jkiss.dbeaver.model.qm.meta.QMMStatementExecuteInfo;
import org.jkiss.dbeaver.model.qm.meta.QMMTransactionInfo;
import org.jkiss.dbeaver.model.qm.meta.QMMTransactionSavepointInfo;
import org.jkiss.dbeaver.model.runtime.DBRProgressMonitor;
import org.jkiss.dbeaver.model.struct.DBSDataContainer;
import org.jkiss.dbeaver.model.struct.rdb.DBSCatalog;
import org.jkiss.dbeaver.model.struct.rdb.DBSSchema;
import org.jkiss.dbeaver.model.websocket.event.WSTransactionalCountEvent;
import org.jkiss.dbeaver.utils.RuntimeUtils;
import org.jkiss.utils.CommonUtils;

import java.lang.reflect.InvocationTargetException;
import java.time.Instant;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;

Expand All @@ -59,6 +73,9 @@

private final AtomicInteger resultId = new AtomicInteger();

public static final DateTimeFormatter ISO_DATE_FORMAT = DateTimeFormatter.ofPattern(DBConstants.DEFAULT_ISO_TIMESTAMP_FORMAT)
.withZone(ZoneId.of("UTC"));

public WebSQLContextInfo(
WebSQLProcessor processor, String id, String catalogName, String schemaName, String projectId
) throws DBCException {
Expand Down Expand Up @@ -216,7 +233,71 @@

}

public WebTransactionLogInfo getTransactionLogInfo() {

Check warning on line 236 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java

View workflow job for this annotation

GitHub Actions / Server / Lint

[checkstyle] reported by reviewdog 🐶 Missing a Javadoc comment. Raw Output: /github/workspace/./server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java:236:5: warning: Missing a Javadoc comment. (com.puppycrawl.tools.checkstyle.checks.javadoc.MissingJavadocMethodCheck)

Check warning on line 236 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java

View check run for this annotation

Jenkins-CI-integration / CheckStyle Java Report

server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java#L236

Missing a Javadoc comment.
DBCExecutionContext context = processor.getExecutionContext();
return getTransactionLogInfo(context);
}

@NotNull
private WebTransactionLogInfo getTransactionLogInfo(DBCExecutionContext executionContext) {
int updateCount = 0;
List<WebTransactionLogItemInfo> logItemInfos = new ArrayList<>();
QMMConnectionInfo sessionInfo = QMUtils.getCurrentConnection(executionContext);
if (sessionInfo.isTransactional()) {
QMMTransactionInfo txnInfo = sessionInfo.getTransaction();
if (txnInfo != null) {
QMMTransactionSavepointInfo sp = txnInfo.getCurrentSavepoint();
QMMStatementExecuteInfo execInfo = sp.getLastExecute();
for (QMMStatementExecuteInfo exec = execInfo; exec != null && exec.getSavepoint() == sp; exec = exec.getPrevious()) {
if (exec.getUpdateRowCount() > 0 ) {

Check warning on line 252 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java

View workflow job for this annotation

GitHub Actions / Server / Lint

[checkstyle] reported by reviewdog 🐶 ')' is preceded with whitespace. Raw Output: /github/workspace/./server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java:252:54: warning: ')' is preceded with whitespace. (com.puppycrawl.tools.checkstyle.checks.whitespace.ParenPadCheck)

Check warning on line 252 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java

View check run for this annotation

Jenkins-CI-integration / CheckStyle Java Report

server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java#L252

) is preceded with whitespace.
DBCExecutionPurpose purpose = exec.getStatement().getPurpose();
if (!exec.hasError() && purpose != DBCExecutionPurpose.META && purpose != DBCExecutionPurpose.UTIL) {
updateCount++;
}
generateLogInfo(logItemInfos, exec, purpose, updateCount);
}
}
}
} else {
QMMStatementExecuteInfo execInfo = sessionInfo.getExecutionStack();
for (QMMStatementExecuteInfo exec = execInfo; exec != null; exec = exec.getPrevious()) {
if (exec.getUpdateRowCount() > 0) {
updateCount++;
DBCExecutionPurpose purpose = exec.getStatement().getPurpose();
generateLogInfo(logItemInfos, exec, purpose, updateCount);
}
}
}
return new WebTransactionLogInfo(logItemInfos, updateCount);
}

private void generateLogInfo(
@NotNull List<WebTransactionLogItemInfo> logItemInfos,
@NotNull QMMStatementExecuteInfo exec,
@NotNull DBCExecutionPurpose purpose,
int id
) {
String type = "SQL / " + purpose.getTitle();
String dateTime = ISO_DATE_FORMAT.format(Instant.ofEpochMilli(exec.getCloseTime()));
String result = ModelMessages.controls_querylog_success;
if (exec.hasError()) {
if (exec.getErrorCode() == 0) {
result = exec.getErrorMessage();
} else if (exec.getErrorMessage() == null) {
result = ModelMessages.controls_querylog_error + exec.getErrorCode() + "]"; //$NON-NLS-1$
} else {
result = "[" + exec.getErrorCode() + "] " + exec.getErrorMessage(); //$NON-NLS-1$ //$NON-NLS-2$
}
}

logItemInfos.add(
new WebTransactionLogItemInfo(id, dateTime, type, exec.getQueryString(),
exec.getDuration(), exec.getUpdateRowCount(), result)
);
}


public WebAsyncTaskInfo commitTransaction() {

Check warning on line 300 in server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java

View workflow job for this annotation

GitHub Actions / Server / Lint

[checkstyle] reported by reviewdog 🐶 Missing a Javadoc comment. Raw Output: /github/workspace/./server/bundles/io.cloudbeaver.server/src/io/cloudbeaver/service/sql/WebSQLContextInfo.java:300:5: warning: Missing a Javadoc comment. (com.puppycrawl.tools.checkstyle.checks.javadoc.MissingJavadocMethodCheck)
DBCExecutionContext context = processor.getExecutionContext();
DBCTransactionManager txnManager = DBUtils.getTransactionManager(context);
WebAsyncTaskProcessor<String> runnable = new WebAsyncTaskProcessor<>() {
Expand All @@ -238,6 +319,17 @@
RuntimeUtils.formatExecutionTime(System.currentTimeMillis() - txnInfo.getTransactionStartTime())
);
}
processor.getWebSession().addSessionEvent(
new WSTransactionalCountEvent(
processor.getWebSession().getSessionId(),
processor.getWebSession().getUserId(),
getProjectId(),
getId(),
getConnectionId(),
0
)
);

}
};
return getWebSession().createAndRunAsyncTask("Commit transaction", runnable);
Expand Down Expand Up @@ -265,6 +357,16 @@
txnInfo.getUpdateCount(),
RuntimeUtils.formatExecutionTime(System.currentTimeMillis() - txnInfo.getTransactionStartTime())
);
processor.getWebSession().addSessionEvent(
new WSTransactionalCountEvent(
processor.getWebSession().getSessionId(),
processor.getWebSession().getUserId(),
getProjectId(),
getId(),
getConnectionId(),
0
)
);
}
}
};
Expand Down
Loading
Loading