Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update grpc to 1.62.0 #152

Merged
merged 6 commits into from
Feb 26, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions cmake/gRPC.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,14 @@ if(GWHISPER_FORCE_BUILDING_GRPC OR GRPC_NOT_FOUND)
unset(PROTC_GRPC_PLUGIN CACHE)

include(FetchContent)

# Workaround for https://github.com/protocolbuffers/protobuf/issues/12185 :
set(ABSL_ENABLE_INSTALL ON)

FetchContent_Declare(
grpc
GIT_REPOSITORY https://github.com/grpc/grpc
GIT_TAG v1.43.0
GIT_TAG v1.62.0
GIT_PROGRESS TRUE
)
set(FETCHCONTENT_QUIET OFF)
Expand All @@ -87,7 +91,7 @@ if(GWHISPER_FORCE_BUILDING_GRPC OR GRPC_NOT_FOUND)
if(CMAKE_SYSTEM_PROCESSOR STREQUAL "s390x")
set(gRPC_SSL_PROVIDER "package" CACHE STRING "force overwritten by gWhisper, as boringSSL does not support s390x -> need to fall-back to system installed libssl" FORCE )
endif()

set(ABSL_PROPAGATE_CXX_STD ON CACHE STRING "force overwritten by gWhisper to conform to Abseil recommendations" FORCE )
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A build warning was given to set this. So setting here.

add_subdirectory(${grpc_SOURCE_DIR} ${grpc_BINARY_DIR} EXCLUDE_FROM_ALL)

# Since FetchContent uses add_subdirectory under the hood, we can use
Expand Down
4 changes: 2 additions & 2 deletions src/libCli/MessageFormatterJson.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ namespace cli

// Will print primitive fields regardless of their values. So e.g. an int32 field set to 0 will not be omitted.
printOptions.always_print_primitive_fields = true;
google::protobuf::util::Status status = google::protobuf::util::MessageToJsonString(f_message, &resultString, printOptions);

auto status = google::protobuf::util::MessageToJsonString(f_message, &resultString, printOptions);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const is your friend ;-)

if(not status.ok())
{
resultString = std::string("Json encoding of Message failed: ") + status.ToString();
Expand Down
2 changes: 1 addition & 1 deletion src/libCli/MessageParserJson.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ std::vector<std::unique_ptr<google::protobuf::Message>> MessageParserJson::parse
buffer << source->rdbuf();
std::string fileContent = buffer.str();
auto status = google::protobuf::util::JsonStringToMessage(
google::protobuf::StringPiece(fileContent),
fileContent,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually you don't need a temporay (fileContent) here, you could pass it directly as parameter to JsonStringToMessage()

message.get(),
google::protobuf::util::JsonParseOptions());
if(not status.ok())
Expand Down
3 changes: 2 additions & 1 deletion tests/functionTests/completionTests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ Possible Candidates:
127.0.0.1 examples.StreamingRpcs (uni- and bi-directional streams)
127.0.0.1 examples.StatusHandling (gRPC error handling)
/^127.0.0.1 grpc.reflection.*$
/^127.0.0.1 grpc.reflection.*$
#END_TEST

#START_TEST complete rpcTimeout
Expand Down Expand Up @@ -316,4 +317,4 @@ RPC succeeded :D
@@CMD@@ --disableCache 127.0.0.1 examples.ScalarTypeRpcs bitwiseInvertBytes data=file://thisFilenameDoesNotExist.iHope
/Error.*
Error parsing method arguments -> aborting the call :-(
#END_TEST
#END_TEST
1 change: 1 addition & 0 deletions tests/functionTests/completionTests_fish.txt
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ Possible Candidates:
127.0.0.1 examples.StreamingRpcs (uni- and bi-directional streams)
127.0.0.1 examples.StatusHandling (gRPC error handling)
/^127.0.0.1 grpc.reflection.*$
/^127.0.0.1 grpc.reflection.*$
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in latest version, grpc server adds two reflection services (one stable and one alpha)

#END_TEST

#START_TEST complete rpcTimeout
Expand Down
4 changes: 1 addition & 3 deletions tests/functionTests/rpcExecutionTests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -200,9 +200,7 @@ RPC succeeded :D

#START_TEST json_input_stdin_malformed
cat @@testResources@@/duplicateEverything1d_malformed.json | @@CMD@@ --disableCache --jsonInput=- 127.0.0.1 examples.NestedTypeRpcs duplicateEverything1d
Warning: Failed to parse JSON file '-': INVALID_ARGUMENT:Unexpected end of string. Expected , or } after key:value pair.

^
/Warning: Failed to parse JSON file '-': .*
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

protobuf error message changed slightly. made the test more flexible

Error parsing method arguments -> aborting the call :-(
#END_TEST

Expand Down
39 changes: 17 additions & 22 deletions third_party/gRPC_utils/cli_call.cc
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
#include <optional>
// END MODIFIED

Copy link
Member Author

@rainerschoe rainerschoe Feb 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took the most recent cli_call.cc and cli_call.hfrom gRPC example source code and adapted it again for our needs (they had some minor updates).

#include <cmath>
#include <iostream>
#include <utility>

Expand All @@ -37,34 +38,29 @@
#include <grpcpp/client_context.h>
#include <grpcpp/support/byte_buffer.h>

// MODIFIED by IBM (Rainer Schoenberger)
// original: #include "src/core/lib/gprpp/crash.h"
// END MODIFIED

namespace grpc {
namespace testing {
namespace {
void* tag(int i) { return (void*)static_cast<intptr_t>(i); }
void* tag(intptr_t t) { return reinterpret_cast<void*>(t); }
} // namespace

Status CliCall::Call(std::shared_ptr<grpc::Channel> channel,
const grpc::string& method, const grpc::string& request,
grpc::string* response,
const OutgoingMetadataContainer& metadata,
Status CliCall::Call(const std::string& request, std::string* response,
IncomingMetadataContainer* server_initial_metadata,
IncomingMetadataContainer* server_trailing_metadata) {
// MODIFIED by IBM (Anna Riesch)
// original: no deadline
std::optional<std::chrono::time_point<std::chrono::system_clock>> deadline;
deadline = std::nullopt;
CliCall call(std::move(channel), method, metadata, deadline);
// END MODIFIED
call.Write(request);
call.WritesDone();
if (!call.Read(response, server_initial_metadata)) {
Write(request);
WritesDone();
if (!Read(response, server_initial_metadata)) {
fprintf(stderr, "Failed to read response.\n");
}
return call.Finish(server_trailing_metadata);
return Finish(server_trailing_metadata);
}

CliCall::CliCall(const std::shared_ptr<grpc::Channel>& channel,
const grpc::string& method,
const std::string& method,
const OutgoingMetadataContainer& metadata,
// MODIFIED by IBM (Anna Riesch)
// original: no argument "deadline"
Expand Down Expand Up @@ -101,19 +97,19 @@ CliCall::~CliCall() {
gpr_mu_destroy(&write_mu_);
}

void CliCall::Write(const grpc::string& request) {
void CliCall::Write(const std::string& request) {
void* got_tag;
bool ok;

gpr_slice s = gpr_slice_from_copied_buffer(request.data(), request.size());
grpc_slice s = grpc_slice_from_copied_buffer(request.data(), request.size());
grpc::Slice req_slice(s, grpc::Slice::STEAL_REF);
grpc::ByteBuffer send_buffer(&req_slice, 1);
call_->Write(send_buffer, tag(2));
cq_.Next(&got_tag, &ok);
GPR_ASSERT(ok);
}

bool CliCall::Read(grpc::string* response,
bool CliCall::Read(std::string* response,
IncomingMetadataContainer* server_initial_metadata) {
void* got_tag;
bool ok;
Expand Down Expand Up @@ -147,7 +143,7 @@ void CliCall::WritesDone() {
GPR_ASSERT(ok);
}

void CliCall::WriteAndWait(const grpc::string& request) {
void CliCall::WriteAndWait(const std::string& request) {
grpc::Slice req_slice(request);
grpc::ByteBuffer send_buffer(&req_slice, 1);

Expand All @@ -171,8 +167,7 @@ void CliCall::WritesDoneAndWait() {
}

bool CliCall::ReadAndMaybeNotifyWrite(
grpc::string* response,
IncomingMetadataContainer* server_initial_metadata) {
std::string* response, IncomingMetadataContainer* server_initial_metadata) {
void* got_tag;
bool ok;
grpc::ByteBuffer recv_buffer;
Expand Down
60 changes: 28 additions & 32 deletions third_party/gRPC_utils/gRPC_utils/cli_call.h
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
/*
*
* Copyright 2015 gRPC authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
//
//
// Copyright 2015 gRPC authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
//

#ifndef GRPC_TEST_CPP_UTIL_CLI_CALL_H
#define GRPC_TEST_CPP_UTIL_CLI_CALL_H
Expand Down Expand Up @@ -47,40 +47,37 @@ namespace testing {
// and thread-unsafe methods should not be used together.
class CliCall final {
public:
typedef std::multimap<grpc::string, grpc::string> OutgoingMetadataContainer;
typedef std::multimap<std::string, std::string> OutgoingMetadataContainer;
typedef std::multimap<grpc::string_ref, grpc::string_ref>
IncomingMetadataContainer;

// MODIFIED by IBM (Anna Riesch)
// original: no argument "deadline"
CliCall(const std::shared_ptr<grpc::Channel>& channel,
const grpc::string& method,
const std::string& method,
const OutgoingMetadataContainer& metadata,
std::optional<std::chrono::time_point<std::chrono::system_clock>> deadline);
// END MODIFIED
~CliCall();

// Perform an unary generic RPC.
static Status Call(std::shared_ptr<grpc::Channel> channel,
const grpc::string& method, const grpc::string& request,
grpc::string* response,
const OutgoingMetadataContainer& metadata,
IncomingMetadataContainer* server_initial_metadata,
IncomingMetadataContainer* server_trailing_metadata);
Status Call(const std::string& request, std::string* response,
IncomingMetadataContainer* server_initial_metadata,
IncomingMetadataContainer* server_trailing_metadata);

// Send a generic request message in a synchronous manner. NOT thread-safe.
void Write(const grpc::string& request);
void Write(const std::string& request);

// Send a generic request message in a synchronous manner. NOT thread-safe.
void WritesDone();

// Receive a generic response message in a synchronous manner.NOT thread-safe.
bool Read(grpc::string* response,
bool Read(std::string* response,
IncomingMetadataContainer* server_initial_metadata);

// Thread-safe write. Must be used with ReadAndMaybeNotifyWrite. Send out a
// generic request message and wait for ReadAndMaybeNotifyWrite to finish it.
void WriteAndWait(const grpc::string& request);
void WriteAndWait(const std::string& request);

// Thread-safe WritesDone. Must be used with ReadAndMaybeNotifyWrite. Send out
// WritesDone for gereneric request messages and wait for
Expand All @@ -90,18 +87,17 @@ class CliCall final {
// Thread-safe Read. Blockingly receive a generic response message. Notify
// writes if they are finished when this read is waiting for a resposne.
bool ReadAndMaybeNotifyWrite(
grpc::string* response,
std::string* response,
IncomingMetadataContainer* server_initial_metadata);

// Finish the RPC.
Status Finish(IncomingMetadataContainer* server_trailing_metadata);

std::string peer() const { return ctx_.peer(); }

private:
std::unique_ptr<grpc::GenericStub> stub_;
// MODIFIED by IBM (Rainer Schoenberger)
// original: grpc_impl::ClientContext ctx_;
grpc::ClientContext ctx_;
// END MODIFIED
std::unique_ptr<grpc::GenericClientAsyncReaderWriter> call_;
grpc::CompletionQueue cq_;
gpr_mu write_mu_;
Expand Down
Loading