Accessing CCF application from outside Azure environment

[psfoley]

I have deployed a CCF application based on version 18.5 into Azure using the DCS4_v2 VM setup instructions in your documentation. Our use case requires that customers should be able to access this service from outside Azure on their local infrastructure. Accessing the application internally on the VM with the local IP works just fine, but requests receive no response (I suspect this may be a certificate issue). Here are the contents of my cchost config.toml file:

                                                                                                                                                                                                                                                                                  enclave-file = "${governor_home}/ccf/lib/libgovernorenc.${enclave_file_type}"
enclave-type = "${enclave_type}"
consensus = "cft"
rpc-address = "${host_ip_address}:6600"
node-address = "${host_ip_address}:7700"
public-rpc-address = "${host_ip_address}:6600"
san = "dNSName:xyz.eastus.cloudapp.azure.com"

[start]
network-cert-file = "${governor_home}/ccf/keys/networkcert.pem"
member-info = "${governor_home}/ccf/keys/memberccf_cert.pem,${governor_home}/ccf/keys/memberccf_enc_pubk.pem"
gov-script = "${governor_home}/ccf/etc/gov.lua"

Inbound requests are open to this VM on port 6600, and I do see that requests are getting through to the VM. I would have expected based on what I have read in prior discussion that the san parameter with the DNSName would be sufficient. Is there anything else that needs to be configured? I would appreciate any help you could provide here.

[achamayou]

public-rpc-address would typically be set either to the public IP that routes to the VM, or to the DNS pointing to that. Setting that value to a local interface will cause problems, for example when asking the network where to reach the primary (a local IP will be returned, typically not accessible to the client).

A typical configuration would look like:

rpc-address = "${local_ip_address}:443"
node-address = "${local_ip_address}:6600"
public-rpc-address = "${public_ip_address}:443"
san = "dNSName:xyz.eastus.cloudapp.azure.com" where xyz.eastus.cloudapp.azure.com resolves to public_ip_address

Are you able to provide any details on the connection issue you are seeing? Can you successfully telnet for example, but not open a TLS session?

[psfoley]

Thank you @achamayou for the quick response.

I made the changes to the public-rpc-address, using the public IP as well as the DNS Name. In both cases I receive the following error from the application:

ccf-governor    | start_ccf_network.sh: configure ccf network : ACK the member, open network, add user
ccf-governor    | 2021-03-11 17:20:53.724 | INFO     | __main__:activate_member_script:42 - none POST /gov/ack/update_state_digest
ccf-governor    | 2021-03-11 17:20:53.724 | INFO     | __main__:activate_member_script:42 - 200 {"state_digest":"08c3b50c4b561cdf5403ba5496b421f27ac350746a9a2cd6e811af08c246fcc3"}
ccf-governor    | 2021-03-11 17:20:53.727 | INFO     | __main__:activate_member_script:43 - none POST /gov/ack {'state_digest': '08c3b50c4b561cdf5403ba5496b421f27ac350746a9a2cd6e811af08c246fcc3'}
ccf-governor    | 2021-03-11 17:20:53.727 | INFO     | __main__:activate_member_script:43 - 401 {"error":{"code":"InvalidAuthenticationInfo","message":"Signer is not a known member"}}
ccf-governor    | 2021-03-11 17:20:53.727 | ERROR    | __main__:activate_member_script:46 - failed to activate the member: {"error":{"code":"InvalidAuthenticationInfo","message":"Signer is not a known member"}}, code: 401
ccf-governor    | 2021-03-11 17:20:53.727 | ERROR    | __main__:activate_member_script:46 - failed to activate the member: {"error":{"code":"InvalidAuthenticationInfo","message":"Signer is not a known member"}}, code: 401
ccf-governor    |  start_ccf_network.sh: operation failed: /tfl/dev/opt/governor/ccf/bin/configure_ccf_network.py --logfile __screen__ --loglevel WARNING

cchost starts, but it appears that now the provided member certificate does not match the expected identity. Is port 443 required for the rpc-address and public-rpc-address? I am using port 6600 for the rpc-addresses, and 7700 for the node-address. Regarding the follow up question you asked about connection details, I'll revert to setting that allowed the application to start up and get back to you.

[achamayou]

Is port 443 required for the rpc-address and public-rpc-address?

There is no port requirement, 443 is typical for https but you can use any port.

it appears that now the provided member certificate does not match the expected identity.

You must sign governance requests with a valid member identity, initially that must be one of the identities you have passed to cchost via the --member-info command line switch.

Running the sandbox with the --verbose flag is a good way to see a simple network startup sequence in action.

[psfoley]

The second error that I posted turned out to be a bug in our code. Making the original changes that you suggested to the public-rpc-address and san solved the problem. Thanks so much for the help @achamayou