Skip to content

Commit

Permalink
Update how-to-run.adoc post-BGP (#4404)
Browse files Browse the repository at this point in the history
Small edits needed to bring this in line with how routes are configured
now. `gateway_ip` is no longer explicitly named as such, so I've added
an explicit comment to signpost how $GATEWAY_IP should be used w.r.t.
the default route.
  • Loading branch information
FelixMcFelix authored Nov 17, 2023
1 parent 2585b88 commit 781ed12
Showing 1 changed file with 33 additions and 7 deletions.
40 changes: 33 additions & 7 deletions docs/how-to-run.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -266,26 +266,52 @@ last = "192.168.1.29"

This is a range of IP addresses on your external network that Omicron can assign to externally-facing services (like DNS and the API). You'll need to change these if you've picked different addresses for your external network. See <<_external_networking>> above for more on this.

You will also need to update route information if your `$GATEWAY_IP` differs from the default.
The below example demonstrates a single static gateway route; in-depth explanations for testing with BGP can be found https://docs.oxide.computer/guides/system/network-preparations#_rack_switch_configuration_with_bgp[in the Network Preparations guide] and https://docs.oxide.computer/guides/operator/configuring-bgp[the Configuring BGP guide]:

[source,toml]
----
# Configuration to bring up boundary services and make Nexus reachable from the
# outside. This block assumes that you're following option (2) above: putting
# your Oxide system on an existing network that you control.
[rack_network_config]
# The gateway for the external network
gateway_ip = "192.168.1.199"
# An internal-only IPv6 address block which contains AZ-wide services.
# This does not need to be changed.
rack_subnet = "fd00:1122:3344:01::/56"
# A range of IP addresses used by Boundary Services on the network. In a real
# system, these would be addresses of the uplink ports on the Sidecar. With
# softnpu, only one address is used.
infra_ip_first = "192.168.1.30"
infra_ip_last = "192.168.1.30"
# Name of the port. This should always be "qsfp0" when using softnpu.
uplink_port = "qsfp0"
uplink_port_speed = "40G"
uplink_port_fec="none"
# Configurations for BGP routers to run on the scrimlets.
# This array can typically be safely left empty for home/local use,
# otherwise this is a list of { asn: u32, originate: ["<v4 network>"] }
# structs which will be be inserted when Nexus is started by sled-agent.
# See the 'Network Preparations' guide linked above.
bgp = []
[[rack_network_config.ports]]
# Routes associated with this port.
# NOTE: The below `nexthop` should be set to $GATEWAY_IP for your configuration
routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
# Addresses associated with this port.
# For softnpu, an address within the "infra" block above that will be used for
# the softnpu uplink port. You can just pick the first address in that pool.
uplink_ip = "192.168.1.30"
addresses = ["192.168.1.30/32"]
# Name of the uplink port. This should always be "qsfp0" when using softnpu.
port = "qsfp0"
# The speed of this port.
uplink_port_speed = "40G"
# The forward error correction mode for this port.
uplink_port_fec="none"
# Switch to use for the uplink. For single-rack deployments this can be
# "switch0" (upper slot) or "switch1" (lower slot). For single-node softnpu
# and dendrite stub environments, use "switch0"
switch = "switch0"
# Neighbors we expect to peer with over BGP on this port.
# see: common/src/api/internal/shared.rs – BgpPeerConfig
bgp_peers = []
----

In some configurations (not the one described here), it may be necessary to update `smf/sled-agent/$MACHINE/config.toml`:
Expand Down

0 comments on commit 781ed12

Please sign in to comment.