Skip to content

Command line options

Jump to bottom
Ori Shalev edited this page Jan 2, 2019 · 27 revisions

The complete list of available options can be obtained with mcrouter --help. Each option has a "command line name" and an "internal name". The internal name is used in [[mcrouter.options admin command|Admin-requests#get-__mcrouter__options]] and in flavor files. In most cases the internal name is the same as the long command line option name with initial dashes removed and separator dashes converted into underscores (i.e. --config-file has config_file internal name). Exceptions are noted in brackets below.

Config management

  • -f <PATH>, --config-file=<PATH> The config file to use. Mcrouter will keep track of updates to the file; restarting the process is not required.
  • --config-str=<JSON> Configure from a string instead of from file.
  • --validate-config Check config for syntax and logic errors and exit immediately with good (zero) or error (non-zero) status.
  • --disable-reload-configs Turn off automatic monitoring of configs (normally mcrouter will track any changes with inotify and update the runtime config on the fly).
  • --runtime-vars-file=<PATH> Path to the runtime variables file (currently used for dynamic shadowing configuration).
  • --file-observer-poll-period-ms=<N> Sleep for this amount between polling inotify for updates on the tracked files.
  • --file-observer-sleep-before-update-ms=<N> Sleep for this amount after an update occurred. This is a hack to avoid reading a partially written config. For example, some text editors save the file in stages, and inotify will alert us on the first save, so we wait for a bit to work around this.
  • --constantly-reload-configs Debug only. Continuously re-parse and re-load the config. Used to exercise the config reload code (for bugs and performance).
  • --config-dump-root Directory to which mcrouter will save last valid configs. Mcrouter will fallback to these saved configs when it starts with a bad configuration.
  • --max-dumped-config-age Tells mcrouter it should not trust dumped configs (see option above) if they are older than what's specified in this argument.

Routing

  • -R <PREFIX>, --route-prefix=<PREFIX> (default_route) Default routing prefix (e.g. /datacenter/cluster/).
  • --disable-miss-on-get-errors (inverse of miss_on_get_errors) By default, mcrouter converts any errors on get commands into misses (i.e. NOT_FOUND) for the client. This option disables this behaviour, forwarding back all actual errors.
  • --send-invalid-route-to-default Normally requests with keys of the form /foo/bar/key will result in an error if /foo/bar/ does not explicitly exist in the config. With this option, mcrouter will send those requests to default route (respecting keep_routing_prefix pool option, and forwarding /foo/bar/ unmodified if it's set).
  • --big-value-split-threshold=<N> Values in update requests over this threshold in bytes will be automatically split into smaller values (re-assembly on get is also automatic). If 0, no splitting is performed.

Timeouts

The simplest way is to specify destination timeouts is with

  • -t <N>, --server-timeout=<N> (server_timeout_ms) Destination timeout in ms.

There are two ways to fine tune timeouts in a geographically distributed setup. A crude way is to add a pool_locality setting to each pool (set to literal strings cluster or region) and specify

  • --cluster-pools-timeout=<N> (cluster_pools_timeout_ms) Destination timeout for cluster pools in ms.
  • --regional-pools-timeout=<N> (regional_pools_timeout_ms) Destination timeout for regional pools in ms.

There is a way to fine tune timeouts in a geographically distributed setup. You can specify region and cluster pool settings. These are compared to the components of the default route (i.e. /some_region/some_cluster/). Depending on whether we're in the same cluster as the pool, same region, or a different region entirely, one of the following timeouts is in effect:

  • --cross-region-timeout-ms=<N> Timeout for talking to pools in another region.
  • --cross-cluster-timeout-ms=<N> Timeout for talking to pools within the same region but a different cluster.
  • --within-cluster-timeout-ms=<N> Timeout for talking to pools within the same cluster.

Health check

See discussion about TKO.

  • --timeouts-until-tko=<N> (failures_until_tko) Mark as soft TKO after this many timeouts.
  • -r <N>, --probe-timeout-initial=<N> (probe_delay_initial_ms); --probe-timeout-max=<N> (probe_delay_max_ms) TKO probes are sent with exponentially increasing interval. These options control the initial size of this interval and the maximum size respectively (in ms). The actual intervals have some random jitter of up to 50% added to them to avoid overloading a single failed host with TKO probes from different mcrouters.
  • --maximum-soft-tkos=<N> Maximum number of destinations allowed to be in soft TKO state at any point in time. This is for cascading failure protection. We stop marking hosts as TKO on timeout once we reach this limit.

Network

  • -p <PORT1>,<PORT2>,..., --port <PORT1>,<PORT2>,... Port(s) to listen on (comma separated).
  • --reset-inactive-connection-interval=<N> A timer with this period (in ms) closes all inactive outgoing connections. 0 to disable.
  • -K <N>, --keepalive-count=<N> (keepalive_cnt) TCP KEEPALIVE count for outgoing connections.
  • -i <N>, --keepalive-interval=<N> (keepalive_interval_s) TCP KEEPALIVE interval for outgoing connections.
  • -I <N>, --keepalive-idle=<N> (keepalive_idle_s) TCP KEEPALIVE idle for outgoing connections.
  • --listen-sock-fd Listen socket fd to take over (used in tests).
  • --no-network Debug only. Return random generated replies to every request, do not use network.

SSL

See SSL support.

  • --ssl-port <PORT1>,<PORT2> SSL Port(s) to listen on (comma separated).
  • --pem-cert-path=<PATH>, --pem-key-path=<PATH>, --pem-ca-path=<PATH> Paths of pem-style certificate/key/CA cert for SSL (used for both incoming and outgoing connections).

Delete stream

See reliable delete stream.

  • --asynclog-disable Disable logging of failed deletes; errors will be returned to the client.
  • -a <PATH>, --async-dir=<PATH> (async_spool) Location for the failed deletes log.
  • --use-asynclog-version2 Enable new format log (the new format logs more information per delete and will become the default in the future).

Runtime

  • --num-proxies=<N> Run with N threads. Typically one thread per core is a good rule of thumb.
  • --fibers-max-pool-size=<N> Maximum number of preallocated free fibers to keep around per thread. Should be set to the expected number of outgoing requests in flight - higher numbers will increase performance on request bursts at the expense of constantly higher memory usage.
  • --fibers-stack-size=<N> The amount of stack to allocate per fiber, in bytes. Should normally not be adjusted unless significant code changes or unusually complicated configs are needed.
  • --fibers-record-stack-size-every Record exact amount of fibers stacks used in the fibers_stack_high_watermark stat counter, which is normally an estimate (location at a likely bottom point of the stack) on every N fiber. This is expensive, so by default runs on every 100000 fiber.
  • --disable-fibers-use-guard-pages By default mcrouter protects limited number of fiber stacks with guard pages (protected piece of a memory at the top of the stack). This way on stack overflow we'll immediately crash, not corrupt memory. This options provides a way to disable this behavior. One particular reason it is useful: 'perf record' may take huge amount of time when running with guard pages.

Queueing

See the discussion on quality of service.

  • --reqs-per-read=<N> If specified, the incoming request buffer size is automatically adjusted to allow through roughly this many requests per event loop iteration. Smaller values may improve latency in a large fan out scenario.
  • --max-client-outstanding-reqs=<N> Maximum requests outstanding (that is, sent by the client but not yet replied by mcrouter) per client connection. Mcrouter stops reading from the connection socket when this limit is reached. This limits overall memory usage by mcrouter.
  • --proxy-max-inflight-requests=<N> Maximum requests that were routed through the config but not replied yet. This is a queue between the incoming parsed requests and individual fibers allocated per request. The rationale is to limit the number of fibers in use, which require more memory than the parsed requests.
  • --destination-rate-limiting Enable rates options in pool configurations, which allow to throttle requests to destination by operation type.
  • --target-max-inflight-requests=<N> Maximum outstanding requests allowed to be sent per destination. Requests over this limit will queue up in mcrouter.
  • --target-max-pending-requests=<N> The size of the outstanding requests queue per destination. Requests over this limit will be returned with an error immediately.

Stats

  • --stats-root=<PATH> Root directory for stats files.
  • --stats-logging-interval=<N> Time in ms between stat file updates, or 0 for no logging.
  • --logging-rtt-outlier-threshold-us=<N> Destination requests with round trip time exceeding this threshold will be considered "outliers" and counted in the outliers stats.

Process management

  • -L <PATH>, --log-path=<PATH> Path for the log file.
  • -n <N>, --connection-limit=<N> File descriptor limit.
  • --debug-fifo-root=<PATH> Root directory for debug fifos. '' to disable.

Mcrouter wiki

Clone this wiki locally