You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
solsitce.reduce(): this should be a function reducing a list of pytrees to a single pytree. This should be a single dispatch generic function which calls .merge() on any Metrics objects (a bit like the CLU implementation) and has customisable behaviour for other types of pytrees (e.g. mean leaves, sum leaves etc...). Could add an implementation for model parallel training ???
solstice.gather(): possibly unnecessary. Utility for reducing and gathering sharded pytrees across devices. This will be most useful for accumulating Metrics, but maybe also for model parallel training ??
The text was updated successfully, but these errors were encountered:
solsitce.reduce()
: this should be a function reducing a list of pytrees to a single pytree. This should be a single dispatch generic function which calls.merge()
on anyMetrics
objects (a bit like the CLU implementation) and has customisable behaviour for other types of pytrees (e.g. mean leaves, sum leaves etc...). Could add an implementation for model parallel training ???solstice.gather()
: possibly unnecessary. Utility for reducing and gathering sharded pytrees across devices. This will be most useful for accumulatingMetrics
, but maybe also for model parallel training ??The text was updated successfully, but these errors were encountered: