Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from tensorflow:master #169

Open
wants to merge 1,655 commits into
base: master
Choose a base branch
from

Conversation

pull[bot]
Copy link

@pull pull bot commented Dec 8, 2024

See Commits and Changes for more details.


Created by pull[bot] (v2.0.0-alpha.1)

Can you help keep this open source service alive? 💖 Please sponsor : )

@pull pull bot added the ⤵️ pull label Dec 8, 2024
toli-y and others added 29 commits January 13, 2025 14:26
…ns when breaking circular cp for better traceability.

PiperOrigin-RevId: 715104228
…is CL makes two minor tweaks to the `BasicStringArray` class (the string array implementation in the PjRt-IFRT backend): (1) `CopyToHostBuffer` now supports the host buffer semantics of `kImmutableUntilTransferCompletes`. (2) `FullyReplicated` now works with `ConcreteSharding`.

PiperOrigin-RevId: 715110567
PiperOrigin-RevId: 715112095
…duce more of its member function to C Api.

PiperOrigin-RevId: 715113720
…ting during instruction fusion.

The computations are not being sorted in a semantically meaningful order; they are sorted by instruction count with ties being broken consistently but arbitrarily (based on a hash of the string representation of the computation). There is therefore no reason why these passes need to traverse the computations in this specific order.

PiperOrigin-RevId: 715122377
PiperOrigin-RevId: 715131342
…ot_handler.cc.

This function creates a copy for the HloInstruction in the given PartitionedHlo and returns a new PartitionedHlo for the copy. This can be reused by other operators (like gather/scatter).

PiperOrigin-RevId: 715138672
Breaking internal tests

Reverts 5e78ccd

PiperOrigin-RevId: 715160248
…stom op.

Attributes will determine behavior

PiperOrigin-RevId: 715161504
PiperOrigin-RevId: 715162849
Only bitcast requires the layout to be known when evaluating HLO.
In all other cases, we can evaluate without knowing the layout.
This is needed for collective pipelining where we have to analyse while loops before layouts were assigned.

PiperOrigin-RevId: 715163612
A gather has two operands, input and indices. If they point to the same instruction, create a copy for indices.

A scatter has n inputs, 1 indices, and n updates (2n+1 operands in total). We allow overlap between n inputs. We also allow overlap between n updates. We need to create a copy if
* indices overlap with any input or update
* update overlap with any input

The added copy will be removed if it is redundant in the following memory related passes (e.g., CopyInsertion).

PiperOrigin-RevId: 715164959
This CL takes care of
1. Migrating external projects dependencies from

```
tensorflow/compiler/xla:test
tensorflow/compiler/xla:test_helpers
tensorflow/compiler/xla/service:pattern_matcher_gmock
```

to `tensorflow/compiler/xla/hlo/testlib:*`

PiperOrigin-RevId: 715167258
…dleDotHelper`.

`HandleDotHelper` is called once for a single dot operation, while `PartitionDot` can be called many times. We need to consider adding a copy only once.

PiperOrigin-RevId: 715189518
Imported from GitHub PR openxla/xla#21104

Transpose folding pass doesn't preserve backend config when creating the new dot with transpose folded. Changing the behavior to copy the old dot's config to the new dot.
Copybara import of the project:

--
d2d6b628af1cab777a210e4ac62184e52fe9f4a9 by TJ Xu <[email protected]>:

Preserve backend config when folding transpose

--
6b5fa3a1cb70a790803e3ac57ff8329690e88e5e by TJ Xu <[email protected]>:

use SetupDerivedInstruction instead of just copying the backend config

Merging this change closes #21104

PiperOrigin-RevId: 715204523
This method was renamed but staging function kept, switch to renamed variant.

PiperOrigin-RevId: 715208433
PiperOrigin-RevId: 715229132
PiperOrigin-RevId: 715249101
PiperOrigin-RevId: 715261168
PiperOrigin-RevId: 715261640
Imported from GitHub PR openxla/xla#21223

This is part-3 of #18785 and #20595.

Motivation: Smaller build files, fewer merge conflicts, and convinience in development (more intuitive targets). For discussion surrounding this, check thread in #18785.
Copybara import of the project:

--
7832f8483091b70214d17789c05c38959d67a515 by Shraiysh Vaishay <[email protected]>:

[nfc] Cleanup build files for expander transforms

This is part-3 of #18785 and #20595.

Motivation: Smaller build files, fewer merge conflicts, and convinience
in development (more intuitive targets). For discussion surrounding
this, check thread in #18785.

Merging this change closes #21223

PiperOrigin-RevId: 715263765
PiperOrigin-RevId: 715267948
PiperOrigin-RevId: 715278564
bmass02 and others added 30 commits January 17, 2025 11:49
Add DutyCycleCombiner for handling intra and inter chip duty cycle aggregation.
Fix DutyCycleTracker bugs with idleness and duplicate active times.

PiperOrigin-RevId: 716741025
…ting behavior unchanged.

Only when coordination_agent_recoverable is set, it tries to reconnect to the cluster and would lead to AlreadyExists error. In this case the already_existing error can be handled by checking the existing topology is same as the new one.

PiperOrigin-RevId: 716748621
…ets.

This step towards encouraging extrenal projects to migrate to the already
migrated hlo sub-components.

PiperOrigin-RevId: 716762154
… reshape ops

Allow a single bounded dynamic dimension. This is likely a short term fix as bounded dynamism as a while likely needs a lot of thought, but this solution with a single bounded dim is unambiguous so should be safe.

PiperOrigin-RevId: 716763010
Updates LLVM usage to match
[bf17016a92bc](llvm/llvm-project@bf17016a92bc)

PiperOrigin-RevId: 716773763
The attribute should be named `channel_handle`, not `channel_id`.

PiperOrigin-RevId: 716822019
…on host memory as host compute. This, of course, excludes DynamicUpdateSlices which are used for host offloading DMAs.

PiperOrigin-RevId: 716839236
PiperOrigin-RevId: 716844952
PiperOrigin-RevId: 716860231
PiperOrigin-RevId: 716914254
PiperOrigin-RevId: 716914439
…size, not

just the size of the newly added values.

PiperOrigin-RevId: 716915408
PiperOrigin-RevId: 716930131
PiperOrigin-RevId: 716939752
PiperOrigin-RevId: 716945103
PiperOrigin-RevId: 716955798
PiperOrigin-RevId: 716956415
PiperOrigin-RevId: 716958380
PiperOrigin-RevId: 717028480
PiperOrigin-RevId: 717039966
PiperOrigin-RevId: 717051750
PiperOrigin-RevId: 717178492
…ding a header inclusion.

As far as I can tell, this does nothing at all.

PiperOrigin-RevId: 717243277
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.