Don't panic in the ECS where we know a fetch is valid #39782

Job	Run time
build (windows-latest)	1m 24s
build (ubuntu-latest)	1m 22s
build (macos-latest)	1m 38s
check-doc	1m 0s
check-cfg	53s
check-bevy-internal-imports	4s
check-missing-examples-in-docs	28s
miri	6m 9s
run-examples-macos-metal	41s
ci	1m 4s
toml	2m 5s
typos	9s
check-missing-features-in-docs	31s
check-compiles	0s
markdownlint	1m 9s
build-wasm	0s
msrv	0s
build-wasm-atomics	0s
	18m 37s

Provide feedback