Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transforms] Batch creating Transforms on a new cluster fails #117995

Open
prwhelan opened this issue Dec 4, 2024 · 1 comment · May be fixed by #118003
Open

[Transforms] Batch creating Transforms on a new cluster fails #117995

prwhelan opened this issue Dec 4, 2024 · 1 comment · May be fixed by #118003
Assignees
Labels
>bug :ml/Transform Transform Team:ML Meta label for the ML team v9.0.0

Comments

@prwhelan
Copy link
Member

prwhelan commented Dec 4, 2024

Repro steps:

  • Launch new cluster
  • Create two Transforms in parallel requests

The first request will block and wait for the Internal system index to create itself, the request will fail to search over the initializing system index.

Failed to execute phase [query], at [email protected]/org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:698) at [email protected]/org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:420) at [email protected]/org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:230) at [email protected]/org.elasticsearch.action.search.TransportSearchAction$AsyncSearchActionProvider.runNewSearchPhase(TransportSearchAction.java:1551) at [email protected]/org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:1300) at [email protected]/org.elasticsearch.action.search.TransportSearchAction.executeLocalSearch(TransportSearchAction.java:1033) at [email protected]/org.elasticsearch.action.search.TransportSearchAction.lambda$executeRequest$4(TransportSearchAction.java:360) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:109) at [email protected]/org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:77) at [email protected]/org.elasticsearch.action.search.TransportSearchAction.executeRequest(TransportSearchAction.java:526) at [email protected]/org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:316) at [email protected]/org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:121) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:135) at [email protected]/org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:54) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/co.elastic.elasticsearch.metering.activitytracking.ActivityTrackerActionFilter.apply(ActivityTrackerActionFilter.java:54) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:54) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.action.support.MappedActionFilters$MappedFilterChain.proceed(MappedActionFilters.java:71) at [email protected]/org.elasticsearch.action.support.MappedActionFilters.apply(MappedActionFilters.java:49) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$4(SecurityActionFilter.java:182) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$DelegatingFailureActionListener.onResponse(ActionListenerImplementations.java:219) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:640) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.ShardSearchRequestInterceptor.intercept(ShardSearchRequestInterceptor.java:23) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.SearchRequestInterceptor.intercept(SearchRequestInterceptor.java:20) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.ValidateRequestInterceptor.intercept(ValidateRequestInterceptor.java:20) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.SearchRequestCacheDisablingInterceptor.intercept(SearchRequestCacheDisablingInterceptor.java:53) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.ResizeRequestInterceptor.intercept(ResizeRequestInterceptor.java:105) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.UpdateRequestInterceptor.intercept(UpdateRequestInterceptor.java:27) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.BulkShardRequestInterceptor.intercept(BulkShardRequestInterceptor.java:85) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.DlsFlsLicenseRequestInterceptor.intercept(DlsFlsLicenseRequestInterceptor.java:106) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634) at [email protected]/org.elasticsearch.xpack.security.authz.interceptor.IndicesAliasesRequestInterceptor.intercept(IndicesAliasesRequestInterceptor.java:127) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.runRequestInterceptors(AuthorizationService.java:634) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.handleIndexActionAuthorizationResult(AuthorizationService.java:619) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$13(AuthorizationService.java:517) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:1033) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:999) at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.lambda$authorizeIndexAction$3(RBACEngine.java:422) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.action.support.SubscribableListener$SuccessResult.complete(SubscribableListener.java:387) at [email protected]/org.elasticsearch.action.support.SubscribableListener.tryComplete(SubscribableListener.java:307) at [email protected]/org.elasticsearch.action.support.SubscribableListener.addListener(SubscribableListener.java:210) at [email protected]/org.elasticsearch.action.support.SubscribableListener.addListener(SubscribableListener.java:175) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$CachingAsyncSupplier.getAsync(AuthorizationService.java:1079) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.authorizeIndexAction(RBACEngine.java:390) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeAction(AuthorizationService.java:510) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.maybeAuthorizeRunAs(AuthorizationService.java:442) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorize$3(AuthorizationService.java:329) at [email protected]/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:257) at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.lambda$resolveAuthorizationInfo$0(RBACEngine.java:156) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRoles$4(CompositeRolesStore.java:202) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRole(CompositeRolesStore.java:210) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRoles(CompositeRolesStore.java:195) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:152) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:345) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$5(SecurityActionFilter.java:178) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$MappedActionListener.onResponse(ActionListenerImplementations.java:97) at [email protected]/org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticate(AuthenticatorChain.java:93) at [email protected]/org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:264) at [email protected]/org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:173) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:174) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$apply$1(SecurityActionFilter.java:115) at [email protected]/org.elasticsearch.xpack.core.security.SecurityContext.executeAsInternalUser(SecurityContext.java:170) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationUtils.switchUserBasedOnActionOriginAndExecute(AuthorizationUtils.java:164) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:111) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.action.support.TransportAction.handleExecution(TransportAction.java:96) at [email protected]/org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:59) at [email protected]/org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:197) at [email protected]/org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:106) at [email protected]/org.elasticsearch.client.internal.node.NodeClient.doExecute(NodeClient.java:84) at [email protected]/org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:140) at [email protected]/org.elasticsearch.xpack.core.ClientHelper.lambda$executeAsyncWithOrigin$3(ClientHelper.java:237) at [email protected]/org.elasticsearch.xpack.core.ClientHelper.executeAsyncWithOrigin(ClientHelper.java:222) at [email protected]/org.elasticsearch.xpack.core.ClientHelper.executeAsyncWithOrigin(ClientHelper.java:237) at [email protected]/org.elasticsearch.xpack.transform.persistence.IndexBasedTransformConfigManager.executeAsyncWithOrigin(IndexBasedTransformConfigManager.java:802) at [email protected]/org.elasticsearch.xpack.transform.persistence.IndexBasedTransformConfigManager.getTransformStoredDoc(IndexBasedTransformConfigManager.java:711) at [email protected]/org.elasticsearch.xpack.transform.persistence.AuthorizationStatePersistenceUtils.persistAuthState(AuthorizationStatePersistenceUtils.java:138) at [email protected]/org.elasticsearch.xpack.transform.action.TransportPutTransformAction.lambda$masterOperation$2(TransportPutTransformAction.java:139) at [email protected]/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:257) at [email protected]/org.elasticsearch.xpack.transform.action.TransformPrivilegeChecker.handlePrivilegesResponse(TransformPrivilegeChecker.java:158) at [email protected]/org.elasticsearch.xpack.transform.action.TransformPrivilegeChecker.lambda$checkPrivileges$0(TransformPrivilegeChecker.java:61) at [email protected]/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:257) at [email protected]/org.elasticsearch.tasks.TaskManager$1.onResponse(TaskManager.java:203) at [email protected]/org.elasticsearch.tasks.TaskManager$1.onResponse(TaskManager.java:197) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$RunBeforeActionListener.onResponse(ActionListenerImplementations.java:336) at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$MappedActionListener.onResponse(ActionListenerImplementations.java:97) at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.lambda$checkPrivileges$7(RBACEngine.java:612) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$DelegatingFailureActionListener.onResponse(ActionListenerImplementations.java:219) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.checkPrivileges(RBACEngine.java:687) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$checkPrivileges$1(AuthorizationService.java:198) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$DelegatingFailureActionListener.onResponse(ActionListenerImplementations.java:219) at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$MappedActionListener.onResponse(ActionListenerImplementations.java:97) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRole$5(CompositeRolesStore.java:220) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$0(RoleReferenceIntersection.java:49) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.action.support.GroupedActionListener.onResponse(GroupedActionListener.java:57) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.buildRoleFromRoleReference(CompositeRolesStore.java:317) at [email protected]/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$1(RoleReferenceIntersection.java:53) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at [email protected]/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.buildRole(RoleReferenceIntersection.java:53) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRole(CompositeRolesStore.java:218) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:165) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.checkPrivileges(AuthorizationService.java:194) at [email protected]/org.elasticsearch.xpack.security.action.user.TransportHasPrivilegesAction.lambda$doExecute$1(TransportHasPrivilegesAction.java:74) at [email protected]/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:257) at [email protected]/org.elasticsearch.xpack.security.authz.store.NativePrivilegeStore.getPrivileges(NativePrivilegeStore.java:197) at [email protected]/org.elasticsearch.xpack.security.authz.store.NativePrivilegeStore.getPrivileges(NativePrivilegeStore.java:160) at [email protected]/org.elasticsearch.xpack.security.action.user.TransportHasPrivilegesAction.resolveApplicationPrivileges(TransportHasPrivilegesAction.java:100) at [email protected]/org.elasticsearch.xpack.security.action.user.TransportHasPrivilegesAction.doExecute(TransportHasPrivilegesAction.java:71) at [email protected]/org.elasticsearch.xpack.security.action.user.TransportHasPrivilegesAction.doExecute(TransportHasPrivilegesAction.java:41) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:135) at [email protected]/org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:54) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/co.elastic.elasticsearch.metering.activitytracking.ActivityTrackerActionFilter.apply(ActivityTrackerActionFilter.java:54) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:54) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.action.support.MappedActionFilters$MappedFilterChain.proceed(MappedActionFilters.java:71) at [email protected]/org.elasticsearch.action.support.MappedActionFilters.apply(MappedActionFilters.java:49) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$4(SecurityActionFilter.java:182) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$DelegatingFailureActionListener.onResponse(ActionListenerImplementations.java:219) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$8(AuthorizationService.java:462) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:1033) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:999) at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$9(AuthorizationService.java:476) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.authorizeClusterAction(RBACEngine.java:191) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeAction(AuthorizationService.java:466) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.maybeAuthorizeRunAs(AuthorizationService.java:442) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorize$3(AuthorizationService.java:329) at [email protected]/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:257) at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.lambda$resolveAuthorizationInfo$0(RBACEngine.java:156) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRoles$4(CompositeRolesStore.java:202) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRole$5(CompositeRolesStore.java:220) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$0(RoleReferenceIntersection.java:49) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.action.support.GroupedActionListener.onResponse(GroupedActionListener.java:57) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.buildRoleFromRoleReference(CompositeRolesStore.java:317) at [email protected]/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$1(RoleReferenceIntersection.java:53) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at [email protected]/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.buildRole(RoleReferenceIntersection.java:53) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRole(CompositeRolesStore.java:218) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRoles(CompositeRolesStore.java:195) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:152) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:345) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$5(SecurityActionFilter.java:178) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$MappedActionListener.onResponse(ActionListenerImplementations.java:97) at [email protected]/org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticate(AuthenticatorChain.java:93) at [email protected]/org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:264) at [email protected]/org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:173) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:174) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:131) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.action.support.TransportAction.handleExecution(TransportAction.java:96) at [email protected]/org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:59) at [email protected]/org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:197) at [email protected]/org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:106) at [email protected]/org.elasticsearch.client.internal.node.NodeClient.doExecute(NodeClient.java:84) at [email protected]/org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:140) at [email protected]/org.elasticsearch.xpack.transform.action.TransformPrivilegeChecker.lambda$checkPrivileges$1(TransformPrivilegeChecker.java:75) at [email protected]/org.elasticsearch.xpack.transform.utils.SecondaryAuthorizationUtils.useSecondaryAuthIfAvailable(SecondaryAuthorizationUtils.java:52) at [email protected]/org.elasticsearch.xpack.transform.action.TransformPrivilegeChecker.checkPrivileges(TransformPrivilegeChecker.java:57) at [email protected]/org.elasticsearch.xpack.transform.action.TransportPutTransformAction.masterOperation(TransportPutTransformAction.java:129) at [email protected]/org.elasticsearch.xpack.transform.action.TransportPutTransformAction.masterOperation(TransportPutTransformAction.java:53) at [email protected]/org.elasticsearch.action.support.master.TransportMasterNodeAction.executeMasterOperation(TransportMasterNodeAction.java:127) at [email protected]/org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.lambda$doStart$3(TransportMasterNodeAction.java:238) at [email protected]/org.elasticsearch.action.ActionRunnable$4.doRun(ActionRunnable.java:101) at [email protected]/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27) at [email protected]/org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:286) at [email protected]/org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.doStart(TransportMasterNodeAction.java:238) at [email protected]/org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:175) at [email protected]/org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:57) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:135) at [email protected]/org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:54) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/co.elastic.elasticsearch.metering.activitytracking.ActivityTrackerActionFilter.apply(ActivityTrackerActionFilter.java:54) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:54) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.action.support.MappedActionFilters$MappedFilterChain.proceed(MappedActionFilters.java:71) at [email protected]/org.elasticsearch.action.support.MappedActionFilters.apply(MappedActionFilters.java:49) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$4(SecurityActionFilter.java:182) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$DelegatingFailureActionListener.onResponse(ActionListenerImplementations.java:219) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$8(AuthorizationService.java:462) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:1033) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:999) at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$9(AuthorizationService.java:476) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.authorizeClusterAction(RBACEngine.java:191) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeAction(AuthorizationService.java:466) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.maybeAuthorizeRunAs(AuthorizationService.java:442) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorize$3(AuthorizationService.java:329) at [email protected]/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:257) at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.lambda$resolveAuthorizationInfo$0(RBACEngine.java:156) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRoles$4(CompositeRolesStore.java:202) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRole$5(CompositeRolesStore.java:220) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$0(RoleReferenceIntersection.java:49) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.action.support.GroupedActionListener.onResponse(GroupedActionListener.java:57) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.buildRoleFromRoleReference(CompositeRolesStore.java:317) at [email protected]/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$1(RoleReferenceIntersection.java:53) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at [email protected]/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.buildRole(RoleReferenceIntersection.java:53) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRole(CompositeRolesStore.java:218) at [email protected]/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRoles(CompositeRolesStore.java:195) at [email protected]/org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:152) at [email protected]/org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:345) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$5(SecurityActionFilter.java:178) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$MappedActionListener.onResponse(ActionListenerImplementations.java:97) at [email protected]/org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticate(AuthenticatorChain.java:93) at [email protected]/org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:264) at [email protected]/org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:173) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:174) at [email protected]/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:131) at [email protected]/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:132) at [email protected]/org.elasticsearch.action.support.TransportAction.handleExecution(TransportAction.java:96) at [email protected]/org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:59) at [email protected]/org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:197) at [email protected]/org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:106) at [email protected]/org.elasticsearch.rest.action.RestCancellableNodeClient.doExecute(RestCancellableNodeClient.java:82) at [email protected]/org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:140) at [email protected]/org.elasticsearch.xpack.transform.rest.action.RestPutTransformAction.lambda$prepareRequest$0(RestPutTransformAction.java:70) at [email protected]/org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:143) at [email protected]/org.elasticsearch.rest.RestController$1.onResponse(RestController.java:465) at [email protected]/org.elasticsearch.rest.RestController$1.onResponse(RestController.java:459) at [email protected]/org.elasticsearch.xpack.security.rest.SecurityRestFilter.doHandleRequest(SecurityRestFilter.java:89) at [email protected]/org.elasticsearch.xpack.security.rest.SecurityRestFilter.lambda$intercept$0(SecurityRestFilter.java:81) at [email protected]/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:257) at [email protected]/org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.lambda$authenticateAndAttachToContext$3(SecondaryAuthenticator.java:99) at [email protected]/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) at [email protected]/org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.authenticate(SecondaryAuthenticator.java:109) at [email protected]/org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.authenticateAndAttachToContext(SecondaryAuthenticator.java:90) at [email protected]/org.elasticsearch.xpack.security.rest.SecurityRestFilter.intercept(SecurityRestFilter.java:75) at [email protected]/org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:459) at [email protected]/org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:623) at [email protected]/org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:306) at [email protected]/co.elastic.elasticsearch.serverless.rest.ServerlessRestController.dispatchRequest(ServerlessRestController.java:133) at [email protected]/org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:488) at [email protected]/org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:584) at [email protected]/org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:461) at [email protected]/org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.handlePipelinedRequest(Netty4HttpPipeliningHandler.java:169) at [email protected]/org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:148) at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at [email protected]/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at [email protected]/io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:107) at [email protected]/org.elasticsearch.http.netty4.Netty4HttpAggregator.channelRead(Netty4HttpAggregator.java:52) at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at [email protected]/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at [email protected]/io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:107) at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at [email protected]/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at [email protected]/org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.forwardData(Netty4HttpHeaderValidator.java:213) at [email protected]/org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.forwardFullRequest(Netty4HttpHeaderValidator.java:156) at [email protected]/org.elasticsearch.http.netty4.Netty4HttpHeaderValidator$1.lambda$onResponse$0(Netty4HttpHeaderValidator.java:128) at [email protected]/io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173) at [email protected]/io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166) at [email protected]/io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) at [email protected]/io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566) at [email protected]/io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at [email protected]/io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.base/java.lang.Thread.run(Thread.java:1575) Caused by: Failed to execute phase [query], Search rejected due to missing shards [[.transform-internal-007][0]]. Consider usingallow_partial_search_results` setting to bypass this error. at [email protected]/org.elasticsearch.action.search.SearchPhase.doCheckNoMissingShards(SearchPhase.java:74) at [email protected]/org.elasticsearch.action.search.SearchPhase.doCheckNoMissingShards(SearchPhase.java:49) at [email protected]/org.elasticsearch.action.search.AbstractSearchAsyncAction.run(AbstractSearchAsyncAction.java:240) at [email protected]/org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:415) ... 310 more

`

@prwhelan prwhelan added :ml/Transform Transform >bug Team:ML Meta label for the ML team v9.0.0 labels Dec 4, 2024
@prwhelan prwhelan self-assigned this Dec 4, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

prwhelan added a commit to prwhelan/elasticsearch that referenced this issue Dec 4, 2024
Putting a new Transform should create the latest system index if it
doesn't already exist, and Transforms should wait for the system index
to become fully available since Transforms immediately starts searching
over the index.

We have existing code to migrate Transforms to new system indices within
the PersistentTaskExecutor, we can reuse that when we create new
Transforms.

This isn't copied to places like Update because:
1. Updates can work on old system indices for existing Transforms, and
   PersistentTaskExecutor will continue to migrate those Transforms.
2. You cannot update a non-existent Transform anyway.

Fix elastic#117995
@prwhelan prwhelan linked a pull request Dec 4, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :ml/Transform Transform Team:ML Meta label for the ML team v9.0.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants