Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

worker 是否可用问题 #127

Open
mua-haha opened this issue Nov 8, 2022 · 6 comments
Open

worker 是否可用问题 #127

mua-haha opened this issue Nov 8, 2022 · 6 comments

Comments

@mua-haha
Copy link

mua-haha commented Nov 8, 2022

大家好，有个问题想请教下：
我部署了三台机器，ip分别是
image
计划是把100作为master，另两台作为worker，但master也作为worker执行任务。
其中78这台机器由于8080端口被占用，所以我用的8081，但是9887是可以的。
我的问题是：
1、如何查看目前哪些worker节点可用？
2、机器组监控 只能看到68这台机器的监控
3、在调度中心，执行任务只能随机选择某个worker执行吗，能不能这个机器组的机器全部执行？
4、调度中心执行任务时，我看到执行任务的机器是100或者68，没有78，是因为端口的原因吗
@mua-haha
Copy link
Author

mua-haha commented Nov 8, 2022

plus，我是用root启动的

@scxwhite
Copy link
Owner

scxwhite commented Nov 8, 2022

1.hera有个接口，凡是连接到master节点的都可以通过 http://localhost:8080/hera/homePage/getJobQueueInfo 接口查看所有机器。具体可以查看：赫拉(hera)分布式任务调度系统之Q&A(五)
2.说明68没连接上，或者68机器的信息采集有问题，建议查看异常日志排查
3.一个任务只能在一个worker上执行，不能在这个机器组的所有机器都执行一下。建议自己修改代码实现
4.根据机器负载随机执行的，只要机器的负载足够就会提交任务，78机器应该只是没有被master分发选到，可以在任务高峰期再观察下。

@mua-haha
Copy link
Author

mua-haha commented Nov 8, 2022 •
edited
Loading

感谢回复
我在Q&A里看到该接口了，但奈何 请求后，后台会有NPE 。

2022-11-08 14:09:31.372 ERROR com.dfire.logs.ErrorLog.error:22 master handler future take error:{}
java.util.concurrent.ExecutionException: java.lang.NullPointerException
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at com.dfire.core.netty.master.MasterHandler.lambda$new$0(MasterHandler.java:58)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
        at com.google.protobuf.Internal.checkNotNull(Internal.java:67)
        at com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:386)
        at com.dfire.protocol.RpcHeartBeatMessage$HeartBeatMessage$Builder.addAllRerunRunning(RpcHeartBeatMessage.java:2487)
        at com.dfire.core.netty.master.response.MasterHandlerWebResponse.lambda$buildJobQueueInfo$0(MasterHandlerWebResponse.java:198)
        at java.util.concurrent.ConcurrentHashMap$ValuesView.forEach(ConcurrentHashMap.java:4707)
        at com.dfire.core.netty.master.response.MasterHandlerWebResponse.buildJobQueueInfo(MasterHandlerWebResponse.java:191)
        at com.dfire.core.netty.master.MasterHandler.lambda$channelRead$8(MasterHandler.java:119)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        ... 3 common frames omitted

我看了源码，可是没看太懂

@mua-haha
Copy link
Author

mua-haha commented Nov 8, 2022

过一会儿请求失败，会有这个错误

2022-11-08 14:25:38.659 ERROR com.dfire.logs.ErrorLog.error:14 三个小时未获得master任务队列的获取信息

2022-11-08 14:25:38.660 ERROR com.dfire.logs.ErrorLog.error:22 请求/hera/homePage/getJobQueueInfo异常:
java.lang.NullPointerException: null
        at com.dfire.core.netty.worker.WorkClient.getJobQueueInfoFromWeb(WorkClient.java:345)
        at com.dfire.controller.SystemManageController.getJobQueueInfo(SystemManageController.java:272)
        at com.dfire.controller.SystemManageController$$FastClassBySpringCGLIB$$b8d4e21.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
        at com.dfire.config.HeraAspect.around(HeraAspect.java:72)
        at sun.reflect.GeneratedMethodAccessor143.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
        at com.dfire.controller.SystemManageController$$EnhancerBySpringCGLIB$$c4bf7163.getJobQueueInfo(<generated>)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at com.alibaba.druid.support.http.WebStatFilter.doFilter(WebStatFilter.java:123)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

2022-11-08 14:25:42.403 INFO  com.dfire.logs.HeraLog.info:12 hold lock and update time

@scxwhite
Copy link
Owner

机器机型的问题。代码不兼容。建议换成相同机型的work。

@mua-haha
Copy link
Author

关于机器组监控问题，目前已解决，只是不知道会不会有其他影响
image

原因是ProcessBuilder 返回数据大，2秒钟内stream又没有被读取，所以总是返回false，改代码如下：
image

因为不知道如何增加ProcessBuilder stream的阈值，所以改为只要不报错直接返回0
望指正

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@scxwhite @mua-haha