Reconnect option

[emiltin]

#149 mentions the {reconnect: true} option. This reconnect option seems to be missing from the documentation in the Readme?

I'm trying to use it, but it doesn't seem to reconnect.

I pass {reconnect: true} as part of the options when calling start_link() and emqtt starts normally. I'm connecting to emqx running locally in a docker container. I can now publish and subscribe to messages.

If I stop the emqx broker, then emqtt does not immediately report any error, even after waiting for several minutes. But if try to publish data, i get [error] shutdown: :econnrefused, and my gen_server dies. This is similar to what happens if the broker is unavailable when I start emqtt.

I would expect emqtt to reconnect, instead of dying?

[emiltin]

Hi, any input on this? Thank you!

[qzhuyan]

[error] shutdown: :econnrefused

Do you mean the client process exists exit or it is just error print out?

which version are you using?

latest version need a retry number like {reconnect, 1000} %% give up after 1000 reconnects.

[emiltin]

Thank you for your reply. It seems the process exits. I'm using emqtt 1.11.0, git ref c815a18.

I'm now using your suggestion of setting the reconnect option to 1000. Is the value 'true' still allowed, and does that imply that it will keep trying?

It's an Elixir/Phoenix project. You can see the code at:
https://github.com/rsmp-nordic/rsmp4_client/tree/reconnect

The settings are in:
https://github.com/rsmp-nordic/rsmp4_client/blob/reconnect/config/config.exs#L63

The module that uses emqtt is at:
https://github.com/rsmp-nordic/rsmp4_client/blob/reconnect/lib/rsmp/rsmp_client.ex

[qzhuyan]

Is the value 'true' still allowed, and does that imply that it will keep trying?

I think we change it to infinity or none_neg_integer()

Do you mean after set it to 1000 you still get [error] shutdown: :econnrefused ?

[emiltin]

Do you mean after set it to 1000 you still get [error] shutdown: :econnrefused ?

Yes.

[qzhuyan]

I am testing a branch here #238 which just make reconnect option accessible from cli.

_build/emqtt/rel/emqtt/bin/emqtt pub  -t a --payload 100 --repeat 1000 --repeat-delay 1 --reconnect 100
2024-01-10T17:09:04.505+01:00 Generated clientid: emqtt-convincing-jellyfish-5cdd0b061a351f78318e
2024-01-10T17:09:04.537+01:00 Connected:
#{'Maximum-Packet-Size' => 1048576,'Retain-Available' => 1,
  'Shared-Subscription-Available' => 1,
  'Subscription-Identifier-Available' => 1,'Topic-Alias-Maximum' => 65535,
  'Wildcard-Subscription-Available' => 1}
2024-01-10T17:09:04.543+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:05.545+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:06.546+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:07.548+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:08.555+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:09.562+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))


%%% EMQX stops here, after few secs, it restarts  %%%


2024-01-10T17:09:25.477+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:26.499+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:27.500+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:28.501+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:29.502+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:30.505+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:09:31.510+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))

I cannot reproduce the issue you found even I set reconnect to 1 I get different error {reconnect_error,econnrefused}

_build/emqtt/rel/emqtt/bin/emqtt pub  -t a --payload 100 --repeat 1000 --repeat-delay 1 --reconnect 1
2024-01-10T17:13:56.275+01:00 Generated clientid: emqtt-convincing-jellyfish-602a7b3323a381826931
2024-01-10T17:13:56.304+01:00 Connected:
#{'Maximum-Packet-Size' => 1048576,'Retain-Available' => 1,
  'Shared-Subscription-Available' => 1,
  'Subscription-Identifier-Available' => 1,'Topic-Alias-Maximum' => 65535,
  'Wildcard-Subscription-Available' => 1}
2024-01-10T17:13:56.312+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:13:57.314+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:13:58.333+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
2024-01-10T17:13:59.345+01:00 Sent PUBLISH (Q0, R0, D0, Topic=a, Payload=...(3 bytes))
=ERROR REPORT==== 10-Jan-2024::17:14:04.988878 ===
** State machine <0.90.0> terminating
** Last event = {state_timeout,1}
** When server state  = {reconnect,
                            {state,undefined,<0.9.0>,undefined,"localhost",
                                1883,[],emqtt_sock,undefined,[],60000,false,
                                <<"emqtt-convincing-jellyfish-602a7b3323a381826931">>,
                                true,undefined,undefined,5,<<"MQTT">>,300,
                                undefined,false,false,undefined,
                                #{'Maximum-Packet-Size' => 1048576,
                                  'Retain-Available' => 1,
                                  'Shared-Subscription-Available' => 1,
                                  'Subscription-Identifier-Available' => 1,
                                  'Topic-Alias-Maximum' => 65535,
                                  'Wildcard-Subscription-Available' => 1},
                                [],#{},
                                #{seq => 1,sent => #{},
                                  max_inflight => infinity},
                                #{},true,30000,undefined,30000,undefined,0,6,
                                false,
                                {none,
                                    #{version => 5,max_size => 268435455,
                                      strict_mode => false}},
                                1,5000,false,undefined,
                                #{count => 0,requests => {[],[]}},
                                #{}}}
** Reason for termination = exit:{reconnect_error,econnrefused}
** Callback modules = [emqtt]
** Callback mode = state_functions
** Postponed = [{info,
                    {publish,default,
                        {mqtt_msg,0,false,false,undefined,<<"a">>,#{},
                            <<"100">>},
                        infinity,
                        {fun emqtt:sync_publish_result/3,
                         [<0.9.0>,#Ref<0.2930057788.2229796868.76848>]}}}]
** Stacktrace =
**  [{gen_statem,loop_state_callback_result,11,
                 [{file,"gen_statem.erl"},{line,1499}]},
     {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,241}]}]

=CRASH REPORT==== 10-Jan-2024::17:14:05.017177 ===
  crasher:
    initial call: emqtt:init/1
    pid: <0.90.0>
    registered_name: []
    exception exit: {reconnect_error,econnrefused}
      in function  gen_statem:loop_state_callback_result/11 (gen_statem.erl, line 1499)
    ancestors: [<0.9.0>]
    message_queue_len: 0
    messages: []
    links: [<0.9.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 10958
    stack_size: 28
    reductions: 18855
  neighbours:

escript: exception exit: {reconnect_error,econnrefused}
  in function  emqtt:publish_via/3 (/home/ubuntu/repo/emqtt/src/emqtt.erl, line 541)
  in call from emqtt_cli:do_publish/3 (/home/ubuntu/repo/emqtt/src/emqtt_cli.erl, line 214)
  in call from emqtt_cli:publish/3 (/home/ubuntu/repo/emqtt/src/emqtt_cli.erl, line 193)
  in call from emqtt_cli:main/2 (/home/ubuntu/repo/emqtt/src/emqtt_cli.erl, line 178)
  in call from escript:run/2 (escript.erl, line 750)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_em/1 
  in call from init:do_boot/3 

Your debug log will be appreciated.

[emiltin]

I think what happens is that I'm using publish() to send data. If the connection is down, this seems to hang. Any other call to my GenServer then also hangs. Any other call to my GenServer then also times out. Is publish() expected to time out at some point if the connection is down?

I guess I should try to use pubish_async().

[emiltin]

Thank you for the answer, I will close the issue. Reconnect works as expected in my app. Although I still wish it will also work during the initial connection: #104.