WiFi dropout loop; Runaway Rx; driver bug?

Hello,

  1. My goal is:
    Sustain continuous WiFi connection.
  2. My actions are:
    WiFi + AWS MQTT config in mos.yml running on an STM32-B-L475E-IOT01A.
  3. The result I see is:
    Established comms with AWS but disconnects due to rx buffer overflow, see log below.
  4. My expectation & question is:

The drdy_gpio on the ISM43362 just isn’t being pulled low, no matter how large I set the rx buffer (1200 default, tried 1000*1000). I’ve tested two boards, so I don’t think it’s the hardware or WiFi fw (which is up-to-date).

I’m able to establish comms with AWS, so it resets properly for some of the transfers, but then after a while of idling gets stuck and causes the drop out - I can make it drop out faster by downloading a small file.

More odd is that once this error occurs, any subsequent reconnects to the WiFi result in almost immediate Runaway again (delay varies), or timeout at other times.

I’m not familiar with the ism43 - can anyone suggest why the data ready pin doesn’t reset (at some undetermined point)? According to the data sheet the pin shouldn’t remain high indefinitely under any conditions, so I’m wondering if there’s a timing issue in the mongoose driver?

[Nov  7 10:53:40.588] mgos_wifi.c:88          WiFi STA: Using config 0 (Moo)
[Nov  7 10:53:40.595] mgos_wifi.c:468         WiFi mode: STA
[Nov  7 10:53:40.598] mgos_wifi.c:270         WiFi STA: Connecting to Moo
[Nov  7 10:53:40.606] mgos_init.c:36          Init done, RAM: 96256 total, 86336 free, 0 min free
[Nov  7 10:53:40.610] mgos_aws_shadow.c:590   Device shadow name: stm32_A8D99C (token cce2acf8)
[Nov  7 10:53:40.617] mgos_event.c:135        ev MOS0 triggered 1 handlers
[Nov  7 10:53:40.622] mgos_event.c:135        ev WFI1 triggered 0 handlers
[Nov  7 10:53:40.626] mgos_net.c:85           WiFi STA: connecting
[Nov  7 10:53:40.630] mgos_event.c:135        ev NET1 triggered 2 handlers
[Nov  7 10:53:41.101] ism43xxx_core.c:62      Inventek eS-WiFi ISM43362-M3G-L44-SPI fw C3.5.2.5.STM
[Nov  7 10:53:41.108] ism43xxx_core.c:75      MAC: C4:7F:51:8D:16:B3
[Nov  7 10:53:41.121] ism43xxx_wifi_hal.c:246 STA connecting...
[Nov  7 10:53:44.446] mgos_wifi.c:136         WiFi STA: Connected, BSSID 00:00:00:00:00:00 ch 0 RSSI -42
[Nov  7 10:53:44.454] mgos_event.c:135        ev WFI2 triggered 0 handlers
[Nov  7 10:53:44.459] mgos_event.c:135        ev WFI3 triggered 0 handlers
[Nov  7 10:53:44.463] mgos_net.c:89           WiFi STA: connected
[Nov  7 10:53:44.467] mgos_event.c:135        ev NET2 triggered 2 handlers
[Nov  7 10:53:44.472] mgos_net.c:101          WiFi STA: ready, IP 192.168.0.111, GW 192.168.0.1, DNS 192.168.0.1
[Nov  7 10:54:49.076] mg_mqtt.c:188           Send PINGREQ
[Nov  7 10:54:49.584] mgos_mqtt.c:135         MQTT event: 213
[Nov  7 10:55:49.098] mg_mqtt.c:188           Send PINGREQ
[Nov  7 10:55:49.572] mgos_mqtt.c:135         MQTT event: 213
[Nov  7 10:56:49.105] mg_mqtt.c:188           Send PINGREQ
[Nov  7 10:56:49.562] mgos_mqtt.c:135         MQTT event: 213
...
[Nov  7 11:15:10.681] ism43xxx_core.c:220     Runaway Rx
[Nov  7 11:15:10.684] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:15:10.688] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:15:10.691] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:15:10.694] ism43xxx_core.c:374     Keeping the module in reset
[Nov  7 11:15:10.699] mgos_mqtt.c:164         MQTT Disconnect
[Nov  7 11:15:10.706] mgos_event.c:135        ev MOS5 triggered 0 handlers
[Nov  7 11:15:10.707] mgos_event.c:135        ev MOS5 triggered 0 handlers
[Nov  7 11:15:10.714] mgos_mqtt.c:507         MQTT connecting after 2007 ms
[Nov  7 11:15:10.722] mgos_wifi.c:119         WiFi STA: Disconnected, reason: 0
[Nov  7 11:15:10.722] mgos_event.c:135        ev WFI0 triggered 0 handlers
[Nov  7 11:15:10.727] mgos_net.c:81           WiFi STA: disconnected
[Nov  7 11:15:10.733] mgos_event.c:135        ev NET0 triggered 2 handlers
[Nov  7 11:15:40.729] mgos_wifi.c:345         WiFi STA: Connect timeout
[Nov  7 11:15:40.734] mgos_wifi.c:88          WiFi STA: Using config 0 (Moo)
[Nov  7 11:15:40.739] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:15:40.745] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:15:40.746] mgos_wifi.c:270         WiFi STA: Connecting to Moo
[Nov  7 11:15:40.958] mgos_net.c:85           WiFi STA: connecting
[Nov  7 11:15:40.962] mgos_event.c:135        ev NET1 triggered 2 handlers
[Nov  7 11:15:41.256] ism43xxx_wifi_hal.c:246 STA connecting...
[Nov  7 11:15:44.607] mgos_wifi.c:136         WiFi STA: Connected, BSSID 00:00:00:00:00:00 ch 0 RSSI -42
[Nov  7 11:15:44.614] mgos_event.c:135        ev WFI2 triggered 0 handlers
[Nov  7 11:15:44.619] mgos_event.c:135        ev WFI3 triggered 0 handlers
[Nov  7 11:15:44.624] mgos_net.c:89           WiFi STA: connected
[Nov  7 11:15:44.628] mgos_event.c:135        ev NET2 triggered 2 handlers
[Nov  7 11:15:44.632] mgos_net.c:101          WiFi STA: ready, IP 192.168.0.111, GW 192.168.0.1, DNS 192.168.0.1
[Nov  7 11:15:44.953] ism43xxx_core.c:220     Runaway Rx
[Nov  7 11:15:44.957] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:15:44.961] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:15:44.964] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:15:44.967] ism43xxx_core.c:374     Keeping the module in reset
[Nov  7 11:15:44.972] mgos_mqtt.c:164         MQTT Disconnect
[Nov  7 11:15:44.975] mgos_event.c:135        ev MOS5 triggered 0 handlers
[Nov  7 11:15:44.980] mgos_mqtt.c:507         MQTT connecting after 54131 ms
[Nov  7 11:15:44.985] mgos_wifi.c:119         WiFi STA: Disconnected, reason: 0
[Nov  7 11:15:44.990] mgos_event.c:135        ev WFI0 triggered 0 handlers
[Nov  7 11:15:44.995] mgos_net.c:81           WiFi STA: disconnected
[Nov  7 11:15:44.999] mgos_event.c:135        ev NET0 triggered 2 handlers
[Nov  7 11:16:14.998] mgos_wifi.c:345         WiFi STA: Connect timeout
[Nov  7 11:16:15.002] mgos_wifi.c:88          WiFi STA: Using config 0 (Moo)
[Nov  7 11:16:15.007] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:16:15.013] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:16:15.018] mgos_wifi.c:270         WiFi STA: Connecting to Moo
[Nov  7 11:16:15.026] mgos_event.c:135        ev WFI1 triggered 0 handlers
[Nov  7 11:16:15.026] mgos_net.c:85           WiFi STA: connecting
[Nov  7 11:16:15.029] mgos_event.c:135        ev NET1 triggered 2 handlers
[Nov  7 11:16:15.525] ism43xxx_wifi_hal.c:246 STA connecting...
[Nov  7 11:16:18.908] mgos_wifi.c:136         WiFi STA: Connected, BSSID 00:00:00:00:00:00 ch 0 RSSI -41
[Nov  7 11:16:18.916] mgos_event.c:135        ev WFI2 triggered 0 handlers
[Nov  7 11:16:18.920] mgos_event.c:135        ev WFI3 triggered 0 handlers
[Nov  7 11:16:18.927] mgos_net.c:89           WiFi STA: connected
[Nov  7 11:16:18.930] mgos_event.c:135        ev NET2 triggered 2 handlers
[Nov  7 11:16:18.934] mgos_net.c:101          WiFi STA: ready, IP 192.168.0.111, GW 192.168.0.1, DNS 192.168.0.1
[Nov  7 11:17:24.518] mg_mqtt.c:188           Send PINGREQ
[Nov  7 11:17:24.982] mgos_mqtt.c:135         MQTT event: 213
[Nov  7 11:18:24.680] mg_mqtt.c:188           Send PINGREQ
[Nov  7 11:18:24.972] mgos_mqtt.c:135         MQTT event: 213
[Nov  7 11:18:29.035] ism43xxx_core.c:220     Runaway Rx
[Nov  7 11:18:29.039] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:18:29.043] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:18:29.045] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:18:29.049] ism43xxx_core.c:374     Keeping the module in reset
[Nov  7 11:18:29.053] mgos_mqtt.c:164         MQTT Disconnect
[Nov  7 11:18:29.066] mgos_event.c:135        ev MOS5 triggered 0 handlers
[Nov  7 11:18:29.067] mgos_event.c:135        ev MOS5 triggered 0 handlers
[Nov  7 11:18:29.067] mgos_mqtt.c:507         MQTT connecting after 1806 ms
[Nov  7 11:18:29.073] mgos_wifi.c:119         WiFi STA: Disconnected, reason: 0
[Nov  7 11:18:29.077] mgos_event.c:135        ev WFI0 triggered 0 handlers
[Nov  7 11:18:29.081] mgos_net.c:81           WiFi STA: disconnected
[Nov  7 11:18:29.085] mgos_event.c:135        ev NET0 triggered 2 handlers
[Nov  7 11:18:59.082] mgos_wifi.c:345         WiFi STA: Connect timeout
[Nov  7 11:18:59.087] mgos_wifi.c:88          WiFi STA: Using config 0 (Moo)
[Nov  7 11:18:59.093] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:18:59.097] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:18:59.101] mgos_wifi.c:270         WiFi STA: Connecting to Moo
[Nov  7 11:18:59.109] mgos_event.c:135        ev WFI1 triggered 0 handlers
[Nov  7 11:18:59.114] mgos_net.c:85           WiFi STA: connecting
[Nov  7 11:18:59.116] mgos_event.c:135        ev NET1 triggered 2 handlers
[Nov  7 11:18:59.613] ism43xxx_core.c:220     Runaway Rx
[Nov  7 11:18:59.615] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:18:59.619] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:18:59.623] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:18:59.626] ism43xxx_core.c:374     Keeping the module in reset
[Nov  7 11:18:59.630] mgos_wifi.c:119         WiFi STA: Disconnected, reason: 0
[Nov  7 11:18:59.636] mgos_event.c:135        ev WFI0 triggered 0 handlers
[Nov  7 11:18:59.641] mgos_net.c:81           WiFi STA: disconnected
[Nov  7 11:18:59.645] mgos_event.c:135        ev NET0 triggered 2 handlers
[Nov  7 11:19:29.107] mgos_wifi.c:345         WiFi STA: Connect timeout
[Nov  7 11:19:29.112] mgos_wifi.c:88          WiFi STA: Using config 0 (Moo)
[Nov  7 11:19:29.117] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:19:29.121] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:19:29.124] mgos_wifi.c:270         WiFi STA: Connecting to Moo
[Nov  7 11:19:29.129] mgos_event.c:135        ev WFI1 triggered 0 handlers
[Nov  7 11:19:29.135] mgos_net.c:85           WiFi STA: connecting
[Nov  7 11:19:29.138] mgos_event.c:135        ev NET1 triggered 2 handlers
[Nov  7 11:19:29.637] ism43xxx_core.c:220     Runaway Rx
[Nov  7 11:19:29.640] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:19:29.645] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:19:29.646] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:19:29.649] ism43xxx_core.c:374     Keeping the module in reset
[Nov  7 11:19:29.656] mgos_wifi.c:119         WiFi STA: Disconnected, reason: 0
[Nov  7 11:19:29.660] mgos_event.c:135        ev WFI0 triggered 0 handlers
[Nov  7 11:19:29.666] mgos_net.c:81           WiFi STA: disconnected
[Nov  7 11:19:29.672] mgos_event.c:135        ev NET0 triggered 2 handlers
[Nov  7 11:19:59.132] mgos_wifi.c:345         WiFi STA: Connect timeout
[Nov  7 11:19:59.135] mgos_wifi.c:88          WiFi STA: Using config 0 (Moo)
[Nov  7 11:19:59.142] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:19:59.146] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:19:59.149] mgos_wifi.c:270         WiFi STA: Connecting to Moo
[Nov  7 11:19:59.154] mgos_event.c:135        ev WFI1 triggered 0 handlers
[Nov  7 11:19:59.162] mgos_net.c:85           WiFi STA: connecting
[Nov  7 11:19:59.164] mgos_event.c:135        ev NET1 triggered 2 handlers
[Nov  7 11:19:59.661] ism43xxx_core.c:220     Runaway Rx
[Nov  7 11:19:59.664] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:19:59.668] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:19:59.672] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:19:59.676] ism43xxx_core.c:374     Keeping the module in reset
[Nov  7 11:19:59.679] mgos_wifi.c:119         WiFi STA: Disconnected, reason: 0
[Nov  7 11:19:59.684] mgos_event.c:135        ev WFI0 triggered 0 handlers
[Nov  7 11:19:59.689] mgos_net.c:81           WiFi STA: disconnected
[Nov  7 11:19:59.693] mgos_event.c:135        ev NET0 triggered 2 handlers
[Nov  7 11:20:29.157] mgos_wifi.c:345         WiFi STA: Connect timeout
[Nov  7 11:20:29.161] mgos_wifi.c:88          WiFi STA: Using config 0 (Moo)
[Nov  7 11:20:29.167] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:20:29.173] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:20:29.174] mgos_wifi.c:270         WiFi STA: Connecting to Moo
[Nov  7 11:20:29.184] mgos_event.c:135        ev WFI1 triggered 0 handlers
[Nov  7 11:20:29.188] mgos_net.c:85           WiFi STA: connecting
[Nov  7 11:20:29.189] mgos_event.c:135        ev NET1 triggered 2 handlers
[Nov  7 11:20:29.688] ism43xxx_core.c:220     Runaway Rx
[Nov  7 11:20:29.690] ism43xxx_core.c:334     Resetting via pin...
[Nov  7 11:20:29.695] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:20:29.697] ism43xxx_net_if.c:507   disconnect
[Nov  7 11:20:29.701] ism43xxx_core.c:374     Keeping the module in reset
[Nov  7 11:20:29.706] mgos_wifi.c:119         WiFi STA: Disconnected, reason: 0
[Nov  7 11:20:29.711] mgos_event.c:135        ev WFI0 triggered 0 handlers
[Nov  7 11:20:29.715] mgos_net.c:81           WiFi STA: disconnected
[Nov  7 11:20:29.720] mgos_event.c:135        ev NET0 triggered 2 handlers