OTA update via mdash give "data required"

General
#1

I am trying to do a ota update. I am using the 3 free devices in the mdash.
When I try to upload my fw.zip file for the ota the dash board shows “Data Required” message
In the device log I see

mg_rpc.c:293 OTA.Begin via WSS_out 148.251.54.236:443
mgos_ota_core.c:252 Starting, timeout 600, commit timeout 600, mem 25112
mg_rpc.c:293 OTA.Write via WSS_out 148.251.54.236:443
mgos_ota_core.c:488 FW: app1 esp8266 1.2 20190519-042139/2.10.0-g237ecfd-master-dirty
esp_ota_backend.c:157 Slot 1, FW: app1.bin -> 0x200000, FS fs.bin -> 0x48000
esp_ota_backend.c:248 Start writing app1.bin (824448) @ 0x200000
mgos_ota_core.c:505 0.16% total, app1.bin 512 of 824448
mgos_ota_core.c:932 Update state: 300
online: true {“ram_free”:16488,“uptime”:705.422568,“btnCount”:0,“on”:false}
mg_rpc.c:293 OTA.Write via WSS_out 148.251.54.236:443
E:M 7288 (912 blocks)

Nothing happens after this. I need to either call OTA.End or wait for it to timeout.

#2

This line means you are out of memory.

1 Like
#3

@Suyash_Mathema
mDash takes your firmware zip, splits into 4k chunks and pushes one by one to your device. Thus your device has to have some free for the update to work - apparently 4k chunks are too big, cause some other buffers are also in play here.

Could you log free RAM periodically to see how much RAM your device has?

#4

I am following this getting started https://mongoose-os.com/docs/mongoose-os/quickstart/setup.md and running the demo js app. I am trying on a wemos D1 mini v3.0

This is what I get for Sys.GetInfo
{
“app”: “app1”,
“fw_version”: “1.2”,
“fw_id”: “20190519-042139/2.10.0-g237ecfd-master-dirty”,
“mg_version”: “2.13.0”,
“mg_id”: “20190519-042139/2.13.0-gd165a78”,
“mac”: “860D8EA825AC”,
“arch”: “esp8266”,
“uptime”: 112,
“ram_size”: 51592,
“ram_free”: 27104,
“ram_min_free”: 10544,
“fs_size”: 233681,
“fs_free”: 99898,
“wifi”: {
“sta_ip”: “192.168.86.28”,
“ap_ip”: “”,
“status”: “got ip”,
“ssid”: “smps”
}
}

This is the log when I start the OTA
online: true {“ram_free”:27820,“uptime”:199.251749,“btnCount”:0,“on”:false}
online: true {“ram_free”:27820,“uptime”:200.251910,“btnCount”:0,“on”:false}
online: true {“ram_free”:27820,“uptime”:201.251804,“btnCount”:0,“on”:false}
online: true {“ram_free”:27820,“uptime”:202.251778,“btnCount”:0,“on”:false}
online: true {“ram_free”:27820,“uptime”:203.251823,“btnCount”:0,“on”:false}
online: true {“ram_free”:27820,“uptime”:204.251791,“btnCount”:0,“on”:false}
mg_rpc.c:293 OTA.Begin via WSS_out 148.251.54.236:443
mgos_ota_core.c:253 Starting, timeout 600, commit timeout 600, mem 24960
mg_rpc.c:293 OTA.Write via WSS_out 148.251.54.236:443
mgos_ota_core.c:489 FW: app1 esp8266 1.2 20190519-042139/2.10.0-g237ecfd-master-dirty
esp_ota_backend.c:157 Slot 1, FW: app1.bin -> 0x200000, FS fs.bin -> 0x48000
esp_ota_backend.c:248 Start writing app1.bin (824448) @ 0x200000
mgos_ota_core.c:506 0.16% total, app1.bin 512 of 824448
online: true {“ram_free”:16340,“uptime”:205.483429,“btnCount”:0,“on”:false}
mg_rpc.c:293 OTA.Write via WSS_out 148.251.54.236:443
E:M 7288 (912 blocks)
mgos_mongoose.c:66 New heap free LWM: 7960
online: true {“ram_free”:16656,“uptime”:206.251891,“btnCount”:0,“on”:false}
online: true {“ram_free”:18260,“uptime”:207.251887,“btnCount”:0,“on”:false}
online: true {“ram_free”:18260,“uptime”:208.251775,“btnCount”:0,“on”:false}
online: true {“ram_free”:18260,“uptime”:209.251765,“btnCount”:0,“on”:false}
online: true {“ram_free”:18260,“uptime”:210.251797,“btnCount”:0,“on”:false}
online: true {“ram_free”:18260,“uptime”:211.251814,“btnCount”:0,“on”:false}

This is the log when I call OTA.End
online: true {“ram_free”:18260,“uptime”:293.251840,“btnCount”:0,“on”:false}
online: true {“ram_free”:18260,“uptime”:294.251812,“btnCount”:0,“on”:false}
online: true {“ram_free”:18260,“uptime”:295.251866,“btnCount”:0,“on”:false}
online: true {“ram_free”:18260,“uptime”:296.251783,“btnCount”:0,“on”:false}
mg_rpc.c:293 RPC.Describe via WSS_out 148.251.54.236:443
online: true {“ram_free”:18260,“uptime”:297.251773,“btnCount”:0,“on”:false}
online: true {“ram_free”:18260,“uptime”:298.251845,“btnCount”:0,“on”:false}
mg_rpc.c:293 OTA.End via WSS_out 148.251.54.236:443
mgos_ota_core.c:811 Update finished, result -13 (Unexpected end of data)
online: true {“ram_free”:27548,“uptime”:299.250587,“btnCount”:0,“on”:false}
online: true {“ram_free”:27820,“uptime”:300.251885,“btnCount”:0,“on”:false}
online: true {“ram_free”:27812,“uptime”:301.251780,“btnCount”:0,“on”:false}
online: true {“ram_free”:27812,“uptime”:302.251709,“btnCount”:0,“on”:false}
online: true {“ram_free”:27812,“uptime”:303.251818,“btnCount”:0,“on”:false}

Does the file system size also matter for the ota or is it just ram?

#5

Thank you for the detailed report.

I believe the solution is to reduce the chunk size for ESP8266, go from 4k to 1k or 512 bytes. It’ll make the OTA slower but it would not crash with out-of-memory. Let us do the change - I will report later in this thread when done.

I am surprised to see that D1 mini can OTA at all. As far as I remember, D1 mini has 1M flash, thus does not have enough space for the second partition. Please correct me if I am wrong.

#6

@cpq D1 mini has 4MB. I also have a few of them.

#7

It does say 4MB but I have read somewhere few of the Chinese clones might have less memory. How can I verify if the memory size.

#8

mos flash detects the size of the flash and during the boot process it is also output in mos console

[May 20 12:53:46.290] rBoot v1.2.1-cesanta1 - richardaburton@gmail.com
[May 20 12:53:46.292] Flash Size:   32 Mbit
[May 20 12:53:46.295] Flash Mode:   DIO
[May 20 12:53:46.295] Flash Speed:  80 MHz
[May 20 12:53:46.298] rBoot Option: Big flash
...
[May 20 12:53:46.408] esp_main.c:174          Mongoose OS f6cbcf9 (20190520-091742/gf6cbcf9-master)
[May 20 12:53:46.417] esp_main.c:178          CPU: ESP8266EX, 160 MHz, RAM: 51920 total, 47280 free
[May 20 12:53:46.422] esp_main.c:180          SDK 2.2.1(1247cc5); flash: 4M
1 Like
#9

It does say 4M flash size.

esp_main.c:174 Mongoose OS 2.13.0 (20190520-041913/2.13.0-gd165a78)
esp_main.c:178 CPU: ESP8266EX, 160 MHz, RAM: 51728 total, 49428 free
esp_main.c:180 SDK 2.2.1(1247cc5); flash: 4M
esp_exc.c:216 Reset cause: 6 (sys reset)
mg_lwip_ev_mgr.c:93 Mongoose 6.14, LwIP 1.4.1
mg_ssl_if_mbedtls.c:57 mbed TLS 2.16.0-cesanta3
mgos_vfs_dev.c:73 sfl0: sysflash (), size 4194304
mgos_vfs_dev.c:73 root: part ({“dev”: “sfl0”, “offset”: 32768, “size”: 262144}), size 262144
mgos_vfs.c:147 /: SPIFFS @ root, opts
mgos_vfs.c:319 /: size 233681, used: 169174, free: 64507
mgos_sys_config.c:231 Loading conf0.json

This is what I get while flashing

Flash size: 4194304, params: 0x024f (dio,32m,80m)

1 Like
#10

Awesome, thank you gents, that’s clear.

#11

A change was pushed to use 4k chunks for ESP32, and 1k chunks for other platforms.

@Suyash_Mathema please retry the OTA.

#12

@cpq @nliviu thank you. :grin::grin:

#13

@Suyash_Mathema please follow up - did it work for you or not,
thanks!

#14

Thanks a lot for posting this thread. I’ve had exactly same issue on Wemos D1 mini. I’ll try again and share the results.

#15

OTA seamless update confirmed two Wemos D1 mini since the update. I’m not sure who made the update, but it solves a clear problem, well done.

1 Like
#16

@cpq yes it worked for me. I should have been clear with my previous reply :sweat_smile:

#17

Just to note, the “data requred” error in reality means “failed to parse incoming JSON-RPC frame with data attribute, because of out-of-memory”. So really, this error means “I am out of RAM”.

1 Like
#18

can ota be done using aws s3 bucket instead of mdash?