Need help understanding/fixing memory leak

General
#1

Hi all,

I need some help understanding and fixing the source of a memory leak on my device.
I’ve built a device (ESP32) that collects info from a few sensors and pulls some data from the web via API, and logs that data to the cloud.

I’ve set debug.level=0, and after about an hour I see:

[Aug  2 16:22:26.839] Heap summary for capabilities 0x00001800:
[Aug  2 16:22:26.845]   At 0x3ffc72d8 len 2356 free 0 allocated 2140 min_free 0
[Aug  2 16:22:26.851]     largest_free_block 0 alloc_blocks 46 free_blocks 0 total_blocks 46
[Aug  2 16:22:26.856]   At 0x3ffbdb28 len 52 free 0 allocated 16 min_free 0
[Aug  2 16:22:26.862]     largest_free_block 0 alloc_blocks 1 free_blocks 0 total_blocks 1
[Aug  2 16:22:26.868]   At 0x3ffb8000 len 6688 free 8 allocated 6536 min_free 0
[Aug  2 16:22:26.871]     largest_free_block 8 alloc_blocks 27 free_blocks 1 total_blocks 28
[Aug  2 16:22:26.879]   At 0x3ffb0000 len 25480 free 56 allocated 24856 min_free 0
[Aug  2 16:22:26.885]     largest_free_block 12 alloc_blocks 126 free_blocks 8 total_blocks 134
[Aug  2 16:22:26.891]   At 0x3ffae6e0 len 6192 free 92 allocated 5664 min_free 8
[Aug  2 16:22:26.897]     largest_free_block 32 alloc_blocks 92 free_blocks 9 total_blocks 101
[Aug  2 16:22:26.902]   At 0x3ffaff10 len 240 free 0 allocated 172 min_free 0
[Aug  2 16:22:26.908]     largest_free_block 0 alloc_blocks 9 free_blocks 0 total_blocks 9
[Aug  2 16:22:26.914] E:M 1692
[Aug  2 16:22:26.914] Heap summary for capabilities 0x00001800:
[Aug  2 16:22:26.919]   At 0x3ffc72d8 len 2356 free 0 allocated 2140 min_free 0
[Aug  2 16:22:26.922]     largest_free_block 0 alloc_blocks 46 free_blocks 0 total_blocks 46
[Aug  2 16:22:26.931]   At 0x3ffbdb28 len 52 free 0 allocated 16 min_free 0
[Aug  2 16:22:26.934]     largest_free_block 0 alloc_blocks 1 free_blocks 0 total_blocks 1
[Aug  2 16:22:26.940]   At 0x3ffb8000 len 6688 free 8 allocated 6536 min_free 0
[Aug  2 16:22:26.945]     largest_free_block 8 alloc_blocks 27 free_blocks 1 total_blocks 28
[Aug  2 16:22:26.951]   At 0x3ffb0000 len 25480 free 56 allocated 24856 min_free 0
[Aug  2 16:22:26.957]     largest_free_block 12 alloc_blocks 126 free_blocks 8 total_blocks 134
[Aug  2 16:22:26.965]   At 0x3ffae6e0 len 6192 free 92 allocated 5664 min_free 8
[Aug  2 16:22:26.971]     largest_free_block 32 alloc_blocks 92 free_blocks 9 total_blocks 101
[Aug  2 16:22:26.977]   At 0x3ffaff10 len 240 free 0 allocated 172 min_free 0
[Aug  2 16:22:26.982]     largest_free_block 0 alloc_blocks 9 free_blocks 0 total_blocks 9
[Aug  2 16:22:26.988]   At 0x3ffb6388 len 7288 free 0 allocated 7160 min_free 0
[Aug  2 16:22:26.994]     largest_free_block 0 alloc_blocks 24 free_blocks 0 total_blocks 24
[Aug  2 16:22:27.000]   At 0x3ffb9a20 len 16648 free 48 allocated 16228 min_free 0
[Aug  2 16:22:27.005]     largest_free_block 24 alloc_blocks 80 free_blocks 5 total_blocks 85
[Aug  2 16:22:27.011]   At 0x3ffd0448 len 64440 free 6488 allocated 55536 min_free 460
[Aug  2 16:22:27.017]     largest_free_block 100 alloc_blocks 468 free_blocks 128 total_blocks 596
[Aug  2 16:22:27.023]   At 0x3ffe0440 len 129984 free 17348 allocated 109696 min_free 16208
[Aug  2 16:22:27.031]     largest_free_block 1448 alloc_blocks 536 free_blocks 191 total_blocks 727
[Aug  2 16:22:27.038]   Totals:
[Aug  2 16:22:27.038]     free 24100 allocated 227944 min_free 16676 largest_free_block 1448
[Aug  2 16:22:27.044]   At 0x3ffb6388 len 7288 free 0 allocated 7160 min_free 0
[Aug  2 16:22:27.050]     largest_free_block 0 alloc_blocks 24 free_blocks 0 total_blocks 24
[Aug  2 16:22:27.058]   At 0x3ffb9a20 len 16648 free 48 allocated 16228 min_free 0
[Aug  2 16:22:27.061]     largest_free_block 24 alloc_blocks 80 free_blocks 5 total_blocks 85
[Aug  2 16:22:27.070]   At 0x3ffd0448 len 64440 free 6488 allocated 55536 min_free 460
[Aug  2 16:22:27.076]     largest_free_block 100 alloc_blocks 468 free_blocks 128 total_blocks 596
[Aug  2 16:22:27.082]   At 0x3ffe0440 len 129984 free 17348 allocated 109696 min_free 16208
[Aug  2 16:22:27.087]     largest_free_block 1448 alloc_blocks 536 free_blocks 191 total_blocks 727
[Aug  2 16:22:27.097]   Totals:
[Aug  2 16:22:27.097]     free 24100 allocated 227944 min_free 16676 largest_free_block 1448
[Aug  2 16:22:27.102] mg_ssl_if_mbedtls.c:31  0x3fff0424 alloc 3821 failed
[Aug  2 16:22:27.108] mg_ssl_if_mbedtls.c:31  0x3fff0424 mbedtls_ssl_fetch_input() returned -32512 (-0x7f00)
[Aug  2 16:22:27.117] mg_ssl_if_mbedtls.c:31  0x3fff0424 ssl_get_next_record() returned -32512 (-0x7f00)
[Aug  2 16:22:27.122] E:M 1692

I’ve caught a dump and I’m not sure I understand what to do with it…

Dump contains FreeRTOS task info
Loaded core dump from last snippet in  /core
0x4008c25b in invoke_abort () at /opt/Espressif/esp-idf/components/esp32/panic.c:156
156             *((int *) 0) = 0;
#0  0x4008c25b in invoke_abort () at /opt/Espressif/esp-idf/components/esp32/panic.c:156
#1  0x4008c3e0 in abort () at /opt/Espressif/esp-idf/components/esp32/panic.c:171
---Type <return> to continue, or q <return> to quit---
#2  0x400d143b in task_wdt_isr (arg=<optimized out>) at /opt/Espressif/esp-idf/components/esp32/task_wdt.c:179
#3  0x400816f5 in _xt_lowint1 () at /opt/Espressif/esp-idf/components/freertos/xtensa_vectors.S:1154
#4  0x40083486 in esp32_uart_tx_fifo_len (uart_no=0) at /mongoose-os/platforms/esp32/src/esp32_uart.c:85
#5  0x400e4a85 in mgos_uart_hal_flush_fifo (us=0x3ffbda38) at /mongoose-os/platforms/esp32/src/esp32_uart.c:275
#6  0x400e3bac in mgos_uart_flush (uart_no=<optimized out>) at /mongoose-os/src/mgos_uart.c:172
#7  0x400e1788 in mgos_debug_write (fd=1, data=0x3ffd210c, len=61) at /mongoose-os/src/mgos_debug.c:79
#8  0x400e4074 in debug_write (fd=1, data=0x3ffd210c, size=61) at /mongoose-os/platforms/esp32/src/esp32_debug.c:62
#9  0x400e1308 in esp_vfs_write (r=<optimized out>, fd=4, data=0x3ffd210c, size=61) at /opt/Espressif/esp-idf/components/vfs/vfs.c:432
#10 0x4000bd86 in ?? ()
#11 0x40001180 in ?? ()
#12 0x40059301 in ?? ()
#13 0x4005937d in ?? ()
#14 0x40058bc2 in ?? ()
#15 0x400d63b8 in __sprint_r (ptr=0x3ffbaa54, fp=0x3ffb65f4, uio=0x3ffda714) at ../../../.././newlib/libc/stdio/vfprintf.c:437
#16 0x400dbba1 in _vfprintf_r (data=<optimized out>, fp=<optimized out>, fmt0=<optimized out>, ap=...)
    at ../../../.././newlib/libc/stdio/vfprintf.c:1782
#17 0x400d2640 in printf (fmt=0x3f402565 "  At 0x%08x len %d free %d allocated %d min_free %d\n") at ../../../.././newlib/libc/stdio/printf.c:58
#18 0x400d1e91 in heap_caps_print_heap_info (caps=6144) at /opt/Espressif/esp-idf/components/heap/heap_caps.c:398
#19 0x400819fd in heap_caps_malloc (size=1692, caps=6144) at /opt/Espressif/esp-idf/components/heap/heap_caps.c:123
#20 0x40081a37 in heap_caps_malloc_default (size=1692) at /opt/Espressif/esp-idf/components/heap/heap_caps.c:154
#21 0x40081d94 in _malloc_r (r=0x3ffbaa54, size=1692) at /opt/Espressif/esp-idf/components/newlib/syscalls.c:37
#22 0x4000beb2 in ?? ()
#23 0x4008919c in wifi_malloc (size=1692) at /opt/Espressif/esp-idf/components/esp32/esp_adapter.c:68
#24 0x40173016 in esf_buf_alloc ()
#25 0x4017d700 in wdevProcessRxSucDataAll ()
#26 0x40090250 in ppTask ()

Any help or suggestions would be appreciated.
Thanks

#2

debug.level=0 will not show you much info. Try debug.level=3 and log free_heap and min_free_heap.
It looks like mbedtls cannot allocate memory.

#3

Thanks for the suggestion @nliviu.
I think I’ve tracked down the source of the memory leak to my mg_connect_http() functions.

Can you recommend a good example sources for mg_connect_http()?
I following a mongoose-os example app from here to perform API calls to a static URL, and an example here (actually written by you @nliviu :slight_smile: ) to programmatically set the URL and send data to a cloud logger.

#4

simple_crawler
http_client
restful_client

#5

Thanks @nliviu.

I found the source of my error: I am using the function mg_asprintf() but at some point I removed the cleanup operation:

char payload_buf[32];
char *payload = payload_buf;
mg_asprintf(&payload, sizeof(payload_buf), "{\"value\": %s}", data);

Needs to be this:

char payload_buf[32];
char *payload = payload_buf;
mg_asprintf(&payload, sizeof(payload_buf), "{\"value\": %s}", data);
...
if (payload != payload_buf)
{
  free(payload);
}