Save remote string from httpGet

General
#1

If you are asking a question, please follow this template:

  1. My goal is: To be able to download a string from an URL and save it persistent to the device in MJS. If the device reboots (electric outage), and the device has no internet connection, the firmware should be able to read the string from the last saved.

  2. My actions are: I have tested FS.Put and FS.Get but I cannot find any function to decode base64 from the get file

  3. The result I see is: Base64 encoded string which I cannot parse for what I need

  4. My expectation & question is: Be be able to save a httpget string directly to a file or to be able to decode the string from FS.Get in RPC Local

Thanks
FBP

#2

Have a read of Mongoose configurations

Sounds like all you need is to call the JS equivalent to mgos_sys_config_set_yourvarname("string") to save the string, then call then call mgos_sys_config_get_yourvarname() to retrieve your string.

Just keen in mind that this will be stored in conf9.json and so will not survive factory reset (if you are using this feature)

In my experience the only reason to use the FS is when the strings become really large

#3

It has become really large (for Mongoose I asume)

The string (if saved as a file) weights 20Kb, that won’t fit into the config json file (have already tried).

I have done a call to a mDash RPC, which requests a get from an URL, then the body (which is already B64), call RPC.Local FS.Put, but I have memory errors, although the whole B64 string is less than 20Kb

[Feb  2 22:59:12.489] mg_rpc.c:293            LM.RequestAuths via WSS_out 148.251.54.236:443
[Feb  2 22:59:12.498] REMOTE AUTHS REQUEST 
[Feb  2 22:59:12.498] requesting auths by RPC... 
[Feb  2 22:59:12.517] http://api.llavemovil.com/services/ws/node.mcu.get.auths.aspx?uid=********************
[Feb  2 22:59:13.330] mgos_mongoose.c:66      New heap free LWM: 41888
[Feb  2 22:59:13.488] mgos_mongoose.c:66      New heap free LWM: 37872
[Feb  2 22:59:13.643] success auth... 
[Feb  2 22:59:13.788] E:M 20368
[Feb  2 22:59:13.788] Heap summary for capabilities 0x00001800:
[Feb  2 22:59:13.788]   At 0x3ffb2730 len 15448 free 0 allocated 15300 min_free 0
[Feb  2 22:59:13.788]     largest_free_block 0 alloc_blocks 29 free_blocks 0 total_blocks 29
[Feb  2 22:59:13.788]   At 0x3ffaff10 len 240 free 0 allocated 172 min_free 0
[Feb  2 22:59:13.788]     largest_free_block 0 alloc_blocks 9 free_blocks 0 total_blocks 9
[Feb  2 22:59:13.788]   At 0x3ffb6388 len 7288 free 0 allocated 7160 min_free 0
[Feb  2 22:59:13.788]     largest_free_block 0 alloc_blocks 24 free_blocks 0 total_blocks 24
[Feb  2 22:59:13.788]   At 0x3ffb9a20 len 16648 free 0 allocated 16368 min_free 0
[Feb  2 22:59:13.788]     largest_free_block 0 alloc_blocks 62 free_blocks 0 total_blocks 62
[Feb  2 22:59:13.788]   At 0x3ffcfed0 len 65840 free 24 allocated 64708 min_free 0
[Feb  2 22:59:13.788]     largest_free_block 12 alloc_blocks 266 free_blocks 3 total_blocks 269
[Feb  2 22:59:13.788]   At 0x3ffe0440 len 129984 free 38656 allocated 89800 min_free 37160
[Feb  2 22:59:13.788]     largest_free_block 18708 alloc_blocks 346 free_blocks 28 total_blocks 374
[Feb  2 22:59:13.788]   Totals:
[Feb  2 22:59:13.788]     free 38680 allocated 193508 min_free 37160 largest_free_block 18708
[Feb  2 22:59:13.788] Guru Meditation Error: Core  0 panic'ed (StoreProhibited). Exception was unhandled.
[Feb  2 22:59:13.788] Core 0 register dump:
[Feb  2 22:59:13.788] PC      : 0x40101d85  PS      : 0x00060530  A0      : 0x801021fd  A1      : 0x3ffca2d0  
[Feb  2 22:59:13.788] A2      : 0x3ffe08e8  A3      : 0x00000000  A4      : 0x3ffea010  A5      : 0x00000000  
[Feb  2 22:59:13.788] A6      : 0x00000000  A7      : 0x00004f90  A8      : 0x00004f90  A9      : 0x3ffca320  
[Feb  2 22:59:13.788] A10     : 0x00000000  A11     : 0x00000000  A12     : 0x3f407a47  A13     : 0x00000001  
[Feb  2 22:59:13.788] A14     : 0x00000000  A15     : 0x0000000c  SAR     : 0x00000004  EXCCAUSE: 0x0000001d  
[Feb  2 22:59:13.788] EXCVADDR: 0x00000000  LBEG    : 0x400834f4  LEND    : 0x400834fe  LCOUNT  : 0x00000000  
[Feb  2 22:59:13.919] 
[Feb  2 22:59:13.919] ELF file SHA256: 4dd4afc10e163d39f9d4acdbc4c29b3391bb644f7e0e1f9cf39bfb0f8d5db257
[Feb  2 22:59:13.919] 
[Feb  2 22:59:13.919] Backtrace: 0x40101d85 0x401021fa 0x4010221d 0x40102290 0x40103d8a 0x40104b91 0x40104d99 0x40104f07 0x401e1a8e 0x401ca6d6 0x401caf97 0x401cb6e6 0x401cb769 0x401ca6d6 0x401cba0e 0x401cec2d 0x401cec5f 0x401e1cd1 0x401c6b5d 0x40083826 0x400839ed
[Feb  2 22:59:13.919] 
[Feb  2 22:59:13.919] 
[Feb  2 22:59:13.919] --- BEGIN CORE DUMP ---
[Feb  2 22:59:13.919] mos: catching core dump
#4

Your firmware is using a lot of RAM. At the moment of the OOM error, there are ~188kB allocated.

You don’t need to use RPC calls to write files on the local filesystem. The standard fopen, fwrite, fclose can be used. If the original string is base64 encoded, you will need additional memory to decode it (~8/6*strlen(b64_string) IIRC).
cs_base64_decode

#5

We are using MJS, we do not have classic fopen/fclose for file handling.

#6

Yes, you have them in api_file.js.

mJS might be the reason for the large memory used.

#7

Might be, because it is not a heavy use app`. It just opens a relay when an RPC method from mDash is received. But now I need some validations from a string. I think the best option is when I request the initial string, to divide it into chunks, and make a json file that list all this files, so later I can check it on every chunk.

From MJS how can we decode the B64 file (where I put the string)?. I will try to test the chunks now so I can reduce the RAM load.