Hi all! So far, this forum helped me a lot with the project I am currently working on for which I am really grateful! Right now, I am trying to implement an OTA mechanism for ESP8266 devices. I use Google Cloud IoT Core for device registry and management (not mDash), Google Cloud functions for backend stuff with data sent by devices, and a Firebase real-time database for storing data acquired via the devices and sent through MQTT and telemetry.
-
My goal is:
Build an OTA mechanism, but without mDash. Unfortunately, we did not start using mDash in the beginning of the project, and now I have a hard time figuring out how to do OTA. -
My actions are:
What I tried so far is to use the Google Cloud IoT Core Python API in which the manager.py script (https://github.com/googleapis/python-iot/tree/master/samples/api-client/manager) helps retrieve device IDs and to send a built-in, MQTT message, called /devices/device_ID/config (https://cloud.google.com/iot/docs/concepts/devices). In the config MQTT message, I include the OTA URL to the fw.zip (the result of mos build). I detect the message in the firmware with a handler for the config topic and retrieve the OTA URL, but the whole process is stuck as shown below.
The OTA code has 2 parts - ota.cpp and ota.h.
The ota.h is as below:
#pragma once
#include <string>
#include <mgos_ota_http_client.h>
namespace my_namespace {
class ota {
public:
ota();
void update(const std::string& url);
private:
struct mgos_ota_opts options;
};
}
The ota.cpp is as below:
#include "ota.h"
#include <mgos.h>
#include <mgos_ota_http_client.h>
#include <string>
namespace my_namespace {
/**
* This is a test
*
* For the OTA constructor
*/
ota::ota() {
options.timeout = 0;
options.commit_timeout = 0;
options.ignore_same_version = true;
}
void ota::update(const std::string& url) {
//https:/github.com/mongoose-os-libs/ota-http-client
mgos_ota_http_start(url.c_str(), &options);
}
}
In the main.cpp, this is the handler for the Google Cloud IoT Core config MQTT message:
my_namespace::mqtt::set_config_handler([](std::string topic, std::string msg) {
(void) topic;
LOG(LL_INFO, ("MQTT config topic: %s", topic.c_str()));
LOG(LL_INFO, ("MQTT config message: %s", msg.c_str()));
bool if_ota;
char *ota_url_c = NULL;
auto scanned_elements = json_scanf(msg.c_str(), strlen(msg.c_str()),
"{if_ota: %B, ota_url: %Q}",
&if_ota, &ota_url_c
);
if (scanned_elements) {
std::string ota_url{ota_url_c};
free(ota_url_c);
LOG(LL_INFO, ("if_ota true: %d", if_ota));
LOG(LL_INFO, ("ota_url: %s", ota_url.c_str()));
if (if_ota == true) {
LOG(LL_INFO, ("if_ota true, url: %s", ota_url.c_str()));
ota->update(ota_url);
}
}
The if_ota variable is a Boolean and the ota_url variable is a string. They are both successfully retrieved from the JSON format.
- The result I see is:
Version 1: fw.zip is on Google Drive
Using port /dev/ttyUSB0
[Jan 4 15:01:03.605] scandone
[Jan 4 15:01:06.758] state: 0 -> 2 (b0)
[Jan 4 15:01:06.758] state: 2 -> 3 (0)
[Jan 4 15:01:06.758] state: 3 -> 5 (10)
[Jan 4 15:01:06.774] add 0
[Jan 4 15:01:06.774] aid 1
[Jan 4 15:01:06.774] cnt
[Jan 4 15:01:07.014]
[Jan 4 15:01:07.014] connected with WIFI NAME, channel 11
[Jan 4 15:01:07.014] dhcp client start…
[Jan 4 15:01:07.014] mgos_wifi.c:136 WiFi STA: Connected, BSSID 28:b3:71:4f:46:c8 ch 11 RSSI -67
[Jan 4 15:01:07.014] mgos_mongoose.c:66 New heap free LWM: 37416
[Jan 4 15:01:07.014] mgos_net.c:89 WiFi STA: connected
[Jan 4 15:01:07.014] sta.cpp:43 Net connected
[Jan 4 15:01:09.879] ip:IP ADDRESS,mask:255.255.254.0,gw:172.16.30.1
[Jan 4 15:01:09.912] mgos_net.c:101 WiFi STA: ready, IP IP ADDRESS, GW 172.16.30.1, DNS 1.1.1.1
[Jan 4 15:01:09.913] mgos_mqtt.c:507 MQTT connecting after 1012 ms
[Jan 4 15:01:09.913] sta.cpp:46 Net got IP address
[Jan 4 15:01:09.913] mgos_mqtt.c:507 MQTT connecting after 941 ms
[Jan 4 15:01:10.852] mgos_mqtt.c:507 MQTT connecting after 944 ms
[Jan 4 15:01:10.952] mgos_sntp.c:95 SNTP query to time.google.com
[Jan 4 15:01:10.952] mgos_mongoose.c:66 New heap free LWM: 36568
[Jan 4 15:01:10.952] mgos_mongoose.c:66 New heap free LWM: 34464
[Jan 4 15:01:10.989] mgos_sntp.c:59 SNTP reply from 216.239.35.4: time 1609768871.309118, local 8.347117, delta 1609768862.962001
[Jan 4 15:01:11.815] mgos_mqtt.c:431 MQTT connecting to mqtt.googleapis.com:8883
[Jan 4 15:01:11.815] mgos_mongoose.c:66 New heap free LWM: 34264
[Jan 4 15:01:11.831] mgos_mongoose.c:66 New heap free LWM: 32216
[Jan 4 15:01:11.879] mgos_mongoose.c:66 New heap free LWM: 31968
[Jan 4 15:01:16.767] pm open,type:0 0
[Jan 4 15:01:18.103] mgos_mqtt.c:141 MQTT TCP connect error (-8)
[Jan 4 15:01:18.103] mgos_mqtt.c:164 MQTT Disconnect
[Jan 4 15:01:18.103] mgos_mqtt.c:507 MQTT connecting after 949 ms
[Jan 4 15:01:19.042] mgos_mqtt.c:431 MQTT connecting to mqtt.googleapis.com:8883
[Jan 4 15:01:19.069] mgos_mongoose.c:66 New heap free LWM: 31712
[Jan 4 15:01:19.085] mgos_mongoose.c:66 New heap free LWM: 31464
[Jan 4 15:01:19.197] mg_ssl_if_mbedtls.c:35 0x3fff056c ciphersuite: TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256
[Jan 4 15:01:19.197] mgos_mongoose.c:66 New heap free LWM: 30768
[Jan 4 15:01:19.849] SW ECDH curve 3
[Jan 4 15:01:23.894] mgos_mongoose.c:66 New heap free LWM: 18352
[Jan 4 15:01:24.181] mgos_mqtt.c:141 MQTT TCP connect ok (0)
[Jan 4 15:01:26.266] mgos_mqtt.c:182 MQTT CONNACK 0
[Jan 4 15:01:26.306] mgos_mqtt.c:125 Subscribing to ‘/devices/esp8266_5A44A7/config’ (QoS 1)
[Jan 4 15:01:26.306] mgos_mqtt.c:125 Subscribing to ‘/devices/esp8266_5A44A7/commands/#’ (QoS 1)
[Jan 4 15:01:26.306] mgos_mqtt.c:125 Subscribing to ‘/devices/esp8266_5A44A7/commands/#’ (QoS 1)
[Jan 4 15:01:26.400] main.cpp:51 BEFORE OTA BUILD
[Jan 4 15:01:26.400] main.cpp:53 MQTT config topic: /devices/device_ID/config
[Jan 4 15:01:26.400] main.cpp:54 MQTT config message: {“if_ota”: true, “ota_url”: “URL to FW ZIP”}
[Jan 4 15:01:26.400] main.cpp:68 if_ota true: 1
[Jan 4 15:01:26.400] main.cpp:69 ota_url: URL to FW ZIP
[Jan 4 15:01:26.427] mgos_ota_http_clien:270 Update URL: URL to FW ZIP
[Jan 4 15:01:26.458] mg_ssl_if_mbedtls.c:35 0x3fff44f4 ciphersuite: TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256
[Jan 4 15:01:27.053] SW ECDSA verify curve 3 hash_len 32 sig_len 72
[Jan 4 15:01:32.291] SW ECDH curve 3
[Jan 4 15:01:37.324] mgos_mongoose.c:66 New heap free LWM: 9816
Version 2: fw.zip is on GitHub (private repository)
… (same until here) …
if_ota true, url: https://github.com/to/repository/blob/main/fw.zip
[Jan 5 13:48:28.613] mgos_ota_http_clien:270 Update URL: https://github.com/to/repository/blob/main/fw.zip
[Jan 5 13:48:28.705] mg_ssl_if_mbedtls.c:35 0x3fff35dc ciphersuite: TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256
[Jan 5 13:48:29.373] SW ECDH curve 3
[Jan 5 13:48:33.656] mgos_mongoose.c:66 New heap free LWM: 14024
[Jan 5 13:48:34.134] mgos_ota_http_clien:102 Invalid HTTP response code
Version 3: fw.zip is on GitHub (public repository)
… (same until here) …
if_ota true: 1
[Jan 5 13:54:48.206] main.cpp:69 ota_url: https://github.com/to/repository/blob/main/fw.zip
[Jan 5 13:54:48.206] main.cpp:71 if_ota true, url: https://github.com/to/repository/blob/main/fw.zip
[Jan 5 13:54:48.206] mgos_ota_http_clien:270 Update URL: https://github.com/to/repository/blob/main/fw.zip
[Jan 5 13:54:48.301] mg_ssl_if_mbedtls.c:35 0x3fff300c ciphersuite: TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256
[Jan 5 13:54:48.986] SW ECDH curve 3
[Jan 5 13:54:53.317] mgos_mongoose.c:66 New heap free LWM: 13904
[Jan 5 13:54:54.113] mgos_ota_http_clien:102 Invalid content-length
- My expectation & question is:
I expect the device to be able to complete the OTA, but it gets stuck at the points shown above.
My first question is if I could use simple online storage, such as Google Drive or GitHub, for storing the firmware ZIP file, or if I need a web server such as mDash. My second question is if I can do OTA with Google Cloud IoT Core given that it does not have a device shadow mechanism for OTA like AWS as mentioned section in https://mongoose-os.com/docs/mongoose-os/userguide/ota.md.
Furthermore, I expect that I will have issues with registering my device after OTA to Google Cloud. So far I have done it with mos gcp-iot-setup …(arguments)… that generates the keys for the device. After OTA, how could I provide the device with these keys or register it to the cloud programmatically?
I appreciate your time and attention! Thank you for any replies and help!
Have a great day!
/Mark