MQTT TLS on Google cloud - invalid JWT generated

I'm trying to connect to our thing on Google Cloud IoT Core, but I get an error when trying to connect using  

status = nxd_mqtt_client_secure_connect(&g_mqtt_client0, &broker_address, NXD_MQTT_TLS_PORT,

I figure out the cause is in nx_secure_tls_session_start.c, function _nx_secure_tls_handshake_process that Fails with 56 (0x38) error (NX_NOT_CONNECTED 0x38)

Then looking at the JWT that is generated, I've checked on (a website) and gets me invalid JWT.

Note that I have RootCA certificate, device certificate and device private key (in pem format) and they work just fine with Python (just checking certificates and key are correct, and server address/ TLS is working), in fact the JWT generated in python gets me "genuine" under the same website validator.

I dont' get any errors while loading certificates and key on netx secure, so I'm guessing is the way I pad or calculate the JWT is not correct, I'm using the same base64 and JWT create functions that you use in the MQTT_tls google cloud example project.

Root certificate and device certificate are converted first to .der (using openssl) and then to binary array using the hexy.exe tool.

Private key is converted first to pkcs1.pem, then to .der and finally to binary array.

All the CA certificate, device certificate and private key are loaded into some header files and included in my project.

Some code, after I establish an IP layer through Wifi, following:

//> SNTP gets time, I save timestamp and call get_time() when I need it (to generate JWT) - no errors here

//> DNS to get IP address of Google Cloud endpoint - no errors here

//>Init of mqtt packet pool and client - all good

Then init and store of certificates, I get no errors in the following:

status = nx_secure_tls_session_create(&g_mqtt_client0.nxd_mqtt_tls_session,
(NX_SECURE_TLS_CRYPTO *) &nx_crypto_tls_ciphers_synergys7,

status = nx_secure_tls_session_packet_buffer_set(&g_mqtt_client0.nxd_mqtt_tls_session,


char *temp_str = "unused";

for(UINT i = 0; i < sizeof(temp_str); i++)
username[i] = temp_str[i];

//Password is the JWT generated using realtime timestamp (epoch) and IoT thing private key
//rsa_private_pkcs1, RSA_PRIVATE_LEN
jwt_expiration = JWT_EXPIRATION_SECS;

/* Generate JWT */
if (jwt_create(password, sizeof(password), &jwt_expiration,
rsa_private_pkcs1, RSA_PRIVATE_LEN, (CHAR*) &project_id))
HSP_APP_LOG(HSP_LOG_LEVEL_ERROR,"Unable to create JWT. Aborting.\r\n");

HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"JWT token created successfully %s\n============", password);
//set login
status = nxd_mqtt_client_login_set(&g_mqtt_client0, username, strlen(username),
(char *)password, strlen((char *)password));
HSP_APP_LOG(HSP_LOG_LEVEL_ERROR,"tnxd_mqtt_client_login_set error %d", status);

//---------------------------------- after this, I get back the error described above.

///> In nx_secure_tls_session_start.c
///> function _nx_secure_tls_handshake_process Fails with 56 (0x38) error
///> (NX_NOT_CONNECTED 0x38)

status = nxd_mqtt_client_secure_connect(&g_mqtt_client0, &broker_address, NXD_MQTT_TLS_PORT,

HSP_APP_LOG(HSP_LOG_LEVEL_ERROR,"nxd_mqtt_client_secure_connect error %d", status);


I include my project file, so maybe somebody can address me on how to properly generate the JWT, as the following is not getting me a proper JWT.

Is the certs/private key conversion and handling correct?

Do I need to change padding/ crypto functions in the JWT header/payload/signature generation?

Unfortunately I can not find any more info on this topic, as normally a jwt library is used to generate hash and create the token, but I want to make use of the hardware secure engine.

Any info/hints very welcome. 

I can share certs/keys if you want to check, but basically If I get a valid JWT generated I think we are half way there.

Generated JWT:


(note the == characters generated, I suspect shouldn't be there)

and so: [0000002235][new_thread0_entry.c:0395] - nxd_mqtt_client_secure_connect error 65541

I'm on S5D9, SSP1.7.8 e2Studio. NetX Duo.

Thanks a lot.


  • Hi Jeremy,

    No, I call x509 for both CA_cert and device cert. should I substitute the ca_cert one?

    Or add nx_secure_tls_ecc_initialize() at the beginning/end of the mqtt_tls_setup?

    Here's my mqtt tls setup function:

    static UINT NX_MQTT_tls_setup(NXD_MQTT_CLIENT *p_client, NX_SECURE_TLS_SESSION *p_tls_session,
    NX_SECURE_X509_CERT *p_cert, NX_SECURE_X509_CERT *p_trusted_cert)
    UINT status = NX_SUCCESS;
    UINT index = 0;

    SSP_PARAMETER_NOT_USED (p_client);

    /* Need to allocate space for the certificate coming in from the remote host. */
    memset(&remote_cert_buffer, 0, sizeof(remote_cert_buffer));
    memset(&remote_certificate, 0, sizeof(remote_certificate));

    status = nx_secure_tls_remote_certificate_allocate(p_tls_session, &remote_certificate,
    remote_cert_buffer, sizeof(remote_cert_buffer));
    if (NX_SUCCESS != status)
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unabe to allocate memory for the certificate buffer %d", status);
    return status;

    memset(remote_int_ca_cert_buffer, 0, sizeof(remote_int_ca_cert_buffer));
    memset(remote_intermediate_ca, 0, sizeof(remote_intermediate_ca));
    for (index = 0; index < ARRAY_SIZE(remote_intermediate_ca); index++)
    status = nx_secure_tls_remote_certificate_allocate(p_tls_session, &(remote_intermediate_ca[index]),
    &(remote_int_ca_cert_buffer[index * CERTIFICATE_BUFFER_SIZE]),
    if (NX_SUCCESS != status)
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unable to allocate memory for interemediate CA certificate %d", status);
    return status;

    /* Add a CA Certificate to our trusted store for verifying incoming server certificates. */
    status = nx_secure_x509_certificate_initialize(p_trusted_cert,
    (UCHAR *) CA_cert_der, (USHORT) CA_CERT_DER_LEN,
    NX_NULL, 0, NULL, 0,
    if (NX_SUCCESS != status)
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unable to initialize CA certificate %d", status);
    return status;

    status = nx_secure_tls_trusted_certificate_add(p_tls_session, p_trusted_cert);
    if (NX_SUCCESS != status)
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unable to add CA certificate to trusted store, %d", status);
    return status;

    if (dev_cert != NULL)
    /* Add the local certificate for client authentication. */
    status = nx_secure_x509_certificate_initialize(p_cert, (UCHAR *) dev_cert, (USHORT) DEV_CERT_LEN,
    NX_NULL, 0, rsa_private_pkcs1, RSA_PRIVATE_LEN,
    if (NX_SUCCESS != status)
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unable to initialize device certificate, %d",status);
    return status;
    status = nx_secure_tls_local_certificate_add(p_tls_session, p_cert);
    if (NX_SUCCESS != status)
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unable to add device certificate to trusted store, %d",status);
    return status;

    /* Add a timestamp function for time checking and timestamps in the TLS handshake. */
    nx_secure_tls_session_time_function_set(p_tls_session, get_time);

    return status;

    Thanks a lot

  • Do you call nx_secure_tls_ecc_initialize() anywhere? Try nx_secure_tls_ecc_initialize() adding after you call nx_secure_tls_session_create();

  • I've called secure_tls_ecc_initialize after session_create(), it returns 0 (I guess is NX_SUCCESS), but still I can not connect with the same 0x10005 error.

    I also tried to load the google certificate in RSA format (and changed broker to "" as advised by IoT Core, but still I get the same error from mqtt connect. Also tried to use port 8883 and 443.

  • Have you taken a Wireshark trace of the network activity to see what is going on in the communication between the Synergy board and the Google servers?

  • Yes, I did, is my uBlox NINA Wifi DHCP address. Asus is our wifi router, external activity is captured from the internal wifi network but I think you can extract the big picture.

    Please find it here:

    I did also 2 additional captures with just my laptop (wireshark running) and uBlox connected to my WiFi hotspost.

    One using ECC certificate and the nx_secure_tls_ecc_initialize() (with the proper endpoint for ECC cert) and the RSA cert version.

    Interestingly, in these two last captures, I can't see the DNS request, even though I get it and resolve the IP of google cloud endpoint correctly from Synergy, also SNTP time query is not showing, and again I get the time correctly from the SNTP server.

    I think something is going on when resolving uBlox external IP, gateway IP, but again, I can perform SNTP queries, DNS queries, and HTTP Post successfully with the same stack.

  • I don't see the DNS query of the Synergy device looking up the address of the Google mqtt server:-




            /* Look up IPv4 addresses to record  IPv4 addresses in record_buffer and return the IPv4 address count.  */
            status = nx_dns_ipv4_address_by_name_get(&g_dns0, (uint8_t *)GCP_IO_SERVER, &gcp_io_ip_address[0], sizeof(gcp_io_ip_address), &record_count, 500/*NX_IP_PERIODIC_RATE*/);

    in any of the the wireshark logs. The only DNS queries in any of the logs you posted are in the file "nina-mqtt-tls-3.pcapng :-

    Also, Have you tried using Google's minimal root CA set (<1 KB) for The minimal root CA set includes a primary and backup certificate.

  • Yes, the minimal set root CA is the one originally used, but as Yep said that is an ECC certificate. And yes, the original address 8883 with the minimal CA (ECC) is the one I was using. Also, I tried adding the nx_secure_tls_ecc_initialize() (when using the minimal cert, ECC) and tried also with the RSA cert version changing the broker accordingly ( 8883).

    Same error comes out.

    Also I tried to manually set the google brokers IP (pinging from my laptop just before compiling) and same happens.

    Is there some IP routing going on in the wrong way? Like, is Google trying to send back packets to my uBlox public addres but it can not find me?

    I don't use any DCHP ssp module, I think I don't need to do that.

    when you say "Also, Have you tried using Google's minimal root CA set (<1 KB) for The minimal root CA set includes a primary and backup certificate.", that means I need to load both certificates into netx secure? Because in python I just use the primary and all goes well as expected.

    Thanks for your help, is becoming very complex to use netx duo framework on IoT Core/ TLS.

    At the moment we are stuck and we need to make sure we can proceed the development on Synergy with this platform.

    Is there a working example using Syergy to interact (connect/publish/receive) with MQTT on IoT Core with these certificates (either minimal root certs or the regular version) through a mqtt tls using JWT?

  • I've noticed a strange thing. When synergy goes to ask ARP resolution, gets no answer.

    I think the whole IP chain down to the synergy stack is not correct/ routed correctly.

    Also. the MAC address I get when I query directly uBlox module is different from the one I see in the wireshark capture (ends with a different number).

    How can I setup the whole uBlox <-> PPP <-> IP in order to work?

    Strangely though, even in this weird configuration, I can get the time from SNTP and google IP from DNS, but I don't see it in the tcp/ip capture...

    Please find here my configuration, the IP I get assigned by my router is, and this is the Peer IPv4 on PPP interface I set up, but I'm not sure about the IP interface configuration.

    Intel core(my laptop) gets an answer, uBlox ARP request gets lost..

    Full extract of the log, you can see the full AT command sequence, MAC address of uBlox, Wifi station init and PPP mode init..

    [0000000331][hsp_wble.c:0418] - WBLE > +STARTUP

    [0000000331][hsp_wble.c:0458] - WBLE < ATE0

    [0000000331][hsp_wble.c:0418] - WBLE > ATE0

    [0000000332][hsp_wble.c:0418] - WBLE >

    [0000000332][hsp_wble.c:0418] - WBLE > OK

    [0000000342][hsp_wble.c:0458] - WBLE < AT+UWAPMACADDR

    [0000000342][hsp_wble.c:0418] - WBLE >

    [0000000342][hsp_wble.c:0418] - WBLE > +UWAPMACADDR:6009C3E05737

    [0000000342][hsp_wble.c:0418] - WBLE > OK

    [0000000352][hsp_wble.c:0458] - WBLE < AT+UWSC=0,0,1

    [0000000352][hsp_wble.c:0418] - WBLE >

    [0000000352][hsp_wble.c:0418] - WBLE > OK

    [0000000362][hsp_wble.c:0458] - WBLE < AT+UWSC=0,2,"----SSID----"

    [0000000362][hsp_wble.c:0418] - WBLE >

    [0000000362][hsp_wble.c:0418] - WBLE > OK

    [0000000372][hsp_wble.c:0458] - WBLE < AT+UWSC=0,5,2

    [0000000372][hsp_wble.c:0418] - WBLE >

    [0000000372][hsp_wble.c:0418] - WBLE > OK

    [0000000382][hsp_wble.c:0458] - WBLE < AT+UWSC=0,8,"----password----"

    [0000000382][hsp_wble.c:0418] - WBLE >

    [0000000382][hsp_wble.c:0418] - WBLE > OK

    [0000000392][hsp_wble.c:0458] - WBLE < AT+UWSC=0,100,2

    [0000000392][hsp_wble.c:0418] - WBLE >

    [0000000392][hsp_wble.c:0418] - WBLE > OK

    [0000000402][hsp_wble.c:0458] - WBLE < AT+UWSC=0,107,1

    [0000000402][hsp_wble.c:0418] - WBLE >

    [0000000402][hsp_wble.c:0418] - WBLE > OK

    [0000000412][hsp_wble.c:0458] - WBLE < AT+UWSC=0,301,1

    [0000000412][hsp_wble.c:0418] - WBLE >

    [0000000412][hsp_wble.c:0418] - WBLE > OK

    [0000000422][hsp_wble.c:0660] - Wifi init SUCCESS
    [0000000422][hsp_wble.c:0500] - WBLE < AT+UWSCA=0,3

    [0000000423][hsp_wble.c:0418] - WBLE >

    [0000000423][hsp_wble.c:0418] - WBLE > OK

    [0000000652][hsp_wble.c:0418] - WBLE >

    [0000000652][hsp_wble.c:0418] - WBLE > +UUWLE:0,FAAC838145DA,1

    [0000000652][hsp_wble.c:0679] - Wifi attach SUCCESS
    [0000000652][hsp_wble.c:0545] - WBLE < AT+UPPPC=101,

    [0000000653][hsp_wble.c:0418] - WBLE >

    [0000000653][hsp_wble.c:0418] - WBLE > OK

    [0000000663][hsp_wble.c:0545] - WBLE < AT+UPPPC=102,

    [0000000663][hsp_wble.c:0418] - WBLE >

    [0000000663][hsp_wble.c:0418] - WBLE > OK

    [0000000673][hsp_wble.c:0545] - WBLE < AT+UPPPC=104,

    [0000000673][hsp_wble.c:0418] - WBLE >

    [0000000673][hsp_wble.c:0418] - WBLE > OK

    [0000000683][hsp_wble.c:0545] - WBLE < AT+UPPPC=105,

    [0000000683][hsp_wble.c:0418] - WBLE >

    [0000000683][hsp_wble.c:0418] - WBLE > OK

    [0000000693][hsp_wble.c:0545] - WBLE < AT+UPPPC=107,1

    [0000000693][hsp_wble.c:0418] - WBLE >

    [0000000693][hsp_wble.c:0418] - WBLE > OK

    [0000000825][hsp_wble.c:0418] - WBLE >

    [0000000825][hsp_wble.c:0418] - WBLE > +UUNU:0

    [0000000835][hsp_wble.c:0545] - WBLE < ATO3

    [0000000835][hsp_wble.c:0418] - WBLE >

    [0000000835][hsp_wble.c:0418] - WBLE > OK

    [0000000845][hsp_wble.c:0697] - PPP init SUCCESS
    [0000000845][hsp_wble.c:0699] - Wifi station link UP and PPP mode active
    [0000001045][hsp_wble.c:0714] - Initialised NX common
    [0000001095][hsp_wble.c:0832] - WBLE module init_connection
    [0000001099][hsp_wble.c:0865] - Initialised IP
    [0000001099][hsp_wble.c:0891] - Initialised PPP
    [0000001099][new_thread0_entry.c:0135] - Interface index 0, IP: 2886994442, NMASK: 4294967295, MTU: 1500, MSW: 0, LSW: 0
    [0000001099][new_thread0_entry.c:0144] - SNTP init and get time
    [0000002154][new_thread0_entry.c:0149] - TIME/DATE >> Sep 4, 2021 16:19:16.796 UTC <<
    [0000002154][new_thread0_entry.c:0150] - EPOCH >> 1630772356 <<
    [0000002154][new_thread0_entry.c:0158] - DNS get MQTT broker IPv4 - attempt #1
    [0000002162][new_thread0_entry.c:0186] - is at
    [0000002174][new_thread0_entry.c:0222] - JWT token created successfully eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJoNG5oNDQtMSIsImV4cCI6MTYzMDc3OTU1NiwiaWF0IjoxNjMwNzcyMzU2fQ.rRBufIU9bdx0gQzckMW-QfIRqzfLruCst27_mIPviDyYkZbIgG10RM3xcKv-n8XY6fKgZrsa56MuoBdg0FX61mi3e9Qbh7f9lD53YDZgienf38qCmArXBgD3O2zwJyt1dnDRqc87fhDdBIsvoYfqnI4EPZyUUElu8UVAX8gVQcorpciHbgoWJjDP1qDK-P7K_ViLNcirj8oXQ7GOabFoEAcLJ3Nnw21FRw19w2ddbvR-h4HuIUC4QYop2CljGMMyRKUeRzGaxD9vQ7x3r56xy_fHPKeRyugnp6BvZl8KVy0yN9dxdLutbCn9p4rJn6VdtthY4YPO2q7DY7xiE1-teg

    [0000002175][new_thread0_entry.c:0237] - Secure TLS session created!
    [0000002175][new_thread0_entry.c:0244] - Secure_tls_ecc_initialize returned : 0x0
    [0000002175][new_thread0_entry.c:0263] - MQTT Client login set successful
    [0000002275][new_thread0_entry.c:0278] - nx_ip_status_check got status 0x0 and actual_status 0x4
    [0000002326][new_thread0_entry.c:0291] - MQTT Client secure connect : ERROR 0x10005
    [0000002326][new_thread0_entry.c:0307] - Interface index 0, IP: 2886994442, NMASK: 4294967295, MTU: 1500, MSW: 0, LSW: 0

  • I think your development network setup you are using to do Wireshark captures is not actually allowing you to see most of the packets going to and from the Synergy device.

    Most networking componets these days are switches that do packet routing, so only send the network packets out on the network interface they need to go out on, rather than a dumb hub that just broadcasts all packets on all network interfaces.

    For this reason I have to use a Network switch that does port mirroring, to send all the network traffic between the Synergy board and the Internet router, to my development PC, to allow me to Capture the packets with Wireshark. I would expect to see something like (this is from an MQTT connection io.adafruit but it shows the DHCP IP allocation, DNS lookup, NTP and TLS handshake for an ethernet connection) : -