MQTT TLS on Google cloud - invalid JWT generated

I'm trying to connect to our thing on Google Cloud IoT Core, but I get an error when trying to connect using  

status = nxd_mqtt_client_secure_connect(&g_mqtt_client0, &broker_address, NXD_MQTT_TLS_PORT,
NX_MQTT_tls_setup, 0, NX_TRUE, NX_WAIT_FOREVER);

I figure out the cause is in nx_secure_tls_session_start.c, function _nx_secure_tls_handshake_process that Fails with 56 (0x38) error (NX_NOT_CONNECTED 0x38)

Then looking at the JWT that is generated, I've checked on jwt.io (a website) and gets me invalid JWT.

Note that I have RootCA certificate, device certificate and device private key (in pem format) and they work just fine with Python (just checking certificates and key are correct, and server address/ TLS is working), in fact the JWT generated in python gets me "genuine" under the same jwt.io website validator.

I dont' get any errors while loading certificates and key on netx secure, so I'm guessing is the way I pad or calculate the JWT is not correct, I'm using the same base64 and JWT create functions that you use in the MQTT_tls google cloud example project.

Root certificate and device certificate are converted first to .der (using openssl) and then to binary array using the hexy.exe tool.

Private key is converted first to pkcs1.pem, then to .der and finally to binary array.

All the CA certificate, device certificate and private key are loaded into some header files and included in my project.

Some code, after I establish an IP layer through Wifi, following:

//> SNTP gets time, I save timestamp and call get_time() when I need it (to generate JWT) - no errors here

//> DNS to get IP address of Google Cloud endpoint - no errors here

//>Init of mqtt packet pool and client - all good

Then init and store of certificates, I get no errors in the following:

status = nx_secure_tls_session_create(&g_mqtt_client0.nxd_mqtt_tls_session,
(NX_SECURE_TLS_CRYPTO *) &nx_crypto_tls_ciphers_synergys7,
crypto_metadata,
sizeof(crypto_metadata));

status = nx_secure_tls_session_packet_buffer_set(&g_mqtt_client0.nxd_mqtt_tls_session,
tls_packet_buffer,
sizeof(tls_packet_buffer));

//---------------------

char *temp_str = "unused";

for(UINT i = 0; i < sizeof(temp_str); i++)
{
username[i] = temp_str[i];
}

//Password is the JWT generated using realtime timestamp (epoch) and IoT thing private key
//rsa_private_pkcs1, RSA_PRIVATE_LEN
jwt_expiration = JWT_EXPIRATION_SECS;

/* Generate JWT */
if (jwt_create(password, sizeof(password), &jwt_expiration,
rsa_private_pkcs1, RSA_PRIVATE_LEN, (CHAR*) &project_id))
{
HSP_APP_LOG(HSP_LOG_LEVEL_ERROR,"Unable to create JWT. Aborting.\r\n");
__BKPT(0);
}

HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"JWT token created successfully %s\n============", password);
//set login
status = nxd_mqtt_client_login_set(&g_mqtt_client0, username, strlen(username),
(char *)password, strlen((char *)password));
if(status){
HSP_APP_LOG(HSP_LOG_LEVEL_ERROR,"tnxd_mqtt_client_login_set error %d", status);
__BKPT(0);
}

//---------------------------------- after this, I get back the error described above.

///> In nx_secure_tls_session_start.c
///> function _nx_secure_tls_handshake_process Fails with 56 (0x38) error
///> (NX_NOT_CONNECTED 0x38)

status = nxd_mqtt_client_secure_connect(&g_mqtt_client0, &broker_address, NXD_MQTT_TLS_PORT,
NX_MQTT_tls_setup, 0, NX_TRUE, NX_WAIT_FOREVER);

if(status){
HSP_APP_LOG(HSP_LOG_LEVEL_ERROR,"nxd_mqtt_client_secure_connect error %d", status);
__BKPT(0);
}

__BKPT(0);

I include my project file, so maybe somebody can address me on how to properly generate the JWT, as the following is not getting me a proper JWT.

https://www.filemail.com/d/hacpvrjcfpuxwnd

Is the certs/private key conversion and handling correct?

Do I need to change padding/ crypto functions in the JWT header/payload/signature generation?

Unfortunately I can not find any more info on this topic, as normally a jwt library is used to generate hash and create the token, but I want to make use of the hardware secure engine.

Any info/hints very welcome. 

I can share certs/keys if you want to check, but basically If I get a valid JWT generated I think we are half way there.

Generated JWT:

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJoNG5oNDQtMSIsImV4cCI6MTYzMDQyMTE3OCwiaWF0IjoxNjMwNDIwODc4fQ==.SbNIiu23b0w9udd8jlICsW/cQqsbMgKvzQlSmga8mseI3hVAwW001YQ9QmLPc7Y3z8bE8ndYJXE3lO4UA39M4D6G/KyrybRnzQkFfevRm4qw8NekOsENPEse25YGNpzydKcVHvomHMBm8r797ApH2Z0Xs3FD2e+EgnS3GkQ4bYVxscMmDFqd/m5IWJPz8mZiwuN3+LXK2gVOF798wObXY7L7q8EWycV2QbeDOQlEkVyG1SFTwudM0Y+Geppsli5uUrscEq5ipT/rgIaIFuqUcvykH9IZm/sLHXh54UYIkaAlHdzGyWTr+50cV4X+9ZWwTcfQdrRcECKIsSgeC/JyHw==

(note the == characters generated, I suspect shouldn't be there)

and so: [0000002235][new_thread0_entry.c:0395] - nxd_mqtt_client_secure_connect error 65541

I'm on S5D9, SSP1.7.8 e2Studio. NetX Duo.

Thanks a lot.

S

Parents
  • Hi S,

    Have you try modifying  the Base64urlEncoder() in Base64Decode.c to strip the padding, "="? Would it pass checking jwt.io?

    The hash/signature would definitely be different without '=', As the Header/payload is available, the receiver could have use it to verify the signature,   

  • Hi,

    Yes I've tried but still doesn't pass the jwt.io check. Is there a valid Base64url encoder library that actually works with Google IoT Core?

    Also, it has something to do with the pkcs1 padding in the jwt_create() ?

    Thanks

  • Hi Rusty,

    I change (in Base64Decode.c)

    base64digits[] = "ABC...789+/"; 

    to

    base64digits[] = "ABC...789-_";

    last 2, '+', '/' change to '-', '_' 

    and in Base64urlEncoder() 

    '=' change to '\0'  to omit the last 1 or 2 padding. (2 instances of this)

            *out++ = (inlen < 2) ? '\0'  /*'='*/   : base64digits[(in[1] << 2) & 0x3c];
            *out++ = '\0';   //'=';

    Now the jwt.io verifies the JWT submitted has correct signature. 

    No changes to pkcs1 padding.

    Can you check if you can reach google server after these changes?

  • Hi Yep,

    I made the changes you suggested, but still I can not generate a valid JWT, here some extract from my logs and the base64 library I'm using.

    Any suggestions on how to get it working much appreciated.

    ----LOG----

    [hsp_wble.c:0690] - PPP init SUCCESS
    [hsp_wble.c:0692] - Wifi station link UP and PPP mode active
    [hsp_wble.c:0707] - Initialised NX common
    [hsp_wble.c:0826] - WBLE module init_connection
    [hsp_wble.c:0859] - Initialised IP
    [hsp_wble.c:0885] - Initialised PPP
    [hsp_wble.c:0908] - Initialised DNS
    [new_thread0_entry.c:0260] - START TIME/DATE >> Sep 2, 2021 12:32:54.841 UTC <<
    [new_thread0_entry.c:0321] - mqtt.2030.ltsapis.goog is at 173.194.76.206:8883
    [new_thread0_entry.c:0344] - Secure TLS session created!
    [new_thread0_entry.c:0375] - JWT token created successfully eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJoNG5oNDQtMSIsImV4cCI6MTYzMDU4NjI3NCwiaWF0IjoxNjMwNTg1OTc0fQ.ce4xJaVwO0HV4ybdnC8-vKTCUrHRroiYY_9R8egWW_izKQPGlUOxv6Ku2-ZLsquh7EJFcARPkg4EkewKymbQDCrBdBYND8Kyd6_NPF_A7nqPRcTwwUTHLOa9xzZxAI_xJpgpicAhGlM7kJCHDeerwSTNu6HWsi1E2uCjwM8NxuyWnsut9Lg12EXcBb8FJ2bF_km8QO6BGqHnlCbQf3byQ58IJ6LIaIy8buCgtu7GHc1FSVbmUHk263QwuFD99yGrh1k-91DkIf5QHUZHnGyn-r7bERQkYev8ef9m_gKiFDUz6sNZ3R1ibW62S6ufTrJGfgs6bQHCkI_n3JxcpCunaw

    [new_thread0_entry.c:0384] - MQTT Client login set successful
    [new_thread0_entry.c:0397] - nxd_mqtt_client_secure_connect error 0x10005

    ---Base64Decode.c

    #include <stdio.h>
    #include <string.h>
    #include <stdint.h>
    #include <assert.h>
    #include "base64.h"

    void Base64urlEncoder(unsigned char *out, const unsigned char *in, int inlen);

    //Base64 char table - used internally for encoding
    static const unsigned char base64digits[] =
    "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_";

    unsigned int base64_int(unsigned int ch)
    {

    /* ASCII to base64_int
    65-90 Upper Case >> 0-25
    97-122 Lower Case >> 26-51
    48-57 Numbers >> 52-61
    43 Plus (+) >> 62
    47 Slash (/) >> 63
    61 Equal (=) >> 64~
    */
    if (ch==43)
    return 62;
    if (ch==47)
    return 63;
    if (ch==61)
    return 64;
    if ((ch>47) && (ch<58))
    return ch + 4;
    if ((ch>64) && (ch<91))
    return ch - 'A';
    if ((ch>96) && (ch<123))
    return (ch - 'a') + 26;
    return 0;
    }

    unsigned int base64_decode(const unsigned char* in, unsigned int in_len, unsigned int* out)
    {

    unsigned int i=0, j=0, k=0, s[4] = {0};

    for (i=0;i<in_len;i++)
    {
    s[j++]=base64_int(*(in+i));
    if (j==4)
    {
    out[k+0] = ((s[0]&255)<<2)+((s[1]&0x30)>>4);
    if (s[2]!=64)
    {
    out[k+1] = ((s[1]&0x0F)<<4)+((s[2]&0x3C)>>2);
    if ((s[3]!=64))
    {
    out[k+2] = ((s[2]&0x03)<<6)+(s[3]); k+=3;
    }
    else
    {
    k+=2;
    }
    }
    else
    {
    k+=1;
    }
    j=0;
    }
    }

    return k;
    }

    unsigned int base64d_size(unsigned int in_size)
    {

    return ((3*in_size)/4);
    }

    ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
    //
    // Function : Base64urlEncoder
    // Purpose : Converts a given string into a base64 encoded buffer.
    // Last updated : 01/09/200515/05/2005
    // Notes :
    //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

    void Base64urlEncoder(unsigned char *out, const unsigned char *in, int inlen)
    /* [OUT] out A pointer to a char to hold the converted string
    [IN] in String to convert
    [IN] inlen Length of the string to be converted
    */
    {
    for (; inlen >= 3; inlen -= 3)
    {
    *out++ = base64digits[in[0] >> 2];
    *out++ = base64digits[((in[0] << 4) & 0x30) | (in[1] >> 4)];
    *out++ = base64digits[((in[1] << 2) & 0x3c) | (in[2] >> 6)];
    *out++ = base64digits[in[2] & 0x3f];
    in += 3;
    }

    if (inlen > 0)
    {
    unsigned char fragment;

    *out++ = base64digits[in[0] >> 2];
    fragment = (in[0] << 4) & 0x30;

    if (inlen > 1)
    fragment |= in[1] >> 4;

    *out++ = base64digits[fragment];
    *out++ = (inlen < 2) ? '\0' : base64digits[(in[1] << 2) & 0x3c];
    *out++ = '\0';
    }

    *out = '\0';
    }

     

  • Correction: 

    The JWT is actually now verified as correct, using jwt.io and the device / private key.

    At this point what could be the cause?

    Timestamp is calculated correctly? (gives me epochs that when converted are in UTC+0)

    I've tried JWT expiration to 1 hour, 2 hours, but seems not to make a difference.

    Again, all the certificate work with a python mqtt project, and I can connect securely to the same broker "mqtt.2030.ltsapis.goog", on port 8883.

    Is there a way to see why this is failing, other that the generic error 0x10005 from nxd_mqtt_client_secure_connect()?

    Debugging to lower level I get that NX is not connected, even though I have an IP layer opened and working.

    ///> In nx_secure_tls_session_start.c
    ///> function _nx_secure_tls_handshake_process Fails with 56 (0x38) error
    ///> (NX_NOT_CONNECTED 0x38)

    Thanks

    my configuration: all IP instances are shared, TLS common is shared between MQTT and TLS Session:

  • Hi,

    Could you get a wireshark logs? Hope to understand why server terminates this tcp connection.

    Just to double check, is the size of your certificate correct?  You mention you convert this certificate into byte array. Could you do a memory dump of this and check if it's correct?

  • Hi Yep, 

    Yes I've done a capture of my wifi network from my PC while application running, the DHCP address assigned to my uBlox NINA is 192.168.1.12, something is going on but I'm not sure at what to look at.

    Following are my google certificate in PEM format (that I use from Python with no problems) and the corresponding byte array used in my FW. Size seems the same. 

    CA_cert.zip

    Should I use a different certificate or check if is loaded correctly into netx secure?

    Can I send the capture file in private to you? 

    Also, would it be possible to create a TLS session project (client) and use the same certificate just to see if it's loaded and checked correctly? I tried also an openssl s_client -connect with this certificate and broker endpoint and status returns 0 from openssl (verified, OK).

     Thanks,

    S

    edit: Looking here:AE-CLOUD2. Google cloud. Could not create tls session. - Synergy - Forum - Renesas Synergy Platform - RenesasRulz

    I have confirmed that I can connect to Azure and Amazon using the 2.1.0 SECT binary, but I cannot connect to Google. There has been a change in the Google certificate chain, that requires a different intermediate CA certificate to be used for Google. This would require a new SECT binary firmware.

    This answer maybe is a clue? Not sure how I load a certificate and an intermediate certificate, as a chain?

  • Hi Rusty,

    The CA_cert_der[] is an ECC cert.  Can you check if ECC is added to nx_crypto_tls_ciphers_synergys7. 

    Could you find/use RSA cert instead? You have install a root/intermediate cert as trusted cert.

    Other intermediate certs in the chain will be downloaded as needed to verify the server cert.

  • Hi,

    Yes all the suites are enabled, including ECC. 

    Do I need to specify that is an ECC cert in tls init?

    That certificate has been given by google, not sure If/how I can find an RSA version of it

  • Do you call nx_secure_tls_ecc_initialize() in the mqtt tls setup function?

    extern const USHORT     nx_crypto_ecc_supported_groups_synergys7[];
    extern const USHORT     nx_crypto_ecc_supported_groups_synergys7_size;
    extern const NX_CRYPTO_METHOD *     nx_crypto_ecc_curves_synergys7[];

     nx_secure_tls_ecc_initialize(p_tls_session,
                                              nx_crypto_ecc_supported_groups_synergys7,
                                              nx_crypto_ecc_supported_groups_synergys7_size,
                                              nx_crypto_ecc_curves_synergys7);

    in the mqtt tls setup function?

  • Hi Jeremy,

    No, I call x509 for both CA_cert and device cert. should I substitute the ca_cert one?

    Or add nx_secure_tls_ecc_initialize() at the beginning/end of the mqtt_tls_setup?

    Here's my mqtt tls setup function:

    static UINT NX_MQTT_tls_setup(NXD_MQTT_CLIENT *p_client, NX_SECURE_TLS_SESSION *p_tls_session,
    NX_SECURE_X509_CERT *p_cert, NX_SECURE_X509_CERT *p_trusted_cert)
    {
    UINT status = NX_SUCCESS;
    UINT index = 0;

    SSP_PARAMETER_NOT_USED (p_client);

    /* Need to allocate space for the certificate coming in from the remote host. */
    memset(&remote_cert_buffer, 0, sizeof(remote_cert_buffer));
    memset(&remote_certificate, 0, sizeof(remote_certificate));

    status = nx_secure_tls_remote_certificate_allocate(p_tls_session, &remote_certificate,
    remote_cert_buffer, sizeof(remote_cert_buffer));
    if (NX_SUCCESS != status)
    {
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unabe to allocate memory for the certificate buffer %d", status);
    return status;
    }

    memset(remote_int_ca_cert_buffer, 0, sizeof(remote_int_ca_cert_buffer));
    memset(remote_intermediate_ca, 0, sizeof(remote_intermediate_ca));
    for (index = 0; index < ARRAY_SIZE(remote_intermediate_ca); index++)
    {
    status = nx_secure_tls_remote_certificate_allocate(p_tls_session, &(remote_intermediate_ca[index]),
    &(remote_int_ca_cert_buffer[index * CERTIFICATE_BUFFER_SIZE]),
    CERTIFICATE_BUFFER_SIZE);
    if (NX_SUCCESS != status)
    {
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unable to allocate memory for interemediate CA certificate %d", status);
    return status;
    }
    }

    /* Add a CA Certificate to our trusted store for verifying incoming server certificates. */
    status = nx_secure_x509_certificate_initialize(p_trusted_cert,
    (UCHAR *) CA_cert_der, (USHORT) CA_CERT_DER_LEN,
    NX_NULL, 0, NULL, 0,
    NX_SECURE_X509_KEY_TYPE_RSA_PKCS1_DER);
    if (NX_SUCCESS != status)
    {
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unable to initialize CA certificate %d", status);
    return status;
    }

    status = nx_secure_tls_trusted_certificate_add(p_tls_session, p_trusted_cert);
    if (NX_SUCCESS != status)
    {
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unable to add CA certificate to trusted store, %d", status);
    return status;
    }

    if (dev_cert != NULL)
    {
    /* Add the local certificate for client authentication. */
    status = nx_secure_x509_certificate_initialize(p_cert, (UCHAR *) dev_cert, (USHORT) DEV_CERT_LEN,
    NX_NULL, 0, rsa_private_pkcs1, RSA_PRIVATE_LEN,
    NX_SECURE_X509_KEY_TYPE_RSA_PKCS1_DER);
    if (NX_SUCCESS != status)
    {
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unable to initialize device certificate, %d",status);
    return status;
    }
    status = nx_secure_tls_local_certificate_add(p_tls_session, p_cert);
    if (NX_SUCCESS != status)
    {
    HSP_APP_LOG(HSP_LOG_LEVEL_DEBUG,"Unable to add device certificate to trusted store, %d",status);
    return status;
    }
    }

    /* Add a timestamp function for time checking and timestamps in the TLS handshake. */
    nx_secure_tls_session_time_function_set(p_tls_session, get_time);

    return status;
    }

    Thanks a lot

  • Do you call nx_secure_tls_ecc_initialize() anywhere? Try nx_secure_tls_ecc_initialize() adding after you call nx_secure_tls_session_create();

  • I've called secure_tls_ecc_initialize after session_create(), it returns 0 (I guess is NX_SUCCESS), but still I can not connect with the same 0x10005 error.

    I also tried to load the google certificate in RSA format (and changed broker to "mqtt.googleapis.com" as advised by IoT Core cloud.google.com/.../mqtt-bridge), but still I get the same error from mqtt connect. Also tried to use port 8883 and 443.

    www.filemail.com/.../cyvrwfffjslgbzp

  • Have you taken a Wireshark trace of the network activity to see what is going on in the communication between the Synergy board and the Google servers?

  • Yes, I did, 192.168.1.12 is my uBlox NINA Wifi DHCP address. Asus is our wifi router, external activity is captured from the internal wifi network but I think you can extract the big picture.

    Please find it here: www.filemail.com/.../mojofybpwdtrpuk

    I did also 2 additional captures with just my laptop (wireshark running) and uBlox connected to my WiFi hotspost.

    One using ECC certificate and the nx_secure_tls_ecc_initialize() (with the proper endpoint for ECC cert) and the RSA cert version.

    www.filemail.com/.../heenqzenkxqxpbx

    Interestingly, in these two last captures, I can't see the DNS request, even though I get it and resolve the IP of google cloud endpoint correctly from Synergy, also SNTP time query is not showing, and again I get the time correctly from the SNTP server.

    I think something is going on when resolving uBlox external IP, gateway IP, but again, I can perform SNTP queries, DNS queries, and HTTP Post successfully with the same stack.

  • I don't see the DNS query of the Synergy device looking up the address of the Google mqtt server:-

    "mqtt.2030.ltsapis.goog"

    or

    "mqtt.googleapis.com"

            /* Look up IPv4 addresses to record  IPv4 addresses in record_buffer and return the IPv4 address count.  */
            status = nx_dns_ipv4_address_by_name_get(&g_dns0, (uint8_t *)GCP_IO_SERVER, &gcp_io_ip_address[0], sizeof(gcp_io_ip_address), &record_count, 500/*NX_IP_PERIODIC_RATE*/);

    in any of the the wireshark logs. The only DNS queries in any of the logs you posted are in the file "nina-mqtt-tls-3.pcapng :-

    Also, Have you tried using Google's minimal root CA set (<1 KB) for mqtt.2030.ltsapis.goog. The minimal root CA set includes a primary and backup certificate.

    cloud.google.com/.../mqtt-bridge

Reply
  • I don't see the DNS query of the Synergy device looking up the address of the Google mqtt server:-

    "mqtt.2030.ltsapis.goog"

    or

    "mqtt.googleapis.com"

            /* Look up IPv4 addresses to record  IPv4 addresses in record_buffer and return the IPv4 address count.  */
            status = nx_dns_ipv4_address_by_name_get(&g_dns0, (uint8_t *)GCP_IO_SERVER, &gcp_io_ip_address[0], sizeof(gcp_io_ip_address), &record_count, 500/*NX_IP_PERIODIC_RATE*/);

    in any of the the wireshark logs. The only DNS queries in any of the logs you posted are in the file "nina-mqtt-tls-3.pcapng :-

    Also, Have you tried using Google's minimal root CA set (<1 KB) for mqtt.2030.ltsapis.goog. The minimal root CA set includes a primary and backup certificate.

    cloud.google.com/.../mqtt-bridge

Children
  • Yes, the minimal set root CA is the one originally used, but as Yep said that is an ECC certificate. And yes, the original address mqtt.2030.ltsapis.goog 8883 with the minimal CA (ECC) is the one I was using. Also, I tried adding the nx_secure_tls_ecc_initialize() (when using the minimal cert, ECC) and tried also with the RSA cert version changing the broker accordingly (mqtt.googleapis.com 8883).

    Same error comes out.

    Also I tried to manually set the google brokers IP (pinging from my laptop just before compiling) and same happens.

    Is there some IP routing going on in the wrong way? Like, is Google trying to send back packets to my uBlox public addres but it can not find me?

    I don't use any DCHP ssp module, I think I don't need to do that.

    when you say "Also, Have you tried using Google's minimal root CA set (<1 KB) for mqtt.2030.ltsapis.goog. The minimal root CA set includes a primary and backup certificate.", that means I need to load both certificates into netx secure? Because in python I just use the primary and all goes well as expected.

    Thanks for your help, is becoming very complex to use netx duo framework on IoT Core/ TLS.

    At the moment we are stuck and we need to make sure we can proceed the development on Synergy with this platform.

    Is there a working example using Syergy to interact (connect/publish/receive) with MQTT on IoT Core with these certificates (either minimal root certs or the regular version) through a mqtt tls using JWT?

  • I've noticed a strange thing. When synergy goes to ask ARP resolution, gets no answer.

    I think the whole IP chain down to the synergy stack is not correct/ routed correctly.

    Also. the MAC address I get when I query directly uBlox module is different from the one I see in the wireshark capture (ends with a different number).

    How can I setup the whole uBlox <-> PPP <-> IP in order to work?

    Strangely though, even in this weird configuration, I can get the time from SNTP and google IP from DNS, but I don't see it in the tcp/ip capture...

    Please find here my configuration, the IP I get assigned by my router is 172.20.10.4, and this is the Peer IPv4 on PPP interface I set up, but I'm not sure about the IP interface configuration.

    Intel core(my laptop) gets an answer, uBlox ARP request gets lost..

    Full extract of the log, you can see the full AT command sequence, MAC address of uBlox, Wifi station init and PPP mode init..

    [0000000331][hsp_wble.c:0418] - WBLE > +STARTUP

    [0000000331][hsp_wble.c:0458] - WBLE < ATE0

    [0000000331][hsp_wble.c:0418] - WBLE > ATE0

    [0000000332][hsp_wble.c:0418] - WBLE >

    [0000000332][hsp_wble.c:0418] - WBLE > OK

    [0000000342][hsp_wble.c:0458] - WBLE < AT+UWAPMACADDR

    [0000000342][hsp_wble.c:0418] - WBLE >

    [0000000342][hsp_wble.c:0418] - WBLE > +UWAPMACADDR:6009C3E05737

    [0000000342][hsp_wble.c:0418] - WBLE > OK

    [0000000352][hsp_wble.c:0458] - WBLE < AT+UWSC=0,0,1

    [0000000352][hsp_wble.c:0418] - WBLE >

    [0000000352][hsp_wble.c:0418] - WBLE > OK

    [0000000362][hsp_wble.c:0458] - WBLE < AT+UWSC=0,2,"----SSID----"

    [0000000362][hsp_wble.c:0418] - WBLE >

    [0000000362][hsp_wble.c:0418] - WBLE > OK

    [0000000372][hsp_wble.c:0458] - WBLE < AT+UWSC=0,5,2

    [0000000372][hsp_wble.c:0418] - WBLE >

    [0000000372][hsp_wble.c:0418] - WBLE > OK

    [0000000382][hsp_wble.c:0458] - WBLE < AT+UWSC=0,8,"----password----"

    [0000000382][hsp_wble.c:0418] - WBLE >

    [0000000382][hsp_wble.c:0418] - WBLE > OK

    [0000000392][hsp_wble.c:0458] - WBLE < AT+UWSC=0,100,2

    [0000000392][hsp_wble.c:0418] - WBLE >

    [0000000392][hsp_wble.c:0418] - WBLE > OK

    [0000000402][hsp_wble.c:0458] - WBLE < AT+UWSC=0,107,1

    [0000000402][hsp_wble.c:0418] - WBLE >

    [0000000402][hsp_wble.c:0418] - WBLE > OK

    [0000000412][hsp_wble.c:0458] - WBLE < AT+UWSC=0,301,1

    [0000000412][hsp_wble.c:0418] - WBLE >

    [0000000412][hsp_wble.c:0418] - WBLE > OK

    [0000000422][hsp_wble.c:0660] - Wifi init SUCCESS
    [0000000422][hsp_wble.c:0500] - WBLE < AT+UWSCA=0,3

    [0000000423][hsp_wble.c:0418] - WBLE >

    [0000000423][hsp_wble.c:0418] - WBLE > OK

    [0000000652][hsp_wble.c:0418] - WBLE >

    [0000000652][hsp_wble.c:0418] - WBLE > +UUWLE:0,FAAC838145DA,1

    [0000000652][hsp_wble.c:0679] - Wifi attach SUCCESS
    [0000000652][hsp_wble.c:0545] - WBLE < AT+UPPPC=101,172.20.10.4

    [0000000653][hsp_wble.c:0418] - WBLE >

    [0000000653][hsp_wble.c:0418] - WBLE > OK

    [0000000663][hsp_wble.c:0545] - WBLE < AT+UPPPC=102,255.255.255.0

    [0000000663][hsp_wble.c:0418] - WBLE >

    [0000000663][hsp_wble.c:0418] - WBLE > OK

    [0000000673][hsp_wble.c:0545] - WBLE < AT+UPPPC=104,172.20.10.1

    [0000000673][hsp_wble.c:0418] - WBLE >

    [0000000673][hsp_wble.c:0418] - WBLE > OK

    [0000000683][hsp_wble.c:0545] - WBLE < AT+UPPPC=105,172.20.10.1

    [0000000683][hsp_wble.c:0418] - WBLE >

    [0000000683][hsp_wble.c:0418] - WBLE > OK

    [0000000693][hsp_wble.c:0545] - WBLE < AT+UPPPC=107,1

    [0000000693][hsp_wble.c:0418] - WBLE >

    [0000000693][hsp_wble.c:0418] - WBLE > OK

    [0000000825][hsp_wble.c:0418] - WBLE >

    [0000000825][hsp_wble.c:0418] - WBLE > +UUNU:0

    [0000000835][hsp_wble.c:0545] - WBLE < ATO3

    [0000000835][hsp_wble.c:0418] - WBLE >

    [0000000835][hsp_wble.c:0418] - WBLE > OK

    [0000000845][hsp_wble.c:0697] - PPP init SUCCESS
    [0000000845][hsp_wble.c:0699] - Wifi station link UP and PPP mode active
    [0000001045][hsp_wble.c:0714] - Initialised NX common
    [0000001095][hsp_wble.c:0832] - WBLE module init_connection
    [0000001099][hsp_wble.c:0865] - Initialised IP
    [0000001099][hsp_wble.c:0891] - Initialised PPP
    [0000001099][new_thread0_entry.c:0135] - Interface index 0, IP: 2886994442, NMASK: 4294967295, MTU: 1500, MSW: 0, LSW: 0
    [0000001099][new_thread0_entry.c:0144] - SNTP init and get time
    [0000002154][new_thread0_entry.c:0149] - TIME/DATE >> Sep 4, 2021 16:19:16.796 UTC <<
    [0000002154][new_thread0_entry.c:0150] - EPOCH >> 1630772356 <<
    [0000002154][new_thread0_entry.c:0158] - DNS get MQTT broker IPv4 - attempt #1
    [0000002162][new_thread0_entry.c:0186] - mqtt.2030.ltsapis.goog is at 74.125.133.206:8883
    [0000002174][new_thread0_entry.c:0222] - JWT token created successfully eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJoNG5oNDQtMSIsImV4cCI6MTYzMDc3OTU1NiwiaWF0IjoxNjMwNzcyMzU2fQ.rRBufIU9bdx0gQzckMW-QfIRqzfLruCst27_mIPviDyYkZbIgG10RM3xcKv-n8XY6fKgZrsa56MuoBdg0FX61mi3e9Qbh7f9lD53YDZgienf38qCmArXBgD3O2zwJyt1dnDRqc87fhDdBIsvoYfqnI4EPZyUUElu8UVAX8gVQcorpciHbgoWJjDP1qDK-P7K_ViLNcirj8oXQ7GOabFoEAcLJ3Nnw21FRw19w2ddbvR-h4HuIUC4QYop2CljGMMyRKUeRzGaxD9vQ7x3r56xy_fHPKeRyugnp6BvZl8KVy0yN9dxdLutbCn9p4rJn6VdtthY4YPO2q7DY7xiE1-teg

    [0000002175][new_thread0_entry.c:0237] - Secure TLS session created!
    [0000002175][new_thread0_entry.c:0244] - Secure_tls_ecc_initialize returned : 0x0
    [0000002175][new_thread0_entry.c:0263] - MQTT Client login set successful
    [0000002275][new_thread0_entry.c:0278] - nx_ip_status_check got status 0x0 and actual_status 0x4
    [0000002326][new_thread0_entry.c:0291] - MQTT Client secure connect : ERROR 0x10005
    [0000002326][new_thread0_entry.c:0307] - Interface index 0, IP: 2886994442, NMASK: 4294967295, MTU: 1500, MSW: 0, LSW: 0

  • I think your development network setup you are using to do Wireshark captures is not actually allowing you to see most of the packets going to and from the Synergy device.

    Most networking componets these days are switches that do packet routing, so only send the network packets out on the network interface they need to go out on, rather than a dumb hub that just broadcasts all packets on all network interfaces.

    For this reason I have to use a Network switch that does port mirroring, to send all the network traffic between the Synergy board and the Internet router, to my development PC, to allow me to Capture the packets with Wireshark. I would expect to see something like (this is from an MQTT connection io.adafruit but it shows the DHCP IP allocation, DNS lookup, NTP and TLS handshake for an ethernet connection) : -

  • Hi Jeremy, I'm using SSP 1.7.8, S5D9, and I got all my pieces of code almost from that project you're talking about.

    I though the same about packet capture, indeed I can not see (ARP replies, and other packets) data directed other than the interface I'm running Wireshark from (my laptop). I read about monitor mode would do the trick (instead of promiscuous that is the one I'm using) but not all wifi interfaces allows that.

    Do you think I need to check further the IP assignment, and maybe set up manually the uBlox MAC address into the msw, lsw of my ip_interface? 

    But since I can resolve DNS, get time from the SNTP server and perform HTTP posts, I think the whole IP stack is actually working. Not sure.

    Is there any simple way to test if I can make a TLS connection to the google server, using one of the certs (reduced ECC or standard RSA)?

    I need to make some progress on this, because If I can't use the NetX Duo stack to connect to MQTT/TLS of IoT Core I need to change my direction of development.

    How could I set up something like your switch to mirror the packets from the Wifi router to my laptop? 

    Thanks a lot

    S

  • Hi S,

    For RSA, you use the Baltimore Cybertrust Root (roots[] = ). Could you use the GTS CA 101 (its also given in the project shared by Jeremy).

    For port mirroring, I have use TL-SG105E | 5-Port Gigabit Easy Smart Switch | TP-Link Singapore with port mirroring feature.

  • Hi Yep, 

    Yes, I loaded the certificate from that project (gcloud_rootCA.pem) and it worked, also managed to publish some data on our GCP device.

    The problem still remains though, as we need a long term certificate (this one expires in december) and a reduce size one because of the data that will be used at every connection. 

    > openssl x509 -enddate -noout -in .\gcloud_rootCA.pem
    notAfter=Dec 15 00:00:42 2021 GMT

    Could you please explain/investigate why we can not use the ECC (reduced) google certificate?

    It is of vital importance that we can make use of the ECC suite and this certificate to create our TLS connection.

    Are you able to try yourself with the ECC reduced one and see if you can connect (using the same projects you developed)?

    Thanks for you help, at least now we got something working!

  • Hi Rusty,

    When using, GTS LTSR (primary) you need to set r_sce_0 SCE COMMON Driver on r_sce to CRYPTO_WORD_ENDIAN_LITTLE.  While verifying an ECC cert, I found it wont pass in ENDIAN_BIG. For RSA certs, both BIG/LITTLE are OK. I dont quite figure out where problem when converting endian.

    The (backup) can't be used. It has a ECC 384bit keys. So any cert ecdsa-with-sha384 is not possible.

    I ask to use RSA, cause your JWT. uses { "alg" : RS256 ...} You may have to change your device cert to ECC as well.

    Also strongly recommend you switch to SSP 2.0.0.. 

  • For big endian, before verification, swapping for input parameters is done, in-situ. If the certificate is in ROM (is a const), bytes cannot be swapped, so operations will fail. You cert need to be in RAM. 

    Sorry my Google account is in a mess, I couldn't set it up to do a proper test.. 

  • I can get the AE CLoud2 version of this project :- 

    https://www.renesas.com/document/apn/synergy-mqtttls-google-cloud-connectivity-solution-application-project?language=en&r=1054626

    to connect to mqtt.2030.ltsapis.goog using the minimal root CA set's primary certificate, if I add the TLS SNI :- 

        static NX_SECURE_X509_DNS_NAME dns_name;

        status = nx_secure_x509_dns_name_initialize(&dns_name, (UCHAR *)p_cfg->p_endpoint, (USHORT)(strlen(p_cfg->p_endpoint)));
        if (NX_SUCCESS != status)
        {
            NX_MQTT_log_msg(p_ctrl, "Unable to initialise SNI DNS name, ret=0x%x\r\n",status);
            return status;
        }

        status = nx_secure_tls_session_sni_extension_set(&(p_ctrl->p_secure_client->nxd_mqtt_tls_session), &dns_name);
        if (NX_SUCCESS != status)
        {
            NX_MQTT_log_msg(p_ctrl, "Unable to add SNI, ret=0x%x\r\n",status);
            return status;
        }

    The requirements to connect to an LTS MQTT domain are here :-

    cloud.google.com/.../mqtt-bridge

  • Thanks Jeremy, I've updated my e2Studio and SSP to 2.0.0 and trying again now to tls using the primary certificate (ECC) to connect to  mqtt.2030.ltsapis.goog.

    Couple of questions if you don't mind:

    1. Do I still need to initialise ECC suite with nx_secure_tls_ecc_initialize(&g_mqtt_client0.nxd_mqtt_tls_session,
      nx_crypto_ecc_supported_groups_synergys7,
      nx_crypto_ecc_supported_groups_synergys7_size,
      nx_crypto_ecc_curves_synergys7) ?
    2. Can I use the converted certificate from PEM->DER->Byte array as I have it at the moment or do I need to convert from the .crt?
    3. At which point do I need to add and set the SNI Dns? In the NX_MQTT_tls_setup() or just before/after the nx_secure_tls_session_create()?
    4. I'm getting the following error now with the new SSP2.0.0, have the function name/arguments changed? 

    ./synergy/ssp/src/framework/sf_el_nx_crypto/nx_crypto_synergys7_ciphersuites.o:(.data.crypto_method_aes_128_gcm_16_sce+0x1c): undefined reference to `_nx_crypto_method_aes_gcm_sce_operation'

    collect2.exe: error: ld returned 1 exit status

    Thanks a lot,

    S

  • I had success using SSP 1.7.8 with the google cloud example project. I have migrated the project to SSP 2.0.0 however I have been unsuccessful in connecting using the LTS certificates.

    In answer to your questions :-

    1) In the application code, the API nx_secure_tls_ecc_initialize() must be called
    after TLS session is created  ( nx_secure_tls_session_create() ). This API notifies the TLS session of the type of
    curves used in the system. During the TLS handshake phase, if ECC
    algorithm is selected, the client and server exchange ECC curve-related parameters, so ECC can be used for the session.

    2) If the certificate is in the correct format in a byte array then it is fine. I have used a similar process before (converted certificate from PEM->DER->Byte array ).

    3) Again, for the SNI, the API to add the SNI information needs to be called after the TLS session is created (nx_secure_tls_session_create()  )

    4) For the compilation issue with SSP 2.0.0, when you enable ECC Cipher, you also need to enable AEAD Cipher for the compilation to be successful. As I said, I have yet to successfully connect to Google with SSP 2.0.0, using the ECC LTS certificates, as there have been some additions to NetX secure, so the configuration is slightly different.

  • Thanks Jeremy, copied all your suggestions:

    • tls_ecc_initialise called after tls_session_create()
    • SNI called after tls_ecc_initialise() (and so after tls_session_create()
    • Enabled AEAD (both options) and now compiles correctly (is that a new feature for SSP2.0.0? This error didn't appear in SSP1.7.8

    As you stated though, now I fail again on the mqtt_secure_connect. Is quite frustrating that this function doesn't return a more precise error, is a generic NX_NOT_CONNECTED : 0x10005.

    Do I need to rollback to 1.7.8 or is there a workaround you guys are working on for the new SSP?

    I'm also trying to bring up another laptop with a Wifi card that should allow monitoring mode, so maybe I'll be able to see what's going on during the handshake.

    Thanks again,

    S

  • I am currently trying to work out why SSP 2.0.0 will not connect to Google using the ECC certificates.

  • I have done a bit more digging as to why the SSP 2.0.0. based project won't connect to the Google cloud MQTT Long term domain, and it appears to be an issue with NetX DUO TLS version in SSP 2.0.0, and the integration of the SCE Crypto HW integration that is used.. There have been alot of changes in NetX DUO TLS between SSP 1.7.8 and SSP 2.0.0. In the Wireshark trace when trying to connect I see an Alert :-

    and the Alert is labelled as an internal error, and the Alert came from the Synergy device.

    After a bit of debugging on the Synergy side, I see the error is coming from a Crypto operation not being successful (the Crypto operation is using the SCE Crypto HW) :-

    It is because of this Crypto operation error, that NetX DUO TLS in SSP 2.0.0 signals the Alert.

    This issue is not going to be fixed quickly, I have raised the issue with the SSP development team, for them to look at.

    As I said, with SSP 1.7.8 I could successfully connect to the Google cloud MQTT Long term domain.