Azure IoT Bug notice (11/2022) : Please read if you are suddenly unable to connect to Microsoft Azure IoT Hub.

Azure IoT Bug notice (11/2022) : Please read if you are suddenly unable to connect to Microsoft Azure IoT Hub.

Overview

Symptom: Your connection to IoT Hub is « CLOSED » and a restart doesn’t change anything

There is a rolling upgrade of Azure IoT Hub services from 'v1' to 'v2', unfortunately it introduces a regression during authentication for which there is no workaround. This note describes the issue, instructs you to open a ticket with Microsoft support. From the ticket number, as we are in contact with Microsoft on this issue, we can handover the ticket to a specific team able to fix your problem. Please contact us with the Microsoft support ticket number. PLEASE VERIFY FIRST THAT YOUR ISSUE IS THE ONE COVERED IN THIS NOTE. 

Impact and Solution

On the TPX User Interface (TPX-UI), in the event panel, you could can see this type of error message:

{

"errorDescription":  "org.apache.qpid.proton.engine.TransportException: connection aborted" ,

"errorCode":  "amqp:connection:framing-error"

}


This «framing-error » appears every time you are trying to restart your IoT Hub connector instance.

This is an issue on Azure IoT-Hub, introduced by a recent rolling upgrade of the Azure platform. There is currently no workaround.

What to do ?
  1. You should open a ticket on Microsoft support at Help + Support - Microsoft Azure.
Use the message template which follows and your iot hub information.
                                    
2. Once the ticket has been created on Microsoft support side, please open an Actility ticket including the Microsoft's ticket reference and details. We will make our Microsoft contacts aware of your ticket so it can be handled faster.
MESSAGE TEMPLATE:

We facing to an AMQP authentication issue using SASL and PLAIN authentication mechanism.

On a regular AMQP, when we are doing the authentication, we are having this kind of dialog :

Request done by Actility:

AzureProtocolTracer : Reactor -> SASL

AzureProtocolTracer : Reactor -> SaslInit{mechanism=PLAIN, initialResponse=\ x00actility-service@SAS.root.actility -prod\x00SharedAccessSignature sig=ybUR7%2FVaa1hK2FOSPinoQmHofdBbaP%3D&se=1694711564&skn=actility-service&sr= actility-prod.azure-devices.net , hostname='null'}

Response from Azure:

AzureProtocolTracer : Reactor <- SASL

AzureProtocolTracer : Reactor <- SaslMechanisms{saslServerMechanisms=[EXTERNAL, MSSBCBS, ANONYMOUS, PLAIN]}

AzureProtocolTracer : Reactor <- SaslOutcome{_code=OK, _additionalData=Welcome!}

If we use wrong credentials the (normal) response is:

Response from Azure with a wrong password:

AzureProtocolTracer : Reactor <- SASL

AzureProtocolTracer : Reactor <- SaslMechanisms{saslServerMechanisms=[EXTERNAL, MSSBCBS, ANONYMOUS, PLAIN]}

AzureProtocolTracer : Reactor <- SaslOutcome{_code=SYS, _additionalData=null}

however in our case we observe the following:

Response from Azure:

AzureProtocolTracer : Reactor <- SASL

AzureProtocolTracer : Reactor <- SaslMechanisms{saslServerMechanisms=[EXTERNAL, MSSBCBS, ANONYMOUS, PLAIN]}

The answer is truncated, we never receive the expected last lines of the response.
This causes an unrecoverable error for the IoT Hub client: Actility ThingPark Enterprise displays "amqp:connection:framing-error"