Websocket call to Unscripted Server API

An introduction to the streaming audio client that uses the ELSA Unscripted API. This implementation facilitates sending an audio file and corresponding sentence to ELSA server. A websocket connection establishes a direct bidirectional connection between the client and the processing speech server. Exchange of information between both is done via messages, which can be either json structures or binary audio packets. This method is particularly beneficial compared to a single-file method when:

• You are implementing ELSA API on an online system where assessment delay is an important factor. ELSA API in streaming mode will be faster to respond as the ELSA Servers start processing your query as soon as the first audio comes in.
• You want to use the automatic endpoint feature to finish the assessment when the server detects completion.

The client streams an audio file to the server and receives a JSON response with the results.

API endpoint

All connections to the Elsa unscripted API start as a websocket request to wss://api.elsanow.io/api/v1/ws/score_audio_plus. Once the connection is authorized, the user can start sending audio data in one of the supported formats. To perform this integration, you can do it in python, javascript and php. However, to this example, we will use python example: https://gitlab.com/elsacorp/aiteam_tools/api-examples/-/blob/main/streaming_unscripted_client.py.

Overview

The ELSA Unscripted Audio Streaming Client is a Python script that leverages the ELSA websocket protocol to stream audio files to the Unscripted API. The key features of this client include:

• Efficient chunking of audio files
• Real-time streaming of audio chunks to the server
• Robust handling of server responses and storage of results in a queue

To achieve this, the script employs several custom classes and functions that manage the audio streaming process and facilitate communication with the server.

Usage

To run the ELSA Unscripted Audio Streaming Client, execute the following command:

python3 streaming_unscripted_client.py --token <CLIENT_TOKEN> --audio_path <audio_path> --return_json

Replace <CLIENT_TOKEN> with your ELSA API token and <audio_path> with the path to your audio file. Set return_json flag to receive the server's response as a JSON object.

Functionality

1. Parse command-line arguments, including the client token, audio file path, API plan, and an optional flag to return JSON in the response.
2. Establish a WebSocket connection with the server.
3. Wait for the WebSocket to connect.

INFO:root:Waiting for websocket connection...
INFO:root:Socket opened! ready for action.
INFO:root:ELSA:wsConnect
INFO:root:Starting a new stream
INFO:root:Sending config
INFO:root:processing stream ID: 

4. Initiate audio streaming by sending audio configuration to the server and start a new audio thread for streaming.

INFO:root:processing: {
    "status": "processing",
    "message": "Finished receiving audio"
}

5. Divide the audio file into manageable chunks.
6. Stream the audio chunks to the server .
7. Handle server responses, and store the results in a queue.
8. Process the results from the queue iteratively, printing them until the audio streaming process is complete or a timeout occurs.

INFO:root:success: {
    "status": "success",
    "result_link": "",
    "audio_link": "",
    "partial_error": "too_short"
}

The ELSA Unscripted Audio Streaming Client provides a powerful and versatile way of integrating real-time speech processing into your applications. Leveraging the WebSocket protocol, it offers quick turnaround times for audio assessments, as it enables processing to begin as soon as the first chunk of audio data is received by the server.

Responses

Response Status	Description
failed	Indicates a failed status with a specific reason
bad_quality	Indicates a bad quality status with a specific reason
processing	Indicates the processing status
paused	Indicates the paused status
success	Indicates the success status
accepted	Indicates the accepted status
Unknown status	Indicates an unknown status that is not covered by the other options