Amazon Chime SDK with over 16 video sessions

This article is also available here.(Japanese)
https://cloud.flect.co.jp/entry/2020/07/08/153456

Previously I talked about the Barcode scanner with semantic segmentation npm.

Implement semantic segmentation with npm

This time I introduce the tricky way to use Amazon Chime SDK. That is, a user join the multiple meeting room in SPA(Single Page Application) at the same time.

This enable us to provide these features available for online seminar.

• A teacher can give lessons to more students than maximum number of the session per meeting room of Amazon Chime.
• Students cna protect their privacy

This is demo for this feature.

In this demo, teacher can join two meeting rooms, and he can see students in both meeting rooms. This number of students is 30, which is more than maximum number of the video session per meeting room of Amazon Chime. And, we can recognize from the demo above, but the movie and voice of one meeting room is not propagated to the other meeting room.

The number of session per meeting room of Amazon Chime against large online seminar

The maximum number of the video session per meeting room of Amazon chime is 16. This number is less than Zoom’s webinar plan. However, Amazon Chime has two type of attendees as shown below, and the maximum number of audio-only user is 250.

And if necessary, we can re-assigne the video session to audio-only session’s user with keeping online. That is, we can show the video of active speacker on demand.

Considering face-to-face lectures and seminars, I don’t think it’s very common for a participant to see another participant’s face outside of the question-and-answer session. So if the behavior of the system is to assign a video call session to the speaker during the presentation and a video call session to the questioner (a specific attendee) during the questioning, the maximum number of video call sessions in the Amazon Chime SDK may be sufficient in many cases.

However, if you have more than 250 participants in an audio call, or if you want to see the reactions of more than 16 speakers on a video call, as shown in the figure below, the Amazon Chime SDK is not enough.

This could be solved if we could spread the attendees across multiple conference rooms and allow speakers to attend those rooms at the same time.

Note that this is a bit of a side note, but if you have more than 16 speakers, Amazon Chime SDK may not be able to handle it.

Protect privacy is needed

Depending on the seminar, there may be some distant family members or relatives who would like to study at an online seminar with you, but don’t want others to know that they are attending. For example, it might be about medical care, exams, marriage, asset management, etc. I think many of these things have serious content, so I think families often take the course while checking their impressions from time to time.

These could also be solved by assigning a conference room to each family or group, as shown below, and allowing speakers to participate in those rooms at the same time, since the conversations and images in the conference room would not be transmitted to other conference rooms.

Point of implementation

The entire demo program is up in the following git repository, so I’ll just go over the points.

However, there is nothing particularly difficult to do.
You can basically just repeat the process of entering a single meeting room and make sure you keep track of each endpoint and participant.

For example, in this demo program, each of the participating conference rooms is managed through the JoinedMeeting interface. This interface contains information specific to each conference room, such as The MeetingSessionConfiguration and MeetingSession, participant information and video tiles.

export interface JoinedMeeting{
    meetingSessionConfiguration: MeetingSessionConfiguration,
    meetingSession: MeetingSession,
    roster: { [attendeeId: string]: Attendee },
    videoTileStates: { [id: number]: VideoTileState },
    outputAudioElement: HTMLAudioElement | null,
    <snip>
}

You should manage these information of each meeting rooms in SPA. In this demo program, we use an associative array with “meetingId” as a key.

export interface AppState {
    joinedMeetings : {[id:string]:JoinedMeeting},
    <snip>
}

And each time you join a new meeting room, you can set up the MeetingSession. audioVideo for that meeting room. You can set up the video input, microphone input and speakers from audioVideo. In the case of device re-selection, the same applies to all participating conference rooms. You can set it up from MeetingSession.audioVideo.

For example, if you want to switch the microphone input, it would look something like this

selectInputAudioDevice = (deviceId: string) => {
        const currentSettings = this.state.currentSettings 
        currentSettings.selectedInputAudioDevice = deviceId

        Object.keys(this.state.joinedMeetings).forEach((x:string)=>{
            this.state.joinedMeetings[x].meetingSession.audioVideo.chooseAudioInputDevice(deviceId); // <- set here
        })
        this.setState({ currentSettings: currentSettings })
    }

Another point

Basically, the above is fine, but if you go into more than one conference room, the amount of data sent and received for that video will increase.

The amount of data sent can be controlled by the Amazon Chime SDK API, so you should be able to adjust it there. Especially on the student’s side, it is better to set lower resolution by default.

selectInputVideoResolution = (value: string) =>{
        <snip>
        Object.keys(this.state.joinedMeetings).forEach((x:string)=>{
            this.state.joinedMeetings[x].meetingSession!.audioVideo.chooseVideoInputQuality(
                videoConfig.width, videoConfig.height, videoConfig.frameRate, videoConfig.maxBandwidthKbps 
            );
        })      
        <snip>
    }

Code

The features described in this article are built into a test bed of new features using video conferencing.
If you are interested in it, please visit the following repository.

FLECT-DEV-TEAM/FLECT_Amazon_Chime_Meeting

Finally

In this article, we experimented with the ability to join multiple conference rooms from an SPA using the Amazon Chime SDK.
There are still some things to consider with respect to networking and other computational resources, but in the grand scheme of things, we found that it is possible.
This gave us the prospect of being able to construct a large scale online seminar venue from multiple meeting rooms.
It also gave us the prospect of protecting the privacy of groups in large venues.

I am very thirsty!!

Reference

Camera images of participants in the conference room were pseudo-created using videos from the following sites.

https://pixabay.com/ja/videos/