Use avatar you like in Zoom !

3 min readJul 15, 2022

Note:
This article is also available here.(Japanese)
https://qiita.com/wok/items/4f51e1a72d735b75f73f

Introduction

Zoom has begun offering an SDK. It’s a lot of excitement. I took the opportunity to experiment with this and will be posting several in a row, as it looked like it could do a lot of interesting things. Please see subsequent posts if you are interested.

In terms of data flow to Zoom, there are input and output, input from the microphone and camera, and output from the speakers. In previous articles, we have experimented mainly how to handle the input. From next article I will to see if we can somehow tweak the output.

Before talking about the output side, however, I would like to introduce two minor features that have been added to the Zoom Meeting SDK.

The first is the motion capture record function. This allows you to reuse what you have recorded. The second is the ability to replace avatars. You can use any avatar you like.

motion capture record

It works like the following video, where once motion capture is done, it can be used again.

The operation is shown in the following figure. upper body capture is quite unstable, so you may want to try it out and disable it if you find it difficult.

Needs improvement, but I’m hoping for a mediapipe official.(ref)

Avatar replacement function

Open the settings dialog from the gear icon in the upper right corner. click on the avatar icon and you will see a button to load a vrm file. click on it and load the file of your choice. You can use avatars other than animals!

The operation is shown in the following figure.

Demo

Again, in addition to creating a Docker image, we are also deploying to Heroku. In the meantime, please try using it.

Docker

$ docker run -p 8888:8888 dannadori/zoom-meeting-plus:v05 
$ docker run --rm -it -p '127.0.0.1:50021:50021' voicevox/voicevox_engine:cpu-ubuntu20.04-latest 
$ docker run -it -p 5500:5500 synesthesiam/opentts:en

Heroku

Heroku will stop if there is no access for a certain period of time, and it will take some time to restart. Please be patient until the application starts. (It will probably take less than a minute.)

https://zoom-meeting-plus.herokuapp.com/

Repository

The demo shown here is available in the following repositories

https://github.com/w-okada/zoom-meeting-plus

The latest version may not be stable as we are constantly updating it.
Please clone and use the following tag.

$ git clone https://github.com/w-okada/zoom-meeting-plus -b v05

I need coffee!!

Disclaimer

In no event shall we be liable for any direct, indirect, consequential, or special damages arising out of the use or inability to use the software on this blog.