You hold your breath,quietly lying on the grass.
The more and moreyelling and wheezing from your right.
you clenched the fingers on the pan, trembling slightly.
Just at the moment that a voice in the front approach you, you jump outof the grass, enemies knockout by the pan—a big victory!
This victory belongs to you, and it also belongs to the 3D positional audio.
Spatial audio is not strange to many game enthusiasts, such as theshooting of gunfire in CS, enemy footsteps sound which means they are approaching, etc.
In the past, these details are based on the audio engine of the SDK.It has a very realistic effect, but it is not universal, such as the 3D real-time voice communication cannot be realized.
So as for the voice communication, his/her voice sounds almost the same no matter whether your teammates are around you - it turns out to be noone.
Now, we can rely on the algorithm to achieve all these.
For example, the 3D positional audio algorithm currently developedby
Tencent Media Lab can not only simulate the 3D sound withoutspecial hardware, but also enable users to experience the effect of 3D real-time speech on any ordinary headset.
Before 3D process:
After 3D process:
We will hear the sound in "3D", because the volume and thedelay of the sound to the left and right ears are different.
This algorithm can not only adjust parameters to simulate human’s sense of orientation in different positions in the space, but also achieve the effect of sound reflection in different environments, such as in the outdoor, concerthall or in a car.
More importantly, this technology can be applied to real-time voice communication to enrich the sound with “position information”, letting you speak the words directly in "3D"!
So, what else can we do with the 3D positional audio that thealgorithm brings with?
Firstly, let’s talk about online video that everyone like. 3D positional audio technology can completely change user experience.
Why is it cool to see the movie in the cinema? In addition to the hyperlarge screen, the key is the astonishing stereophonic effect.
What do that stereo sound effects depend on? An array of sound boxesin N different positions. The same principle applies to home theater.
But nowadays who will lie on the couch and turn on your speaker towatch an online video? Most people watch videos on their phones directly with headphones.
So, regardless of how many channels of sound recorded, we can onlyhear the sound from left to right with a headset. Many details are blurred out. I do not know how many people blamed this.
Our 3D positional audio algorithm can virtualize each speaker indifferent positions to restore each audio channel in appropriate locations. Itsounds like a cinema on your mobile phone, and you only need an ordinary stereoheadset, super value pack!
3D positional audio algorithm also helps scenarios like VR telepresence.
For example, when you look at the VR video, there is a monster infront of you. But if the sound was not in 3D, it would sound like that themonster just stand still which was kind of embarrassing..
This requires that the VR device can not only track objects throughhead movement, but also offer realistic 3D positional audio, so that people can hear the voice that matches the video, which can virtualize the location information of the objects. As a result we can create a three-dimensional spaceand make people believe what we want them to.
For applications in which there is screaming from the roller coaster,or approaching footsteps behind the haunted house, VR can't do without the 3D positional audio. It is said that Tencent will release a new VR game that applied withthis 3D positional audio algorithm. Welook forward to it bringing users with unprecedented new experience.
And even more, this algorithm can lead 3D real-time voice communication to diverse applications in online games.
For the most popular multiplayer online games nowadays, if weintroduce 3D real-time voice communication, we can get the dramatic effect describedat the beginning of this article. The location information will be embeddedwith the voice we hear which means that the voice we make and hear will change along with the position shift in real-time.
That is to say, when we are excited to shout, provoke, and talk tothe nearby people, don't forget that our position will be exposed. This makesthe players’ communication and fighting experience become very involving. Listen!The northeast corner of the 30-degree direction someone is approaching.
3D positional audio can also be used in multiple party online games suchas online chess and card rooms. Each player’s voice will sound like coming fromcorresponding position. You will play like with real people in the same room.
For instance, we can’t distinguish the players when everybody is talking in online playing cards game. However, you can clearly realize that somebody says “bomb” in the left while another person says “rocket” in the right with the application of 3D positional audio. What an exciting game!
3D posional Audio in cards game:
Furthermore, let’s brainstorm. Maybe after years of development, when the 3D positional audio becomesmore accurate, the visually disabled people can play “audio games” by"listening."
We draw a big picture of what our algorithm can do as talked above. Wehope, in the near future, all of the ideas will come true, opening the door tothe new hearing experience for us.
