“Google’s Live Caption app uses the new AudioPlaybackCapture API to capture currently playing audio”
Android Q will feature a new API for developers known as the AudioPlaybackCapture API. To make everyone aware of this feature, Google has detailed the use case and functioning of this API in a new blog post. The API enables on-device audio capture, which can be used by one app to record audio of another app. It will make it easier for developers that want to develop apps to record game audio or for Live captioning.
In the blog post, Google provides use case scenarios in Live captioning and Game recording/ streaming. The search giant has already showcased Live Captions, which can record device audio and then show it as captions on-screen. Google has used this API in Live Captions and it can be very helpful when you’re trying to watch something in a noisy place. The API will convert the device audio to captions and make it easier for you to understand what’s going on in the video.
Google also explained that game recording and streaming apps can make use of this API to record in-game sounds. The AudioPlaybackCapture API requires user permission before it can start recording device audio. The app doing the audio capturing will seek users’ permission first. Once the user grants permission, a Screen Capture Intent pop-up will be shown to the user. After reading through this, the user needs to tap on Start Now to actually start recording audio.
After hitting the Start Now button, the audio capturing app can record both audio and the screen. That will be pretty useful for those who want to stream games from smartphones. Google also outlined how this API can be used to disallow certain apps from capturing audio by default. Developers can use this when the third-party app has sensitive information or copyrighted material such as music, video, etc. Also, developers of apps that use API version 28 and older, will need to make some changes to the manifest file of their app. After this, the app can allow other apps to record its audio.