Merge from AndroidX Media main branch #27
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds missing `MessageType` constants to the `@IntDef` annotation and the class Javadoc. PiperOrigin-RevId: 828500265
When an ad has a content resume offset, some positions of the primary content stream may be skipped. This change makes sure that player messages that must be triggered are not scheduled within such content positions. PiperOrigin-RevId: 828646704
Also add a release note and some nullness fixes
This is necessary for a follow-up task that will use the forwarding video sink to intercept calls to flush, and notify CompositionPlayer's texture listeners of a seek. PiperOrigin-RevId: 828867711
PiperOrigin-RevId: 828870252
This value was split when sniffing was first introduced in <unknown commit> But at that time, the sniffing order was fixed for **all** files, so it was important that each sniffing implementation was quite conservative in how much data it consumed, in order to not delay playback start-up for files "later" in the sniffing order. Since then, extractor sniffing order has been optimized to prefer extractors that match the URI's file extension or MIME type (from a response header): * File extension: <unknown commit> * MIME type: <unknown commit> This optimization means that each individual extractor impl no longer has to be so conservative in its sniffing implementation. The effect of increasing the sniffing limit in `Mp3Extractor` today is: 1. For well-formed MP3 files with no/minimal garbage at the start, sniffing continues to return early when a sync byte is found - no change in behaviour. 2. For non MP3 files with a recognized file extension or MIME type, their preferred extractor will be sniffed before `Mp3Extractor`, so no change in behaviour. 3. For poorly-formed MP3 files with more than 32kB (but less than 128kB) of garbage at the start, playback previously failed and will now succeed. 4. For non-MP3 files with no recognized file extension or MIME type that are successfully sniffed by an extractor later than MP3 in the [default order](https://github.com/androidx/media/blob/05227c9591c6133abf4540e5460396919197ff45/libraries/extractor/src/main/java/androidx/media3/extractor/DefaultExtractorsFactory.java#L108-L133) - this will now take slightly longer, and involve reading 128kB data instead of 32kB (in order for the MP3 extractor to reject it). This is extractors for the following formats: * AVI * MIDI * Images Issue: #2713 PiperOrigin-RevId: 828884642
Add a new CompositionTextureListener which: * receives per-frame metadata and flush commands on playback thread * receives output textures and flush-completed signal on GL thread An output texture reads the next metadata that was enqueued by the playback thread A flush-completed removes all pending metadata and the next flush command. Flush is intercepted via ForwardingVideoSink Metadata is sent via renderers. PiperOrigin-RevId: 828890556
PiperOrigin-RevId: 828916716
It is already defined in `Application.mk` as: APP_ABI :=armeabi-v7a x86 arm64-v8a x86_64 PiperOrigin-RevId: 828943115
This change is part of a fix to the issue "Player Errors during Ad playback not attributed correctly" The change introduced in commit 7cf2fd9 was a partial fix to allow ad playback errors to be reported. It delayed removing the listener when an error released the AdsMediaSource. What remained was the larger issue: When an error occurs **during content preparation while an ad is still playing**, ExoPlayer cannot correctly attribute the error to the content source. Instead, the error bubbles up through `ExoPlayerImplInternal.stopInternal()` and causes the **entire CompositeMediaSource (AdsMediaSource + content)** to be torn down. This results in the `ImaAdsLoader` being stopped prematurely — even when the ad is not at fault — and breaks proper ad lifecycle handling. **User-Facing Issues** 1. **The ad is incorrectly marked as failed** A content preparation error is misattributed to the currently playing ad, resulting in an invalid `AD_STATE_ERROR`. 2. **The ad is never marked as completed** Even though `ALL_ADS_COMPLETED` is dispatched by IMA, the player is torn down before ExoPlayer updates `AdPlaybackState` to reflect completion. 3. **A single failing ad in a pod is not skipped** If one ad in a pod fails (e.g., due to a 404 or DRM error), the player stops entirely instead of skipping to the next ad in the group. 4. **A failing ad does not fall back to content** When an ad fails, the player should skip to content. Instead, the entire playback pipeline is torn down via `stopInternal()`. This change fixes the first two User-Facing issues. When combined with error recovery logic it also fixes the 4th issue. The 3rd issue may require some work on the IMA SDK side, the mid-pod failure aborts all ad playback with this fix, rather than skipping just the failing ad.
PiperOrigin-RevId: 828952608
Without this change, this test fails with `NoSuchElementException` when run with: ```shell ./gradlew :lib-exoplayer:testReleaseUnitTest ``` But pass with: ```shell ./gradlew :lib-exoplayer:testDebugUnitTest ``` With this change, both versions pass. #cherrypick PiperOrigin-RevId: 828967141
PiperOrigin-RevId: 828980223
Scrubbing suppresses playback, and in a Robolectric test time can auto-advance very quickly, which can easily exceed the default timeout. A couple of tests in `ExoPlayerScrubbingTest` were triggering stuck-while-playing, so that's disabled for those tests. Many tests in `ScrubbingPlaybackTest` use a blocking renderer implementation which interferes with the stuck-without-ending detection, so that is disabled for these tests. #cherrypick PiperOrigin-RevId: 828982225
PiperOrigin-RevId: 828996907
This ensures these standard commands are fully backwards compatible with platform custom actions or older Media3 custom layouts. The main code change is converting the standard commands to custom ones, and when receiving them, triggering the same standard action as if the normal standard command was received. PiperOrigin-RevId: 829010396
The `frameReadyWithoutSurface` flag in `VideoFrameReleaseControl` was incorrectly set to true when first frames were skipped or ignored due to being late while no output surface was available. This caused `MediaCodecVideoRenderer.isReady()` to return true even when no frame was actually ready to be rendered. The fix ensures `frameReadyWithoutSurface` is only set when a frame is genuinely ready but cannot be released because of the missing surface, exemplified by the `FRAME_RELEASE_TRY_AGAIN_LATER` result. PiperOrigin-RevId: 829368901
PiperOrigin-RevId: 829385327
This was suggested by a static analysis check, although it's not super-relevant here because this `WebView` is only used to render HTML that is crafted inside this class and `HtmlUtils` - so we never directly display untrusted user-provided HTML that could reference a `content://` URI unexpectedly. #cherrypick PiperOrigin-RevId: 829395425
The Composable underneath is `BasicText` which is not smart enough to ensure that dynamic text, like video timestamp, doesn't cause layout to reflow. Usually, there are a couple of ways to ensure the width remains constant: * `Monospaced` font (every character has the same width) * `fontFeatureSettings` parameter of the `TextStyle` (each digit has the same width, normal characters remain proportional) * `TextMeasurer` (measure the widest "worst-case" string like 88:88, apply that to all other numbers) The default font on Android (Roboto) is well-designed, and the width differences between its numerals are subtle to the naked eye. What is making it harder to test in the demo app is the flexible `Spacer` that takes up the *remaining* space. Ideally, adding the tests that check the width of various strings **would** show that, as of now, the implementation is width-independent and robust. The problem is that the Android test environment is intentionally designed to prevent this kind of test from working. It does **not** use the standard "Roboto" font, but a basic, non-proportional test font (i.e. all characters have the same width). This is done to prevent visual inconsistencies and flakiness in screenshot and UI tests. In fact, loading custom fonts in this environment is a known difficulty, so no matter what FontFamily we specify, the test environment will likely override it with a font that has fixed-width numerals. Given all this, we shall still apply one of the fixes to ensure the solution will work on real world devices. #cherrypick PiperOrigin-RevId: 829452658
Before this change, the `(.|\f)*` pattern can lead to exponential backtracking because it has to try many different combinations. Using the extra `+` makes the matching 'possessive', so once a match is found no other alternatives are considered. This is safe for these regexes, because the pattern in question is always at the end of the match, so there's no trailing characters that might need to be 'removed' from the match during a backtrack. This is a follow-up to 2bd2600 and 92677e9. #cherrypick PiperOrigin-RevId: 829457102
This is consistent with the rest of the library PiperOrigin-RevId: 830344997
startRenderingInternal() and stopRenderingInternal() should not be exposed as public methods. This is a non-functional change. PiperOrigin-RevId: 830374247
PiperOrigin-RevId: 830396036
When MediaController connects to a platform session, it reads bitmap data from the platform MediaMetadata and MediaDescription. These structures may contain Bitmaps that need to be compressed to be stored in Media3 structures like MediaMetadata. This currently happens repeatedly, both for the same instance of Bitmap, but also often for equivalent but not equal instances of Bitmap (which are created by sessions sending updates over binder, which create a new Bitmap instance every time even if the data is the same). This can be made more efficient by storing the already compressed data in MediaDescriptionCompat and MediaMetadataCompat. And in addition, we can attempt to preserve this data for new instances if their original Bitmap is the same as the new one. Calling Bitmap.sameAs is also not recommended for the main thread, but is very fast (<1ms) for the Bitmap sizes returned from platform media sessions, compared to the easily 100ms+ for every compress operation. PiperOrigin-RevId: 830410548
The core issue is `flush()` and `configure()` can run concurrently.
This is problematic because
- `flush()` relies on the TextureManager to complete the correct flushing
sequence
- `configure()`, if switching input type, can change the actual working
`TextureManager` in the shader chain
- Consequently, if `flush()` and `configure()` runs concurrently,
- `flush()` acquires `textureManager1`, sets the `onFlushCompletedListener` on
it, expecting this to run after the pipeline is flushed, and this unblocks the frame processor
- `configure()` on the other hand, modifies the pipeline to use
`textureManager2`, who doesn't have the `onFlushCompletedListener` set
- Hence, `onFlushCompletedListener` is never executed if the actual
texture manager is `textureManager2`, leading to a deadlock
Notice that `configure()` is invoked from two places:
1. As the last step of `flush()`, and this doesn't trigger the problem this CL
aims at solving
2. From `FinalShaderProgramWrapper.Listener()` which can be invoked at EOS, for
example when registering new input streams after user seeks
PiperOrigin-RevId: 830429582
PlaybackAudioGraphWrapper should only be accessed from the playback thread. Before this change, CompositionPlayer was releasing the PlaybackAudioGraphWrapper from the application thread, causing a race condition between the registration and unregistration of platform callbacks in AudioCapabilitiesReceiver. PiperOrigin-RevId: 830432413
We cannot rely on these resource names continuing to exist in future versions of `androidx.core`, so we must not reference them from here in order to avoid a runtime crash due to "diamond dependencies". #cherrypick PiperOrigin-RevId: 830435116
Also tweak one of the test cases so we exercise both the positive & negative offset paths. Issue: #2929 #cherrypick PiperOrigin-RevId: 842148345
The IntDef @Player.Command is defined as an int and hence the arguments passed in as an int should be of type int as well. PiperOrigin-RevId: 842220027
This allows apps to access the lifecycle scoping utils of the service Also downgrade the current AndroidX lifecycle version to the last version that did not force-include Kotlin in case apps are not using it. Issue: #187 PiperOrigin-RevId: 842292078
In order to share the OutputLayout type and the util, getOutputLayoutForChannelMask, this CL pulls them out of the IAMF decoder extension. In the future, these will be used by MediaCodecAudioRenderer for IAMF. PiperOrigin-RevId: 842301275
PiperOrigin-RevId: 842473353
PiperOrigin-RevId: 842495038
PiperOrigin-RevId: 842733818
This is a non-functional refactor and prework for adding a preprocessing pipeline in AudioGraphInput. PiperOrigin-RevId: 842759197
This CL adds - complete representation of IAMF layouts for conversion from channel mask to OutputLayout - the reverse conversion, from OutputLayout to channel mask - Thorough testing of both PiperOrigin-RevId: 842849446
Same as the OUTPUT_LAYOUT, move this value to IamfUtil to be shared by both extension decoder and MediaCodecAudioRenderer (when it has additional IAMF functionality). PiperOrigin-RevId: 842867825
PiperOrigin-RevId: 842887055
Migrate the MediaRouteButtonViewProvider to use the media3's MediaRouteButtonFactory class. PiperOrigin-RevId: 842932069
If a frame rate is set on `EditedMediaItem`, then render will drop decoded frames to match the frame rate. The logic can only decrease the frame rate but can not increase it. PiperOrigin-RevId: 843149488
Android platform treats sRGB and SMPTE 170M color spaces as equivalent. However, the internal SDR shader in `DefaultVideoFrameProcessor` was performing an explicit conversion from sRGB to SMPTE 170M. This led to a visible color shift when processing sRGB inputs. This change modifies the SDR shader `fragment_shader_transformation_sdr_internal_es2.glsl` to use the input color space as the working color space, effectively skipping the conversion. Consequently, the logic for converting to the output color space has also been updated to handle both sRGB and SMPTE 170M as possible working color spaces. PiperOrigin-RevId: 843156441
They were added to support `tryQueuePacket()`. But as the method is removed, the two classes are obsolete. PiperOrigin-RevId: 843160209
There is a generic catch exception to mark a generic XmlPullParserException as "malformed manifest". However, these exception are also thrown when the XML parser encounters an upstream data source issue, where this classification is not helpful. Instead, we can detect the underlying IOException and rethrow it directly. PiperOrigin-RevId: 843214758
This loop happens if the manifest download or the index load runs into PriorityTooLowException, which are retried on the same Future, whose state is still failed, and thus is retried forever. Issue: #2931 PiperOrigin-RevId: 843233678
PiperOrigin-RevId: 843340020
PiperOrigin-RevId: 843397399
This is done by forking the `VideoSampleExporter` to work specically with
`PacketProcessor`.
The flow is now:
- `PacketConsumerVideoSampleExporter` creates `GraphInputs` to interface with
`ExoAssetLoaderRenderer` (essentially to receive renderer output)
- The new `GraphInputs` is backed still by `SingleInputVideoGraph`, unchanged
- But the `SingleInputVideoGraph` outputs to a texture listener, subsequently
connected to the `PacketConsumer` complex, sequentially they are:
- `FrameAggregator`
- `PacketProcessor`
- `GlTextureFrameRenderer`
- Note there's no `ReleaseControl` in this process
- `GlTextureFrameRenderer` renderers onto the encoder input surface
**Other additions** include
- Demo app change to showcase the usage
PiperOrigin-RevId: 843465852
- Don't release composited frame - Suspend until capacity is returned to texture pool Without this change the compositor fails with error: `Textures are all in use. Please release in-use textures before calling useTexture.` PiperOrigin-RevId: 843549402
PiperOrigin-RevId: 843605471
This refactoring is for using another field `EditedMediaItem.frameRate` for adding frame dropping logic. PiperOrigin-RevId: 843618810
When `EditedMediaItem.frameRate` is set, then renderer will drop frames accordingly. PiperOrigin-RevId: 843634898
Remove a couple of setters, remove mutable state, simpler if/else in CompositionPlayer PiperOrigin-RevId: 843651006
The pre-processing pipeline will allow the `AudioGraphInput` to modify the input audio stream before it reaches user-provided audio effects. This transparent processing is required for use cases like speed adjustment, that needs to apply processing transparently to the audio stream output by the renderer. The pre-processing pipeline will also be useful for input format normalizing. Only internal components can add `AudioProcessor` instances to the pre-processing pipeline. PiperOrigin-RevId: 843657974
Previously, the order of elements in the assertions was ignored. PiperOrigin-RevId: 843670752
To create timebar functionality, we need to unite the Material3 `Slider` Composable and `ProgressIndicator` Composable. One gives the user ability to pick a value by scrubbing the `Thumb`. Another is driven by a state-backed value to move the `Thumb` along the `Track` automatically. There is currently no out-of-the-box solution for it in Material3. Sliders are used for scrub-and-done kind of action (e.g. volume, brightness) and progress indicators are used to show a non-interactive bar (e.g. download progress). Playback case is special and needs to both display the progress of the Player, but also let the user seek within the `MediaItem`. Note that the current design does not allow for displaying of "buffering position" as a different colour on the track. See: * https://m3.material.io/components/sliders * https://m3.material.io/components/progress-indicators http://Issue: #2288 #cherrypick PiperOrigin-RevId: 843677725
This change migrates the `SpeedChangingAudioProcessor` instance used by `CompositionPlayer` to apply a speed adjustment effect on an audio stream to the pre-processing `AudioGraphInput` pipeline. This keeps the `SpeedChangingAudioProcessor` instance separate from user-provided processors. PiperOrigin-RevId: 843688512
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.