-
Notifications
You must be signed in to change notification settings - Fork 361
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for playing Kotlin Multiplatform resources #1405
Comments
I can see two possible options here:
(2) seems to solve the problem described here, without the fiddliness introduced by (1). The downside of (2) seems to be that we can only read the entire file into memory in one go, which might not be great for large media files. On the other hand, large media files probably shouldn't be baked into the APK in the first place... I'm tempted to go with (2), and maybe in future Compose Multi-Platform will add some first-class support for reading resources 'incrementally' and we can update our implementation to use that. |
Ah, I didn't realise that the
This makes (2) a little trickier, though we can either take a super-type of this generated |
EDIT: The suggestion in this comment of creating a separate
public final class ResolvingByteArrayDataSource implements DataSource {
public static final class Factory implements DataSource.Factory {
private final Function<Uri, byte[]> uriResolver;
public Factory(Function<Uri, byte[]> uriResolver) {
this.uriResolver = uriResolver;
}
@Override
public DataSource createDataSource() {
return new ResolvingByteArrayDataSource(uriResolver);
}
}
private final Function<Uri, byte[]> uriResolver;
private final ArrayList<TransferListener> listeners;
@Nullable private ByteArrayDataSource delegate;
@Nullable private Uri uri;
private ResolvingByteArrayDataSource(Function<Uri, byte[]> uriResolver) {
this.uriResolver = uriResolver;
this.listeners = new ArrayList<>(/* initialCapacity= */ 1);
}
@Override
public long open(DataSpec dataSpec) throws IOException {
uri = dataSpec.uri;
delegate = new ByteArrayDataSource(uriResolver.apply(uri));
for (TransferListener listener : listeners) {
delegate.addTransferListener(listener);
}
return delegate.open(dataSpec);
}
@Override
public int read(byte[] buffer, int offset, int length) throws IOException {
return delegate.read(buffer, offset, length);
}
@Override
public void addTransferListener(TransferListener transferListener) {
if (!listeners.contains(transferListener)) {
listeners.add(transferListener);
}
if (delegate != null) {
delegate.addTransferListener(transferListener);
}
}
@Nullable
@Override
public Uri getUri() {
return uri;
}
@Override
public void close() throws IOException {
uri = null;
delegate.close();
delegate = null;
}
} Then you would set your player up with this (see https://developer.android.com/media/media3/exoplayer/customization) - this makes your player only work with 'kotlin multi platform' URIs: ExoPlayer player =
new ExoPlayer.Builder(/* context= */ this)
.setMediaSourceFactory(
new DefaultMediaSourceFactory(
new ByteArrayDataSource.Factory(uri -> Res.readBytes(uri.getPath())))
.build(); And you should then be able to play a resource like this: player.addMediaItem(MediaItem.fromUri("files/music.mp3")); |
First thing - great thank to you for all the help and ideas so far! I really appreciate it. I have tested code above and it's working. I have few things if someone will reuse it too - possibly from Kotlin code: readBytes() is suspend function so it needs to be called similar to this
b) Don't forget to add also As you mentioned above about delaying instantiation of In case that option (2) is way to go - should I create new issue on Compose Multiplatform side about Res.readBytes() reading in on go as you mentioned above? Or it will be more clear what kind of interface will be needed as this will be implemented on this side? In short - let me know if I can do something more. And thanks again. |
@Woren I am assuming you don't have a lot of files at hand, a little workaround approach to do is via offloading the files inside the jar to the app's contextual private files directory (Which on Android equals to |
@yuroyami Before Compose Multiplatform Resources generation (since 1.6.0) I was using mentioned But as mentioned before workaround using uriResolver seams to be working for now so hopefully until proper solution by Media3/Exoplayer library there will be no need another one. But thanks for the idea! I will keep in mind for another possible problems in other areas. And surely there will be some. |
This is a relatively small change, and massively simplifies the work needed for an app to consume Kotlin Multiplatform resources (without a full `KmpResourceDataSource` implementation, which poses some dependency challenges for now). Issue: #1405 PiperOrigin-RevId: 638991375
4dd8360 changes the library's We are not currently planning any tighter integration with Kotlin Multiplatform Resources - partly because it looks like it will require some additional dependencies that are currently tricky to resolve in the environments where we develop and deploy this library (and it would probably also need to be in a new extension module). I will leave this issue open to track doing something more closely integrated in future. |
Thank you for changes with simplifications and more info about the future. I will check mentioned commit when the new version is out. Version 1.6.0 of Compose/Kotlin Multiplatform is one of the first versions with this support (now for example lacking handling larger files) so it will be more mature in coming months/years.
|
If someone is still looking at this, and want a more optimized solution that doesn't load the whole content of the file to the memory, here is an attempt at creating a custom class JarResourceDataSource : BaseDataSource(false) {
private var uri: Uri? = null
private var inputStream: InputStream? = null
private var bytesRemaining = 0
override fun read(buffer: ByteArray, offset: Int, length: Int): Int {
if (length == 0) return 0
if (bytesRemaining == 0) return C.RESULT_END_OF_INPUT
val read = inputStream!!.read(buffer, offset, min(bytesRemaining, length))
if (read == -1) {
return C.RESULT_END_OF_INPUT
} else {
bytesRemaining -= read
bytesTransferred(read)
}
return read
}
override fun open(dataSpec: DataSpec): Long {
uri = dataSpec.uri
transferInitializing(dataSpec)
inputStream = URL(dataSpec.uri.toString()).openConnection().getInputStream()
bytesRemaining = if (dataSpec.length == C.LENGTH_UNSET.toLong()) {
inputStream!!.available() - dataSpec.position.toInt()
} else {
dataSpec.length.toInt()
}
return bytesRemaining.toLong()
}
override fun getUri(): Uri? = uri
override fun close() {
inputStream?.close()
uri = null
inputStream = null
}
} I think this is similar to the approach (1) that @icbaker initially suggested, just instead of having to think about where the Jar is located and other stuff, we just delegate the call to Now, we can consume this directly same as the suggestions above by passing a custom class ExtendedMediaFactory(private val defaultMediaFactory: DefaultMediaSourceFactory) :
MediaSource.Factory by defaultMediaFactory {
private val jarResourceMediaSourceFactory = ProgressiveMediaSource.Factory {
JarResourceDataSource()
}
override fun createMediaSource(mediaItem: MediaItem): MediaSource {
return if (mediaItem.localConfiguration?.uri?.scheme == "jar") {
jarResourceMediaSourceFactory.createMediaSource(mediaItem)
} else {
defaultMediaFactory.createMediaSource(mediaItem)
}
}
}
----
ExoPlayer.Builder(context).setMediaSourceFactory(ExtendedMediaFactory(DefaultMediaSourceFactory(context))) Then, you should be able to play the media: player.addMediaItem(MediaItem.fromUri(Res.getUri("files/media.mp3"))); |
Thanks for the code suggestions @hichamboushaba - a couple of thoughts if you or others are interested:
|
Thank you @icbaker, this is quite helpful, I didn't know about these two points. |
No problem - some other thoughts: One of the problems with building a
Instead of this, I wonder if you can use You current implementation also ignores |
Thanks again @icbaker
I used
That's correct, I missed this, thanks for pointing it. |
* Remove `@UnstableApi` from demo apps This annotation is only needed on public classes that are part of a distributed library. Switch to `@OptIn` for the one file where `@UnstableApi` was suppressing lint errors. PiperOrigin-RevId: 633858516 * Remove `@UnstableApi` from package-private files This annotation is only needed on public classes. This change also removes the `/* package */` comment from some `public` classes. PiperOrigin-RevId: 633864544 * Remove `/* package */` comment from `public` classes Also make one class truly package-private and keep the comment instead. This comment should only appear on elements with default (package-private) visibility. PiperOrigin-RevId: 633911914 * Add unit test for seeking in clipped MediaItem PiperOrigin-RevId: 633912487 * Clear the PreloadMediaSource when deprioritized by the preload manager PiperOrigin-RevId: 633917110 * Fix test flakiness The test currently resets the time too far in the past and then has to run through ~30000 additional iterations of doSomeWork to reach the end, sometimes triggering the test timeout. Fix it by resetting the time to the intended start position when transitioning items. PiperOrigin-RevId: 633918706 * Publish `CompositionPlayer` for playing compositions This class is not ready for production app usage yet, so it is still marked `@RestrictTo(LIBRARY_GROUP)` for now. Apps can experiment with it in a non-prod context by suppressing the associated lint error. * Issue: androidx/media#1014 * Issue: androidx/media#1185 * Issue: androidx/media#816 PiperOrigin-RevId: 633921353 * Support AVIF in exoplayer https://developer.android.com/media/platform/supported-formats#image-formats was updated to include AVIF support in API 34+, so <unknown commit> updated our associated Util's to reflect this. After that change, ExoPlayer's BitmapFactoryImageDecoder will be able to decode AVIF, but the player won't be able to detect or extract it. Add this support for completeness, so that ExoPlayer can continue to say it supports all formats in https://developer.android.com/media/platform/supported-formats#image-formats. PiperOrigin-RevId: 633956245 * Add debug trace logs for AudioMixer & AudioGraph events. PiperOrigin-RevId: 633973723 * Default to parse subtitles while extracting, instead of while rendering To override this change, and go back to parsing during rendering, apps must make two method calls: 1. `MediaSource.Factory.experimentalParseSubtitlesDuringExtraction(false)` 2. `TextRenderer.experimentalSetLegacyDecodingEnabled(true)` PiperOrigin-RevId: 634262798 * Update PgsParser.java Fix "subtitles rendered with border and no fill color" problem: https://github.com/moneytoo/Player/issues/413 because '0' means it's an index to the palette table, not an RGBA value of 0 * Format with google-java-format and add release note * Handle playToEndOfStream called before configuring the audio sink ExoPlayer sometimes calls AudioSink.playToEndOfStream before configuring the sink. Before this CL, the composition player was failing if this happened. PiperOrigin-RevId: 634306592 * Fix image not ignored for non-images in setImageDurationMs PiperOrigin-RevId: 634345071 * Use List in `createRenderers` for better readability Also Use `Iterables.toArray()` to void the confusing `List.toArray()` method call PiperOrigin-RevId: 634351844 * Check for .heif extension in File types inference PiperOrigin-RevId: 634409758 * Fix and/or bug in `XingSeeker` This was accidentally introduced in https://github.com/androidx/media/commit/4fde35c9cc03947aecd3f8713f633f3d4dcd6aa0 PiperOrigin-RevId: 634465380 * Add debug trace log for AssetLoader Renderer format events. PiperOrigin-RevId: 634474584 * Add toggle for DebugTraceUtil to Transformer demo. PiperOrigin-RevId: 634495944 * Throw errors from `WebvttPlaybackTest.stallUntilPlayerCondition` Before this change, if a playback error is thrown the test fails with a timeout and no additional info: ``` java.util.concurrent.TimeoutException at androidx.media3.exoplayer.e2etest.WebvttPlaybackTest.stallPlayerUntilCondition(WebvttPlaybackTest.java:361) ``` After this change, the test failure includes a much more useful stack trace, e.g. from https://github.com/androidx/media/commit/0352db9a375cff16b14e3e8a653e30dcfa589b74: ``` Caused by: java.lang.IllegalStateException: Legacy decoding is disabled, can't handle text/vtt samples (expected application/x-media3-cues). at androidx.media3.common.util.Assertions.checkState(Assertions.java:100) at androidx.media3.exoplayer.text.TextRenderer.assertLegacyDecodingEnabledIfRequired(TextRenderer.java:587) at androidx.media3.exoplayer.text.TextRenderer.onStreamChanged(TextRenderer.java:210) ``` PiperOrigin-RevId: 634672138 * Add missing check before calling discardBuffer The method is only allowed to be called on prepared items. This check was currently missing and also causing the corresponding test to be flaky in ExoPlayerTest. PiperOrigin-RevId: 634694077 * Fix flakiness in ExoPlayerTest The two affected tests where playing until a specific position to enable the player to read ahead. The method pauses at exactly the target position, but then has temporarily undetermined behavior because the playback thread uses player.getClock().onThreadBlocked() that lets the playback thread make progress in parallel to the test thread. The tests were flaky because they sometimes made so much progress that they ended playback before we could query the updated renderer state. This can be fixed by using run(player).untilBackgroundThreadCondition instead, which is guaranteed to be fully deterministic, but may not be able to stop at exactly the desired position (which we don't really need anyway for this test). PiperOrigin-RevId: 634699752 * Fix AV sync for sequences with audio track shorter than video For each item, AudioGraphInput now pads the input audio track with silence to the duration given in onMediaItemChanged. Possibly resolves Issue: androidx/media#921 . PiperOrigin-RevId: 634753721 * Clean up unused members in `XingFrame` PiperOrigin-RevId: 634817614 * Handle timeline updates where all periods in window have been replaced This case is most likely to happen when re-preparing a multi-period live stream after an error. The live timeline can easily move on to new periods in the meantime, creating this type of update. The behavior before this change has two bugs: - The player resolves the new start position to a subsequent period that existed in the old timeline, or ends playback if that cannot be found. The more useful behavior is to restart playback in the same live item if it still exists. - MaskingMediaSource creates a pending MaskingMediaPeriod using the old timeline and then attempts to create the real period from the updated source. This fails because MediaSource.createPeriod is called with a periodUid that does no longer exist at this point. We already have logic to not override the start position and need to extend this to also not prepare the real source. Issue: androidx/media#1329 PiperOrigin-RevId: 634833030 * Refactor pending clock sync logic in DashMediaSource This should be no-op overall and helps to disentangle the clock sync update from the state of the manifest. We currently check oldPeriodCount==0 to trigger the clock sync load, which only works because the manifest happens to be null whenever we need a new clock sync. We can decouple these concepts by directly checking whether we have an existing elapsedRealtimeOffsetMs. This also requires to set this value explicitly at the point where we consider it set to the local device clock fallback when the timing element load fails. PiperOrigin-RevId: 634844921 * Keep manifest in DashMediaSource after release This is a fix for the fix in https://github.com/androidx/media/commit/319854d624a95d3f71a26c96ecb4e5827d4a7f4e. The original fix did not reset the firstPeriodId to avoid any id clashes with future updates. This however only works under the assumption that the next manifest load at the next call to prepare() is exactly the same as the current manifest. This is not true unless the call happens very quickly (and may fail even then). Instead we should keep the existing manifest directly as a reference so we can use it to find the number of removed periods when we get a new manifest at the next call to prepare(). Issue: androidx/media#1329 PiperOrigin-RevId: 634853524 * Fix image seeking Queue image again after position reset, and reset timestamp iterator. PiperOrigin-RevId: 635049953 * Allow LoadControl.shouldContinueLoading accept playWhenReady as a parameter * Move parameters inside LoadControl and use it for shouldStartPlayback +additional formatting and Javadoc changes * Add debug trace for Muxer completely ending. PiperOrigin-RevId: 635459757 * Make flag for debug traces in logcat private and final. PiperOrigin-RevId: 635465538 * Fix issue links from `google/ExoPlayer` to `androidx/media` PiperOrigin-RevId: 635469173 * Add OptIn annotations to declarations in demo app files #minor-release PiperOrigin-RevId: 635469477 * Create equals method for gainmaps Gainmaps don't currently have an equals override, only reference equality is checked by Objects.equals(gainmap1, gainmap2). Create an equals method for gainmaps with the fields we care about to ensure we don't incur false positives in our equality checks. PiperOrigin-RevId: 635510451 * Work around SurfaceTexture implicit scale If MediaCodec allocates passes an image buffer with a cropped region, SurfaceTexture.getTransformMatrix will cut off 2 pixels from each dimensions. The resulting videos will appear a little stretched. This patch inspects the SurfaceTexture transform matrix, and guesses what the unscaled transform matrix should be. Behind experimentalAdjustSurfaceTextureTransformationMatrix flag PiperOrigin-RevId: 635721267 * Move bitmap coordinate flip out of fragment shader Fragment shaders in OpenGL ES shader language aren't guaranteed to support highp, required to correctly represent pixel coordinates inside large images (e.g. 1920x1080). This change moves coordinate mirroring for images out of fragment shader. Fixes http://Issue: androidx/media#1331 PiperOrigin-RevId: 635732208 * Fix javadoc formatting PiperOrigin-RevId: 635737466 * Move license to top for consistency PiperOrigin-RevId: 635742699 * Assert file exists before trying to re-decode for test assertions. PiperOrigin-RevId: 635748820 * MP3: Add test CBR sample with 'too small' `PCUT` frame This shows ExoPlayer currently wrongly reports the duration of this sample, because it assumes every frame is 32kbps (104 bytes) due to the `PCUT` frame immediately after the `Info` frame. A follow-up change will modify `Info` frame handling to resolve this issue. This sample was crafted using a hex editor to insert the additional `PCUT` frame (the pattern of `null` and `x` is taken from the sample file in Issue: androidx/media#1376, the header is modified to set the channel count to 1 to match the rest of the file), and then update the frame count and data size of the `Info` header to match. Issue: androidx/media#1376 PiperOrigin-RevId: 635772837 * Re-apply CEA-708 `rowLock/columnLock` fix This change was originally made in https://github.com/androidx/media/commit/6f8249184bd69b126c93479f918768b0cf2d0889 It was then accidentally lost in when `Cea708Parser` was merged back into `Cea708Decoder` in https://github.com/androidx/media/commit/51b4fa2cc83b60fcb313fd0e6afd2d45fe64e535. This is the only change made to the actual 'decoding' logic in `Cea708Parser` between it being split from `Cea708Decoder` and merged back in again, all the other changes in this period relate to the implementation of the `SubtitleParser` interface, so don't need to be preserved in `Cea708Decoder`: https://github.com/androidx/media/commits/51b4fa2cc83b60fcb313fd0e6afd2d45fe64e535/libraries/extractor/src/main/java/androidx/media3/extractor/text/cea/Cea708Parser.java `Cea608Parser` was also merged back into `Cea608Decoder` in https://github.com/androidx/media/commit/25498b151ba298ef359f245e2ed80718b4adf556 and so is vulnerable to the same risk of accidental loss of changes. To be sure, I also checked the history of this file: https://github.com/androidx/media/commits/25498b151ba298ef359f245e2ed80718b4adf556/libraries/extractor/src/main/java/androidx/media3/extractor/text/cea/Cea608Parser.java The only 'decoding logic' change there is https://github.com/androidx/media/commit/379cb3ba540d7044785f8355b2df9220e7f5eb8f, which was also lost in https://github.com/androidx/media/commit/25498b151ba298ef359f245e2ed80718b4adf556. I will send a separate change to resolve this. PiperOrigin-RevId: 635796696 * Re-apply CEA-608 `validDataChannelTimeoutMs` assertion This change was originally made in https://github.com/androidx/media/commit/379cb3ba540d7044785f8355b2df9220e7f5eb8f. It was then accidentally lost in when `Cea608Parser` was merged back into `Cea608Decoder` in https://github.com/androidx/media/commit/25498b151ba298ef359f245e2ed80718b4adf556. This was spotted when re-doing a similar lost change to `Cea708Decoder`, reported in https://github.com/androidx/media/pull/1315. See reasoning on https://github.com/androidx/media/commit/e2847b3b80112e7bce0e0706604bdee8126704e5 about why this is the only 'lost' CEA-608 change. PiperOrigin-RevId: 635803536 * Add new line between printing glsl source code in error PiperOrigin-RevId: 635812838 * Migrate debug trace logs to track generic events for Muxer. Track information is added to the details string where relevant. PiperOrigin-RevId: 635815866 * Remove VideoFrameReleaseControl setter from SinkProvider Move the parameter to the constructor instead. PiperOrigin-RevId: 636077477 * Set image duration on all media types This was previously only set on images because it was not ignored on other media types. This parameter was made no-op for non-images in https://github.com/androidx/media/commit/7b2a1b444312953f9518868f1cfc3e0b6c400d7b. PiperOrigin-RevId: 636078142 * Make getIconResIdForIconConstant public This allows controller apps to map these constants to suitable icons without creating a CommandButton instance first. PiperOrigin-RevId: 636096841 * Work around 1080p export failures on certain devices Fall back to using software decoder for 1920x1080 for certain devices. PiperOrigin-RevId: 636132298 * MP3: Derive duration and bitrate from frame count in `Info` header `Info` header is used for CBR files, but in some cases not **every** frame in these files is the same size. This change stops using the single frame after the `Info` frame as the 'template' (and assuming all subsequent frames are the same size/bitrate), and instead derives the bitrate from fields in the `Info` header. This works for files which are 'almost' constant bitrate, like the one in Issue: androidx/media#1376 where every frame is either 1044 or 1045 bytes except the one immediately after the `Info` frame which is 104 bytes (32kbps), resulting in a wildly incorrect duration calculation. PiperOrigin-RevId: 636151605 * Parse dashif:Laurl license url in mpd * Add test and formatting changes * Trigger silence generation when end of stream is encountered This change avoids a muxer deadlock when: 1. Sequence of items 2. First item has audio track that is shorter than video 3. Audio finishes, and muxer refuses to write more than 500ms of video consecutively. SequenceAssetLoader fails to progress to the second item. A muxer deadlock is possible when the audio of the first item finishes, audio end-of-stream is not propagated through AudioGraph, and muxer blocks video, preventing SequenceAssetLoader to move to the next item in sequence. By triggering silence generation early as soon as audio EOS is encountered, we ensure SequenceAssetLoader can progress to the next item. PiperOrigin-RevId: 636179966 * MP3: Make a defensive copy of the header in `XingFrame` constructor This is currently set from `Mp3Extractor.synchronizedHeader` which gets overwritten every time we read a new frame. It seems safer to make this defensive copy (and there will be at most one `XingFrame` instance per-playback, so this is not prohibitively expensive). PiperOrigin-RevId: 636181038 * Add DV profile 10 handling in getAlternativeCodecMimeType() method * Fragmented Mp4Muxer: add support to B-frame Muxing Add composition time offset parameter to TRUN box to support muxing of videos containing B-frames by FragmentedMp4Muxer. Update TRUN box version from 0 to 1 in order to manage signed composition time offset. PiperOrigin-RevId: 636426397 * Update session module registration PiperOrigin-RevId: 636482934 * Assert file size in E2E android tests that claim to complete. Remove redundant test logic to add file size to ExportResult because the file size is already added to export result as part of an export finishing. PiperOrigin-RevId: 636499236 * Add missing module registrations PiperOrigin-RevId: 636506860 * Ignore rowLock and numLock as define in CTA-708 spec. Update current row value when new line is added. * revert rowLock and colomnLock changes since it will be done separately * remove test comment * Add cea708Decoder test for setPenLocation command and newline handling * Rename and reshuffle tests * Add release note * Remove jetifier config from gradle config None of the AndroidX libs we depend on require jetification any more. Issue: androidx/media#1362 PiperOrigin-RevId: 636544337 * Add more details to thread assertion in onAudioCapabilitiesChanged This helps to debug issues reported in https://github.com/androidx/media/issues/1191 PiperOrigin-RevId: 636545970 * Migrate `buildConfig` from `properties.gradle` to `build.gradle` This is generated in response to a deprecation warning in AS: ``` The option setting 'android.defaults.buildfeatures.buildconfig=true' is deprecated. The current default is 'false'. It will be removed in version 9.0 of the Android Gradle plugin. You can resolve this warning in Android Studio via `Refactor` > `Migrate BuildConfig to Gradle Build Files` ``` PiperOrigin-RevId: 636546985 * Remove deprecated `setContentTypePredicate()` methods Use the suggested alternative on the respective `XXXDataSource.Factory` instead. PiperOrigin-RevId: 636560182 * Remove `OkHttpDataSource` constructors & `OkHttDataSourceFactory` Use `OkHttpDataSource.Factory` instead. PiperOrigin-RevId: 636585523 * Print underlying extractor name in `UnrecognizedInputFormatException` If subtitle-parsing-during-extraction is enabled (now defaults to on), the 'outer' extractor class name is often `SubtitleTranscodingExtractor`, leading to some slightly useless error messages like: `None of the available extractors (FragmentedMp4Extractor, Mp4Extractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, TsExtractor, MatroskaExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, AviExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor, SubtitleTranscodingExtractor)` PiperOrigin-RevId: 636834354 * Remove `PlayerMessage.setHandler(Handler)` Use `setLooper(Looper)` instead. PiperOrigin-RevId: 636840566 * Maintain a consistent luminance range across HDR content in effects PQ and HLG have different luminance ranges (max 10k nits and max 1k nits resp). In GL, colors work in a normalised 0 to 1 scale, so for PQ content, 1=10k nits and and for HLG content, 1=1k nits. This cl scales and normalises PQ content appropriately so that all HDR content works in the HLG luminance range. This fixes two things 1. Conversions between HLG and PQ are "fixed" (before the output colors looked too bright or too dark depending on which way you are converting) 2. color-altering effects will be able to work consistently across HLG and PQ content 1 is tested in this cl. 2 will be tested when ultra HDR overlays are implemented, both cases have been manually tested to ensure the output looks correct on a screen. PiperOrigin-RevId: 636851701 * Remove `ShadowLog` references from tests These were accidentally submitted after being added for local debugging. PiperOrigin-RevId: 636865825 * Remove deprecated `Timeline.Window.isLive` field PiperOrigin-RevId: 636870982 * Remove deprecated `DefaultHttpDataSource` constructors Use `DefaultHttpDataSource.Factory` instead. PiperOrigin-RevId: 636875524 * Remove deprecated `DashMediaSource.DEFAULT_LIVE_PRESENTATION_DELAY_MS` Use `DashMediaSource.DEFAULT_FALLBACK_TARGET_LIVE_OFFSET_MS` instead. PiperOrigin-RevId: 636906922 * Remove deprecated `MediaCodecInfo.isSeamlessAdaptationSupported(...)` Use `MediaCodecInfo.canReuseCodec(...)` instead. PiperOrigin-RevId: 636918479 * Remove deprecated `DrmSessionManager.DUMMY` and getter method Use `DRM_UNSUPPORTED` constant instead. PiperOrigin-RevId: 636937592 * UltraHdr: Fix calculations to apply gainmap PiperOrigin-RevId: 636964921 * Ensure single-frame videos are correctly exported Add a wait in DefaultCodec.signalEndOfInputStream when no video encoder output has been seen. This avoids a thread synchronization problem between writing frames to video surface, and signaling end of stream, which was hit for video input of only one frame on some devices. PiperOrigin-RevId: 637844690 * Remove deprecated format changed methods Use the overloads with an additional `@Nullable DecoderReuseEvaluation` parameter instead. PiperOrigin-RevId: 637851937 * Remove unused `Util.getCommaDelimitedSimpleClassNames` method PiperOrigin-RevId: 637854422 * Add support for ultra HDR overlays PiperOrigin-RevId: 637863706 * Reset input capacity when setting external shader program Before this CL, externalShaderProgramInputCapacity was not reset when the external shader program was reset (which occurs when the InputSwitcher switches to an input with a different ColorInfo). This is due to a regression introduced in https://github.com/androidx/media/commit/bef3d518d2dd11155a759d4e62bd649e32752a2d. PiperOrigin-RevId: 637869215 * Tighten exception handling in `TestPlayerRunHelper` This removes `throws Exception` from public methods in favour of more specific exception types (`TimeoutException` and `PlaybackException`). PiperOrigin-RevId: 637880546 * Remove deprecated `RendererSupport.FormatSupport` IntDef & constants Use `C.FormatSupport` and associated constants instead. PiperOrigin-RevId: 637890304 * Speed up image to video Export Only sample from input bitmap when the input image has changed. Introduce GainmapShaderProgram.newImmutableBitmap API that signals input bitmap changes to GainmapShaderProgram (DefaultShaderProgram). PiperOrigin-RevId: 637920207 * Add a flag to control whether input bitmap resampling can be skipped Add DefaultVideosFrameProcessor experimental flag that controls whether input Bitmaps are sampled once for a repeating sequence of output frames with the same contents, or once for each output frame. PiperOrigin-RevId: 637921350 * Add an experimental analyzer mode to Transformer. PiperOrigin-RevId: 637926059 * Import string translations for session module #minor-release PiperOrigin-RevId: 638224207 * Rename test so that it runs on MH PiperOrigin-RevId: 638237657 * Use file inserts and string replacements in overlay shaders PiperOrigin-RevId: 638251955 * Increase h.264 buffer size in `ShadowMediaCodecConfig` Some test media has samples larger than 100kB PiperOrigin-RevId: 638598553 * Rename `PreloadMediaSource.PreloadControl` methods The IntDefs in `DefaultPreloadManager.Stage` are also renamed accordingly. PiperOrigin-RevId: 638631357 * Suppress incorrect linter error in DebugRenderersFactory method Method should use @RequiresApi annotation instead of @SdkSuppress PiperOrigin-RevId: 638639225 * Regenerate TransformerHdrTest goldens after removing degammaing Degammaing has been removed in https://github.com/androidx/media/commit/cb4b2ea55c70efa80adc3e664301b1990e89eaf3. The goldens for TransformerHdrTest (previously TransformerSequenceEffectTestWithHdr) were not regenerated because the test wasn't running due to its name (fixed in https://github.com/androidx/media/commit/e41a966237528148590712048dc735dabfd344ca). PiperOrigin-RevId: 638645635 * Remove unnecessary sortKey from Mp4Muxer.addTrack() PiperOrigin-RevId: 638647112 * Use INDEX_UNSET instead of LENGTH_UNSET for next media sequence/part Both constants have the same value, but the method returning initial values for the media sequence/part uses `INDEX_UNSET`, so it makes sense to use it. PiperOrigin-RevId: 638673282 * Schedule exoplayer work task to when renderers can make progress Currently ExoPlayer schedules its main work loop on a 10 ms interval. When renderers cannot make any more progress(ex: hardware buffers are fully written with audio data), ExoPlayer should be able to schedule the next work task further than 10Ms out. Through `experimentalSetDynamicSchedulingEnabled`, ExoPlayer will dynamically schedule its work tasks based on when renderers are expected to be able to make progress. PiperOrigin-RevId: 638676318 * Call PreloadControl.onSourcePrepared only once for each preload request PiperOrigin-RevId: 638677090 * Schedule exoplayer work to when MediaCodecAudioRenderer can progress Currently ExoPlayer schedules its main work loop on a 10 ms interval. When renderers cannot make any more progress (ex: hardware buffers are fully written with audio data), ExoPlayer should be able to schedule the next work task further than 10ms out into the future. Through `experimentalSetDynamicSchedulingEnabled` and these changes to `MediaCodecAudioRenderer`, ExoPlayer can use the data provided by the audio renderer to dynamically schedule its work tasks based on when it is expected that progress can be made. PiperOrigin-RevId: 638677454 * Add references in javadocs to relevant listeners for Player fields PiperOrigin-RevId: 638688864 * Use `kotlinx-coroutines-guava` in session demo app I originally tried switching to `Futures.addCallback` (as a follow-up to Issue: androidx/media#890), but it seemed like a good chance to go further into Kotlin-ification. Before this change, if the connection to the session failed, the app would hang at the 'waiting' screen with nothing logged (and the music keeps playing). This behaviour is maintained with the `try/catch` around the `.await()` call (with additional logging). Without this, the failed connection causes the `PlayerActivity` to crash and the music in the background stops. The `try/catch` is used to flag to developers who might be using this app as an example that connecting to the session may fail, and they may want to handle that. This change also switches `this.controller` to be `lateinit` instead of nullable. Issue: androidx/media#890 PiperOrigin-RevId: 638948568 * Schedule doSomeWork when MediaCodec signals available buffers When running in asynchronous mode, MediaCodec will be running the CPU to signal input and output buffers being made available for use by the player. With ExoPlayer.experimentalSetDynamicSchedulingEnabled set to true, ExoPlayer will wakeup to make rendering progress when MediaCodec raises these signals. In this way, ExoPlayer work will align more closely with CPU wake-cycles. PiperOrigin-RevId: 638962108 * Allow `ByteArrayDataSource` to resolve the byte array when opened This is a relatively small change, and massively simplifies the work needed for an app to consume Kotlin Multiplatform resources (without a full `KmpResourceDataSource` implementation, which poses some dependency challenges for now). Issue: androidx/media#1405 PiperOrigin-RevId: 638991375 * Fix a race condition in AudioGraphInput AudioGraphInput.onMediaItemChanged is called on input thread. Pending media item changes are processed on processing thread, inside calls to getOutput(). This change allows multiple pending media item changes to be enqueued, and processed in sequence. PiperOrigin-RevId: 638995291 * Add `FOREGROUND_SERVICE_MEDIA_PLAYBACK` permission for test session app Foreground service type `mediaPlayback` requires permission `android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK`. The `MockMediaSessionService`, `LocalMockMediaSessionService`, `MockMediaLibraryService` and `LocalMockMediaLibraryService` declared in the manifest are in the `mediaPlayback` type. PiperOrigin-RevId: 639013810 * Move effect final values and preset names and paths to arrays file. PiperOrigin-RevId: 639034630 * Fix CMCD data assignment for init segment The CMCD data was incorrectly added to the `dataSpec` of the media segment instead of the init segment. Also relaxed the condition for playbackRate to be C.RATE_UNSET when creating an instance of CmcdData.Factory as there was nothing enforcing this check. #minor-release PiperOrigin-RevId: 639046080 * Drop API requirement from SeparableConvolution Switch from 4-channel RGBA_16F lookup texture to 1-channel R_16F. Do not use a bitmap when creating the lookup table texture. Instead, fill the texture directly. Do not manually convert 32-bit float to 16-bit. Instead, let OpenGL libraries do this for us. PiperOrigin-RevId: 639717235 * Make progress tests stricter. These changes are possible because getProgress is no longer a blocking operation on transformer. * Tests call getProgress after every looper message executed. * Use longer media assets for getProgress tests to give more progress intervals. * Remove conditional assertions. PiperOrigin-RevId: 639734368 * Remove module registration from HttpEngineDataSource It used to be its own module in the first revision, but then moved into the DataSource module as of https://github.com/androidx/media/commit/250fc80419b4b8ef1b753824e53ebdac176968f5. PiperOrigin-RevId: 639745498 * Add an analyzer mode option to Transformer demo. PiperOrigin-RevId: 639746452 * Fix the RELEASENOTES.md This [commit](https://github.com/androidx/media/commit/d175223cc632f9a927a0cfe62bc808d336a8c83a) was introduced not long ago (after 1.4.0-alpha01), but has the release note added to the alpha01 release. Moving this piece of note to "unreleased changes" so that it will be included in the 1.4.0-alpha02 release note as a new change. PiperOrigin-RevId: 639768495 * Move CEA-708 release note from 1.3.0 to 'unreleased' (1.4.0) This was added in the wrong place as part of https://github.com/androidx/media/commit/0a58832d85a51a8ee8f50144c25d92ab8d8c0e78 PiperOrigin-RevId: 639783174 * Allow customisation of various icons in PlayerControlView Before this change: The only way to customize the icons was to override the drawables, e.g. * `exo_styled_controls_play` * `exo_styled_controls_pause` However, that would set the drawables globally and prevent users from customizing the icons **per** PlayerView. After the change, it is possible to provide drawable icons in the xml layout directly via `<androidx.media3.ui.PlayerView>` and * `app:play_icon="@drawable/...` * `app:pause_icon="@drawable/...` * `app:vr_icon="@drawable/...` * `app:fullscreen_exit_icon="@drawable/...` * `app:next_icon="@drawable/...` Note: Two buttons that are left out of this change are fast-forward and rewind. They are more complicated due to layout insertion and customization with seek back/forward increments in the TextView. Issue: androidx/media#1200 PiperOrigin-RevId: 639832741 * Support Separable Convolutions with negative coefficients fragment_shader_separable_convolution_es2.glsl had optimizations that assumed all convolution coefficients are positive. Support negative coefficients, and add tests. PiperOrigin-RevId: 640104741 * Add isReleased() to the Exoplayer This avoids having to add AnalyticsListener and catching the `onPlayerReleased` callback. The `Exoplayer.release()` method is blocking and one can be sure that the player is released if the call returned. However, the method is useful for UI testing and asserting that the player is released after a certain UI action, e.g. closing the activity or detaching a window. PiperOrigin-RevId: 640114416 * Skip AV1 test on devices that don't support AV1 HDR encoding Skip AV1 HDR test on some devices. PiperOrigin-RevId: 640135455 * Update release notes for 1.4.0-alpha02 PiperOrigin-RevId: 640138307 * Version bump to media3:1.4.0-alpha02 #minor-release PiperOrigin-RevId: 640196724 * Add icons for fastforward and rewind buttons PiperOrigin-RevId: 640206040 * Decide whether to use the VideoSink when enabling renderer This will simplify moving the release control inside the video sink PiperOrigin-RevId: 640416128 * Add LanczosResample effect to scale videos in Transformer Add SeparableConvolution.configure(inputSize) to allow effect configuration depending on input dimensions. Add LanczosResample.scaleToFit method to scale input images to fit inside given dimensions. PiperOrigin-RevId: 640498008 * MCVR: use shouldUseVideoSink where possible Before this CL, we were checking whether the video sink was initialized to determine whether it should be used. In the meantime, shouldUseVideoSink has been introduced. Use this boolean to check whether to use the video sink as it's a clearer signal. PiperOrigin-RevId: 640499147 * Update FILE_UNKNOWN_DURATION to a longer fmp4 (~15s), Removes the flakiness of MediaItemExportTest.getProgress_unknownDuration_returnsConsistentStates by using a longer input asset, such that ExoPlayer does not determine the duration of the media. PiperOrigin-RevId: 640502470 * Remove @UnknownInitialization DefaultCodec parameter Using an uninitialised object seems unusual. PiperOrigin-RevId: 640514501 * Move renderer state methods from release control to sink PiperOrigin-RevId: 640515298 * Remove direct usages of release control when video sink is used Usages removed in this CL are: - onProcessedStreamChange, which was already called from the VideoSink (via VideoFrameRenderControl) - setOutputSurface, which was also already called from the VideoSink - setFrameRate, which this CL now sets in the VideoSink PiperOrigin-RevId: 640530903 * Fix invalid state transitions for trim optimization getProgress. Ensures valid progress state is returned. Should not return NOT_STARTED once transformer.start has been called, until export ends. PiperOrigin-RevId: 640533805 * Refactor trim optimization getProgress for improved readability. This change is a no-op refactor to improve the readability of the states returned in this method. PiperOrigin-RevId: 640538374 * Remove unnecessary `throws Exception` from `HlsChunkSourceTest` PiperOrigin-RevId: 640552488 * Add missing null check PiperOrigin-RevId: 640555113 * Exit early if buffer becomes invalid When the frame release control invalidates a buffer and returns that the buffer must be ignored, we need to exit early before performing additional checks that may result in method calls using the invalid buffer. PiperOrigin-RevId: 640555688 * MuxerWrapper: Fix spelling mistake in method name PiperOrigin-RevId: 640838741 * Stabilize offload related error codes PiperOrigin-RevId: 640839273 * Add CodecDetails to ExportException This will replace the existing free-form string in the error message PiperOrigin-RevId: 640954158 * Unify timestamp handling Before this change, the timestamps output from composition playback is offset with the renderer offset. After this change, the offset is removed and the timestamp behaviour converges with Transformer, that is, the timestamps of video/images frames will follow that of the composition. For example, with a composition of two 10-s items, clipping the first with 2s at the start, the timestamp of the first frame in the second item, will be 8s. PiperOrigin-RevId: 641121358 * Fix a concurrency issue that shader capacity is set off GL thread Also add test to cover transitions between BT709 and 601. PiperOrigin-RevId: 641224971 * Fix timestamps in tests https://github.com/androidx/media/commit/38a7229d9684970f3475eff203255b7d1c8cf747 changed only some of the timestamps, in fact all of the timestamps should have the offset removed. PiperOrigin-RevId: 641226102 * Fix test names PiperOrigin-RevId: 641249128 * Mark `PlayerWrapper` as `final` This package-private class isn't currently extended, so this change makes it not possible to be extended in future. PiperOrigin-RevId: 641270464 * Change externalShaderProgramInputCapacity to int The field is only accessed on the GL thread. PiperOrigin-RevId: 641273595 * Add `PlayerSurface` Compose component Underneath, it delegates to either * `AndroidExternalSurface` (equivalent of SurfaceView), which is generally better for power and latency * `AndroidEmbeddedExternalSurface` (equivalent of TextureView) which is better for interactions with other widgets or applying visual effects through the graphicsLayer Note: the resulting surface is stretched across the whole screen. Aspect ratio handling will be addressed in the follow-up changes. PiperOrigin-RevId: 641285482 * Add support for setting and getting volume for composition preview. PiperOrigin-RevId: 641822822 * Remember Player instance on recompositions PiperOrigin-RevId: 641823733 * Remove incorrect 'bundleable' reference from `stringMapToBundle` param This was accidentally copy-pasted in https://github.com/androidx/media/commit/5008417c8cd12680a3945945e00399cca05915c5 PiperOrigin-RevId: 641823991 * Set VideoSink listener when enabling MCVR Before, the listener was set in onReadyToInitializeCodec, which means that it was reset every time a new codec was used. We need to set the listener every time MCVR is enabled (not only the first time), because it might have been set by another renderer. PiperOrigin-RevId: 641825717 * Add Transformer HDR-only assets sequence effect tests. PiperOrigin-RevId: 641828107 * Rename "preset file" to "preset" in composition demo. This change is made for consistency with transformer demo. PiperOrigin-RevId: 641835714 * HDR-only assets sequence effect test: rename golden files rename to match test name PiperOrigin-RevId: 641836522 * Support upscaling with LanczosResample Avoid scaling the lanczos windown function when scaling up. When scaling up (output > input), we shouldn't scale the window function or we risk not sampling any of the input pixels. PiperOrigin-RevId: 641838149 * Call VideoFrameReleaseControl.setPlaybackSpeed from sink when enabled Do not call VideoFrameReleaseControl.setPlaybackSpeed directly from MCVR when the video sink is enabled. PiperOrigin-RevId: 641840894 * Clarify docs on `ExoPlayer.setVideoEffects()` re calling `prepare()` The previous wording suggested that `setVideoEffects()` may **only** be called before `prepare()`, i.e. the effect cannot be changed during playback. The intent is instead that `setVideoEffects()` must be called once before playback in order to configure the effects pipeline, but the effect can then be changed during playback by further calls to `setVideoEffects()`. Issue: androidx/media#1393 PiperOrigin-RevId: 641853629 * Remove most nullness marking from Transformer demos The nullness checking is very verbose and redundant in some places, for example, the ensures-non-null marking in the transformer activity, and it makes changing features in the demo app more time consuming. The cost/benefit balance seems to be in favor of removing this for demo code. Note: the ExoPlayer main demo has nullness checks turned off. PiperOrigin-RevId: 641890618 * Tidy passing views in transformer demos PiperOrigin-RevId: 641891515 * Rename "preset file" to just "preset" in transformer demo The new name means we can add streams and other sources that aren't files in future, for example, screen recording input. PiperOrigin-RevId: 641894319 * Fix an invalid javadoc link PiperOrigin-RevId: 641895590 * Create explicit key for testing 'notification controller' Previously these tests were setting the test-only `KEY_CONTROLLER` bundle value to the prod value of `MediaController.KEY_MEDIA_NOTIFICATION_CONTROLLER_FLAG`, which is a bit confusing because this is intended to be used as a `Bundle` **key** (for a boolean value), and not a value itself. Instead this CL creates a test-only value that can be used by `RemoteMediaSession` and related test-only friends to indicate that something (e.g. error or extras) should only be set on the notification controller. This CL also changes the logic in `MediaSessionProviderService` to avoid needing to special-case checking for this controller key. PiperOrigin-RevId: 641939283 * Add helper function in Track.java to copy track without edit lists. PiperOrigin-RevId: 642038117 * Add MediaSession.Callback.onPlayerInteractionFinished This callback is useful for advanced use cases that care about which Player calls are batched together. It can be implemented by triggering this new callback every time a batch of Player interactions from a controller finished executing. PiperOrigin-RevId: 642189491 * Add stress test for image transform PiperOrigin-RevId: 642213253 * Reduce flakiness of getProgress tests around percentage values. There is no requirement for the first progress value to be 0, if progress is made instantly. PiperOrigin-RevId: 642245457 * Add SessionError and use it in service results This change adds `SessionError` and uses it in `SessionResult` and `LibraryResult` to report errors to callers. Constructors and factory method that used a simple `errorCode` to construct error variants of `SessionResult` and `LibraryResult` have been overloaded with a variant that uses a `SessionError` instead. While these methods and constructors are supposed to be deprecated, they aren't yet deprecated until the newly added alternative is stabilized. PiperOrigin-RevId: 642254336 * Remove `Bundleable` type & `Bundleable.Creator<Foo> CREATOR` fields This interface is not used in the library. Callers can use the `Bundle toBundle()` and `static Foo fromBundle(Bundle)` methods defined directly on each type instead. PiperOrigin-RevId: 642271609 * Increase sleep duration when video encoder ends with no output frames This duration was not sufficient on a Pixel 3a PiperOrigin-RevId: 642276212 * Publish the media3 session controller test app Issue: androidx/media#78 PiperOrigin-RevId: 642277900 * Add `@CallSuper` to `TrackSelectionParameters.toBundle` PiperOrigin-RevId: 642292328 * Clean up unecessary zero-arg `toBundle()` methods These are no longer needed now that the `Bundleable` interface has been removed. Public methods are deprecated, package-private ones are removed. Callers are migrated in both cases (except where tests explicitly exist for the deprecated method). PiperOrigin-RevId: 642294451 * Propagate media3 session extras to media1 `PlaybackStateCompat` PiperOrigin-RevId: 642299178 * Update `@UnstableApi` docs with a link to DAC The code snippets in javadoc don't render correctly due to https://issuetracker.google.com/342557694 PiperOrigin-RevId: 642309632 * Make ExportResult.processedInputs package private Rational for keeping field package private: Since the overall export already very complex, by looking at the output file its hard to know if the desired processing happened or not. Since we now support sequences with "n" media items, its even more important to know if all the media items were processed or not. Although the field exposes implementation details, it seems ok as we get benefit of detailed testing. PiperOrigin-RevId: 642337888 * Pass clock to release control from sink provider In order to do that, make the VideoSink nullable in MCVR. We want to avoid calling VideoFrameReleaseControl.setClock directly from MCVR when the sink is enabled. The goal is to handle all the communication with the release control from the sink/sink provider. PiperOrigin-RevId: 642542063 * Remove useHdr from overlaySettings useHdr is unused option and doesn't make sense is the dynamic range of the overlay and video must match Also reorders and adds javadoc in line with coding conventions PiperOrigin-RevId: 642555396 * Make ImageOutput clearable It's currently not possible to remove a previously set image output on ExoPlayer, although the underlying renderer already supports receiving null to clear the output. Marking the parameter as nullable allows apps to clear it as well. PiperOrigin-RevId: 642569081 * Suppress warning about unused return value The return value is intentionally unused because the check is only done to see if the reflection operation succeeds. PiperOrigin-RevId: 642579373 * Fix minor timestamp handling issue - Video release should check for buffer timestamp (which is renderer-offsetted), rather than the frame timestamp - ImageRenderer should report ended after all of it's outputs are released, rather than when finished consuming its input. Add tests for timestamp handling PiperOrigin-RevId: 642587290 * Sync and map fatal and non-fatal errors from and to the legacy session A fatal `PlaybackException` is mapped to a legacy playback state in state `STATE_ERROR` with error code, message and extras. A non-fatal error sent to controllers with `MediaSession.sendError` is synced to the legacy session by setting error code and message and merging the extras while preserving the rest of the state in sync with the session player. Vice versa, a `MediaController` connected to a legacy session receives fatal errors through `Player.onPlayerErrorChanged()` and non-fatal errors through `MediaController.Listener.onError()`. Error codes are mapped in `LegacyConversions`. Values of error codes in `@SessionError.ErrorCode` come from `@PlaybackExceptino.ErrorCode` with the exception of `@SessionError.ERROR_IO` and `@SessionError.ERROR_UNKNOWN`. These already exist in `@PlaybackException.ErrorCode` and are mapped accordingly to avoid semantic duplicates. PiperOrigin-RevId: 642595517 * Refactors the seek test and add more test cases PiperOrigin-RevId: 642597298 * Make nit improvements in muxer code PiperOrigin-RevId: 642599475 * Move usages of allowReleaseFirstFrameBeforeStarted inside sink PiperOrigin-RevId: 642611061 * Add image samples and track selection to demo app Also move the track selection header strings to the demo app as they are only used there (except for audio, which stays in UI) PiperOrigin-RevId: 642616037 * Clarify semantics of MSG_SET_IMAGE_OUTPUT The value can be null, which isn't mentioned in the docs yet PiperOrigin-RevId: 642622583 * Split muxer android tests between parameterized and non parameterized This will be helpful for adding more tests which need not to be parameterized. PiperOrigin-RevId: 642638702 * Fix single sample handling ProgressiveMediaPeriod when disabling tracks When deselecting the single sample track and later re-selecting this track, the current shortcuts in ProgressiveMediaPeriod don't handle this case correctly and cause assertion failures. In particular, this change fixes 3 issues: 1. When re-selecting the single sample track, we have cleared the SampleQueue and need to reload the sample. The existing shortcut should only be applied to avoid the reload when starting from a non-zero position. 2. When de-selecting the track, ProgressiveMediaPeriod is left in an inconsistent state where the sample queues are empty but loadingFinished is still true. Fix this by resetting loadingFinished to false. 3. When seeking, we avoid reloading the stream if we can keep inside the existing samples. This logic assumes that all remaining samples will continue to be loaded in the queue. This condition isn't true though for single sample tracks that have been de-selected. They appear to support the seek inside the queue (=no seek necessary, always supported), but still require a new load if there is no ongoing one to load the sample. Fix this by checking this implicit assumption (still loading, or loading finished). PiperOrigin-RevId: 642650248 * Schedule refresh for all the playing playlists for HLS live stream Issue: androidx/media#1240 PiperOrigin-RevId: 642927082 * Add a checkbox to use Media3 muxer in Transformer demo app PiperOrigin-RevId: 642941439 * Fix bug where enabling CMCD for HLS live streams causes error Determine `nextMediaSequence` and `nextPartIndex` based on the last `SegmentBaseHolder` instance, as it can update `mediaSequence` and `partIndex` depending on whether the HLS playlist has trailing parts or not. Issue: androidx/media#1395 PiperOrigin-RevId: 642961141 * Fix AVI extractor WAVE format extraction Allow `WAVEFORMAT` (in addition to `WAVEFORMATEX`), which omits initialization data. Fix reading of bits per sample to use little endian byte order like the other reads. See also the wave format docs linked from https://learn.microsoft.com/en-us/windows/win32/directshow/avi-riff-file-reference#avi-stream-headers. PiperOrigin-RevId: 642962893 * Effect: Delay end of stream until all frames are rendered. Previously, if renderFramesAutomatically = false, DefaultVideoFrameProcessor may call onInputStreamProcessedListener after all frames have been onOutputFrameAvailableForRendering, but before they had all been rendered and freed. Delay onInputStreamProcessedListener being called and subsequent DVFP reconfiguration of effects, until all frames are rendered. Tested using exoplayer setVideoEffects demo with playlist PiperOrigin-RevId: 642963100 * Remove the code to restore current output position Muxer never uses latest output position but always writes to a specific location, so restoring its position does not add any value. PiperOrigin-RevId: 642996941 * Refactor mixedInput E2E test to use parameterized logic. This CL is a pure refactor of the existing tests, iterative changes will be done in follow-ups. The only logic change is having all MP4 assets have the presentaton of height=360. The old code had some with height 480. PiperOrigin-RevId: 643020773 * Add a drop-in replacement for `MediaExtractor` in Media3 This change introduces a new class in Media3 `MediaExtractorCompat`, designed to be a drop-in replacement for platform `MediaExtractor`. While not all APIs are currently supported, the core functionality for the most common use cases of `MediaExtractor` is now available. Full API compatibility will be achieved in the future. PiperOrigin-RevId: 643045429 * SpeedChangingAP: synchronize fields accessed from multiple threads PiperOrigin-RevId: 643046385 * support hdr text overlays adds luminance multiplier to allow the luminance (i.e. brightness) over a text overlay to be scaled PiperOrigin-RevId: 643047928 * Parse 'max num reorder samples' values from h.264 and h.265 videos This value is used in a follow-up change to re-order SEI messages containing CEA-6/708 data from decode order to presentation order. PiperOrigin-RevId: 643296338 * Remove PreloadConfiguration from public API This feature isn't completed, so we should remove the public facing API to avoid confusion. #minor-release PiperOrigin-RevId: 643318692 * Image support in PlayerView Images are rendered into an ImageView (on top of the video shutter). The image view is set to the images emitted by ExoPlayer's ImageOutput and cleared when there is no longer a selected image track. In order to keep the existing behavior of video tracks to only clear the old output once the new first frame is rendered (avoiding short periods of black between playlist items), we have to reorder this code slightly to make it work for video and images. Both are treated in the same way. If both are enabled, video takes precedence. As the UI module only depends on the common module, we can't direcly add the ImageOutput to ExoPlayer. This is done via reflection if the provided Player is an ExoPlayer. #cherrypick PiperOrigin-RevId: 643320666 * Rollback of https://github.com/androidx/media/commit/cd9b914c423900edcfe1e64bfb775a601bc23844 PiperOrigin-RevId: 643329324 * Add `demo-composition` to `settings.gradle` This was missed as part of https://github.com/androidx/media/commit/0e5a5e029455ea5e456cbb6028e9be1e87cdfe09 #cherrypick PiperOrigin-RevId: 643331307 * Add more cases for parameterized video android tests. PiperOrigin-RevId: 643346610 * Write moov box at the start of the file if possible This is done by reserving some space for moov box at the start of the file and writing it there if it fits. If it doesn't fit, then it is written at the end of the file. PiperOrigin-RevId: 643944698 * Reset release control from video sink when enabled PiperOrigin-RevId: 643950097 * Merge pull request #1437 from MGaetan89:add_exoplayer_setMaxSeekToPreviousPosition PiperOrigin-RevId: 643987403 (cherry picked from commit 67a7b41fa7453ac4fa06d4d9c7df68e72fbc58ea) * Fix linter errors in `DemoMediaLibrarySessionCallback.kt` PiperOrigin-RevId: 644002147 (cherry picked from commit ed07ac5d7de31a610b63d51242fc383d4bad9ffe) * Update release notes for 1.4.0-beta01 #cherrypick PiperOrigin-RevId: 644351958 (cherry picked from commit 794731607d2fd267d45f8f587812d11e655e59df) * Version bump to media3:1.4.0-beta01 PiperOrigin-RevId: 644352776 (cherry picked from commit c07bbd333ca88ec93a9efdfcc037e406e1aa4099) * Improve audio focus handling tests with ExoPlayer There are a lot of tests for AudioFocusManager in isolation, but almost none for the handling in ExoPlayer. Add test coverage for all the common cases, including some currently broken behavior that is indicated by TODOs. PiperOrigin-RevId: 644319251 (cherry picked from commit e20e94fde277a8e9e9a09e502a5760772e9eb006) * Audio focus player command clean up The 'player commands' returned to ExoPlayerImpl instruct the player on how to treat the current audio focus state. The current return value when playWhenReady==false is misleading because it implies we are definitely not allowed to play as if we've lost focus. Instead, we should return the actual player command corresponding to the focus state we are in. This has no practical effect in ExoPlayerImpl as we already ignore the 'player command' completely when playWhenReady=false. To facilitate this change, we also introduce a new internal state for FOCUS_NOT_REQUESTED to distinguish it from the state in which we lost focus. #cherrypick PiperOrigin-RevId: 644416586 (cherry picked from commit 66c19390e2b31106a2a467366f7945c37a273fc6) * Allow session activity to be set per controller PiperOrigin-RevId: 644693533 (cherry picked from commit 856d394c281cabc2b1932b003d752631549e7231) * Use a localized fallback message for known error codes In the case of a legacy session app providing an error code different to `ERROR_UNKNOWN` without an error message, a localized fallback message is provided instead of ignoring the error. #cherrypick PiperOrigin-RevId: 644704680 (cherry picked from commit 6cc6444dd9be34ea3a40d2bb296247178ee86b1d) * Use `SurfaceSyncGroup` to ensure resize transaction isn't dropped This workaround is only applied on API 34, because the problem isn't present on API 33 and it is fixed in the platform for API 35 onwards. Issue: androidx/media#1237 #cherrypick PiperOrigin-RevId: 644729909 (cherry picked from commit 968f72fec6d5452a34976824ac822782eeeb8f45) * Move playWhenReadyChangeReason inside PlaybackInfo This helps to keep the reason always together with the state it is referring to, avoiding any side channels and making sure there are no accidental inconsistencies. #cherrypick PiperOrigin-RevId: 644969317 (cherry picked from commit c0abd6f91ec62bc5347561c3f3174b3a660a7ba6) * Clarify that onPlayWhenReadyChanged can be called again with new reason Sometimes the reason for the current state may change. If we don't report this again, users have no way of knowing that the reason changed. Also adjust ExoPlayerImpl and MediaControllerImplBase accordingly. SimpleBasePlayer already adheres to this logic. #cherrypick PiperOrigin-RevId: 644970236 (cherry picked from commit 1d26d1891e6e3cd49a4b479723b8aba7be746c1c) * Fix audio focus handling in ExoPlayerImpl Some cases are not handled correctly at the moment: - Pausing during suppressed playback should not clear the suppression state. - Transient focus loss while paused should be reported as a playback suppression. Issue: androidx/media#1436 #cherrypick PiperOrigin-RevId: 644971218 (cherry picked from commit e84bb0d21cd114c5b726c6c3d799b194df068a14) * Remove direct `AspectRatioFrameLayout` usage from session demo This class is a lower-level UI component that isn't directly needed if apps are using `PlayerView` to handle their video output (it is used as an implementation detail of `PlayerView`). Removing its unecessary usages from this demo avoids developers copying this as an example when building their own apps. #cherrypick PiperOrigin-RevId: 644972454 (cherry picked from commit cb8f87e05e73d72ce7d2c405c4c5899dc31ade5e) * Add null-check to `PlayerView` to avoid NPE in edit mode Previously we assumed that `surfaceSyncGroupV34` was always non-null on API 34, but this isn't true in edit mode. Instead we add an explicit null-check before accessing it. We don't need to null-check it at the other usage site because we are already null-checking `surfaceView` (via `instanceof` check). Issue: androidx/media#1237 #cherrypick PiperOrigin-RevId: 645008049 (cherry picked from commit 99803066ea6d375f5527e0953aad4ecd91fbac3c) * Use HttpEngineDataSource when supported in demo app. HttpEngineDataSource is the recommended HttpDataSource when it is available. It uses the [HttpEngine](https://developer.android.com/reference/android/net/http/HttpEngine). #cherrypick PiperOrigin-RevId: 646051562 (cherry picked from commit e591c37b1e60d5138a612040efba752b7b7ab8e5) * Fix flakiness in MediaCodecVideoRendererTest The test is flaky because the decoding process in the renderer depends on some timing from MediaCodec beyond our control and the new keyframe added in the test is sometimes 'dropped' when it arrives too late. We can fix this by controlling the test progress a bit more tightly: first rendering with the same current time until the key frame is processed and then start increasing the time until we've reached the end. #cherrypick PiperOrigin-RevId: 646064352 (cherry picked from commit ada4dc982fac32a57db069341f1f51c6a3bb6cc5) * In DemoUtil, don't set cookie handler when using HttpEngineDataSource. HttpEngine does not support cookie storage. #cherrypick PiperOrigin-RevId: 646084702 (cherry picked from commit bb568b5150781457fe22c3d4a877e8c99bddc4a2) * Add fail-early checks for TrackSelectorResult correctness The two arrays need to have the same length and the selection must match in their nullness (unless for TYPE_NONE renderers). Clarify this more clearly in the docs and add new asssertions for it. This avoids that the player is failing in obscure ways much later. Issue: androidx/media#1473 #cherrypick PiperOrigin-RevId: 646086833 (cherry picked from commit 71ef848ec3fa4409f3cdf797226c2a51ecd15da6) * Merge pull request #1416 from khouzam:customFormat PiperOrigin-RevId: 646121082 (cherry picked from commit b026271c8438f227f4e15e0eaedd7223b4492037) * Merge pull request #1479 from dryganets:sdryanets/fix-handler-usage PiperOrigin-RevId: 646402268 (cherry picked from commit 04667284975e9fc71c137b4a8c133ca5cc626c19) * Add guard against additional tracks reported by Extractors Extractors should not report additional tracks once they called ExtractorOutput.endTracks. This causes thread safety issues in ProgressiveMediaPeriod where the array of sample queues is extended while the playback thread accesses the arrays. Detecting this problem early is beneficial to avoid unexplained exceptions later one. In most cases where this may happen (namely TS extractors finding new tracks), it's better to ignore the new tracks instead of failing completely. So this change adds a warning log message and assigns a placeholder output. Note: The same workaround already exists in HlsSampleStreamWrapper and MediaExtractorCompat. Issue: androidx/media#1476 #cherrypick PiperOrigin-RevId: 646427213 (cherry picked from commit 18e631ff790e0e9b3630ea8e36efb468474d0877) * Rename DummyTrackOutput and DummyExtractorOutput #cherrypick PiperOrigin-RevId: 646434450 (cherry picked from commit 867410fece006122820e26be4e0bea87309a4f2e) * Use `MediaCodec.stop()` before `release()` for surface switching bug ExoPlayer used to call `stop()` before `release()`. This was removed in <unknown commit>. A framework bug introduced in Android 11 (API 30) resulted in some DRM -> clear transitions failing during `MediaCodec.configure()`. An investigation in Issue: google/ExoPlayer#8696 and b/191966399 identified that this was due to `release()` returning 'too early' and the subsequent `configure()` call was then trying to re-use a `Surface` that hadn't been fully detached from the previous codec. This was fixed in Android 13 (API 33) with http://r.android.com/2094347. ExoPlayer worked around the framework bug by adding an arbitrary 50ms sleep after a failed codec initialization, followed by retrying. This was enough to resolve the problem in the test scenario on a OnePlus AC2003. Issue: androidx/media#1497 points out that 50ms might not be the appropriate delay for all devices, so it's an incomplete fix. They suggested re-adding the `MediaCodec.stop()` call instead. This also reliably resolves the issue on the OnePlus AC2003 (with neither workaround in place, the problem repros almost immediately). PiperOrigin-RevId: 646461943 (cherry picked from commit 5fcc7433a1f188c9644d4430a828a49d06c993cb) * Use `removeKey` method instead of setting `null` for KEY_CODECS_STRING Setting a `null` value doesn't remove the key as expected per the `MediaFormat` API documentation, using the `removeKey` method instead which is only available starting API level 29. PiperOrigin-RevId: 646462402 (cherry picked from commit 12c42585d25d4c51bc5d022aa282d4d36b8e1ff2) * Merge pull request #1487 from colinkho:main PiperOrigin-RevId: 646917527 (cherry picked from commit 6244d8605f48d1b548201a8a964a9879961b3c68) * Send pending updates before adding discontinuity for error When handling a playback error that originates from a future item in the playlist, we added support for jumping to that item first, ensuring the errors 'happen' for the right 'current item'. See https://github.com/androidx/media/commit/79b688ef30d4a3306d4d321ddf4464b428074ea2. However, when we add this new position discontinuity to the playback state, there may already be other position discontinuities pending from other parts of the code that executed before the error. As we can't control that in this case (because it's part of a generic try/catch block), we need to send any pending updates first before handling the new change. Issue: androidx/media#1483 PiperOrigin-RevId: 646968309 (cherry picked from commit 727645179b3c26e495540f4445216fd035cc7654) * Cache audio timestamp frame position across track transition reset Upon track transition of offloaded playback of gapless tracks, the framework will reset the audiotrack frame position. The `AudioTrackPositionTracker`'s `AudioTimestampPoller` must be made to expect the reset and cache accumulated …
Use case description
Main use case is about playing files inside APK with not only for example Uri:
Uri.parse(("file:///android_asset/music.mp3"))
But to be able to use files located in jar archive. For example Uri:
jar:file:/data/app/~~4hhLse7uFXE7V7sA==/com.example.composetest-jD6eQ3BeyvfQ==/base.apk!/composeResources/com.example.shared.resources/files/music.mp3
This approache can be used for multiplatform applications (e.i. multiplatform media players using AVPlayer on iOS and Exoplayer on Android with shared code) using Compose Multiplatform which will generated resources Uri for files inside common/shared directory. Mentioned directory "android_asset" is not accessible on iOS and generated resources Uri can't be used on Android. Media files needs to be duplicated to proper platform dependend directories.
More about Compose Multiplatform resources:
https://www.jetbrains.com/help/kotlin-multiplatform-dev/compose-images-resources.html
Proposed solution
Be able to use following code:
val mediaItem = MediaItem.Builder().setUri(Res.getUri("files/music.mp3")).build()
Alternatives considered
Only solution I can think of is using custom MediaSource and handling InputStream using method from Compose Multiplatform:
Res.readBytes("files/music.mp3")
with solution mentioned for example here (Exoplayer issue)
Similar issue is opened in Compose Multiplatform repository here and support from this side is not coming. The reasons are mentioned there.
Thank you for consideration!
The text was updated successfully, but these errors were encountered: