/** * Copyright 2012-2024 JogAmp Community. All rights reserved. * * Redistribution and use in source and binary forms, with or without modification, are * permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, this list of * conditions and the following disclaimer. * * 2. Redistributions in binary form must reproduce the above copyright notice, this list * of conditions and the following disclaimer in the documentation and/or other materials * provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY JogAmp Community ``AS IS'' AND ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND * FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL JogAmp Community OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * The views and conclusions contained in the software and documentation are those of the * authors and should not be interpreted as representing official policies, either expressed * or implied, of JogAmp Community. */ package com.jogamp.opengl.util.av; import com.jogamp.opengl.GL; import com.jogamp.opengl.GLEventListener; import com.jogamp.opengl.GLException; import jogamp.opengl.Debug; import java.io.PrintStream; import java.util.List; import com.jogamp.common.av.AudioSink; import com.jogamp.common.av.PTS; import com.jogamp.common.av.TimeFrameI; import com.jogamp.common.net.Uri; import com.jogamp.math.Vec4f; import com.jogamp.opengl.util.texture.Texture; import com.jogamp.opengl.util.texture.TextureSequence; /** * GLMediaPlayer interface specifies a {@link TextureSequence} state machine * using a multiplexed audio/video stream as it's source. *
* Audio maybe supported and played back internally or via an {@link AudioSink} implementation. *
** Audio and video streams can be selected or muted via {@link #playStream(Uri, int, int, int, int)} * or {@link #playStream(Uri, int, String, int, String, int, int)} * using the appropriate stream id's. *
** Camera input can be selected using the {@link #CameraInputScheme} Uri. *
* ** Most of the stream processing is performed on the decoding thread, a.k.a. StreamWorker: *
* Caught exceptions on StreamWorker are delivered as {@link StreamException}s, * which either degrades the {@link State} to {@link State#Uninitialized} or {@link State#Paused}. *
** An occurring {@link StreamException} triggers a {@link GLMediaEventListener#EVENT_CHANGE_ERR EVENT_CHANGE_ERR} event, * which can be listened to via {@link GLMediaEventListener#attributesChanged(GLMediaPlayer, int, long)}. *
** An occurred {@link StreamException} can be read via {@link #getStreamException()}. *
* * **
Action | {@link State} Before | {@link State} After | {@link EventMask#Bit Event} |
---|---|---|---|
{@link #playStream(Uri, int, int, int, int)} | {@link State#Uninitialized Uninitialized} | {@link State#Initialized Initialized}1, {@link State#Uninitialized Uninitialized} | {@link EventMask.Bit#Init Init} or ( {@link EventMask.Bit#Error Error} + {@link EventMask.Bit#Uninit Uninit} ) |
{@link #initGL(GL)} | {@link State#Initialized Initialized}, {@link State#Uninitialized Uninitialized} | {@link State#Playing Playing}, {@link State#Uninitialized Uninitialized} | {@link EventMask.Bit#Play Play} or ( {@link EventMask.Bit#Error Error} + {@link EventMask.Bit#Uninit Uninit} ) |
{@link #pause(boolean)} | {@link State#Playing Playing} | {@link State#Paused Paused} | {@link EventMask.Bit#Pause Pause} |
{@link #resume()} | {@link State#Paused Paused} | {@link State#Playing Playing} | {@link EventMask.Bit#Play Play} |
{@link #stop()} | {@link State#Playing Playing}, {@link State#Paused Paused} | {@link State#Uninitialized Uninitialized} | {@link EventMask.Bit#Pause Pause} |
{@link #seek(int)} | {@link State#Paused Paused}, {@link State#Playing Playing} | {@link State#Paused Paused}, {@link State#Playing Playing} | none |
{@link #getNextTexture(GL)} | any | same | none |
{@link #getLastTexture()} | any | same | none |
{@link TextureFrame#END_OF_STREAM_PTS END_OF_STREAM} | {@link State#Playing Playing} | {@link State#Paused Paused} | {@link EventMask.Bit#EOS EOS} + {@link EventMask.Bit#Pause Pause} |
{@link StreamException} | any | {@link State#Paused Paused}, {@link State#Uninitialized Uninitialized} | {@link EventMask.Bit#Error Error} + ( {@link EventMask.Bit#Pause Pause} or {@link EventMask.Bit#Uninit Uninit} ) |
{@link #destroy(GL)} | any | {@link State#Uninitialized Uninitialized} | {@link EventMask.Bit#Uninit Uninit} |
*
value | request | get |
---|---|---|
{@link #STREAM_ID_NONE} | mute | not available |
{@link #STREAM_ID_AUTO} | auto | unspecified |
≥0 | specific stream | specific stream |
* Current implementations (check each API doc link for details): *
* Implementations of this interface must implement: *
* public static final boolean isAvailable(); ** to be properly considered by {@link GLMediaPlayerFactory#create(ClassLoader, String)} * and {@link GLMediaPlayerFactory#createDefault()}. * *
*
* Timestamp type and value range has been chosen to suit embedded CPUs * and characteristics of audio and video streaming. See {@link TimeFrameI}. *
* ** The class follows a passive A/V synchronization pattern. * Audio is being untouched, while {@link #getNextTexture(GL)} delivers a new video frame * only, if its timestamp is less than {@link #MAX_VIDEO_ASYNC} ahead of time. * If its timestamp is more than {@link #MAX_VIDEO_ASYNC} ahead of time, * the previous frame is returned. * If its timestamp is more than {@link #MAX_VIDEO_ASYNC} after time, * the frame is dropped and the next frame is being fetched. *
** https://en.wikipedia.org/wiki/Audio_to_video_synchronization *
* d_av = v_pts - a_pts; ** *
* Recommendation of audio/video pts time lead/lag at production: *
* Recommendation of av pts time lead/lag at presentation: *
*
Big Buck Bunny 24f 16:9 | ||||
---|---|---|---|---|
Big Buck Bunny | 320p | h264 | aac 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4 |
Big Buck Bunny | 240p | h264 | aac 48000Hz 2 chan | http://archive.org/download/BigBuckBunny_328/BigBuckBunny_512kb.mp4 |
Big Buck Bunny | 720p | mpeg4 | ac3 48000Hz 5.1 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_surround.avi |
Big Buck Bunny | 720p | msmpeg4v2 | mp3 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.avi |
Big Buck Bunny | 720p | theora | vorbis 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.ogg |
Big Buck Bunny | 1080p | mpeg4 | ac3 48000Hz 5.1 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_surround.avi |
WebM/Matroska (vp8/vorbis) | ||||
Big Buck Bunny Trailer | 640p | vp8 | vorbis 44100Hz 1 chan | http://video.webmfiles.org/big-buck-bunny_trailer.webm |
Elephants Dream | 540p | vp8 | vorbis 44100Hz 1 chan | http://video.webmfiles.org/elephants-dream.webm |
You Tube http/rtsp | ||||
Sintel | http://www.youtube.com/watch?v=eRsGyueVLvQ | rtsp://v3.cache1.c.youtube.com/CiILENy73wIaGQn0LpXnygYbeRMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp | ||
Audio/Video Sync | ||||
Five-minute-sync-test1080p | https://www.youtube.com/watch?v=szoOsG9137U | rtsp://v7.cache8.c.youtube.com/CiILENy73wIaGQm133VvsA46sxMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp | ||
Audio-Video-Sync-Test-Calibration-23.98fps-24fps | https://www.youtube.com/watch?v=cGgf_dbDMsw | |||
sound_in_sync_test | https://www.youtube.com/watch?v=O-zIZkhXNLE |