Media elements, such as
Correlating Media Events and Network Activity
Sometimes media playback may become stalled due to network activity, such as if a byte range request takes too long to load. Previously there was no easy way to identify these issues solely from Web Inspector. Now, however, Web Inspector provides an option called “Group Media Requests” that organizes network activity based on the related top-level media element. This makes it easier to keep track of related resources, such as byte range requests or items that are part of a media playlist.
Since media elements aren’t resources it doesn’t make sense to have the usual “waterfall”. Instead, we’ve created a special timeline that keeps a log of playback related events fired by the media element, such as
stalled, as well as some basic state about the media element, such as whether it’s paused. A dashed line represents “paused”, a solid line represents “playing”, and no line represents “stopped”. If a
<video> element enters fullscreen a grey area is shown indicating the time in which that
<video> remained in fullscreen mode. On macOS
<video> elements can also be rendered in a special way, called Power Efficient Video Playback, which is indicated by a green area.
Diagnosing issues with Power Efficient Video Playback
<video> element is about to render a frame on macOS, it makes a decision about whether to use the computer’s CPU/GPU or the display itself to composite the frame. In cases where the display can be used for frame compositing, it is often more power efficient. If your website contains full-screen video content, you should strive to follow these guidelines to ensure that your
<video> is able to use display compositing as often as possible.
<video>must be playing.
<video>must be full-screen.
<video>must not be occluded in any way (e.g. not showing controls or any content “above” it).
Keep in mind that these are ways in which the web content itself can prevent power efficient video playback. There are other reasons that aren’t controllable from web content as to why a video may not enter a power efficient playback state, such as being connected to an external display or the device itself not supporting it.
Starting in macOS Mojave 10.14.4, WebDriver is able to access the total count of display composited frames which can be used for continuous integration testing.
let video = document.querySelector("video"); let videoPlaybackQuality = video.getVideoPlaybackQuality(); videoPlaybackQuality.displayCompositedVideoFrames
Each time a frame is display composited, that count will increment, so if the count does not increment, the
<video> is not using display compositing.
If you develop websites where the main content is a
<video>, consider adopting these guidelines to provide a better experience for your users.
These enhancements are available starting in macOS Mojave 10.14.4 and iOS 12.2. Is there additional information you’d like us to visualize with respect to media elements? Are there other elements that fit into this “category” for which you’d like to see similar instrumentation? Let us know! Please feel free to send us feedback on Twitter (@dcrousso or @jonathandavis) or by filing a bug.