Fix frame seeking with negative DTS values in FFMPEG backend #27878

0lekW · 2025-10-07T22:16:44Z

Merge with extra: opencv/opencv_extra#1279

Fix for issue #27819.
Accompanied by PR on opencv_extra:

The FFmpeg backend fails to correctly seek in H.264 videos that contain negative DTS values in their initial frames. This is a valid encoding practice used by modern video encoders (such as DaVinci Resolve's current export) where B-frame reordering causes the first few frames to have negative DTS values.

When picture_pts is unavailable (AV_NOPTS_VALUE), the code falls back to using pkt_dts:
picture_pts = packet_raw.pts != AV_NOPTS_VALUE_ ? packet_raw.pts : packet_raw.dts;

If this DTS value is negative (which is legal per H.264 spec), it propagates through the frame number calculation:
frame_number = dts_to_frame_number(picture_pts) - first_frame_number;

This results in negative frame numbers, messing up seeking operations.

Solution implemented in this branch is a timestamp normalization similar to FFmpegs -avoid_negative_ts_make_zero flag:

Calculate a global offset once on the first decoded frame by getting the minimum timestamp in either:
- Container start_time
- Stream start_time
- First observed timestamp (PTS, then DTS).
Apply the offset consistently to all timestamps, shifting negative values to begin at 0 while keeping relative timing.
Simplify timestamp converters to remove start_time subtractions since timestamps are pre-normalized.

This also includes a new test videoio_ffmpeg.seek_with_negative_dts
This test verifies that seeking behavior performs as expected on a file which has negative DTS values in the first frames.
A PR on opencv_extra accompanies this one with that testing file: opencv/opencv_extra#1279

opencv_extra=ffmpeg-videoio-negative-dts-test-data

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

Add test case to verify behavior when seeking in videos that contain negative DTS values in initial fames. Is accompanied by test media "negdts_h264.mp4" in a opencv_extra commit

Implement timestamp normalization to handle videos with negative decoding time stamp values, which are produced by modern encoders like DaVinci Resolve. Calculates a constant timestamp offset once on the first decoded frame, then applies it to all timestamps to shift them to start from 0. The approach reflects what FFMPEG -avoid_negative_ts_make_zero might do. Offset is determined by finding the min timestamp across the container start time, stream start time, or first observed frame timestamp. If the minimum is negative the offset is set to negate it, otherwise is zero (no normalization).

…FFmpeg wrapper update

…n in raw mode The offset needs to be computed for timestamps which are positive as well - this used to be applied in dts_to_sec but now that we normalize in grabFrame it makes sense to do it here. Also changed it so rawMode wont apply the normalization as this doesn't make much sense

revert change which didn't allow normalization to happen in raw mode, and instead correctly apply the normalization to the dts as well as pts, which should solve problems where the dts < pts because of a dts delay

opencv-alalek

Thank you for contribution!

Perhaps ts_offset_avtb should be also handled in the "seek" operation (for reverting timestamp changes back). Please note that this operation is not accurate on non-key frames (see #9053). Lets ensure that it properly works at least with zero input value (request to seek to start)

opencv-alalek · 2025-10-08T05:34:50Z

modules/videoio/src/cap_ffmpeg_impl.hpp

+
+                // Compute offset to shift timestamps to zero
+                if (min_start_avtb != INT64_MAX) {
+                    ts_offset_avtb = -min_start_avtb;


if (min_start_avtb < 0) is preferable to keep original timestamp on "non-issue" videos with all non-negative timestamps.

Hi, thanks for taking a look at this - I'm very happy to contribute.
I agree with the above. The alternative would probably be for CvCapture_FFMPEG::dts_to_sec(int64_t dts) to have a condition which checks if ts_offset_decided is false (no normalization has happened) and take away the stream start_time like it did previously:

double CvCapture_FFMPEG::dts_to_sec(int64_t dts) const { const AVStream* st = ic->streams[video_stream]; int64_t ts = dts; if (ts_offset_avtb == 0 && st->start_time != AV_NOPTS_VALUE_) ts -= st->start_time; return ts * r2d(st->time_base); }

That way positive start time files will still work correctly

And I guess seek will do something like this to revert the timestamps back?

AVStream* st = ic->streams[video_stream]; int64_t time_stamp = st->start_time; double time_base = r2d(st->time_base); int64_t ts_norm = (int64_t)(sec / time_base + 0.5); if (ts_offset_avtb != 0) { // map normalized target back to original demux timeline time_stamp += ts_norm - from_avtb(ts_offset_avtb, st->time_base); } else { time_stamp += ts_norm; }

I went ahead and made the changes - the Linux x64 build failed, but seems unrelated.
Some GStreamer test stalling

Keep original timestamp for all non-negative timestamp files by only computing and offset for a negative case + handle this situation in dts_to_sec by explicitly checking if the offset was applied. Additionally, seek operation now considers the offset so that the timestamp is reverted

opencv-alalek · 2025-10-08T16:10:10Z

modules/videoio/test/test_ffmpeg.cpp

+
+    if (!cap.isOpened())
+        throw SkipTestException("Video stream is not supported");
+


Please add .grab() + cap.get(CAP_PROP_POS_MSEC) + non-negative check

There is Windows build configuration which doesn't update FFmpeg plugin during PR validation build and this test doesn't fail (passes without fix above).

I've added those checks into the test now.
Note, this test is within a #ifndef _WIN32 check because I read about some prebuilt binaries not working here.

opencv-alalek · 2025-10-10T07:35:08Z

Main problem of this approach that we can't use timestamps received from cv::VideoCapture in external tools (e.g. ffmpeg command), because they don't expect "shifts".

Also need to check GStreamer backend behavior (users expect the same result from the same videos)

0lekW · 2025-10-10T11:41:52Z

Main problem of this approach that we can't use timestamps received from cv::VideoCapture in external tools (e.g. ffmpeg command), because they don't expect "shifts".

This is technically true I guess, but the timestamps for the problem files in question weren't correct to begin with.
Take the broken file I've been testing with for example:

Frame 0: seek=OK pos=0 timestamp=0
Frame 1: seek=OK pos=1 timestamp=0.0813802
Frame 2: seek=OK pos=-3 timestamp=0.16276
Frame 3: seek=OK pos=-3 timestamp=0.244141
Frame 4: seek=OK pos=-3 timestamp=0.325521
Frame 5: seek=OK pos=-3 timestamp=0.406901

Compared to now: (not using grab in these btw just seek)

Frame 0: seek=OK pos=0 timestamp=0
Frame 1: seek=OK pos=1 timestamp=0
Frame 2: seek=OK pos=2 timestamp=41.6667
Frame 3: seek=OK pos=3 timestamp=83.3333
Frame 4: seek=OK pos=4 timestamp=125
Frame 5: seek=OK pos=5 timestamp=166.667

If you use ffprobe with -frame=best_effort_timestamp_time you also get those results.

Behavior for normal files is already the same since we only do this for the negative case.
As for other backends, I haven't tested much. I know at least that GStreamer was totally unable to read any positions or timestamps from my file correctly and that MSMF could read positions but got bogus timestamps.

0lekW added 3 commits October 8, 2025 10:43

add ffmpeg videoio test for seeking with negative dts values

de82dde

Add test case to verify behavior when seeking in videos that contain negative DTS values in initial fames. Is accompanied by test media "negdts_h264.mp4" in a opencv_extra commit

move negative_dts test into #ifndef WIN32 while they wont work untul …

b06d5e4

…FFmpeg wrapper update

0lekW mentioned this pull request Oct 7, 2025

Add test video with negative DTS values opencv/opencv_extra#1279

Open

0lekW added 2 commits October 8, 2025 12:23

fix trailing whitespaces and indentations

d4eb57a

0lekW force-pushed the ffmpeg-negative-dts-seeking-fix branch from cba0185 to 5a48502 Compare October 8, 2025 02:42

opencv-alalek added bug feature category: videoio labels Oct 8, 2025

apply normalization to dts as well

0f7ab59

revert change which didn't allow normalization to happen in raw mode, and instead correctly apply the normalization to the dts as well as pts, which should solve problems where the dts < pts because of a dts delay

opencv-alalek added this to the 4.13.0 milestone Oct 8, 2025

opencv-alalek reviewed Oct 8, 2025

View reviewed changes

asmorkalov assigned opencv-alalek Oct 8, 2025

opencv-alalek reviewed Oct 8, 2025

View reviewed changes

update test to check negative value after a grab

23d487a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Fix frame seeking with negative DTS values in FFMPEG backend #27878

Fix frame seeking with negative DTS values in FFMPEG backend #27878

0lekW commented Oct 7, 2025 •

edited by opencv-alalek

Loading

Uh oh!

opencv-alalek left a comment

Uh oh!

opencv-alalek Oct 8, 2025

Uh oh!

0lekW Oct 8, 2025 •

edited

Loading

Uh oh!

0lekW Oct 8, 2025

Uh oh!

opencv-alalek Oct 8, 2025

Uh oh!

0lekW Oct 8, 2025

Uh oh!

opencv-alalek commented Oct 10, 2025

Uh oh!

0lekW commented Oct 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		if (!cap.isOpened())
		throw SkipTestException("Video stream is not supported");

Uh oh!

Fix frame seeking with negative DTS values in FFMPEG backend #27878

Are you sure you want to change the base?

Fix frame seeking with negative DTS values in FFMPEG backend #27878

Conversation

0lekW commented Oct 7, 2025 • edited by opencv-alalek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Uh oh!

opencv-alalek left a comment

Choose a reason for hiding this comment

Uh oh!

opencv-alalek Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

0lekW Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

0lekW Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

opencv-alalek Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

0lekW Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

opencv-alalek commented Oct 10, 2025

Uh oh!

0lekW commented Oct 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

0lekW commented Oct 7, 2025 •

edited by opencv-alalek

Loading

0lekW Oct 8, 2025 •

edited

Loading