DeDup 0.17 by Loren Merritt, Copyright 2004 based on Dup 2.20 beta 1 by Donald Graft/Klaus Post DeDup is an Avisynth 2.5 filter, intended to remove duplicate frames in the interest of compression quality and speed. In some cases, it can also reduce noise or compression artifacts in the source. DeDup differs from Dup in that it completely drops the duplicate frames, rather than keeping several copies of each in the output stream. Then, to compensate for the dropped frames, DeDup outputs a Matroska-compatible timecode file to put the remaining frames back in the right places. This is a two pass filter. While my algorithm could work with one pass, it wouldn't fit in the Avisynth API. Usage example: --- first pass --- LoadPlugin("DeDup.dll") AVISource("blah.avi") DupMC(log="blah.dup.txt") ------------------- --- second pass --- LoadPlugin("DeDup.dll") AVISource("blah.avi") DeDup(threshold=0.3, maxcopies=10, maxdrops=3, log="blah.dup.txt", times="blah.times.txt") ------------------- First Pass Options: DupMC(clip c, bool chroma, string log) chroma: default: true Use chroma planes when calculating differences. (This is the same as in Dup) log: default: none. required. File to save dup metrics for the second pass. DupMC does not modify the frames passed through it. It only generates the log. There is no harm in seeking during the first pass: The log will be out of order, but DeDup doesn't care. Unlike Dup, which compares the last frame in a run to the first frame, DupMC always compares adjacent frames. You don't decide on a threshold until the second pass. Second Pass Options: DeDup(clip c, float threshold, float threshold2, int range2, float trigger2, bool show, bool dec, int maxcopies, int maxdrops, bool blend, string log, string times, string timesin, string ovr, string debug) threshold: range: 0 - 100 default: 0.3 Consecutive frames that differ by less than or equal to this much will be merged into a single output frame, taken from the end of the merged sequence. (This uses the same units as in Dup) threshold2: range: 0 - 100 default: 0.5 (but disabled due to range2) Use this instead of threshold if there are nearby high-motion frames in both directions. (Actually, linearly interpolate between threshold and threshold2 based on the amount of motion.) range2: range: 0 - lots (sane values: 0 - maxdrops) default: 0 (disabled) Search range for threshold2. trigger2: range: 0 - 100 default: 5.0 What counts as "high-motion" for threshold2. show: default: false Print diagnostic information into each frame. dec: default: true Drop frames. Without this, DeDup won't do much. But you might want to disable this with show=true, to see the frames that would be dropped. maxcopies: range: 1 - 20 default: 12 Break any string of duplicates longer than this. (This is the same as in Dup.) maxdrops: range: 1 - maxcopies default: 1 (no dropped frames (like the original Dup)) Max input to output frame ratio. (So maxdrops=3 means a max of 2 consecutive dropped frames.) Don't set this above 3-4 or so if you want to display subtitles, because many media players can't update the subtitles until they get a new video frame. decwhich: Which frame from a run of duplicates do we keep? 0: first frame 1: middle, round down 2: middle, round up (default) 3: last log: default: none. The file created in the first pass with DupMC(). Required, unless you want to specify every frame with ovr. times: default: none. File to write matroska timecodes to. Not strictly required, but you'll need it for muxing unless maxdrops=1. timesin: default: none. Read timestamps for a VFR input. Matroska timecode formats v1 and v2 accepted. ovr: default: none. File to override settings on a per-frame basis. See below for format. debug: default: none. File to write the same information as with show=true. Override File Format: Example: 0-350 threshold=.3 350-900 threshold=.7 threshold2=1.5 trigger2=8.0 100 k 200-220 ddk Any line not beginning with a frame number is ignored as a comment. There is no notation for 'last frame', aside from using it's real frame number. However, it's OK for ranges to extend past the end of the movie. If two lines override the same property of the same frame, the one later in the ovr file takes precedence. Each line is of the format: startframe-endframe option [more options] or frame option [more options] Option overrides accepted: A string matching /[kd]+/: This forces the frame(s) to be (k) kept or (d) considered dups. The string of [kd]s will be repeated as necessary to fill the frame range. Forcing a dup does not necessarily cause that frame to be dropped; it's still subject to maxdrops. This option modifies thresholds, so put any [kd] option after any relevant threshold override. threshold=%f threshold2=%f trigger2=%f These are the same as the global options. range2=%d The range2 parameter is taken from the frame being considered from dropping, not from the frames being searched over. maxdrops=%d maxcopies=%d These values are taken from the frame at the end of the clump of dups, but it's better to just set them the same over the whole clump. Tips: If you're doing other CPU-intensive filtering, you can save time (at the cost of space) by saving the first pass (with DupMC) to a huffyuv, and then run the second pass (with DeDup) from that, instead of repeating the other filters. Don't set threshold too high (especially if maxcopies is large): Unlike the original Dup, I don't do sanity checking on slow zooms/pans/fades. The second pass options 'times' and 'debug' take effect as soon as you open the avs. You don't have to play/encode the whole video in order to produce those files.