21-v-001
Disclaimer and Conditions Regarding Use of SWGDE Documents:
SWGDE documents are developed by a consensus process that involves the best efforts of relevant subject matter experts, organizations, and input from other stakeholders to publish suggested best practices, practical guidance, technical positions, and educational information in the discipline of digital and multimedia forensics and related fields. No warranty or other representation as to SWGDE work product is made or intended.
As a condition to the use of this document (and the information contained herein) in any judicial, administrative, legislative, or other adjudicatory proceeding in the United States or elsewhere, the SWGDE requests notification by e-mail before or contemporaneous to the introduction of this document, or any portion thereof, as a marked exhibit offered for or moved into evidence in such proceeding. The notification should include: 1) The formal name of the proceeding, including docket number or similar identifier; 2) the name and location of the body conducting the hearing or proceeding; and 3) the name, mailing address (if available) and contact information of the party offering or moving the document into evidence. Subsequent to the use of this document in the proceeding please notify SWGDE as to the outcome of the matter. Notifications should be sent to secretary@swgde.org.
From time to time, SWGDE documents may be revised, updated, or sunsetted. Readers are advised to verify on the SWGDE website (www.swgde.org) they are utilizing the current version of this document. Prior versions of SWGDE documents are archived and available on the SWGDE website.
Redistribution Policy:
SWGDE grants permission for redistribution and use of all publicly posted documents created by SWGDE, provided that the following conditions are met:
- Redistribution of documents or parts of documents must retain this SWGDE cover page containing the Disclaimer and Conditions of Use.
- Neither the name of SWGDE nor the names of contributors may be used to endorse or promote products derived from its documents.
- Any reference or quote from a SWGDE document must include the version number (or creation date) of the document and also indicate if the document is in a draft status.
Requests for Modification:
SWGDE encourages stakeholder participation in the preparation of documents. Suggestions for modifications are welcome and must be forwarded to the Secretary in writing at secretary@swgde.org. The following information is required as a part of any suggested modification:
- Submitter’s name
- Affiliation (agency/organization)
- Address
- Telephone number and email address
- SWGDE Document title and version number Scientific Working Group on Digital Evidence
- Change from (note document section number)
- Change to (provide suggested text where appropriate; comments not including suggested text will not be considered)
- Basis for suggested modification
Intellectual Property:
Unauthorized use of the SWGDE logo or documents without written permission from SWGDE is a violation of our intellectual property rights.
Individuals may not misstate and/or over represent duties and responsibilities of SWGDE work. This includes claiming oneself as a contributing member without actively participating in SWGDE meetings; claiming oneself as an officer of SWGDE without serving as such; claiming sole authorship of a document; use the SWGDE logo on any material and/or curriculum vitae.
Any mention of specific products within SWGDE documents is for informational purposes only; it does not imply a recommendation or endorsement by SWGDE.
Date submitted: February 16, 2021
Submitter: Andrew Fredericks
Affiliation: iNPUT-ACE
Date adjudicated by SWGDE: September 16, 2021
Comment 1
Section 3
Original language: It is recommended that examiners acquire both proprietary and open file formats from the source, if available. This allows for both validation of a determined frame time and an additional resource to analyze.
Proposed change: It is recommended that examiners acquire both proprietary and open file formats from the source, if available. This allows for additional resources to be analyzed and may provide more information than a single source.
Rationale for change: Analyzing multiple formats from the source can certainly provide additional information and potentially highlight metadata that is inaccurate or concerning – but it does not help to validate whether frame timing is accurate. A single system could provide multiple export options that all purport the same inaccurate frame timing.
SWGDE Resolution: Change accepted.
Comment 2
Section 3
Original language: When multiple file formats are available, the examiner should exercise caution in identifying the file format with the intended frame rate timing.
Proposed change: Regardless of the file format, the examiner should exercise caution when reviewing timing metadata, as timing metadata in any format can be inaccurate.
Rationale for change: Any (or all) files from a system can have inaccurate timing, so there may not be a file on the system that properly reports an “intended” frame timing. Since this section is intended to be about limitations, I believe it is important to use stronger wording about the potential inaccuracies in file metadata.
Note: Additional rationale was provided on September 11, 2021 by the submitter via four explanation videos.
SWGDE Resolution: The explanation videos provided on September 11, 2021 included examples of timing inconsistencies when a video was transcoded/re-wrapped using software outside the original recording device as well as an example of multiple time sources found on video files exported from a proprietary DVR system in different formats. Based on those examples, the following language was added to Section 3, “Additionally, this document is not intended for use on files that have been transcoded from their camera original file format by software outside the original recording device.”
Based upon the feedback received in the September 11, 2021 videos regarding the ability to decode timing in different file formats, the following language was also added to Section 7, “Consideration should be given to cases where proprietary systems are offering exports in nonproprietary formats, e.g., ISO base media file format. In these circumstances, it is also possible that the software developers who write custom code to create these ISO base media file format wrappers may create inaccurate calculations as they insert data into the internal ISO base media file format structures. Manual decoding of the file in a hex editor or the use of an external timing source as discussed in Section 8 may assist in this evaluation.”
Section 8 was also updated to state, “Consideration should be given to confirm timing information given to non-camera original files (i.e., files transcoded to .MP4 by a recording device), either through the use of an external timing source and/or manual decoding of a file in a hex editor.”
The SWGDE video committee reviewed and discussed the submitters proposed change and felt that it was too broad a statement to the intended audience given the specific examples provided. It is believed that the above changes are reflective of the specific issues demonstrated in the explanation videos and are more beneficial to the reader with the additional language than the generic statement regarding all video files.
Comment 3
Section 4
Original language: Included in the standard is specific encoding language pertaining to the timing and presentation of video frames. Using this information, the specific time intervals between frames can be calculated.
Proposed change: Included in the standard is specific encoding language pertaining to the timing and presentation of video frames. Using this information, the time intervals between frames can be calculated; however, it is important to note that these time intervals represent what is stored in the file and may not represent the “actual time” that elapsed between the events depicted in the video images.
Rationale for change: Since the document is most likely to be used as a basis for performing speed calculations, I believe the language needs to be clear that the “specific time” that can be calculated is simply the time stored within the file – and not the time the actual events occurred.
SWGDE Resolution: Based upon the comment and supplied explanation videos, the language in Section 4 was changed to, “Included in the standard is specific encoding language pertaining to the decoding and presentation timing of video frames; adherence to this encoding language should be evaluated by the examiner. Using this information, it is possible to calculate the specific time intervals between displayed frames.”
The SWGDE video committee agrees that the original statement can be misleading depending on interpretation. The new language is more reflective of the information presented in the format specifications and there is no correlation noted to “actual time” in this section. The correlation between elapsed time in displayed video against “actual time”/ground truth is discussed in Section 8.
Comment 4
Section 4
Original language: footnote referenced “State of New Hampshire v. Wiley…”
Proposed change: “State of New Hampshire v. Witty…”
Rationale for change: Incorrect case citation
SWGDE Resolution: Change accepted
Comment 5
Section 7.1
Original language: By looking at the difference between pkt_pts_time values for sequential frames, examiners can determine the time that has elapsed between those frames.
Proposed change: By looking at the difference between pkt_pts_time values for sequential frames, examiners can determine the time that the decoder purports has elapsed between those frames.
Rationale for change: If the other recommendations above are accepted, this additional recommendation may be unnecessary – but in the interest of being complete, I have included it.
SWGDE Resolution: Language changed to, “By looking at the difference between pkt_pts_time values for sequential frames, examiners can determine the reported elapsed time between frames.” Rationale for change for change as noted in comment 3 above.
Comment 6
Section 7.2
Original language: It is important to note that the aforementioned ffprobe command is deriving information from the video file that has not been decoded. For that reason, the packet decode timestamp (pkt_dts_timestamp) and the packet presentation timestamp (pkt_pts_timestamp) will most likely be the same value.
Proposed change: None noted
Rationale for change: The statement is unclear. First, there is a minor typo as the attributes should be “pkt_dts_time” and “pkt_pts_time”. Second, FFprobe derives pkt_dts and pkt_pts from the decompressed AVframe, so it does technically decode the packets in order to derive the timing metadata.
It may be helpful to note as well that the frame’s “pkt_pts” has been deprecated for some time in FFmpeg’s code. Even though the FFprobe report still labels the column with “pkt_pts”, it actually derives the data from the frame’s “pts”. The difference being that the frame’s “pkt_pts” was originally a value copied directly from the AVPacket – while “pts” is calculated off of the codec (after decoding the frame from one or more packets). This is often a confusing area because the deprecated Libav “pkt_pts” attribute is distinct from the “pkt_pts” values that are presented within FFprobe.
I’m not sure what the original statement is trying to say, but I hope my comments above will assist if an edit to this unclear statement is created.
SWGDE Resolution: The intent of the original language was to address the manner in which ffprobe is parsing data from a video file. As such the presence of bi-directional frames in a video file may cause a discrepancy in the ffprobe output (discussed in the document’s following paragraph). For clarity, the language was changed to, “It is important to note that the aforementioned ffprobe command is deriving information from the video file as stored in the frames data, prior to a file being actively decoded. For that reason, the packet decode timestamp (pkt_dts_time) and the packet presentation timestamp (pkt_pts_time) may be the same value.”
Comment 7
Section 7.2 (footnote 5)
Original language: Packet duration time displays the total time that an individual frame is to be displayed (the value is expressed in the timescale of the media). Examiners should validate this time against packet presentation time before use in an examination. Due to the nature of how certain video files are encoded, differences between packet presentation times of sequential frames and packet duration time may occur.
Proposed change: Packet duration time displays the total time that an individual frame is to be displayed (the value is expressed in the timescale of the media). While packet presentation time is derived from the Libav library as part of the AVPacket, packet duration time is calculated by FFmpeg via the decoder. Each FFmpeg decoder can calculate packet duration time in a unique way, so it is not uncommon for packet presentation time and packet duration time to differ.”
Rationale for change: The recommended change may be too technical – but the current statement that examiners should validate the packet duration time against packet presentation time before use in an examination is unclear. Since the two values are calculated via different means, they potentially answer different questions. The original statement in the document appears to suggest that the values can validate one another – but since both values are unreliable for the purpose of speed calculations without other validation methods (as discussed already), I believe further clarification would be helpful to reduce potential confusion/misinterpretation by the reader.
SWGDE Resolution: The intent of the original language was to alert the reader to differing values represented by duration vs. presentation time. As demonstrated in a specific explanation video supplied by the submitter there are considerations for the interpretation of both values in certain recordings. Subsequently the language was changed to, “Examiners should evaluate this time against packet presentation time before use in an examination. However, as these values are given at different times in the frame processing, differences between packet presentation times of sequential frames and packet duration time may occur and should be examined and documented.”
Comment 8
Section 8
Original language: When ISO Base Media File metadata analysis cannot be conducted, an external timing source can be used to evaluate frame timing. Even when there is metadata available, one should still validate findings against a known time source. While deriving frame time from file metadata may have more precision than other methods, there are occurrences when that is not an option (e.g., inability to decode proprietary container information).
Proposed change: External timing sources can be used to evaluate the accuracy of frame timing metadata within a file. Since file metadata can be an inaccurate measure of “real time”, external validations should be performed prior to using the information in speed or force calculations.
Rationale for change: The current statement appears to suggest that file metadata is a more precise measure of timing than an external method, and that external methods are only required when ISO Base Media File metadata analysis cannot be conducted.
SWGDE Resolution: The language was changed based upon the submitter’s comments and supplied explanation videos. The intention of the section is to advise that any use of file metadata for timing analysis should be verified using an alternate means of examination. A specific explanation video provided by the submitter raised questions of discrepancies in the duration and presentation times, as such the language was changed to highlight potential discrepancies but the SWGDE video committee feels that addressing the specific issue in this document is too specific and would detract from the overall guidance provided. As such the language was changed to, “Regardless of the ability to decode ISO Base Media File metadata, a known timing source can be used to evaluate frame timing. While deriving frame time from file metadata may have a higher degree of precision than other methods, there are occurrences when that is not an option (e.g., inability to decode proprietary container information, transcoding within the recording device, or a stream copy that removes pertinent metadata). In the event of discrepancies between the documented frame data and an external source, additional testing may be necessary. Special consideration should be given to confirm timing information given to non-camera original files (i.e., files transcoded to .MP4 by a recording device), either through the use of an external timing source and/or manual decoding of a file in a hex editor.”
Additionally, a footnote was added to clarify the use of the term “precision” to say, “In this context, precision is the quality of the being exact (e.g., the smallest unit of measure of time that can be discerned). It should not be conflated with accuracy, which is how close the value is to the correct value (ground truth).”
Comment 9
Entire document
Original language: Document uses the term “ffmpeg”
Proposed change: The document contains a few mentions of “ffmpeg”, but the proper spelling should start with two capital letters: “FFmpeg”
Rationale for change: See bullet 15 from: https://ffmpeg.org/legal.html.
SWGDE Resolution: The usage switches depending on the reference. When the library is called, “FFmpeg” is used, but when the application is called, “ffmpeg” is used. This is consistent with other SWGDE documents.
Additional comment: Thank you for reviewing these recommended changes. As identified above, my primary concern centers around the fact that timing metadata is often inaccurate (even in ISO files), and I believe the current document does not sufficiently limit the scope with the methodology presented. Examiners who are not well versed in these limitations may misinterpret the intent of the article and rely on its contents to calculate inaccurate speed/force. If there is any disagreement, and the SWGDE board rejects the premise that “ISO files often store inaccurate time”, please let me know and I can follow up to this response with several case examples that include MP4 files that were written directly by surveillance systems and have demonstrably inaccurate timing metadata. I hope it is clear that I present my comments above with full respect and appreciation to the SWGDE team.
SWGDE response: SWGDE extends the deepest gratitude and thanks to Mr. Fredericks for taking the time to not only review the document, but provide comments and numerous communications with the SWGDE video committee to improve the document. Based upon the overall comments and submitted videos, the SWGDE video committee also included language in Section 2 – Scope indicating, “This document specifically refers to the functionality of the metadata within files from recording devices, and not the reliability of the devices themselves”.
The video committee also believes that it is important to highlight the need for empirical, published research on many of the concepts introduced in this document as well as the meaningful comments. Like many other consensus-based standards/best practice documents, these concepts are based upon the information available at the time of publishing and are subject to change as additional research is conducted.
History
| Revision | Issue Date | Section | History |
|---|---|---|---|
|
1.0 DRAFT |
2021-09-16 |
Video |
Initial draft created, released for public
comment |
|
1.0 |
2022-06-09 |
Video |
No comments received, released as a final
publication |
Version: 1.0 (June 9, 2022)