Methodology for State Safety Data Quality Methodology Enhancements for State Safety Data Quality (SSDQ)

(Results as of: April 27, 2012 )

Definitions  
Revision Notes   		Click here for a Printer Friendly Version of this page. Print Friendly Version
The Methodology for State Safety Data Quality (SSDQ) was developed by FMCSA to evaluate the completeness, timeliness, accuracy, and consistency of the State-reported commercial motor vehicle crash and inspection records in the Motor Carrier Management Information System (MCMIS). The SSDQ evaluation uses a 12-month time period that ends three months prior to the MCMIS snapshot for each measure, unless otherwise stated in the rating description. Crash and inspection records were used in this evaluation if the date of the event occurred within the 12-month time period, not when the records were uploaded to MCMIS. The quality of this data is evaluated with each monthly snapshot and the States receive ratings of "Good," "Fair," or "Poor" for nine SSDQ Measures. Based on these individual ratings, plus the Overriding Indicator, each State receives an Overall State Rating. States also receive a Crash Rating that considers only the SSDQ Crash Measures and the Overriding Indicator. The methodology used to determine these ratings is provided below.

Overall State Rating
    State Safety Data Quality Measures
         Crash
    1. Crash Record Completeness
      1. Driver Identification
      2. Vehicle Identification
    2. Non-Fatal Crash Completeness
    3. Fatal Crash Completeness
    4. Crash Timeliness
    5. Crash Accuracy
         Inspection
    1. Inspection Record Completeness
      1. Driver Identification
      2. Vehicle Identification
    2. Inspection VIN Accuracy
    3. Inspection Timeliness
    4. Inspection Accuracy

    Overriding Indicator
    1. Crash Consistency
Crash Rating
    State Safety Data Quality Measures
         Crash
    1. Crash Record Completeness
      1. Driver Identification
      2. Vehicle Identification
    2. Non-Fatal Crash Completeness
    3. Fatal Crash Completeness
    4. Crash Timeliness
    5. Crash Accuracy

    Overriding Indicator
    1. Crash Consistency
State Safety Data Quality Ratings
Overall State Rating: Considers all nine SSDQ measures and the Overriding Indicator, except measures with a rating of "Insufficient Data." States receive an overall score based on ratings in each of the measures and the Overriding Indicator. A State that has received a "red flag" will be automatically rated "Poor". A State with at least one "Good" crash measure, one "Good" inspection measure, and no "Poor" measures receives a "Good" rating. A State with only one "Poor" measure will receive a "Fair" rating, and any State with two or more "Poor" measures will receive a "Poor" rating. (See image below.)
    Overall State Rating is based on ratings in each of the measures and the Overriding Indicator.

     

    The Overall State Rating is determined as follows:
    Rating Criteria
    Good  Good Rating Minimum of 1 Good Crash Measure, 1 Good Inspection Measure, AND 0 Poor 
    Fair  Fair Rating Maximum of 1 Poor 
    Poor  Poor Rating 2+ Poor OR Red Flagged 
    * States that are red flagged are automatically rated POOR overall.

    Back to Top
Crash Rating: Considers the five SSDQ crash measures and the Overriding Indicator, except measures with a rating of "Insufficient Data." States receive an overall score based on ratings in each of the crash measures and the Overriding Indicator. A State that has received a "red flag" will be automatically rated "Poor". A State with at least one "Good" measure and no "Poor" measures receives a "Good" rating. A State with only one "Poor" measure will receive a "Fair" rating, and any State with two or more "Poor" measures will receive a "Poor" rating. (See image below.)
    The Crash Rating is based on ratings in each of the five SSDQ crash measures and the Overriding Indicator.


    The Crash Rating is determined as follows:
    Rating Criteria
    Good  Good Rating Minimum of 1 Good AND 0 Poor 
    Fair  Fair Rating Maximum of 1 Poor 
    Poor  Poor Rating 2+ Poor OR Red Flagged 
    * States that are red flagged are automatically rated POOR overall.

    Back to Top

 

State Safety Data Quality Measures: Crash
Crash Record Completeness: Average of Driver and Vehicle Identification Completeness Evaluations
    The Crash Record Completeness Measure evaluates fatal and non-fatal crash records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on the completeness of driver and vehicle crash data reported to FMCSA. A State's rating is determined by evaluating the completeness of the driver data and vehicle data separately and then averaging these results together. The completeness of the driver data is determined by the Driver Identification Completeness Evaluation and the completeness of the vehicle data is determined by the Vehicle Identification Completeness Evaluation.

    The Crash Record Completeness measure is the average of the Driver and Vehicle Identification Completeness Evaluations.
     
    Driver Identification Completeness EvaluationThis evaluation determines the percentage of State-reported fatal and non-fatal crash records in the MCMIS database with complete driver information (i.e., the number of crash records with complete driver information divided by the number of crash records reported) over a 12-month time period. A State-reported crash record is considered complete when the following information is provided: driver license number, driver date-of-birth, driver first name, driver last name, and license class. If any of this information is missing, a record is considered incomplete.
    Vehicle Identification Completeness Evaluation This evaluation determines the percentage of State-reported fatal and non-fatal crash records in the MCMIS database with complete vehicle information (i.e., the number of crash records with complete vehicle information divided by the number of crash records reported) over a 12-month time period. A State-reported crash record is considered complete when the following information is provided: vehicle identification number, license plate number, vehicle configuration, cargo body type, and gross vehicle weight rating. If any of this information is missing, a record is considered incomplete.

    The Crash Record Completeness rating is determined as follows:
    Rating Criteria
    Good  Good Rating Percentage of completed driver and vehicle information is >= 85%
    Fair  Fair Rating Percentage of completed driver and vehicle information is 70 - 84%
    Poor  Poor Rating Percentage of completed driver and vehicle information is < 70%

    Back to Top
Non-Fatal Crash Completeness: Percentage of Non-Fatal Crash Records Reported
    The Non-Fatal Crash Completeness Measure evaluates crash records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on a ratio of reported to predicted non-fatal crash records reported to MCMIS. The number of reported non-fatal crash records was calculated using a 12-month time period that ends six months prior to the MCMIS snapshot date.

    A statistical model was developed using data from eight States to predict the number of non-fatal reportable crash involvements. A three-year average of fatal crash records was entered into this model to derive the predicted non-fatal crash record values. Due to the inherent variability of crash data reporting by individual States, the Non-Fatal Crash Completeness Measure is intended to serve as a guideline to assess whether a State's non-fatal crash reporting falls within an expected range, as determined by the model.

    As data from more States become available for analysis, the sample size for the model will increase, which should result in more precise predictions.

    The percent non-fatal crash completeness is determined by dividing the number of reported non-fatal crash records by the number of predicted non-fatal crash records:
     % Non-Fatal Crash Completeness   =    # Reported Non-Fatals 
     # Predicted Non-Fatals 

    The Non-Fatal Crash Completeness rating is determined as follows:
    Rating Criteria
    Good  Good Rating Percentage of non-fatal crash records reported is >= 75%
    Fair  Fair Rating Percentage of non-fatal crash records reported is 50 - 74%
    Poor  Poor Rating Percentage of non-fatal crash records reported is < 50%
    Insufficient Data  Insufficient Data State has < 15 average # of fatal crash records AND Percentage of non-fatal crash records reported is < 50%

    Back to Top
Fatal Crash Completeness: Percentage of Fatal Crash Records Reported
    The Fatal Crash Completeness Measure evaluates only those records that represent large trucks involved in fatal crashes that occurred within the calendar year. This measure determines a rating based on a comparison of the number of State-reported fatal crash records in MCMIS to the number of fatal crash records reported in the Fatality Analysis Reporting System (FARS). FARS is the national database of fatal motor vehicle crashes maintained by the National Highway Traffic Safety Administration (NHTSA).

    The Fatal Crash Completeness rating is determined as follows:
    Rating Criteria
    Good  Good Rating MCMIS as a % of FARS is >= 90%
    Fair  Fair Rating MCMIS as a % of FARS is 80 - 89%
    Poor  Poor Rating MCMIS as a % of FARS is < 80%
    Insufficient Data  Insufficient Data State has < 15 FARS records AND MCMIS as a % of FARS is < 80%

    Back to Top
Crash Timeliness: Percentage of Crash Records Reported within 90 Days
    The Crash Timeliness Measure evaluates fatal and non-fatal crash records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on the percentage of crash records reported to FMCSA within 90 days over a 12-month period.

    The Crash Timeliness rating is determined as follows:
    Rating Criteria
    Good  Good Rating Percentage reported within 90 Days is >= 90%
    Fair  Fair Rating Percentage reported within 90 Days is 65 - 89%
    Poor  Poor Rating Percentage reported within 90 Days is < 65%
    Insufficient Data  Insufficient Data State has < 15 records reported in current timeframe AND percentage reported within 90 Days is < 65%

    Back to Top
Crash Accuracy: Percentage of Matched Crash Records
    The Crash Accuracy Measure evaluates fatal and non-fatal crash records that represent interstate carriers and intrastate carriers transporting hazardous material and includes large truck and bus vehicle types. This measure determines a rating based on the percentage of crash records reported by the State over a 12-month period that were matched to a company registered in MCMIS. (Crash records entered per FMCSA's "Procedures for Entering Crashes without Carrier Identification into SAFETYNET" are not evaluated by this measure.)

    The Crash Accuracy rating is determined as follows:
    Rating Criteria
    Good  Good Rating Percentage of matched records is >= 95%
    Fair  Fair Rating Percentage of matched records is 85 - 94%
    Poor  Poor Rating Percentage of matched records is < 85%
    Insufficient Data  Insufficient Data State has < 15 records reported in current timeframe AND percentage of matched records is < 85%

    Back to Top


State Safety Data Quality Measures: Inspection
Inspection Record Completeness: Average of Driver and Vehicle Identification Completeness Evaluations
    The Inspection Record Completeness Measure evaluates level 1, 2, and 3 roadside inspection records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on the completeness of driver and vehicle inspection data reported to FMCSA. A State's rating is determined by evaluating the completeness of the driver data and vehicle data separately and then averaging these results together. The completeness of four (4) driver data elements are determined by the Driver Identification Completeness Evaluation and the completeness of two (2) vehicle data elements are determined by the Vehicle Identification Completeness Evaluation..

    The Crash Record Completeness measure is the average of the Driver and Vehicle Identification Completeness Evaluations.
     
    Driver Identification Completeness Evaluation This evaluation determines the percentage of State-reported level 1, 2, and 3 roadside inspection records in the MCMIS database with complete driver information (i.e., the number of inspection records with complete driver information divided by the number of inspection records reported) over a 12-month time period. A State-reported inspection record is considered complete when the following information is provided: driver license number, driver date-of-birth, driver first name, and driver last name. If any of this information is missing, a record is considered incomplete.
    Vehicle Identification Completeness Evaluation This evaluation determines the percentage of State-reported level 1, 2, and 3 roadside inspection records in the MCMIS database with complete vehicle information (i.e., the number of inspection records with complete vehicle information divided by the number of inspection records reported) over a 12-month time period. A State-reported inspection record is considered complete when the following information is provided: license plate number and gross vehicle weight rating. If any of this information is missing, a record is considered incomplete.

    The Inspection Record Completeness rating is determined as follows:
    Rating Criteria
    Good  Good Rating Percentage of completed driver and vehicle information is >= 85%
    Fair  Fair Rating Percentage of completed driver and vehicle information is 70 - 84%
    Poor  Poor Rating Percentage of completed driver and vehicle information is < 70%

    Back to Top
Inspection VIN Accuracy: Percentage of Valid Vehicle Identification Numbers Reported on the First Vehicle Unit within Inspection Records
    The Inspection VIN Accuracy Measure evaluates level 1-6 roadside inspection records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on the completeness and accuracy of the vehicle identification number reported on the first vehicle unit reported to FMCSA – all trailing units are excluded from this measure. A State’s rating is determined by evaluating the 17-character VIN using the "checksum" digit in the 9th character position. The checksum digit is used to determine if the VIN is accurate based upon an algorithm that uses the other 16 characters in the VIN. Any VIN with invalid characters (i.e. I, O, or Q) or an incomplete field (i.e. less than 17 characters) does not pass the checksum algorithm and is invalid. For this measure, records with all the same numbers (i.e. 99999999999999999) are also counted as invalid.

    The Inspection VIN Accuracy rating is determined as follows:
    Rating Criteria
    Good  Good Rating Percentage of completed and accurate VIN is >= 85%
    Fair  Fair Rating Percentage of completed and accurate VIN is 70 - 84%
    Poor  Poor Rating Percentage of completed and accurate VIN is < 70%

    Back to Top

Inspection Timeliness: Percentage of Inspection Records Reported within 21 Days

    The Inspection Timeliness Measure evaluates inspection records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This measure determines a rating based on the percentage of inspection records reported to FMCSA within 21 days over a 12-month period.

    The Inspection Timeliness rating is determined as follows:
    Rating Criteria
    Good  Good Rating Percentage reported within 21 Days is >= 90%
    Fair  Fair Rating Percentage reported within 21 Days is 65 - 89%
    Poor  Poor Rating Percentage reported within 21 Days is < 65%

    Back to Top
Inspection Accuracy: Percentage of Matched Inspection Records
    The Inspection Accuracy Measure evaluates inspection records that represent interstate carriers and intrastate carriers transporting hazardous material and includes large truck and bus vehicle types. This measure determines a rating based on the percentage of inspection records reported by the States over a 12-month period that were matched to a company registered in MCMIS.

    The Inspection Accuracy rating is determined as follows:
    Rating Criteria
    Good  Good Rating Percentage of matched records is >= 95%
    Fair  Fair Rating Percentage of matched records is 85 - 94%
    Poor  Poor Rating Percentage of matched records is < 85%

    Back to Top
Overriding Indicator
Crash Consistency †: Percentage of State-Reported Non-Fatal Crash Records
    The Crash Consistency Overriding Indicator evaluates non-fatal crash records that represent interstate and intrastate carriers and includes large truck and bus vehicle types. This "red flag" indicates States that have reported less than 50% of non-fatal crash records for the current 12-month period compared to the yearly average, based on the previous 36-months.

    The Crash Consistency Overriding Indicator "flag" is determined as follows:
    Rating Criteria
    No Flag    Estimate Reported is >= 50%
    Red Flag Red Flag Estimate Reported is < 50%
    Insufficient Data  Insufficient Data State has < 15 records reported in current timeframe AND State has < 15 records reported in previous 3 year average AND Estimate Reported is <= 50%

    † States that have an obvious and significant decline in crash record reporting will be categorized as Poor in the Overall State Rating and Crash Rating, without regard to their rating on other measures.

    Back to Top


Definitions:
Bus: A motor vehicle with seats for at least nine (9) people, including the driver's seat.

Checksum algorithm: The 9th digit of a 17-character VIN can be used to determine if the field is valid.

Fatal Crash: A crash where one or more persons dies within 30 days of the crash. The fatality does not have to occur at the scene of the crash. It includes any person involved in the crash, including pedestrians and bicyclists, as well as occupants of the passenger cars and trucks.

Interstate Carrier: Carriers that transport a commodity outside the state of its place of business.

Intrastate Carrier: Carriers that transport a commodity only within the state of its place of business.

Large truck: Any truck having a gross vehicle weight rating (GVWR) of more than 10,000 pounds or a gross combination weight rating (GCWR) over 10,000 pounds.

Non-fatal crash: A crash where one or more persons has non-fatal injuries requiring transportation by a vehicle for the purpose of obtaining immediate medical attention; or one or more of the vehicles were towed away from the scene due to "disabling damage". The towed vehicle need not be the truck involved in the crash.

Back to Top
Revision Notes: Improvements or changes made to the "State Safety Data Quality" methodology are documented below.

    September 24, 2010
    • Two (2) new measures have been developed that focus on inspection reporting. These measures evaluate the completeness of driver and vehicle inspection data (Inspection Record Completeness Measure), as well as the completeness and accuracy of inspection record vehicle identification numbers (VINs) (Inspection VIN Accuracy Measure).
    • The standards used to evaluate states' crash and inspection timeliness have been modified. The "good", "fair" and "poor" criteria have each increased by a value of five (5).
    • The SSDQ evaluation is based upon five (5) crash measures, four (4) inspection measures and one crash indicator.
    • The criteria used to provide a state's overall state rating has been modified. A state will receive a "good" overall rating if it has at least one "good" crash measure, one "good" inspection measure, and no "poor" measures. Those states that do not meet the "good" measure criteria and do not have any "poor" measures will receive a "fair" rating. The "poor" criteria have not changed.
    October 30, 2007
    The SSDQ Overall State Rating methodology has changed. It now includes the Crash Record Completeness Measure and the Non-Fatal Crash Completeness Measure. A Crash Rating has been developed to evaluate only crash data quality.
    June 23, 2006
    The Crash Accuracy Measure has changed. The crash accuracy analysis will now exclude all "Carrier Non-Identifiable" crash records. Changing this analysis will prevent records that are entered using FMCSA's guidelines on entering crash records without carrier identification from being included in the calculation of a State's accuracy measure.
    March 31, 2006
    The Crash and Inspection Timeliness Measure has changed. The timeliness analysis will now include "all" (both 'add' and 'change') records. Changing this analysis will allow all crash and inspection records uploaded to MCMIS within a specific timeframe to be evaluated. Previous releases only included 'add' records.
    December 23, 2004
    The crash and roadside inspection accuracy measures will include interstate carriers and intrastate hazardous material carriers in its analysis to determine the percentage of records reported in the MCMIS database that were matched to a motor carrier in MCMIS. Previous releases only included interstate carriers.

    Back to Top