It's nice to know DoD hasn't got anything else they need to spend money & resources on.

I can see what they use MD5 and SHA-1 hashes for - they want to find partially overwritten or erased files. But can you imagine Mr. FBI going before a jury to argue that a computer has a kiddie pic, admitting there isn't really a pic, but a small chunk that possibly came from a kiddie pic, and then defense talking about SHA-1 hash collisions (one has been found!), etc etc etc? If I'm a juror it would be time to catch some and in deliberations they've either got a pic or they don't. And that's if the judge even lets the case get that far.

Moreover, what are they going to do when Internet worms start dropping kiddie pics on unknowing systems by the millions? That's mentioned below but it's a huge problem. The consensus in the AV industry is that even the best anti-virus programs have no more than 20%-30% coverage - it will never work for the FBI to say "we didn't find a worm so there wasn't one".

PS. On the second thought these are cops - I may be overestimating what they're using hashes for. They might just be matching ordinary files which is no more than a high school class project in CS, not something you pay General Dynamics for many man-months of development.

http://story.news.yahoo.com/news?tmpl=story&cid=1093&ncid=1093&e=6&u=/pcworld/20050125/tc_pcworld/119434

Defense Department Uses New Tools to Fight Child Porn
Tue Jan 25, 6:00 PM ET
Paul Roberts, IDG News Service

Faced with competing demands for efforts to combat terrorism, the U.S. Department of Defense (news - web sites) has spent $500,000 and put its top cybercrime researchers on a program to make the fight against child pornography more efficient, according to officials at the agency.

The DOD's Defense Cyber Crime Center launched the Known Image Database System, or KIDS, last July to hasten the identification of pornographic images depicting children. Another benefit of the program is to relieve the workload on swamped computer crime investigators.

Use of the system is pioneering investigative strategies and tools for cases involving huge quantities of seized data, and may yield techniques that help the DOD prosecute other kinds of cases as well, including cyberterrorism and espionage, according to U.S. Air Force Lieutenant Colonel Ken Zatyko, director of the DOD's Computer Forensic Laboratory.

The emphasis on fighting child pornography is the result of a flood of such cases that has swamped DOD forensics examiners, as well as their counterparts in federal, state, and local law enforcement, said Bill Harback, a senior forensic examiner at the DOD Computer Forensic Lab in Linthicum, Maryland, who spoke in January at a DOD Cyber Crime Conference in Florida.

In fact, as much as half of all criminal forensic investigations done by staff at the Defense Cyber Crime Center involve child pornography, said Steven Shirley, executive director of DC3. Behind the DOD program is a surge in child pornography cases, driven by the availability of inexpensive computer hard drives, digital cameras and scanners. The new technology has pushed child pornography cases into the hands of computer crime investigators, Shirley said.

"Before the PC, the only people who were concerned with child pornography were customs and the postal inspectors. Now it's every police agency," he said.

Sifting Through Hard Drives

Forensic investigations of child pornography cases typically require investigators to sift through images on seized computer hard drives and identify pornographic images that depict minors. The work can take weeks or months to complete for a single case, which can jeopardize some criminal investigations and wear on investigators, Harback said.

Work on the child pornography cases siphons investigators from other high-priority cases such as terrorism, homicides, espionage, and major government procurement fraud, Zatyko said.

For KIDS, The DOD contracted with General Dynamics in May to create a large database of known child porn images that can be identified by message-digest algorithms, also known as "hash sets," which are unique alphanumeric values that identify each image based its content. General Dynamics staff worked for a month to develop new, accurate hash sets for the database of images, which the DOD maintains on a high-capacity storage area network, Zatyko said.

DOD provided General Dynamics with the equipment and facilities to develop the new system, and a person to manage the hash sets created for the program, he said.

The KIDS hash sets are used to rapidly compare a suspect image or images on a hard drive to known child porn images, freeing up investigators to focus on other images and data found in the search, such as Web surfing and Internet search history, which can be used to establish that the computer owner was actively searching for child pornography. Investigators also look for malicious code or Trojan horse programs, which could result in images being planted on a computer without the owner's notice, he said.

The new hash sets shorten the time it takes for forensic examiners to study seized images from 90 days to two weeks. The goal is to tell case agents as soon as possible if there is evidence of child pornography so that they can decide whether or not to pursue the case, Zatyko said.

Using hash sets to hasten image comparisons is nothing new. Both the National Center for Missing and Exploited Children and the U.S. Federal Bureau of Investigation already maintain databases of images and hash sets. But the DOD is using newer, highly secure mathematical algorithms, such as the MD5 Message-Digest Algorithm and Secure Hash Algorithm, or "SHA-1," to create hash values that are more accurate and that will provide more reliable evidence in court cases, Zatyko said.

The DOD is also researching the legal ramifications of making the image database available to other government law enforcement agencies, such as the FBI or state and local law enforcement, Zatyko said.

Mathematics to find kidporn. Yeah, right.