Gichin Funakoshi (the person who brought karate to North America), through his travels, met with, and compared system and philosophy notes with Jigaro Kano (the founder of Judo). This is when striking skills were added to Judo, and conversely, grappling skills to Karate. It was Kano that developed the "belt system" that martial arts have adopted today. Both Kano, and Funakoshi were school teachers. Kano developed a grading system comparable to school, consisting of "belts" that represented levels of training. Funakoshi copied that idea, and developed a system of 10 preparatory classes ("kyu"), and 10 grades of competence ("dan"). The kyu levels counted backward from 10 (Jukyu - 10th class - white belt) to 1 (Ikkyu - 1st class - brown belt). The degrees of black belt count upward from 1 (shodan - 1st degree black belt) to 10 (judan - 10th degree black belt).
Today, belt colors, their order, and the uniform colors associated with training will vary from school to school. Traditionally, however, martial artists wore white, therefore, for cosmetic purposes, they would start with a white belt. In China, a white sash was worn while practicing, so this is probably where this idea stemmed from.
Originally the belts were:
1st white (jukyu) 2nd black (nidan) 2nd white (kukyu) 3rd black (sandan) yellow (hachikyu) 4th black (yondan) orange (shichijyu) 5th black (godan) green (rokkyu ) 6thblack (rokdan) blue (gokyu) 7th black (shichidan) purple (yonkyu) 8th black (hachidan) 1st brown (sankyu) 9th black (kudan) 2nd brown (nikyu) 10th black (judan) 3rd brown (ikkyu) 1ST BLACK (SHODAN)
Note: In many Japanese systems, the red belt is worn by those above black belt. In Chinese arts, typically a purple belt is worn to signify the level above black belt. Again though - this is highly personal, and varies from school to school.