Someone looking very ragged and dirty is often called haggard.