What you have so far is a great first pass to break up the smoothness of the cloth. Now you will have to go in and add more defining wrinkles. These will start to define the materials.
(Note: The folds under the butt arn't very pronounced in this picture)
Pay special attention to these areas. These wrinkles are where you will get most of your weight from. Stick close to your reference to make sure it look like HIS pants, but most of the reference I looked up is pretty dark like the image Benjyshwa posted.
Good work so far! Keep it up
