If you didn't install the gutters, look at the way they're put together, particularly how the downspout attaches to the gutters, before you begin. Then you'll know if you can use a screwdriver or power drill to remove screws, or if you'll have to drill out rivets instead.
Remove the Downspout
Remove the downspout first. Remove the elbow where it's attached to the drop outlet on the gutter, and then the downspout assembly can easily be removed and placed out of your way. You need to do this step first because an attached downspout makes removing the gutter more difficult and awkward. It can even pull it down and potentially damage the fascia where the rest of the gutter is attached. Next, remove the attachment strap that holds the spout to the siding.
Remove the Gutters
This is as simple as removing each gutter hanger or other device that holds the gutter in place. This is usually done by removing screws or using a pry bar to remove nails. Remove the gutter once all hangers are gone.
You can use caulk to fill holes left in the siding once the gutters and downspouts are removed. Smooth this flat and paint with a matching color when dry. If the fascia was damaged, use wood putty to fill holes. Once dry, sand and repaint the wood.