To clean hot water baseboard heaters, you need to turn it off first, to prevent electric shock and injury. Remove the covers, which can be done easily by undoing the screws holding them down. Wipe the cover clean with a wet cloth. Next is to clean the dissipation fins with a vacuum brush, until the first layer of dust is removed. Then, get a can of pressurized air and blow around this area, in order to remove small dust particles found in between the fins. Get your vacuum cleaner and do another pass, so you can get the last remaining dust particles that were blown off using the can of compressed air.