Here is a simple power regulator circuit you can solder together in a few minutes from common components (available at Radio Shack). It will take the output of a standard AC adapter that produces DC from 8 to 12 volts and turn that into a regulated 5 volts. For other voltages, other versions of the main part (the 7805) can be used to provide other voltages, as long as the input voltage is around half again as much as the rated voltage of the device.
You need a 7805, (RS #276-1770) and a pair of 0.1uF capacitors (RS #272-131), And some source of 9 to 12V power. That could be an AC adapter that puts out 9 to 12 volts DC (e.g. RS #273-1774 or any of the ones you have laying around) or a battery, set of solar panels, etc... And you need a soldering iron and solder, or some really tiny screw terminals. You can try just wapping the leads around each other tightly and then bending the result over to keep it from unwrapping.
Here is how to wire up the parts without a circuit board, kit or whatever.
The picture on the right shows a version with two capacitors of different value on the right side, and with the center and right leg left straight so they can be plugged into the power strip on a standard "White board" breadboarding system. Here is the schematic diagram:
Depending on what you are powering, you may find that the 7805 gets hot. If it gets hot enough to cause a burn, shut it down and try adding a heatsink to the back of the unit, bolted to the large metal tab. There is already a hole in the tab for a bolt and nut to hold it tightly to the heatsink. Your heatsink can be just about any substance that will conduct away the heat from the 7805, but some things work (much) better than others. Usual metal is better than any other material.
If your supply voltage is more than +12V, or if you are drawing more than a few hundred milli Amps, you should probably add a heatsink (it doesn't take a big one) as the 7805 may get hot and shut down.
If your only source of power is more than 12 volts, you can augment this kit by replacing the 7805 with a switching regulator like the Murata OKI-78SR-5/1.5-W36-C^ which can take up to 36 volts and still provide 1.5 amps at 5volts.
Interested: