If you are not doing power supply measurements every day, you probably don't have any specialized probe for it. At best you might be using two probes and math channels to get a cheap differential-like probe as the only thing needed for this is your standard probes and a pair of pass-through 50 Ω terminators. Dave went over it on EEVBlog channel better than I ever could.
While this approach might not be perfect, it has advantage of a decent result with minimum parts needed. Assuming you already have the probes, only other parts needed are pass-through 50 Ω terminators that come in handy for other stuff too. If you are lucky, your scope might even have them built-in.
Due to low cost and ease of measurement I actually prefer it to more common single-ended measurements with short ground spring. Not only I find probing with two hands easier than dealing with springy ground wire but it also alleviates the need to worry as much about noise sources. With advent of switched LED lightning single-ended measurements just got too annoying.
Disadvantage, beside the precision, is that all your measurements need scaling as your probes and termination resistor make a voltage divider. Since we already know one side of voltage divider (50 Ω terminator), we need to measure probe's resistance. From tip to BNC connector that will be in a few hundreds ohms range. Using the voltage divider formula, we get the ratio (for example I measured about 360 Ω):
ratio = (Rprobe + 50) / 50 = (360 + 50) / 50 = 8.2
While simply multiplying by ratio resulting numbers is not too much of a hassle and you could be done here, quite a few oscilloscopes will allow you to create a custom probe. As I usually use PicoTech 2206B, I will explain how to configure custom probe in PicoScope 6 so our on-screen result is already scaled correctly.
In Tools menu the first entry leads to Custom Probes window. There clicking on New Probe will start a wizard dialog. We use the standard volts unit and a linear equation. Our precalculated scaling ratio (7.6 in my case) goes under gradient and offset stays 0. Automatic range management is fine but software frequency filter we set to 20 MHz as PicoScope 2000-series has no hardware bandwidth selection. After naming the probe we can now select it in channel's configuration.
When dealing with the "standard" probes, be aware that most x1 probes have around 10 MHz bandwidth. As power supply noise is usually measured with 20 MHz bandwidth limit this is something to be aware of. However, the whole measurement setup is on a cheap side and this will rarely be a deal breaker. Yes, you might not see the best fidelity but you're gonna be in the ballpark regardless.
Further more, we are not interested in the DC component. If oscilloscope supports it, use AC coupling as we're not interested in DC values - only noise. This will allow you to use all ADC bits for signal you want to see. If oscilloscope supports only DC coupling, you won't be able to see much as, at ±5V, your 8 bits give you only 40 mV resolution and it goes worse from there. Unless you can get your analog offset configured (unlikely if your scope is so low end that AC coupling is unavailable), you can forget about any meaningful measurement.
Those doing it on cheap can check $140 PicoScope 2204A as it does support AC coupling. For example, assuming 100 mV range, you get quite adequate 0.4 mV resolution. As you go up in model selection you will get better bandwidth and bigger sample buffer but I would think even this works well for what we can get with this testing setup.
And I am mentioning PicoScope here specifically as other cheap scopes (whether PC-based or in MP3 player case) often don't have AC coupling and lie about bandwidth on a grand scale at a comparable price. While you can get around missing AC coupling by placing capacitor in series, it is much harder to work around bandwidth restrictions and a missing analog frontend. PicoScope is not perfect (and it cannot be at that price point) but they don't lie about their capabilities and their PC software is the best I've seen.
That aside, we're not done yet. To see our "differential" signal noise, we have one more step. Under Tools there is an option called Math Channels. Here we can select already predefined A-B function and finally we can see the noise in all its glory.
PS: To automate measurements a bit, you can use Measurements, Add Measurement. Selecting A-B as a measurement channel will allow you to add both peak-to-peak and AC RMS as the most useful values.
PPS: The lowest PicoScope model where you can do these measurement comfortably is $350 2206B with 32 MS buffer. Lower models have only either 8 kS ($140 2204A) or 16kS ($225 2205A) making any analysis a bit annoying.
PPPS: Great thing about using PicoScope for measuring power supply is that it can be easily isolated. Just disconnect laptop from power supply and your USB (and thus PicoScope) is floating. :)