That is a great idea, and if it's that small it should fit into one of your on-board tool bags.
Typically, that light comes on when the voltage difference between the battery and the alternator is somewhat close, meaning that the alternator isn't contributing much to charging the battery. It will sometimes glow faintly at night if the alternator is just loafing along or if the connections are corroded or something. The only way to cure the connections is to shine them up, including: Both Battery connections, battery cable at the starter, both connections on the alternator, the connection at the indicator lamp in the Tach and the +12V feeding the other side of the dash indicator, and the ground cable between the transmission and the chassis.
If THAT doesn't cure it, then you can have someone (an automotive electrical shop) do a load test on the alternator (usually $25 bucks) to see if it's putting out proper voltage under load (which Jack's voltmeter can do, if you turn on everything in the car while it's running - the Auto shop uses a big-ass resistor as a load).
The only way to tell what's going on is with a voltmeter, used as Jack mentions. The "resting" (non-running) battery voltage should be around 11.4-11.9 VDC. The running (say, around 3K rpm) voltage should be 2 volts higher, or around 13.5-14.2 volts. All of those voltages can be tested anywhere that you can find 12 volts - a "Power Plug" would be perfect.
Let us know what you find for a volt meter like Jack mentioned. Sounds like something I could use in my tool bag.