I see that most answers so far have focused, quite reasonably!, on not breaching any contract (or law -- jurisdictions vary greatly!) with respect to your employer's rights -- including some unwritten expectations that your employer may have. Employers' attitudes may vary greatly based on what kind of outside activity you're doing -- for example, some will look benignly at work done to help charities or other non-profits, and/or open-source work, or such things as writing programming books, being active on Stack Overflow or other programming venues, and the like -- perhaps up to the point of benignly tolerating the use of work-related resources for such activities that bring reputation and goodwill but probably not much money (I guess some technical books do make money, but typically it's not much anyway;-).
Employers are less likely to smile and nod approvingly, at the other extreme, if your freelancing is (or is perceived to be) in direct competition to the employer's business (and they're quite likely in such cases to have laws on their side, even apart from plenty of employment documents they've probably been careful to make you sign). Of course, there's a large gray area in the middle. In general, asking your manager may be a good way to get specific information on whether any specific activity would, or wouldn't, get you fired, sued, and/or otherwise penalized (formally or informally) at your place of work.
But, making sure that no official or unofficial penalties will come from your outside activities is just the beginning, of course -- the lack of such penalties is a necessary but not sufficient condition to make such outside activities worthwhile. To make them worthwhile, in addition, you need specific motivations -- such as, such burning passion for programming that you can't get enough of it at work, or fierce love for some specific technology that you don't get to use at work... and/or, a strong attachment to something good that you think will result for your programming, rather than for the programming itself.
For example, say that you'd like to volunteer some hours every week to help at your local soup kitchen -- I don't know of any employer who'd object to your doing that in your spare time! -- and after spending a while volunteering for such duties as kitchen cleaning and serving, you find out that they're spending an inordinate amount of volunteers' time to manually keep track of inventories, donations and warehousing. You consult with the folks managing the kitchen and they agree it would be great to have an inventory-tracking program running on their old beat-up PC, they just don't think there is any such program, suitable for their needs and resources, that's available for free. There may not be, but, if you used to be good at DOS programming, you could make one for them, and they'd love you to. Now what?
Now, dusting off your ancient DOS-programming skills and doing a totally mundane, text-screen inventory management application for the soup kitchen is definitely not the sexiest programming work in the world -- but then, neither is serving food and cleaning kitchens, but, they're no less needful and helpful kinds of volunteer work: here, the motivation for your "freelancing" would rather be a selfless one... using your spare hours, that you decide are available for volunteering, in the most effective way to help keeping needy people decently fed. Whether you WANT to do this, just like whether you want to do any other kind of volunteer charity work, is obviously totally up to you, but if that's the way you choose to help your community, I don't think anybody could gainsay that, and I'd be astonished if an employer did (though, again, checking w/your manager is still advisable).
Sure, it IS more work and less "personal life" -- just like any other charity volunteer work you might be doing -- and it would be very wrong to express any disapproval of people who choose NOT to spend any of their time this way, and would rather relax, chill out, spend time with friends... perfectly natural and normal preferences. But, if your choice IS to devote some time and energy to volunteer work, programming may in some cases be a way you can leverage your specific skills to be most helpful and effective in your volunteering.
Other motivations you bring into play, such as learning new technologies, could also be combined with non-profit work in some cases (e.g. you could do the inventory app as a way to learn Google App Engine -- as long as that old PC can run a browser and has net access, that might actually work better than running it locally, AND give you a chance to learn GAE which you might not get at work). Only "making more money" seems to be a desire you could satisfy exclusively with commercial engagements... and I suspect it's also the one that's likeliest to put you in trouble with your employer. Unless you're really hurting for more money (in which case looking for a new job or finding out what you'd need to do at your current one to get a promotion and a raise might be safer and more effective approaches), I do recommend you consider opportunities that can give you "everything except" the "more money" bit!-)