Bookkeeper
As soon as you can afford it, which should be within several months of launching your fledgling business, get yourself a bookkeeper.
You could save a few bucks by doing your bookkeeping yourself, but in my opinion that's wasted money. Unless of course you are running a bookkeeping consultancy!
For everybody else it is better to stick to the thing they can do best and spend a little bit of money on a professional bookkeeper.
The way to pick a good bookkeeper is to go and visit a bunch of them. Look at their offices. Pick a company that matches yours in size. No point in going to a major firm, all you'll end up doing is pay too much and be a very small fish in a large pond. By matching the size of your bookkeeper to your own you get direct contact with another owner of a small business, one that has lots of insight in how to run small businesses, for one because he runs one, secondly because he sees in to the books of lots of other small businesses, and even if you can't have the benefit of that directly the collected wisdom of such a person is not something that you could easily come by in another way.
When you find 'your' bookkeeper, you'll know. This is a very important step because your relationship with your bookkeeper will probably last as long as you are in business. So, don't pick one that's 63, and don't pick one that is your own age either if you're under 30. Pick one that has lots of experience, is mid 30's to early 50's. Their office should be orderly (if a bookkeeper can't keep things orderly run like hell) and there should be one or two other people working there besides the owner (and sometimes their spouse).
Ask some questions, see if you get answers that you can understand and if not if you can do follow up and learn. Keep in mind that if you really understand something that you can explain it to almost anybody that actually has an interest in the subject. If someone can not explain to you what you wish to know (and what they should know) then that's probably a sign that you should continue your search. Don't forget, you are in the drivers seat, you decide which bookkeeper fits you.
If after the first couple of weeks you decide that you and your bookkeeper are not a good match then by all means, switch! Find a better one. It's annoying to have to switch in the beginning, but it is *much* harder to switch later on. So better do it while it is easy, until you find a bookkeeper that matches you.
What does a bookkeeper do:
- they'll keep your legally required books for you
- they will file your corporate taxes
- they may file your personal taxes
- they will give you an annual report on your business
- they will help you structure your administration
- they will answer questions you have with respect to taxes and the general administrative side of your business
- In some countries they can assist with incorporation
If you read this far you should 
Good Stuff
Thanks for the input. Always good to learn from those who have made the mistakes and 'walked the line'...
Bookkeeper
I believe that your goals and intentions in this are written in good faith, but I don't believe you've had much experience in the realm of accounting. To provide the services of a bookkeeper requires a very rudimentary understanding of accounting that many college students posses shortly after taking their first position. To provide the service of tax consultancy requires a CPA or federal certification Also, If a person is just starting out he or she isn't likely to need an annual report issued by a separate objective firm or person. Should the person decide to take on debt, only then will the need for an annual report need to be fulfilled. The reports generated by the in house or outsourced bookkeeper would suffice for any person that is operating a small business without any debt, so long as the bookkeeper is providing quality service. For someone that is truly starting out as a consultant, I would recommend finding someone with about 1-2 years of experience working at a smaller public accounting firm or a small-mid size corporation. These persons would likely have a strong understanding of accounting in general and the requirements of a small business or a person working as a consultant.
I think you misread the 'get yourself a bookkeeper' bit
I don't mean that you should hire a bookkeeper as an employee, I mean that you find a bookkeeper that offers his/her services to others. Usually these are small offices with 2 to 5 people working there, most of these are CPA's as well.
Usually you can find these companies in the yellow pages under 'accounting services' or something to that effect. No need to go for KPMG, that will just cost you an arm and a leg, simply a small 'mom-and-pop' accounting office with a good track record will suffice.
As for needing the annual reports, this depends on the country, but in most places that I've got experience with you really need them, even if you are not taking on debt. They're part of your tax filing, they also help you to guide your business, and should you ever decide to become larger they can be used to establish the value of the entity you wish to incorporate (which can give you a huge tax advantage).