As mentioned above, you don't tell us of any specialized skills or knowledge that you might have.
At one time, if one had a source to buy goods wholesale, the mail order business was probably the easiest to start with limited funds. Location was not important; one's kitchen table was sometimes sufficient. The main start up costs where advertising and printing a catalog. Not much inventory was needed because if people received their order in a couple of weeks, they considered that good service.
That has all changed; people now expect to receive their orders in just a few days and I think they expect most any business to have a web presence. I think you would have to have a rather unique product, or at least have sufficient skills at copy writing that you can make a pedestrian product seem unique.
I don't know the cost of having a professional web site produced, but I would not be surprised if it were not around the total of the amount of funds that you have available.
I imagine that if you have some specialized knowledge and a source of goods to sell that eBay would be one avenue to explore, but you have a lot of competition there. Some folks are probably doing very well selling at eBay, but it is probably like so many business endeavors that 10% of the sellers are making 90% of the money.
I have seen one small business that seems to be growing and my guess is that he is doing well and will do even better in the future and that is Kent Wang
www.kentwang.com , so I think it is possible, just not easy. Of course, if it was easy, everyone would be doing it.
If buying and selling is not feasible, personal services is another area that one can enter with a relatively small investment. Things like party planning, wedding planning, and image consulting come to mind. I expect that if you had the knowledge and contacts to provide this type of service that you would not be asking us for ideas.
I don't mean to discourage you but I think if someone here could see a good business that they could start for 5 or 10K that they might do it themselves - I probably would.
I don't know that any of the above has been helpful, but there is one piece of advice that I can give you that I strongly believe in, and that is to avoid partnerships. Often in fields like law and medicine the advantages of a partnership outweigh the disadvantages, but for the rest of us, they seldom work out.
Best of luck, Jim.