I have used exercise balls for many types of exercises and found them to be a really useful piece of exercise equipment. Being a person who has always had very active work life involving constant movement, Yoga, dance and physical activity (never worked sitting at a desk before) I find that I get really uncomfortable sitting at my computer desk and I can really feel the terrible effect it has had on my posture in the last few years since I have been doing a lot of administration work, the most significant and noticeable physical difference being joints becoming stiff.
I did, originally, have an exercise ball as an office chair until my cat punctured two (whilst sharpening his claws) and I got fed up with replacing them! I tried an ergonomic kneeling chair which was good for posture but I found it cut off the circulation to my legs. Somehow I ended up with a weird plastic office chair which is really uncomfortable and encourages me to slouch.
I have finally taken action to replace my cheap office chair with another Yoga Ball and it is heaven. I love the fact that my body is in motion while I am on the computer, the circulation in my legs is much better and my body feels much more comfortable. I find that I get up and down more often and bounce a bit from time to time. If I need to get something I get up off my chair rather than roll to it.
If you do use a Yoga Ball as a chair I would recommend you first take a look at this Wikipedia article which has sensible safety and ergonomics suggestions http://www.wikihow.com/Use-an-Exercise-Ball-As-a-Chair. Especially the fact that you need to swap over from time to time to a normal chair to avoid muscle fatigue. I use two computers in the office throughout the day so I hop from a chair to a Yoga Ball. You also need to take care as a fall off a Yoga Ball is not much fun and you can really hurt yourself if it rolls out from underneath you.
I have read articles and studies about this, as usual some say there is no difference between an office chair and an exercise ball, some recommend the use of an exercise ball and others say its bad for you. Personally I am really happy and much more comfortable with this arrangement, it suits me. Of course, we are all different and this may not suit everyone. I always advise clients to make sure they listen to their body and stay aware of how they feel at all times. This way they can avoid doing something which is not suitable for them as an individual. If you have any kind of injury or physical problem I would always advise that you check with your health professional first before choosing the right kind of chair for your work station as one size does not fit all. See the link below to some comments from different people about their experience with this.
(P.S. The cat is banished from the office now!)