Can you give more details about what precisely you want to do? The whole field of computer graphics is rather large. You're right about CG being heavy on mathematics though, but what maths you need does depend on what you want to do.