What penalties would I face?
Yes Aidan, you can. Assuming you are referring to a Non-Qualified Annuity (for example: not IRA, or Roth dollars), the first dollars out would be considered your gain and taxed at ordinary income tax rates. Additionally if you are under age 59 1/2 , an early withdrawal 10% penalty tax would also apply to the gain. The penalty is only assessed on the amount you distribute that represents a gain.
That penalty, but not the tax, can be avoided if you elect distributions under section 72(t) of the IRS code. In doing so, you agree to take distributions for a minimum of 5 years or to age 59 ½, whichever is longer. The distributions have to be level. If you do not go the full time period required, the 10% penalty is applied retro-actively to the very first distribution. So you have to be careful if you are using an annuity where the value can fluctuate as if you run out of money before the five year period (or age 59 ½ if later), bingo – you owe the penalty and back penalties. Depending on your needs, annuities with guaranteed income benefits may work best for you to ensure avoidance of the penalty if that is the route you care to consider.
I certainly agree with David's comments and I add one additional caveat. Many annuities include a surrender charge depending on how long you've owned your annuity. Check your annuity contract to make sure you are past the surrender period. If not, annuities generally allow a certain amount (%) to be distributed annually before the surrender charge kicks in.