By default, with delivery turned on & a delivery charge set, all orders going to that zone will be charged the delivery fee. The only way you can get around that is if you create a coupon that credits the dollar amount of the delivery charge.
For example: Customer is being charged a $5 delivery charge. You create a coupon that offers $5 discount based on ordering $X amount. The customer can apply that specific coupon code & get a $5 credit on their order. This will have the same end result effect.