VAT on Lunch

I found out pretty quickly after going to lunch a few times that when places where asking me was I eating in or taking out, it wasn't just so they could offer me a paper cup or a tray. There's 20% VAT on any food just to sit down and have it there! Whether the shop wants to charge you or not - I'm used to the idea of having to pay more to sit in and drink you're coffee, but this is a little ridiculous. Obviously the places selling the food agree. How many times have said I was getting take away and then changed my mind and sat down..