Panic is a horrible nonsensical mechanic. In a time of crisis like an alien invasion, the panic of the public wouldn't mean jack. Your soldiers, scientists and engineers aren't going to just stop working because some people are panicking.
This is like the AngryJoe review of Xcom:
Noah: "I regret to inform you that Naru, Palau, Belize and Papa New Guinea cut their funding and withdrawn from the Xcom project."
AngryJoe: "God I really should have paid attention more in history class. Where the hell are these countries? Uh that's unfortunate sir. But it's hard to defend the entire planet. I'm losing men left and right. I have what like 30 guys left. but at least the major countries that everybody knows is still with us and frankly sir we're the best hope that you have."
Noah: "That may be so Commander but we have decided that you failed to combat the alien threat."
AngryJoe: "But you don't understand, there will be nothing left."
Noah: "We've heard enough Commander Joe."
AngryJoe: "No, no. That doesn't make any sense. Who will protect you?"
Noah: "We're taking away your toys and then we're letting the aliens have their way with us."
AngryJoe: "But. But."
Noah: "Oh yes, in the butt. The Counsel has spoken."
Panic should be replaced by a type of beleaguered status tracking the state of the war against the aliens. The less you do the more that the militaries of the war get attacked by aliens and have to fight back themselves. By shooting down UFOs, launching attacks against UFOs in orbit, launching strikes on strategic defensive or production or logistical points in territory controlled by the aliens you would reduce the ability of the aliens to engage in warfare on Earth's surface. This would also mean that the aliens are launching fewer or even no missions on Earth. Maybe there are even other planets facing the same predicament or a secret rebel alien faction that disagrees with the leadership constantly invading and destroying other species. The better you do at combating the alien menace the higher the chance that these rebelling groups will try to contact you.