"Inwardly" doesn't sound terribly idiomatic to me here, probably because it's a somewhat formal or literary word and "freaking out" is very colloquial. I'd use "internally".

That being said, it would sound more natural to my ears to say something like, "He was freaking out, but tried not to show it." That expresses the same idea.