The paceman was found guilty of breaching Cricket Australia's code of conduct during Victoria's Sheffield Shield game against Queensland last week.
It was not clear what he said, but the governing body characterised it as "personal abuse of a player while fielding".
Given it was his third breach in the past 18 months, the incident triggered a one-match suspension.
"We have a duty to uphold the highest standards of behaviour and the action taken in this matter demonstrates that," CA's head of integrity and security Sean Carroll said in a statement.
Pattinson's ban opens the door to Mitchell Starc to play in the Test at Brisbane starting Thursday.
The two were seen to be competing for one bowling slot to partner Josh Hazelwood, Pat Cummins, and Nathan Lyon.
"I made a mistake in the heat of the moment," said Pattinson, who admitted the offence.
"Straight away I realised I was in the wrong, and I apologised immediately, both to the opponent and to the umpires.
"I have done the wrong thing and accept the penalty. I'm gutted to miss a Test match, but the standards are there for a reason and the fault is mine." CA said no replacement will be called up for the Test squad.
For in-depth, objective and more importantly balanced journalism, Click here to subscribe to Outlook Magazine