I recently created my own Google Apps for Business account for free (my company has less than 5 employees). I want to test Google Play Private Channel but it isn't possible with a free account, I can get a free trail for 30 days. I was wondering if I have to pay after the trial or I go back to my free account?
No. There's no longer a free edition of Google Apps. If you had signed up for a trial full account before December 6 2012, you could have cancelled the trial and gone back to the free edition. Since Google no longer offers a free edition, and December 6th is more than 30 days ago, you cannot revert to the free edition.
From the Google Support article:
After the last day of your trial period, you can no longer downgrade to the Google Apps free edition.
You can cancel your account or delete your domain.
If you have not entered your billing information and you do not explicitly cancel, your account will expire. However, if you've already entered your billing information, you should cancel to ensure that you're not charged at the end of your trial.