Your first reaction of what to do when you actually catch your puppy "in the act" is critical. It is very important to know that you should under no circumstances ever divide your puppy for something that you did not see him do. It is your fault. Yes, that is right. If your puppy has made a mess somewhere he should not have it is because you were not watching close enough and you allowed it to happen.
It is no good getting upset at the puppy. It is yourself you should be angry with. What you should do in this situation is calmly pick up your puppy and say "no" in a stern manner. This must be done without raising your voice or trying to indicate to the pup that you are mad. Then simply carry them to where it is that they should have done their business. If this occurs while the puppy is in the act it is always a good suggestion to tuck his tail under him when you pick him up. This will help them to be able to stop in mid-action.
Careful consideration should be given to what you do when you do catch them in the act. Just remember the number one rule and never punish them for something they did if you do not catch them in the act. And remember to praise them with "good dog!" when they finish doing what they are supposed to be doing. It is important to realize that just as in physics so it is with training puppies. For every action there is an equal and opposite reaction.
For the purposes of training puppies it is important that your actions are always beneficial. When you approach your puppy and severely scold them for what you perceive as a mistake, you will scour the heck out of them. At this point you are moving in a wrong direction. Be careful not to over react. The same goes for the idea of rubbing the puppy's nose in his mess. It is extremely difficult for the puppy to understand why he is in trouble for doing what is natural to him. This is never a good idea whether or not you catch them in the act or not.
When something like that is experienced by your puppy he can only perceive it to the limits of his intelligence. He does not know the difference between having his nose rubbed in his mess left an hour ago from rubbing it in the mess at the park left by a dog a whole day ago. Punishment is never a very effective part of training puppies. It does not speed up the process and more often than not will cause your pet to become afraid of you. In the least your pet may develop a nervous habit every time he needs to use the bathroom.