It helps to mentally concentrate on the muscle you are working. Focus on feeling it while it is being worked. That can help to adjust your form to maximally work the muscle. Also, be sure that your form is correct and that you are using a moderately slow eccentric rep, i.e. 3 to 4 seconds on the lowering of the weight.
You mention multiple sets. I am personally not a fan of them. I prefer to perform multiple exercises than multiple sets of the same exercise. My opinion is that gives your muscle a more balanced and full workout when hypertrophy, as opposed to strength, is you primary goal. There are others that disagree so you need to experiment and see what seems to work best for you. In either case, I am not sure that there is much benefit to, and perhaps there is even a negative to, working that muscle more after that pump has been reached.
As far as your muscle seeming to be smaller the next day, I assume that you mean smaller than when it was pumped up during your workout. That is normal. The pumped up feel and look comes from blood coming to the rescue of the muscles that have been assaulted through work. If you mean that your muscle is smaller than it was before you worked out, I am not sure what could cause that unless you are not adequately replenishing the glycogen stores used in your workout. If that is the case, you might want to increase your carb intake.
I have also seen instances where beginners seem to shrink a bit when they first start working out. Usually this is temporary (no m ore than a month) and they begin to grow when they get their diet straightened out.
Hope this helps a little.
Just don't give up. You will grow.