I'm not really sold on the impact of dietary phytoestrogens on the male body. I accept that this may be dependent on those men who already have some amount of gyno from puberty, who would be especially sensitive to estrogen. My question about this would be do you feel like the actual gland inside is tender and swollen? Or are you just getting increased fat storage in the chest area? For the most part, your fat storage patterns are based on your genetics, through your natural hormone levels, so it is conceivable that an increase in estrogen levels could lead to a slightly more feminine fat storage pattern. So if you are simply experiencing more fat storage in that area and not actual gyno, then I can possibly see this happening but I do think it is definitely unlikely to be occurring.
Based on my own experience, I've eaten a lot of soy at one a few points in the past and I've had periods where I regularly drank beer and never grew titties. It might be worth noting that I've never really gotten too high on the bodyfat percentage scale though in comparison to the average male, so perhaps that has some bearing on my anecdote. Clearly you will find more aromatization in men with higher amounts of adipose tissue, so that could have an impact on all of this as well.
However, over the past 8 years or so of experimentation, I find that when I partake in alcohol (beer and mead primarily, sometimes wine and very occasionally hard liquor) that I actually tend to drop weight rather than store any additional fat. I have used beer during refeeds in the past when doing RFL style diets with some surprising results, in that it seems to trigger a whoosh effect when drinking it at the end of a carb load. But no boob growth at all.
I don't think occasional beer intake will cause you to grow boobs and I've never seen this happen in people unless they were packing in calories. If you are controlling calories and only drinking in moderation, it really shouldn't be an issue. Obviously in your example, you were talking about daily intake for a few weeks straight. I'm not really sure that is long enough for altered estrogen levels to have such a pronounced effect though who knows. I know a trans women who started estrogen and spiro quite some time ago and she didn't notice anything really dramatic in that area for at least a month. Granted she was not on higher calories, but hormone therapy for transitioning is without a doubt a much larger change to estrogen levels than simply drinking beer each day for a few weeks.