First of all, let me say this: Madison has a significant brunch selection. Among our favorites is Graze (for the varied menu and great view of the capitol through a huge plate-glass front), La Baguette (for their amazing Quiche), Zuzu (because it is around the corner and has a breakfast egg sandwich special that's pretty cheap), Lazy Jane's (for the Willy Street hipster in us), and most recently, Sophias (for the pastries and sweets).
All that being said, Madison Sourdough stands out. First, it looks like it's in an old car repair garage. It has two huge front windows that fill the bay doors and let in lots of light. Second, it's fundamentally a bakery, and so their breads are the best. When we go back, we're going to make sure we require bread so we have an excuse to... um... buy their bread.
Thirdly, their breakfast sandwich. It's a breakfast egg sandwich. Like Zuzu. Except amazing. Part of it is the croissant, which would have been a great brunch even on its own. But the chessy-eggy goodness inside was just as important. If I had to visit the places we've already been for brunch, I would go to Madison Sourdough, Madison Sourdough, Graze, Madison Sourdough, La Baguette, Graze, Madison Sourdough, and Madison Sourdough. I'd do that 100 times, and then go to Zuzu, to remember what normal food tastes like, kind of reset the palate.