Anchor text can at times convey more about a referred site than the content on the referred site put together. It helps Google understand the context of the content present on the referre3d site, as Brian Dean states it in his blog about Google ranking factors.
Anchor text and the content around it plays a great role in helping Google to gauge the importance of the content present on the referred page.
Cheers.