Recently, my installation of Witch had been really sluggish. Being a window switcher, Witch relies on applications to report their open windows. If an individual app is slow then Witch feels slow. Last.fm does not play nice in this regard. 
Thanks to the Witch user forums, I discovered this issue and found that you can have Witch ignore certain apps. Skipping Last.fm leads to normal zippy behavior in Witch. I’m posting about the solution mainly so that it can better surface in Google.