This has happened to me in the past - sometimes randomly, sometimes consistently. After a quick search I found that all you need to do is make sure you select "Workflow" as the type of code you want to attach to, as described nicely in this article:
How to solve Visual studio crash on attach process for debugging a sharepoint workflow