Is there a way to delete the attachements without having to hunt through the tasks to find them?
Also it would be nice if it intergrated with live as there is 25Gb of space and you can have things like office live so you can edit the docs. Thinking along the lines of attaching a live document as an attachment rather than just putting it in the notes.
Yes, you can delete a file. Just click on the "Files" link in the left side of the website and then use the trashcan icon to delete the file. This will delete it from all tasks that it is attached to.