Windows Mail by default plays a sound when a new message arrives. It also provides option to disable this sound. This can be done by using the Windows Mail options.
Click the Tools menu –> Select Options from the menu list
Navigate to General tab and then to the Send / Receive Messages section. Un mark the checkbox with label as Play sound when new messages arrive. This would disable the sound when a new messages arrives in Windows Mail.