After a bit of Google, however, I found a useful tool: sc.exe (Service Controller Tool) come with Windows Resource Kit. This command line tool make deleting service very easy, just run:
sc.exe delete [service name]
Notice: the service name for the command line is not the same as "Name" you see in the list (this is usually the "Display Name"). You need to double click the service to open the properties page. And there you will find "Service Name".