Yeah, you just need to replace the tensioner. You can do this with a stock automatic one which will eventually just go bad again, or you can use a manual adjuster which wont go bad.
You can get one here, and get a little more info as to why a manual will work opposed to an automatic.
http://cbrzone.com/sprockets.html