I used to have a 2007 WRX, and even drove a 2013 before settling on my Charger!
They are just two totally different cars, each with it's own benefits. I personally think the Charger is the overall better car (all things considered), but thats not a knock on the WRX either.
They drive very different. The WRX is very nimble on the road, a combination of it's AWD setup, smaller size, and very quick steering. It is a like a go-cart on wheels and extremely entertaining.
A car the size and weight of the Charger is not going to compare the same. This much car just won't feel that nimble. That said, the handles very well in it's own right. (Honestly, it handles amazingly considering how well it cruises, and how big it is, easily outhandling some other expensive and good handling cars I tested such as a Mercedes C300 with sport package, and Volvo S60).
I needed the size, so the Charger was a no brainer. Beyond that, IMO, the Charger is made with much higher grade materials, is far smoother and quieter in daily use, has better/more available features, is more comfortable, and has a sporty demeanor to it's drive (if not a race car feel like the WRX).
I loved my WRX, and find the current one a lot of fun! I suppose individual taste and priority will dictate cross shopping these two due to their numerous and vast differences.