Right clicking on a channel button in the texture viewer will select it alone. If it is already the only channel selected, the meaning is inverted and all others will be selected.
This is most useful for quickly toggling between RGB and Alpha-only views.