Failed Travis builds don't produce a notification or a yellow cat icon.
I only just installed CatLight, so this was never working for me. I get notifications for started builds and successful builds, just not failed builds, which are the only notifications I truly care about. I'm using Travis for Open Source. My notification settings are set to show for all builds, and when builds start, succeed, and fail.
Edit: I had also set CatLight to poll every five seconds for the sake of testing.