Shooting with your camera plumb-n-level, that is, keeping the vertical and horizontal lines in your viewfinder plumb and/or level, is often a good idea.
Shooting a pretty, bikini-clad girl at the beach is a good example: If the ocean's horizon is sloping, even a small amount, it is often not a good thing. In fact, it can be downright distracting and mess up an otherwise good capture.
Yeah, you can make leveling adjustments when cropping but, sometimes, that inhibits other ways you might want to crop. My advice: Better to keep things plumb/level when you shoot--leastwise, if that's what you're looking to do--in order to make your work in post go a bit more smoothly.
There are plenty of times, however, when shooting with a canted camera (not to be confused with a candid camera) adds appealing dynamics to a photo. I have no clue what the human psychology might be that drives the heightened interest that sometimes is a result of Dutch Angles and canted cameras. It's enough for me to know they can be interesting and worth doing. Certainly not all the time but, at least, some of the time.
Diagonal lines can also add much interest to a photo. They often add a sense of depth, drama, interesting juxtaposition, and can perform aesthetic wonders for your composition. Same goes for canted cameras, also known as oblique angles, whether the lines are curvy (as with shooting pretty girls) or generally straight.
While plumb-and-level tends to maintain order in the world, at least the world as seen through your viewfinder, diagonals and oblique camera angles unbalance that order and, often enough, creates more interest in the photo. It's all about the mood or viewer perception you're hoping to effect.
Like shooting from less-seen angles and positions, shooting with an eye for diagonals, whether to lead the viewer or to simply heighten interest, and utilizing canted or oblique angles, can be powerful, interesting, and draw more attention to your work. I'm not saying these are always the "makes-sense" ways to approach your subjects. Instead, they are alternative ways of doing things. Ways that can be quite appealing and powerful, photographically speaking.
The pretty girl at the top is Sasha, snapped at a very dark and cramped interior location in Hollywood last year. Subtle yet effective diagonals can be included in posing models. Forming triangular shapes with arms or legs is a way to accomplish that as triangles always have at least one diagonal line as part of their form. A Rembrandt patch makes a nice triangle too!