In addition to the obvious peeling, run your hand across the painted surface. Did you get some white residue on your hand? If so, this is called "Chalking" This is a sign that it's time to repaint. The paint film is breaking down and losing it's protective qualities. It's usually accompanied by a "washed out" look. Chalking is a normal part of paint aging. High quality 100% Acrylic exterior paints don't do this as much as lower quality paint. This is why you do NOT skimp on the quality of your paint!
2: The Caulking. This can be a tough one. There are many places where caulk belongs and many places where it does not. I won't cover all of that here, but I will in a future post. Caulk is designed to be a flexible sealant that helps to keep water and air from intruding into your home. The two most important places to look are around your windows and doors and above all of your horizontal trim boards. There should be NO gaps or cracks at these joints. Cheap caulk (also known as "painter's caulk" for some reason) will crack within a couple of years. It's just nor very flexible. I like ti use Urethane Modified Latex Caulk with a 55-year rating.
3: Siding and Trim. If you have wood trim or any siding made from wood or wood products, you probably have a problem somewhere! Much of these problems are caused by poor installation methods or poor finishing. Many of these problems can also be caused by defective or missing caulking. Look for buckling in the siding, moist or rotting wood, excessive insect activity, etc.You can clearly see the buckling in the siding in this picture. Unfortunately there is really nothing that can be done to fix this besides replace the damaged siding. Many times this doesn't create any problems, but it can also open up gaps to allow water to get in. Generally there is no damage to the framing. It's usually a pretty easy fix, we do it all the time.
This is typical of "hardboard" siding that is so prevalent in the Midwest. You can see the bottom has taken on water and has deteriorated beyond repair. Simply painting the bottom edge a few years ago would have prevented this. Now it will need to be replaced.
It's a little hard to see, but the sill on this double-hung window is badly rotten. Normally, this is relatively easy to fix. Unfortunately, the middle window sash is rotten as well. This window had to be completely replaced. This goes to show you that if you spend a little money on maintenance and prevention you won't have to spend a lot on replacement. The repair would have cost less than $500.00. The cost to replace and finish the window were over $2,000.