When you look at a random piece of sky, you're seeing light that is being emitted by gas molecules in the atmosphere. They are radiating light in all directions, including towards you. The energy for this comes from sunlight that they absorb. Because they tend to absorb short-wavelength blue light more than longer wavelength light, their radiation is also blue.
The ocean is a little more complicated... First, as you mentioned it reflects the blue light from the sky. Second, just as the gas molecules absorbed and redirected blue light towards you, water molecules in the ocean will do the same thing. Also, the ocean contains microorganisms that are blue-green and contribute to the color.
|