Exactly how does a comet's tail form? Why doesn't a comet's gravity hold onto the gas?

The most important things acting on the comet and its tail are the gravitational field of the sun, the solar wind (which is a flow of fast-moving charged particles such as protons out of the sun), solar radiation (photons from the sun hitting the comet and particles in the comet's tail) and the gravitational field of the comet itself.

The gravitational field of the sun just causes the comet to orbit the sun. The solar wind and solar radiation “blow” the comet's tail so that it always points away from the star. Comets are commonly referred to as “dirty snowballs,” meaning that they are mostly ice with some dust and other “dirt” mixed in. The part of the comet's tail that we can see is mostly due to the sun's light reflecting off of dust in the comet's tail.

In principle there could be a stable “atmosphere” around a comet if the comet's gravitational field were strong enough to hold on to water and dust. However, comets are very small (typically a few kilometers in diameter), and it turns out that their gravitational field is just too small to hold on to any material that's not physically stuck to the comet.

To be a bit more quantitative, the escape velocity from the surface of a comet (how fast something needs to be moving to leave the comet and never return) is about 1 meter per second, around 2 miles per hour. So, if you were standing on the surface of a comet, you wouldn't find it too difficult to just pick up a snowball and throw it straight up so that it never came back. In fact, you could probably bend down and jump, and you'd exceed the escape velocity and therefore float off into space and never return to the surface of the comet. (Such a thing happened to an astronaut in the movie “Deep Impact.”)

Now, an object's temperature is one way of measuring the average speed of the particles that make up that object. Hot objects are made up of very energetic, fast moving particles, while cool objects are made up of sluggish, slowly moving particles. In order to make the average speed of water molecules faster than 1 meter per second, you have to have the gas warmer than about 0.001 degrees Kelvin, very close to absolute zero (this is about -459 degrees Fahrenheit). This is extremely cold, and a comet in the solar system will never get this cold, even when it's far from the sun.

The upshot of all this is that as soon as the sun heats up the comet enough to cause some of the ice to turn to water vapor, these water molecules are moving fast enough to completely overwhelm the gravitational field of the comet and escape. Then the dust and water vapor are blown back by the solar wind and solar radiation so that the comet's tail points away from the sun.

So no, there is no point where the comet's gravitational field balances the sun's radiation pressure. But this is only because comets are typically so small and therefore have weak gravitational fields. If comets were much more massive, there would be such a point. A good example of this is the Earth, with gravity strong enough to hold on to a fairly thick, warm atmosphere.