What does it mean for a gas furnace to be inefficient?

You are right, burning a unit of gas always produces the same amount of heat. But what is done with the energy is what you are concerned with here. In "efficient" furnaces, a lot of that heat is used for some purpose, i.e.

, making your house warm. In "inefficient" furnaces, a lot of that heat goes out the exhaust, or makes the furnace itself warm. It produces heat, but the heat is not useful.

An extremely efficient furnace wastes almost no heat through the exhaust. It would have exhaust that was nearly the same temperature as the incoming air used to combust the gas. In fact it might not even need a chimney, just a small tube running out the side.

The exhaust is so cool that the water created by combustion condenses out, so these furnaces are called condensing furnaces. They need some way to get rid of that water, which is dirty and corrosive. These guys might use 30% less gas to maintain a given level of usable heat than older furnaces.

The usual unit for measuring the "efficiency" of furnaces is "annual fuel utilization efficiency" (AFUE). Old ones might be in the 60% range, while new ones might be 90% and up. Now mind you, a more efficient furnace might not save much in fuel costs if your house is poorly insulated, or if you have leaky ducts, etc. The AFUE just measures how much useful heat the furnace produces, not what happens to the heat after it leaves the furnace.

If that heat ends up outside, then it is not doing you much good.