That's the thing. Even before the line was extended to Darwin, The Ghan has been a premium grade tourist experience. The last time it was even slightly close to being a conventional public transport service was in the late 1970s when it last ran on the old route which went through a few towns and hamlets.
For decades people have known that if you want to get to Alice Springs or Darwin cheaply, get a flight on a low service discount airline like Tiger or Cattle-Star or, if you want to travel there slowly and uncomfortably sitting up the whole way, then get a bus.
Essentially the part of the public transport market that wants to travel by rail is so miniscule that it makes no sense to serve it when aeroplanes are so vastly cheaper than any rail service could ever be. However there ARE people prepared to fork out for a premium tourist experience on a train. So quite sensibly, GSR caters to the market that does exist and is prepared to pay for the big cost of operating the service.