Malaysians are not allowed dual citizenship. There are those who are not careful and thus lost their Malaysian citizenship at custom. The idea is to how well one hides one's foreign (non-Malaysian) passport from Malaysian custom.
Why would Malaysians do this? To enjoy the benefits of not one (their non-Malaysian country) but both (or more) countries' benefits, especially when it comes to retirement. And it's definitely a hush hush thing to do, because, Malaysian government encourages people to report any Malaysian who is suspected of carrying multiple citizenship.
But many have done so, and here's the trick around getting caught at custom (by dyl88 of the forum linked above), unless of course, if you get reported by a friend or family or an enemy who knows about your multiple nationalities:
If you want to use both passports then the best way to go is follow this process:
1. Book the tickets under your Malaysian passport
-Travelling to Malaysia
2. Show Malaysian passport at airline check in (OZ)
3. Show Australian passport at leaving immigration (OZ)
4. Show Malaysian passport at arriving immigration (Malaysia)
-Travelling back to Australia
5. Show Australian passport at airline check in (Malaysia)
6. Show Malaysian passport at leaving immigration (Malaysia)
7. Show Australian passport at arriving immigration (Australia)
That's the standard process for travelling on two passports. It's pretty rare for immigration to check stamps now so I don't think you'll run into any problems there. Also obviously keep your Australian passport hidden when you're dealing with Malaysian authorities.