Kellie castle was built in 1909 by William Kellie Smith as a symbol of his love for his wife, Agnes. The design is similar to the Moghul Castle in Agra India. There are 14 rooms in the castle and also a secret 500 meter underground tunnel which end to hindu temple.

The Kellie's Castle was meant to be the center of the wealthy colonial planters and administrators to socialize. The mansion was built with a six storey tower, magnificent columns and even a wine cellar. There was also to be a rooftop courtyard for holding parties and an elevator which was at that time, the first in the country of Malaysia. Though plans were grand and full of luxury, the building was never completed. Construction began in 1915 but came to a halt in December 1926 with the unexpected death of its owner, William Kellie Smith. After the death of Smith, the estate on which the mansion was located was sold and the mansion, too became part of the deal in the selling of the estate.