Settlement in Mittagong developed early in the history of the area. In 1816 Macquarie referred to Mittagong as "Marragan or Minnikin and than James Meehan spelt it Mittagong in March 1818. James Oxley also spelt it this way two years earlier. In its early application Mittagong referred to the range, which ends in the Gib but in March 1818, Meehan mentioned "Mittagong Flat". A village was formed at the Iron Works called "Nattai" and than known as "New Sheffield". In 1867 the railway was completed and called "Mittagong". In 1848 Thomas Moore gave evidence before the Select Committee on Railways that the neighbourhood of Chalkers Flat was a place where a station could be formed. This area is now known as Lower Mittagong where William Chalker had his property in 1821. A permit was granted to him in 1821 that allowed him to go through the cow pastures with 93 head of cattle with 2 stock keepers to graze his stock on the Mittagong Range. William Chalker was the Principal Overseer of Government Stock at the Cow Pastures and for his services received 200 acres of land but he died before the grant was surveyed. This land was later granted to Elizabeth Sheckell, his widow, who remarried after his death. William Chalker is regarded as a pioneer of the district of Mittagong and his land climbed over the Mittagong Ranges that is now known as Old South Rd.

In 1827 George Cutler built a building to be used as an inn, which was originally granted to Robert Plumb. This land is known as Portion 73, Parish of Mittagong.

Cutler ended up being fined as he anticipated the issue of a license and began selling alcohol. In July 1830 a free license was issued for 1 year and the inn was named The Kangaroo.

For some years Charles Sturt resided on a property on Old South Road. In 1836, Sturt as a dairyman employed the notorious bushranger Martin Cash. He began milking two cows, which later increased to eight. Cash said Sturt was a quiet man and was rarely seen.

On Rowe's Hill in the 1850's the first church was built which was also used as a school. The building was a slab and barks structure, which was replaced by a better building with the foundation stone being, laid on the 28th August 1861. In 1873 services ceased and the building demolished with two of the stones being built into the extensions of St Stephens Church, Mittagong in 1928.

From 1820 to 1835 traffic to the south used the old road that went through Lower Mittagong. When the new line through Berrima opened, the highway over the Mittagong Range and to Bong Bong was deserted. Lower Mittagong was the first pioneer settlement of the district.

In 1869 a visitor to the area describes passing through a number of farms and homesteads with cleared pasture country and fertile lands in a more or less satisfactory state of cultivation.

The Government leased 100 acres in 1895 to supply the cottage homes with wood and vegetables. The Southwood Estate was later leased with these properties covering an area of 540 acres being purchased. Twenty-two men from an asylum cleared and cultivated the land. Some of these men were then employed in boot repairing and making clothes for the Homes.

This property was then used as a home for wayward boys with a building for 50-60 boys being built in 1906. Since then many more buildings have been erected. A modern school was built in 1959 with 12 houses, which can accommodate 250 boys now standing on the property.

The Marist Brothers with the idea of building a sanatorium purchased Dr Chisholm Ross' Rowes Hill farm with 400 acres. In 1917 The Australian Juniorate of the Order was moved to Lower Mittagong and additions were made to the building in July 1923. In April 1930 new buildings for the Juniorate were opened and blessed.

Major Mitchell's new road was constructed through Mittagong and to cater for travellers, Edward Chalker got a license for the King Arms on the 27th June 1836 and than George Cutler got a license "on the new line of road" for the Kangaroo Inn on the 13th August 1836. In 1837 Chalker renewed his license and than on the 26th June 1832 John Chalker got a license for the Woodpack In at the Nattai Bridge. William McGrath held the license for the Kangaroo Inn in 1838 but in 1839 the licensee was Ann Cutler and she continued to hold the license, as did John Chalker for the Woodpack Inn. The Kangaroo Inn was situated at the southern end of Mittagong with the Woodpack Inn at the northern end. Ann Cutler employed Alexander Brand in 1845 to build a new inn, which was named the Fitzroy Inn. This building became known as "Oaklands" and still standing and is used as a guesthouse.

A subdivision of land known as "Town of Gainsbrough" had an attempt at being sold in February 1841, and than a subdivision called "Livingstone Township" was also offered for sale but both appeared to have failed. No land was then subdivided until the iron works began with company using some of its land as housing.

In 1865 the iron works were situated on the right hand side going south with the workers housing located on the opposite side. The houses were on the side of a gently sloping hill and were either slab or bark huts.

In August 1861 it was suggested by Surveyor Campbell that a portion of land close to the mine be reserved for village purposes as he thought that in time the position might become the center of a large manufacturing district. The reserve was made in 1862 and named "Village of Fitzroy".

In 1863 the Government intended to form a principal station on the southern line near the iron works. A subdivision with 245 blocks in the township of "New Sheffield" was offered for sale in May 1865. Many attended the sale with £3000 worth of land sold.

In January 1866 a news report stated that during the previous months the importance of the "New Sheffield" had increased with many dwellings made out of slab, bark or corrugated iron the township would know have at least two hotels, stores, a Wesleyan Church and some weatherboard cottages. Many structures were erected with not a week passing were a building being started or finished. In March 1867 the station in Mittagong came into use and adjoined the town on a small flat with bush all around it.

The prosperity of the town relied on the iron works and when it periodically closed it stagnated.

John McCullum, a native of Argylshire arrived in New South Wales in 1860 founded the coach building in Mittagong that was very prosperous for many years. He worked on a Government contract and than ASN Co employed him to fit up a steamer. He worked on railway construction for five years and than entered into a partnership with a man by the name of Stobo at the blacksmiths and general wheelwrights in 1866. After the dissolution of the partnership McCallum continued the business till he died in 1882. McCallums two sons Hector and Archibald Duncan continued the business for many years. The business manufactured all descriptions of vehicles as well as ploughs and ironwork. The brothers erected a building in Bowral between 1903/1904 where business was conducted until 1908 when it was sold.

There was a demand for building sites in the 1880's with the land required not being available. The Mittagong Land Company had acquired1500 acres of iron works land and in 1883 they subdivided 140 acres. In April 1884 they put the land up for auction and sold 120 acres for £13809. Business site sold from £1/15/- to £10/10/- per foot with housing sites varying from £8/- to £1/6/- per foot. Houses began to be erected later in the year.

In 1890 many splendid buildings were erected and in 1910 the building trade was reported to be busy with cottages being built. In 1921 there was a large building boom with Horton's and WH Jones' new stores built, the Roman Catholic Church was built at a cost of almost £2000 and number of cottages were erected. In 1926 84 building applications were approved with a new convent and wing being added at Frensham.