The parish of Fersfield sits in southern Norfolk, about 4 miles northwest of the market town of Diss. Fersfield is not a distinct village as such but a collection of farms and cottages spread along a broad lane which runs north from the A1066 at South Lopham. Fersfield is sited at about 40 metres above sea level and is on rich agricultural land to the north of the Waveney flood plain.

St Andrew's church lies adjacent to the broad lane running through the settlement and is largely Perpendicular in style like most Norfolk churches. There are indication of an earlier building as the southern doorway and western windows of the aisle are of the 13th century. The chancel is a rebuild from the Victorian period. The rather strange tower had even Pevsner baffled stating that it could be "of any date". The church is sited in a graveyard with mature trees providing much shade and restricting the angles for photography.