The location is stongly associated with the Battle of Sedgemoor which occurred on the 6th of July 1685. This was the last battle of the Monmouth rebellion where the Duke and his rebels lost against the royalists. Many of the prisoners were held in the church before being taken off for trial and ultimately, execution.
The church in question is St Marys church which is a 15th century perpendicular style building. Inside is an area devoted to information about the Battle of Sedgemoor.
Just down from the church is a medieval pub called The Sedgemoor Inn.
Just outside of the town is Westonzoyland Pumping Station Museum which houses some working examples of steam pumps.
To the east of the village is a small decommissioned WWII airstrip where enthusiasts now fly microlight aircraft.
Photos relating to Westonzoyland can be found here.