According to Wikipedia:

The Reformed Church of Beacon, originally the Reformed Dutch Church of Fishkill Landing, is a congregation of the Reformed Church in America. It claims to be the oldest church in Beacon, New York, United States. It is located on NY 9D about 0.5 miles (1 km) south of Beacon’s municipal complex and downtown area. It overlooks the Hudson River from the top of a bluff.

It was designed by Frederick Clarke Withers (who also designed nearby Tioronda and the Tioronda School) and built in 1859. It is one of the rare churches he designed in the Victorian Gothic style, and one of the earlier uses of that style in the United States. The church and its cemetery were added to the National Register of Historic Places in 1988. It is one of three Withers buildings in the city on the Register.

The Church’s own website provides information on the church’s history along with some pictures of the interior (it was closed the day I went so I couldn’t get any interior shots).

Behind the church is a fascinating abandoned cemetery.