This location was the Callander train station for many years. On July 19, 1884, it was announced in the New York Times that the Northern and Northwest Railway would be extended north from Gravenhurst to Callander, to be open by 1886. The first train reached Callander on January 27, 1886, although regular service, by this time starting in North Bay, did not start until 1888. The train line was purchased by the Grand Trunk Railway in 1887, which in 1923 became part of the Canadian National Railway.
One of the issues was agreeing with Callander council which streets would be closed by the tracks and who would pay for the access road beside the tracks. It was finally agreed to have level crossings at Lansdowne and Swale Streets, and have Catherine, Isabella (never built) and Burritt Streets closed at the tracks.
On April 21, 1931, a fire started at the Pacific Hotel, beside the tracks, at the northeast corner of Main and Lansdowne Streets, which burned much of the downtown area. First noticed at 10:00 am, the fire was under control by 1:00 pm. Only because the winds changed were the Orange Hall, churches and school saved.
For several years afterwards, a train car was used as the train station, until a new single story station was built.
After the mills closed, there was less train traffic, the station was closed and the tracks removed in 1973. The path of the tracks can still be seen through town, especially behind the municipal offices. The Shannon Trail follows the old train tracks north of Swale Street.